fengsilin
3 weeks ago
1 changed files with 166 additions and 0 deletions
Split View
Diff Options
+ 166
- 0
docs/superpowers/specs/2026-06-18-e2e-test-foundation-design.md
View File
| @@ -0,0 +1,166 @@ | |||
| # E2E Test Foundation Design | |||
| ## Goal | |||
| Build a reusable end-to-end test foundation for the admin console, then use the model multidimensional pricing page as the first covered workflow. | |||
| The foundation must verify the real frontend/backend contract, authentication, persistence, and page routing. It should be easy to run locally and suitable for CI later. | |||
| ## Scope | |||
| In scope: | |||
| - Add a Playwright-based E2E test foundation under `web/`. | |||
| - Run the React frontend through Vite. | |||
| - Run the Go backend against an isolated SQLite database. | |||
| - Log in as the generated root user. | |||
| - Cover the model multidimensional pricing workflow under `/console/setting?tab=ratio`. | |||
| - Keep tests deterministic by using temporary files, fixed ports, and isolated storage state. | |||
| Out of scope for the first pass: | |||
| - Docker Compose E2E. | |||
| - MySQL/PostgreSQL E2E matrix. | |||
| - Full relay calls to upstream AI providers. | |||
| - Broad admin console coverage outside the model pricing workflow. | |||
| ## Recommended Approach | |||
| Use real backend + temporary SQLite + Playwright. | |||
| The project already has `@playwright/test` in `web/package.json`. The backend can start with `go run main.go`, and when no users exist it creates the root user with username `root` and password `123456`. Running against a temporary SQLite database gives each test run a clean system without requiring external services. | |||
| This approach is slower than API mocks, but it catches the issues that matter for this feature: frontend/backend response shape, save/load behavior, auth redirects, and persistence across page reloads. | |||
| ## Runtime Architecture | |||
| Playwright will run from the `web/` directory. | |||
| Backend: | |||
| - Started by a Playwright global setup helper. | |||
| - Command: `go run main.go` from the repository root. | |||
| - Environment: | |||
| - `PORT=3001` | |||
| - `SQLITE_PATH=<temp-dir>/e2e.db?_busy_timeout=30000` | |||
| - `SESSION_SECRET=e2e_session_secret_for_local_tests` | |||
| - The helper waits for `/api/setup` or another lightweight HTTP endpoint before running tests. | |||
| - The helper stores backend process metadata so global teardown can stop it. | |||
| Frontend: | |||
| - Started by Playwright `webServer`. | |||
| - Command: `bun run dev -- --host 127.0.0.1 --port 5173` | |||
| - Environment: | |||
| - `VITE_REACT_APP_SERVER_URL=http://127.0.0.1:3001` | |||
| - Base URL: `http://127.0.0.1:5173` | |||
| Browser auth: | |||
| - A setup project logs in once as `root / 123456`. | |||
| - The authenticated browser state is saved to `web/e2e/.auth/root.json`. | |||
| - Feature tests reuse that storage state. | |||
| ## Test Layout | |||
| Proposed files: | |||
| - `web/playwright.config.js` | |||
| - `web/e2e/global-setup.js` | |||
| - `web/e2e/global-teardown.js` | |||
| - `web/e2e/auth.setup.js` | |||
| - `web/e2e/model-pricing.spec.js` | |||
| - `web/e2e/utils/process.js` | |||
| - `web/e2e/utils/wait.js` | |||
| Proposed package scripts: | |||
| - `e2e`: `playwright test` | |||
| - `e2e:ui`: `playwright test --ui` | |||
| - `e2e:headed`: `playwright test --headed` | |||
| - `e2e:install`: `playwright install chromium` | |||
| ## First Workflow: Model Multidimensional Pricing | |||
| The first E2E test should cover the critical user path: | |||
| 1. Open `/console/setting?tab=ratio`. | |||
| 2. Confirm the admin setting page loads. | |||
| 3. Open the `模型多维计费` tab. | |||
| 4. Create or edit pricing for a deterministic test model name, for example `e2e-video-pricing-model`. | |||
| 5. Add dimensions such as `duration` and `quality`. | |||
| 6. Use the generator to produce a linear pricing table. | |||
| 7. Save the config. | |||
| 8. Reload the page. | |||
| 9. Reopen the same model config. | |||
| 10. Assert that dimensions and generated rows are loaded. | |||
| The test should also observe the network response for save/load requests and assert successful API envelopes where practical. This makes contract mismatches visible even if the UI shows a generic toast. | |||
| ## Selectors | |||
| Prefer stable selectors instead of brittle text-only selectors for controls that are hard to disambiguate. | |||
| The implementation should add `data-testid` only where current markup cannot be selected reliably: | |||
| - model pricing tab container | |||
| - create/edit model pricing button | |||
| - model name input | |||
| - dimension editor add row button | |||
| - generator open/apply buttons | |||
| - save button | |||
| Visible text can still be used for high-level navigation when the text is unique and user-facing. | |||
| ## Data Isolation | |||
| Each E2E run uses a new temporary SQLite database path. This prevents tests from depending on local developer data and avoids cleanup requirements for most tables. | |||
| Within the UI, test-created models should use an `e2e-` prefix. If the same database is reused during debugging, tests can safely identify their own records. | |||
| ## Failure Artifacts | |||
| Playwright should collect useful artifacts on failure: | |||
| - screenshot: only on failure | |||
| - trace: retain on failure | |||
| - video: off by default for speed | |||
| These defaults keep normal runs fast while still making UI failures diagnosable. | |||
| ## CI Path | |||
| The initial implementation should work locally first. CI can later run: | |||
| 1. `cd web && bun install` | |||
| 2. `cd web && bun run e2e:install` | |||
| 3. `cd web && bun run e2e` | |||
| No external database or service should be required for the first workflow. | |||
| ## Risks And Mitigations | |||
| Backend startup can be slow. | |||
| - Mitigation: wait on an HTTP endpoint with a clear timeout and useful logs. | |||
| Ports can be occupied locally. | |||
| - Mitigation: use fixed defaults first for simplicity, but allow overrides through `E2E_BACKEND_PORT` and `E2E_FRONTEND_PORT`. | |||
| Chinese UI text may be rendered through i18n and encoding-sensitive files. | |||
| - Mitigation: prefer `data-testid` for critical controls and keep user-facing text assertions limited to stable labels. | |||
| Playwright browsers may not be installed. | |||
| - Mitigation: provide `bun run e2e:install` and document it in script names. | |||
| ## Acceptance Criteria | |||
| - `cd web && bun run e2e` starts the backend and frontend automatically. | |||
| - The run uses an isolated SQLite database. | |||
| - The test logs in as root without manual steps. | |||
| - The model multidimensional pricing workflow can save and reload a generated config. | |||
| - Failing tests produce a trace and screenshot. | |||
| - Existing unit tests and frontend build remain runnable without depending on E2E setup. | |||