diff --git a/docs/superpowers/specs/2026-06-18-e2e-test-foundation-design.md b/docs/superpowers/specs/2026-06-18-e2e-test-foundation-design.md new file mode 100644 index 0000000..2184fc1 --- /dev/null +++ b/docs/superpowers/specs/2026-06-18-e2e-test-foundation-design.md @@ -0,0 +1,166 @@ +# E2E Test Foundation Design + +## Goal + +Build a reusable end-to-end test foundation for the admin console, then use the model multidimensional pricing page as the first covered workflow. + +The foundation must verify the real frontend/backend contract, authentication, persistence, and page routing. It should be easy to run locally and suitable for CI later. + +## Scope + +In scope: + +- Add a Playwright-based E2E test foundation under `web/`. +- Run the React frontend through Vite. +- Run the Go backend against an isolated SQLite database. +- Log in as the generated root user. +- Cover the model multidimensional pricing workflow under `/console/setting?tab=ratio`. +- Keep tests deterministic by using temporary files, fixed ports, and isolated storage state. + +Out of scope for the first pass: + +- Docker Compose E2E. +- MySQL/PostgreSQL E2E matrix. +- Full relay calls to upstream AI providers. +- Broad admin console coverage outside the model pricing workflow. + +## Recommended Approach + +Use real backend + temporary SQLite + Playwright. + +The project already has `@playwright/test` in `web/package.json`. The backend can start with `go run main.go`, and when no users exist it creates the root user with username `root` and password `123456`. Running against a temporary SQLite database gives each test run a clean system without requiring external services. + +This approach is slower than API mocks, but it catches the issues that matter for this feature: frontend/backend response shape, save/load behavior, auth redirects, and persistence across page reloads. + +## Runtime Architecture + +Playwright will run from the `web/` directory. + +Backend: + +- Started by a Playwright global setup helper. +- Command: `go run main.go` from the repository root. +- Environment: + - `PORT=3001` + - `SQLITE_PATH=/e2e.db?_busy_timeout=30000` + - `SESSION_SECRET=e2e_session_secret_for_local_tests` +- The helper waits for `/api/setup` or another lightweight HTTP endpoint before running tests. +- The helper stores backend process metadata so global teardown can stop it. + +Frontend: + +- Started by Playwright `webServer`. +- Command: `bun run dev -- --host 127.0.0.1 --port 5173` +- Environment: + - `VITE_REACT_APP_SERVER_URL=http://127.0.0.1:3001` +- Base URL: `http://127.0.0.1:5173` + +Browser auth: + +- A setup project logs in once as `root / 123456`. +- The authenticated browser state is saved to `web/e2e/.auth/root.json`. +- Feature tests reuse that storage state. + +## Test Layout + +Proposed files: + +- `web/playwright.config.js` +- `web/e2e/global-setup.js` +- `web/e2e/global-teardown.js` +- `web/e2e/auth.setup.js` +- `web/e2e/model-pricing.spec.js` +- `web/e2e/utils/process.js` +- `web/e2e/utils/wait.js` + +Proposed package scripts: + +- `e2e`: `playwright test` +- `e2e:ui`: `playwright test --ui` +- `e2e:headed`: `playwright test --headed` +- `e2e:install`: `playwright install chromium` + +## First Workflow: Model Multidimensional Pricing + +The first E2E test should cover the critical user path: + +1. Open `/console/setting?tab=ratio`. +2. Confirm the admin setting page loads. +3. Open the `模型多维计费` tab. +4. Create or edit pricing for a deterministic test model name, for example `e2e-video-pricing-model`. +5. Add dimensions such as `duration` and `quality`. +6. Use the generator to produce a linear pricing table. +7. Save the config. +8. Reload the page. +9. Reopen the same model config. +10. Assert that dimensions and generated rows are loaded. + +The test should also observe the network response for save/load requests and assert successful API envelopes where practical. This makes contract mismatches visible even if the UI shows a generic toast. + +## Selectors + +Prefer stable selectors instead of brittle text-only selectors for controls that are hard to disambiguate. + +The implementation should add `data-testid` only where current markup cannot be selected reliably: + +- model pricing tab container +- create/edit model pricing button +- model name input +- dimension editor add row button +- generator open/apply buttons +- save button + +Visible text can still be used for high-level navigation when the text is unique and user-facing. + +## Data Isolation + +Each E2E run uses a new temporary SQLite database path. This prevents tests from depending on local developer data and avoids cleanup requirements for most tables. + +Within the UI, test-created models should use an `e2e-` prefix. If the same database is reused during debugging, tests can safely identify their own records. + +## Failure Artifacts + +Playwright should collect useful artifacts on failure: + +- screenshot: only on failure +- trace: retain on failure +- video: off by default for speed + +These defaults keep normal runs fast while still making UI failures diagnosable. + +## CI Path + +The initial implementation should work locally first. CI can later run: + +1. `cd web && bun install` +2. `cd web && bun run e2e:install` +3. `cd web && bun run e2e` + +No external database or service should be required for the first workflow. + +## Risks And Mitigations + +Backend startup can be slow. + +- Mitigation: wait on an HTTP endpoint with a clear timeout and useful logs. + +Ports can be occupied locally. + +- Mitigation: use fixed defaults first for simplicity, but allow overrides through `E2E_BACKEND_PORT` and `E2E_FRONTEND_PORT`. + +Chinese UI text may be rendered through i18n and encoding-sensitive files. + +- Mitigation: prefer `data-testid` for critical controls and keep user-facing text assertions limited to stable labels. + +Playwright browsers may not be installed. + +- Mitigation: provide `bun run e2e:install` and document it in script names. + +## Acceptance Criteria + +- `cd web && bun run e2e` starts the backend and frontend automatically. +- The run uses an isolated SQLite database. +- The test logs in as root without manual steps. +- The model multidimensional pricing workflow can save and reload a generated config. +- Failing tests produce a trace and screenshot. +- Existing unit tests and frontend build remain runnable without depending on E2E setup.