Преглед изворни кода

docs: add e2e test foundation design

master
fengsilin пре 3 недеља
родитељ
комит
915efefaec
1 измењених фајлова са 166 додато и 0 уклоњено
  1. +166
    -0
      docs/superpowers/specs/2026-06-18-e2e-test-foundation-design.md

+ 166
- 0
docs/superpowers/specs/2026-06-18-e2e-test-foundation-design.md Прегледај датотеку

@@ -0,0 +1,166 @@
# E2E Test Foundation Design

## Goal

Build a reusable end-to-end test foundation for the admin console, then use the model multidimensional pricing page as the first covered workflow.

The foundation must verify the real frontend/backend contract, authentication, persistence, and page routing. It should be easy to run locally and suitable for CI later.

## Scope

In scope:

- Add a Playwright-based E2E test foundation under `web/`.
- Run the React frontend through Vite.
- Run the Go backend against an isolated SQLite database.
- Log in as the generated root user.
- Cover the model multidimensional pricing workflow under `/console/setting?tab=ratio`.
- Keep tests deterministic by using temporary files, fixed ports, and isolated storage state.

Out of scope for the first pass:

- Docker Compose E2E.
- MySQL/PostgreSQL E2E matrix.
- Full relay calls to upstream AI providers.
- Broad admin console coverage outside the model pricing workflow.

## Recommended Approach

Use real backend + temporary SQLite + Playwright.

The project already has `@playwright/test` in `web/package.json`. The backend can start with `go run main.go`, and when no users exist it creates the root user with username `root` and password `123456`. Running against a temporary SQLite database gives each test run a clean system without requiring external services.

This approach is slower than API mocks, but it catches the issues that matter for this feature: frontend/backend response shape, save/load behavior, auth redirects, and persistence across page reloads.

## Runtime Architecture

Playwright will run from the `web/` directory.

Backend:

- Started by a Playwright global setup helper.
- Command: `go run main.go` from the repository root.
- Environment:
- `PORT=3001`
- `SQLITE_PATH=<temp-dir>/e2e.db?_busy_timeout=30000`
- `SESSION_SECRET=e2e_session_secret_for_local_tests`
- The helper waits for `/api/setup` or another lightweight HTTP endpoint before running tests.
- The helper stores backend process metadata so global teardown can stop it.

Frontend:

- Started by Playwright `webServer`.
- Command: `bun run dev -- --host 127.0.0.1 --port 5173`
- Environment:
- `VITE_REACT_APP_SERVER_URL=http://127.0.0.1:3001`
- Base URL: `http://127.0.0.1:5173`

Browser auth:

- A setup project logs in once as `root / 123456`.
- The authenticated browser state is saved to `web/e2e/.auth/root.json`.
- Feature tests reuse that storage state.

## Test Layout

Proposed files:

- `web/playwright.config.js`
- `web/e2e/global-setup.js`
- `web/e2e/global-teardown.js`
- `web/e2e/auth.setup.js`
- `web/e2e/model-pricing.spec.js`
- `web/e2e/utils/process.js`
- `web/e2e/utils/wait.js`

Proposed package scripts:

- `e2e`: `playwright test`
- `e2e:ui`: `playwright test --ui`
- `e2e:headed`: `playwright test --headed`
- `e2e:install`: `playwright install chromium`

## First Workflow: Model Multidimensional Pricing

The first E2E test should cover the critical user path:

1. Open `/console/setting?tab=ratio`.
2. Confirm the admin setting page loads.
3. Open the `模型多维计费` tab.
4. Create or edit pricing for a deterministic test model name, for example `e2e-video-pricing-model`.
5. Add dimensions such as `duration` and `quality`.
6. Use the generator to produce a linear pricing table.
7. Save the config.
8. Reload the page.
9. Reopen the same model config.
10. Assert that dimensions and generated rows are loaded.

The test should also observe the network response for save/load requests and assert successful API envelopes where practical. This makes contract mismatches visible even if the UI shows a generic toast.

## Selectors

Prefer stable selectors instead of brittle text-only selectors for controls that are hard to disambiguate.

The implementation should add `data-testid` only where current markup cannot be selected reliably:

- model pricing tab container
- create/edit model pricing button
- model name input
- dimension editor add row button
- generator open/apply buttons
- save button

Visible text can still be used for high-level navigation when the text is unique and user-facing.

## Data Isolation

Each E2E run uses a new temporary SQLite database path. This prevents tests from depending on local developer data and avoids cleanup requirements for most tables.

Within the UI, test-created models should use an `e2e-` prefix. If the same database is reused during debugging, tests can safely identify their own records.

## Failure Artifacts

Playwright should collect useful artifacts on failure:

- screenshot: only on failure
- trace: retain on failure
- video: off by default for speed

These defaults keep normal runs fast while still making UI failures diagnosable.

## CI Path

The initial implementation should work locally first. CI can later run:

1. `cd web && bun install`
2. `cd web && bun run e2e:install`
3. `cd web && bun run e2e`

No external database or service should be required for the first workflow.

## Risks And Mitigations

Backend startup can be slow.

- Mitigation: wait on an HTTP endpoint with a clear timeout and useful logs.

Ports can be occupied locally.

- Mitigation: use fixed defaults first for simplicity, but allow overrides through `E2E_BACKEND_PORT` and `E2E_FRONTEND_PORT`.

Chinese UI text may be rendered through i18n and encoding-sensitive files.

- Mitigation: prefer `data-testid` for critical controls and keep user-facing text assertions limited to stable labels.

Playwright browsers may not be installed.

- Mitigation: provide `bun run e2e:install` and document it in script names.

## Acceptance Criteria

- `cd web && bun run e2e` starts the backend and frontend automatically.
- The run uses an isolated SQLite database.
- The test logs in as root without manual steps.
- The model multidimensional pricing workflow can save and reload a generated config.
- Failing tests produce a trace and screenshot.
- Existing unit tests and frontend build remain runnable without depending on E2E setup.

Loading…
Откажи
Сачувај