17 KiB

Raw Blame History

E2E Test Foundation Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a reusable Playwright E2E foundation and cover the model multidimensional pricing save/reload workflow.

Architecture: Playwright runs from web/, starts the React app through Vite, and starts the Go backend through a global setup helper. The backend uses a temporary SQLite database and a fixed test session secret so each E2E run is isolated and repeatable.

Tech Stack: Playwright, Bun, React/Vite, Go backend, SQLite.

File Structure

Create web/playwright.config.js: Playwright configuration, projects, web server, artifacts, and env wiring.
Create web/e2e/global-setup.js: start backend with temporary SQLite and wait for readiness.
Create web/e2e/global-teardown.js: stop backend process and remove temporary database directory.
Create web/e2e/auth.setup.js: log in as root / 123456 and save storage state.
Create web/e2e/model-pricing.spec.js: first real E2E workflow for model multidimensional pricing.
Create web/e2e/utils/process.js: cross-platform process spawn/kill helpers.
Create web/e2e/utils/wait.js: HTTP polling helper.
Modify web/package.json: add E2E scripts.
Modify web/src/components/settings/RatioSetting.jsx: add data-testid to the model pricing tab content.
Modify web/src/pages/Setting/Operation/ModelPricing/index.jsx: add stable selectors for the page, add button, search input, and table.
Modify web/src/pages/Setting/Operation/ModelPricing/Editor.jsx: add stable selectors for model input, save, generator, and JSON editor.
Modify web/src/pages/Setting/Operation/ModelPricing/DimensionEditor.jsx: add stable selectors for add dimension and editable cells.
Modify web/src/pages/Setting/Operation/ModelPricing/GeneratorModal.jsx: add stable selectors for base price, add ratio, apply, and ratio inputs.

Task 1: Add Playwright Scripts

Files:

Modify: web/package.json
Step 1: Add E2E scripts

Add these entries inside scripts:

"e2e": "playwright test",
"e2e:ui": "playwright test --ui",
"e2e:headed": "playwright test --headed",
"e2e:install": "playwright install chromium"

Step 2: Verify package scripts parse

Run:

cd web
bun pm pkg get scripts.e2e

Expected output includes:

"playwright test"

Step 3: Commit

git add web/package.json
git commit -m "test: add e2e scripts"

Task 2: Add Backend Process Helpers

Files:

Create: web/e2e/utils/process.js
Create: web/e2e/utils/wait.js
Step 1: Create process helper

Create web/e2e/utils/process.js:

import { spawn } from 'node:child_process';

export function startProcess(command, args, options = {}) {
  const child = spawn(command, args, {
    cwd: options.cwd,
    env: { ...process.env, ...(options.env || {}) },
    shell: process.platform === 'win32',
    stdio: ['ignore', 'pipe', 'pipe'],
  });

  const output = [];
  child.stdout.on('data', (chunk) => output.push(chunk.toString()));
  child.stderr.on('data', (chunk) => output.push(chunk.toString()));

  child.once('exit', (code) => {
    if (code !== null && code !== 0) {
      output.push(`\nprocess exited with code ${code}\n`);
    }
  });

  return { child, output };
}

export async function stopProcess(child) {
  if (!child || child.killed) return;
  await new Promise((resolve) => {
    child.once('exit', resolve);
    child.kill(process.platform === 'win32' ? undefined : 'SIGTERM');
    setTimeout(() => {
      if (!child.killed) child.kill('SIGKILL');
      resolve();
    }, 5000).unref();
  });
}

Step 2: Create HTTP wait helper

Create web/e2e/utils/wait.js:

export async function waitForHttp(url, options = {}) {
  const timeoutMs = options.timeoutMs || 60000;
  const intervalMs = options.intervalMs || 500;
  const start = Date.now();
  let lastError;

  while (Date.now() - start < timeoutMs) {
    try {
      const response = await fetch(url, { cache: 'no-store' });
      if (response.ok) return response;
      lastError = new Error(`HTTP ${response.status}`);
    } catch (error) {
      lastError = error;
    }
    await new Promise((resolve) => setTimeout(resolve, intervalMs));
  }

  throw new Error(`Timed out waiting for ${url}: ${lastError?.message || 'no response'}`);
}

Step 3: Commit

git add web/e2e/utils/process.js web/e2e/utils/wait.js
git commit -m "test: add e2e process helpers"

Task 3: Add Playwright Runtime Setup

Files:

Create: web/e2e/global-setup.js
Create: web/e2e/global-teardown.js
Create: web/playwright.config.js
Step 1: Create global setup

Create web/e2e/global-setup.js:

import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { startProcess } from './utils/process.js';
import { waitForHttp } from './utils/wait.js';

const statePath = path.resolve('.playwright/e2e-state.json');

export default async function globalSetup() {
  const repoRoot = path.resolve('..');
  const backendPort = process.env.E2E_BACKEND_PORT || '3001';
  const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'new-api-e2e-'));
  const sqlitePath = path.join(tempDir, 'e2e.db');

  await fs.mkdir(path.dirname(statePath), { recursive: true });

  const backend = startProcess('go', ['run', 'main.go'], {
    cwd: repoRoot,
    env: {
      PORT: backendPort,
      SQLITE_PATH: `${sqlitePath}?_busy_timeout=30000`,
      SESSION_SECRET: 'e2e_session_secret_for_local_tests',
    },
  });

  try {
    await waitForHttp(`http://127.0.0.1:${backendPort}/api/setup`, { timeoutMs: 90000 });
  } catch (error) {
    console.error(backend.output.join(''));
    backend.child.kill();
    throw error;
  }

  await fs.writeFile(
    statePath,
    JSON.stringify({
      backendPid: backend.child.pid,
      tempDir,
      backendPort,
    }),
    'utf8',
  );
}

Step 2: Create global teardown

Create web/e2e/global-teardown.js:

import fs from 'node:fs/promises';
import path from 'node:path';

const statePath = path.resolve('.playwright/e2e-state.json');

export default async function globalTeardown() {
  try {
    const state = JSON.parse(await fs.readFile(statePath, 'utf8'));
    if (state.backendPid) {
      try {
        process.kill(state.backendPid);
      } catch {}
    }
    if (state.tempDir) {
      await fs.rm(state.tempDir, { recursive: true, force: true });
    }
    await fs.rm(statePath, { force: true });
  } catch {}
}

Step 3: Create Playwright config

Create web/playwright.config.js:

import { defineConfig, devices } from '@playwright/test';

const frontendPort = process.env.E2E_FRONTEND_PORT || '5173';
const backendPort = process.env.E2E_BACKEND_PORT || '3001';

export default defineConfig({
  testDir: './e2e',
  globalSetup: './e2e/global-setup.js',
  globalTeardown: './e2e/global-teardown.js',
  timeout: 60_000,
  expect: { timeout: 10_000 },
  fullyParallel: false,
  retries: process.env.CI ? 1 : 0,
  reporter: [['list'], ['html', { open: 'never' }]],
  use: {
    baseURL: `http://127.0.0.1:${frontendPort}`,
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'off',
  },
  webServer: {
    command: `bun run dev -- --host 127.0.0.1 --port ${frontendPort}`,
    url: `http://127.0.0.1:${frontendPort}`,
    reuseExistingServer: !process.env.CI,
    env: {
      VITE_REACT_APP_SERVER_URL: `http://127.0.0.1:${backendPort}`,
    },
    timeout: 90_000,
  },
  projects: [
    { name: 'setup', testMatch: /auth\.setup\.js/ },
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        storageState: 'e2e/.auth/root.json',
      },
      dependencies: ['setup'],
      testIgnore: /auth\.setup\.js/,
    },
  ],
});

Step 4: Run Playwright list to verify config loads

Run:

cd web
bunx playwright test --list

Expected: command exits successfully and lists no feature tests yet, or only setup if Task 4 has already been completed.

Step 5: Commit

git add web/playwright.config.js web/e2e/global-setup.js web/e2e/global-teardown.js
git commit -m "test: add e2e runtime setup"

Files:

Create: web/e2e/auth.setup.js
Step 1: Create auth setup test

Create web/e2e/auth.setup.js:

import { test, expect } from '@playwright/test';
import fs from 'node:fs/promises';

test('authenticate as root', async ({ page }) => {
  await fs.mkdir('e2e/.auth', { recursive: true });
  await page.goto('/login');

  await page.getByPlaceholder(/用户名|username/i).fill('root');
  await page.getByPlaceholder(/密码|password/i).fill('123456');

  const loginResponse = page.waitForResponse((response) =>
    response.url().includes('/api/user/login') && response.request().method() === 'POST',
  );
  await page.getByRole('button', { name: /登录|login/i }).click();
  const response = await loginResponse;
  expect(response.ok()).toBeTruthy();

  await expect(page).toHaveURL(/\/console/);
  await page.context().storageState({ path: 'e2e/.auth/root.json' });
});

Step 2: Run setup only

Run:

cd web
bunx playwright test --project=setup

Expected: setup test passes and creates web/e2e/.auth/root.json.

Step 3: Ignore generated auth state

If web/e2e/.auth/root.json appears in git status, add this to .gitignore:

web/e2e/.auth/
web/.playwright/
web/playwright-report/
web/test-results/

Step 4: Commit

git add web/e2e/auth.setup.js .gitignore
git commit -m "test: add e2e root login setup"

Task 5: Add Stable Selectors To Model Pricing UI

Files:

Modify: web/src/components/settings/RatioSetting.jsx
Modify: web/src/pages/Setting/Operation/ModelPricing/index.jsx
Modify: web/src/pages/Setting/Operation/ModelPricing/Editor.jsx
Modify: web/src/pages/Setting/Operation/ModelPricing/DimensionEditor.jsx
Modify: web/src/pages/Setting/Operation/ModelPricing/GeneratorModal.jsx
Step 1: Add tab content selector

In web/src/components/settings/RatioSetting.jsx, wrap the model pricing tab content:

<Tabs.TabPane tab={t('模型多维计费')} itemKey='model_pricing'>
  <div data-testid='model-pricing-tab'>
    <ModelPricing />
  </div>
</Tabs.TabPane>

Step 2: Add page-level selectors

In web/src/pages/Setting/Operation/ModelPricing/index.jsx, use these props:

<Card style={{ marginTop: 10 }} data-testid='model-pricing-page'>

<Button data-testid='model-pricing-add-model' icon={<Plus size={16} />} type='primary' onClick={() => setEditingModel('')}>

<Input data-testid='model-pricing-search' placeholder={t('搜索模型')} value={keyword} onChange={setKeyword} style={{ width: 280 }} />

<Table data-testid='model-pricing-table' columns={columns} dataSource={data} rowKey='model' pagination={{ pageSize: 10 }} loading={loading} />

Step 3: Add editor selectors

In web/src/pages/Setting/Operation/ModelPricing/Editor.jsx, add:

<SideSheet
  data-testid='model-pricing-editor'

<Button data-testid='model-pricing-save' type='primary' icon={<Save size={16} />} onClick={save} loading={saving}>

<Input data-testid='model-pricing-model-name' prefix={t('模型')} value={name} onChange={setName} style={{ width: 260 }} disabled={Boolean(modelName)} />

<Button data-testid='model-pricing-open-generator' icon={<Wand2 size={16} />} onClick={() => setGeneratorVisible(true)}>

<TextArea data-testid='model-pricing-json-editor' value={jsonText} onChange={setJsonText} autosize={{ minRows: 18, maxRows: 28 }} />

Step 4: Add dimension selectors

In web/src/pages/Setting/Operation/ModelPricing/DimensionEditor.jsx, add indexed data-testid attributes to editable inputs:

<Input data-testid={`dimension-key-${index}`} value={value} onChange={(key) => update(index, { key })} />

<Input
  data-testid={`dimension-source-path-${index}`}
  value={parsed.path}
  placeholder={parsed.sourceType === DIMENSION_SOURCE_TYPES.advanced ? 'request.options.size' : 'duration'}
  onChange={(path) => update(index, { source: composeDimensionSource(parsed.sourceType, path) })}
/>

<Button data-testid='dimension-add' icon={<Plus size={16} />} onClick={add}>

Step 5: Add generator selectors

In web/src/pages/Setting/Operation/ModelPricing/GeneratorModal.jsx, add:

<Modal
  data-testid='model-pricing-generator'

<InputNumber
  data-testid='generator-base-price'

<Button data-testid='generator-add-ratio' icon={<Plus size={16} />} onClick={addEntry}>

<Button data-testid='generator-apply' type='primary' onClick={apply} disabled={preview.rows.length === 0}>

Step 6: Run frontend build

Run:

cd web
bun run build

Expected: build exits 0. Existing Vite chunk warnings are acceptable.

Step 7: Commit

git add web/src/components/settings/RatioSetting.jsx web/src/pages/Setting/Operation/ModelPricing/index.jsx web/src/pages/Setting/Operation/ModelPricing/Editor.jsx web/src/pages/Setting/Operation/ModelPricing/DimensionEditor.jsx web/src/pages/Setting/Operation/ModelPricing/GeneratorModal.jsx
git commit -m "test: add model pricing e2e selectors"

Task 6: Add Model Pricing E2E Workflow

Files:

Create: web/e2e/model-pricing.spec.js
Step 1: Write the E2E test

Create web/e2e/model-pricing.spec.js:

import { test, expect } from '@playwright/test';

test('model multidimensional pricing can be saved and loaded', async ({ page, request }) => {
  const modelName = `e2e-video-pricing-model-${Date.now()}`;

  await page.goto('/console/setting?tab=ratio');
  await page.getByTestId('model-pricing-tab').waitFor();
  await page.getByTestId('model-pricing-add-model').click();
  await page.getByTestId('model-pricing-editor').waitFor();

  await page.getByTestId('model-pricing-model-name').fill(modelName);

  await page.getByTestId('dimension-key-0').fill('duration');
  await page.getByTestId('dimension-source-path-0').fill('duration');
  await page.getByTestId('dimension-add').click();
  await page.getByTestId('dimension-key-1').fill('quality');
  await page.getByTestId('dimension-source-path-1').fill('quality');

  await page.getByTestId('model-pricing-open-generator').click();
  await page.getByTestId('model-pricing-generator').waitFor();
  await page.getByTestId('generator-base-price').fill('0.2');
  await page.getByTestId('generator-apply').click();

  const saveResponsePromise = page.waitForResponse((response) =>
    response.url().includes(`/api/option/model_pricing/${encodeURIComponent(modelName)}`) &&
    response.request().method() === 'PUT',
  );
  await page.getByTestId('model-pricing-save').click();
  const saveResponse = await saveResponsePromise;
  expect(saveResponse.ok()).toBeTruthy();
  const saveBody = await saveResponse.json();
  expect(saveBody.success).toBe(true);

  await page.reload();
  await page.getByTestId('model-pricing-search').fill(modelName);
  await expect(page.getByTestId('model-pricing-table')).toContainText(modelName);

  const apiResponse = await request.get(`/api/option/model_pricing/${encodeURIComponent(modelName)}`);
  expect(apiResponse.ok()).toBeTruthy();
  const body = await apiResponse.json();
  expect(body.success).toBe(true);
  expect(body.data.schema_version).toBe(1);
  expect(body.data.scope).toBe('model');
  expect(body.data.dimensions.map((item) => item.key)).toEqual(['duration', 'quality']);
  expect(body.data.table.length).toBeGreaterThan(0);
});

Step 2: Run E2E

Run:

cd web
bun run e2e

Expected: setup project logs in, chromium project runs the model pricing workflow, and all tests pass.

Step 3: Commit

git add web/e2e/model-pricing.spec.js
git commit -m "test: cover model pricing e2e workflow"

Task 7: Final Verification

Files:

No new files.
Step 1: Run model pricing unit tests

Run:

cd web
bun test src/pages/Setting/Operation/ModelPricing/pricingConfig.test.js

Expected: all tests pass.

Step 2: Run frontend build

Run:

cd web
bun run build

Expected: build exits 0. Existing chunk warnings are acceptable.

Step 3: Run E2E

Run:

cd web
bun run e2e

Expected: all Playwright projects pass.

Step 4: Review status

Run:

git status --short --branch
git log --oneline -5

Expected: only intentional changes remain uncommitted, or all E2E commits are present if each task committed successfully.

17 KiB Raw Blame History

E2E Test Foundation Implementation Plan

File Structure

Task 1: Add Playwright Scripts

Task 2: Add Backend Process Helpers

Task 3: Add Playwright Runtime Setup

Task 4: Add Login Storage Setup

Task 5: Add Stable Selectors To Model Pricing UI

Task 6: Add Model Pricing E2E Workflow

Task 7: Final Verification

17 KiB

Raw Blame History