test(e2e): improve auth coverage and authoring support (#34920)

.agents/skills/e2e-cucumber-playwright/SKILL.md

@@ -0,0 +1,79 @@
+---
+name: e2e-cucumber-playwright
+description: Write, update, or review Dify end-to-end tests under `e2e/` that use Cucumber, Gherkin, and Playwright. Use when the task involves `.feature` files, `features/step-definitions/`, `features/support/`, `DifyWorld`, scenario tags, locator/assertion choices, or E2E testing best practices for this repository.
+---
+
+# Dify E2E Cucumber + Playwright
+
+Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
+
+## Scope
+
+- Use this skill for `.feature` files, Cucumber step definitions, `DifyWorld`, hooks, tags, and E2E review work under `e2e/`.
+- Do not use this skill for Vitest or React Testing Library work under `web/`; use `frontend-testing` instead.
+- Do not use this skill for backend test or API review tasks under `api/`.
+
+## Read Order
+
+1. Read [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) first.
+2. Read only the files directly involved in the task:
+   - target `.feature` files under `e2e/features/`
+   - related step files under `e2e/features/step-definitions/`
+   - `e2e/features/support/hooks.ts` and `e2e/features/support/world.ts` when session lifecycle or shared state matters
+   - `e2e/scripts/run-cucumber.ts` and `e2e/cucumber.config.ts` when tags or execution flow matter
+3. Read [`references/playwright-best-practices.md`](references/playwright-best-practices.md) only when locator, assertion, isolation, or waiting choices are involved.
+4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
+5. Re-check official docs with Context7 before introducing a new Playwright or Cucumber pattern.
+
+## Local Rules
+
+- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
+- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
+- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
+- Browser session behavior comes from `features/support/hooks.ts`:
+  - default: authenticated session with shared storage state
+  - `@unauthenticated`: clean browser context
+  - `@authenticated`: readability/selective-run tag only unless implementation changes
+  - `@fresh`: only for `e2e:full*` flows
+- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
+
+## Workflow
+
+1. Rebuild local context.
+   - Inspect the target feature area.
+   - Reuse an existing step when wording and behavior already match.
+   - Add a new step only for a genuinely new user action or assertion.
+   - Keep edits close to the current capability folder unless the step is broadly reusable.
+2. Write behavior-first scenarios.
+   - Describe user-observable behavior, not DOM mechanics.
+   - Keep each scenario focused on one workflow or outcome.
+   - Keep scenarios independent and re-runnable.
+3. Write step definitions in the local style.
+   - Keep one step to one user-visible action or one assertion.
+   - Prefer Cucumber Expressions such as `{string}` and `{int}`.
+   - Scope locators to stable containers when the page has repeated elements.
+   - Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
+4. Use Playwright in the local style.
+   - Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
+   - Use web-first `expect(...)` assertions.
+   - Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
+5. Validate narrowly.
+   - Run the narrowest tagged scenario or flow that exercises the change.
+   - Run `pnpm -C e2e check`.
+   - Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
+
+## Review Checklist
+
+- Does the scenario describe behavior rather than implementation?
+- Does it fit the current session model, tags, and `DifyWorld` usage?
+- Should an existing step be reused instead of adding a new one?
+- Are locators user-facing and assertions web-first?
+- Does the change introduce hidden coupling across scenarios, tags, or instance state?
+- Does it document or implement behavior that differs from the real hooks or configuration?
+
+Lead findings with correctness, flake risk, and architecture drift.
+
+## References
+
+- [`references/playwright-best-practices.md`](references/playwright-best-practices.md)
+- [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md)

.agents/skills/e2e-cucumber-playwright/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "E2E Cucumber + Playwright"
+  short_description: "Write and review Dify E2E scenarios."
+  default_prompt: "Use $e2e-cucumber-playwright to write or review a Dify E2E scenario under e2e/."

.agents/skills/e2e-cucumber-playwright/references/cucumber-best-practices.md

@@ -0,0 +1,93 @@
+# Cucumber Best Practices For Dify E2E
+
+Use this reference when writing or reviewing Gherkin scenarios, step definitions, parameter expressions, and step reuse in Dify's `e2e/` suite.
+
+Official sources:
+
+- https://cucumber.io/docs/guides/10-minute-tutorial/
+- https://cucumber.io/docs/cucumber/step-definitions/
+- https://cucumber.io/docs/cucumber/cucumber-expressions/
+
+## What Matters Most
+
+### 1. Treat scenarios as executable specifications
+
+Cucumber scenarios should describe examples of behavior, not test implementation recipes.
+
+Apply it like this:
+
+- write what the user does and what should happen
+- avoid UI-internal wording such as selector details, DOM structure, or component names
+- keep language concrete enough that the scenario reads like living documentation
+
+### 2. Keep scenarios focused
+
+A scenario should usually prove one workflow or business outcome. If a scenario wanders across several unrelated behaviors, split it.
+
+In Dify's suite, this means:
+
+- one capability-focused scenario per feature path
+- no long setup chains when existing bootstrap or reusable steps already cover them
+- no hidden dependency on another scenario's side effects
+
+### 3. Reuse steps, but only when behavior really matches
+
+Good reuse reduces duplication. Bad reuse hides meaning.
+
+Prefer reuse when:
+
+- the user action is genuinely the same
+- the expected outcome is genuinely the same
+- the wording stays natural across features
+
+Write a new step when:
+
+- the behavior is materially different
+- reusing the old wording would make the scenario misleading
+- a supposedly generic step would become an implementation-detail wrapper
+
+### 4. Prefer Cucumber Expressions
+
+Use Cucumber Expressions for parameters unless regex is clearly necessary.
+
+Common examples:
+
+- `{string}` for labels, names, and visible text
+- `{int}` for counts
+- `{float}` for decimal values
+- `{word}` only when the value is truly a single token
+
+Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
+
+### 5. Keep step definitions thin and meaningful
+
+Step definitions are glue between Gherkin and automation, not a second abstraction language.
+
+For Dify:
+
+- type `this` as `DifyWorld`
+- use `async function`
+- keep each step to one user-visible action or assertion
+- rely on `DifyWorld` and existing support code for shared context
+- avoid leaking cross-scenario state
+
+### 6. Use tags intentionally
+
+Tags should communicate run scope or session semantics, not become ad hoc metadata.
+
+In Dify's current suite:
+
+- capability tags group related scenarios
+- `@unauthenticated` changes session behavior
+- `@authenticated` is descriptive/selective, not a behavior switch by itself
+- `@fresh` belongs to reset/full-install flows only
+
+If a proposed tag implies behavior, verify that hooks or runner configuration actually implement it.
+
+## Review Questions
+
+- Does the scenario read like a real example of product behavior?
+- Are the steps behavior-oriented instead of implementation-oriented?
+- Is a reused step still truthful in this feature?
+- Is a new tag documenting real behavior, or inventing semantics that the suite does not implement?
+- Would a new reader understand the outcome without opening the step-definition file?

.agents/skills/e2e-cucumber-playwright/references/playwright-best-practices.md

@@ -0,0 +1,96 @@
+# Playwright Best Practices For Dify E2E
+
+Use this reference when writing or reviewing locator, assertion, isolation, or synchronization logic for Dify's Cucumber-based E2E suite.
+
+Official sources:
+
+- https://playwright.dev/docs/best-practices
+- https://playwright.dev/docs/locators
+- https://playwright.dev/docs/test-assertions
+- https://playwright.dev/docs/browser-contexts
+
+## What Matters Most
+
+### 1. Keep scenarios isolated
+
+Playwright's model is built around clean browser contexts so one test does not leak into another. In Dify's suite, that principle maps to per-scenario session setup in `features/support/hooks.ts` and `DifyWorld`.
+
+Apply it like this:
+
+- do not depend on another scenario having run first
+- do not persist ad hoc scenario state outside `DifyWorld`
+- do not couple ordinary scenarios to `@fresh` behavior
+- when a flow needs special auth/session semantics, express that through the existing tag model or explicit hook changes
+
+### 2. Prefer user-facing locators
+
+Playwright recommends built-in locators that reflect what users perceive on the page.
+
+Preferred order in this repository:
+
+1. `getByRole`
+2. `getByLabel`
+3. `getByPlaceholder`
+4. `getByText`
+5. `getByTestId` when an explicit test contract is the most stable option
+
+Avoid raw CSS/XPath selectors unless no stable user-facing contract exists and adding one is not practical.
+
+Also remember:
+
+- repeated content usually needs scoping to a stable container
+- exact text matching is often too brittle when role/name or label already exists
+- `getByTestId` is acceptable when semantics are weak but the contract is intentional
+
+### 3. Use web-first assertions
+
+Playwright assertions auto-wait and retry. Prefer them over manual state inspection.
+
+Prefer:
+
+- `await expect(page).toHaveURL(...)`
+- `await expect(locator).toBeVisible()`
+- `await expect(locator).toBeHidden()`
+- `await expect(locator).toBeEnabled()`
+- `await expect(locator).toHaveText(...)`
+
+Avoid:
+
+- `expect(await locator.isVisible()).toBe(true)`
+- custom polling loops for DOM state
+- `waitForTimeout` as synchronization
+
+If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
+
+### 4. Let actions wait for actionability
+
+Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
+
+Good pattern:
+
+- assert a meaningful visible state when that is part of the behavior
+- then click/fill/select via locator APIs
+
+Bad pattern:
+
+- stack arbitrary waits before every action
+- wait on unstable implementation details instead of the visible state the user cares about
+
+### 5. Match debugging to the current suite
+
+Playwright's wider ecosystem supports traces and rich debugging tools. Dify's current suite already captures:
+
+- full-page screenshots
+- page HTML
+- console errors
+- page errors
+
+Use the existing artifact flow by default. If a task is specifically about improving diagnostics, confirm the change fits the current Cucumber architecture before importing broader Playwright tooling.
+
+## Review Questions
+
+- Would this locator survive DOM refactors that do not change user-visible behavior?
+- Is this assertion using Playwright's retrying semantics?
+- Is any explicit wait masking a real readiness problem?
+- Does this code preserve per-scenario isolation?
+- Is a new abstraction really needed, or does it bypass the existing `DifyWorld` + step-definition model?

.claude/skills/e2e-cucumber-playwright

@@ -0,0 +1 @@
+../../.agents/skills/e2e-cucumber-playwright

e2e/AGENTS.md

@@ -165,3 +165,132 @@ Open the HTML report locally with:
 ```bash
 open cucumber-report/report.html
 ```
+
+## Writing new scenarios
+
+### Workflow
+
+1. Create a `.feature` file under `features/<capability>/`
+2. Add step definitions under `features/step-definitions/<capability>/`
+3. Reuse existing steps from `common/` and other definition files before writing new ones
+4. Run with `pnpm -C e2e e2e -- --tags @your-tag` to verify
+5. Run `pnpm -C e2e check` before committing
+
+### Feature file conventions
+
+Tag every feature or scenario with a capability tag. Add auth tags only when they clarify intent or change the browser session behavior:
+
+```gherkin
+@datasets @authenticated
+Feature: Create dataset
+  Scenario: Create a new empty dataset
+    Given I am signed in as the default E2E admin
+    When I open the datasets page
+    ...
+```
+
+- Capability tags (`@apps`, `@auth`, `@datasets`, …) group related scenarios for selective runs
+- Auth/session tags:
+  - default behavior — scenarios run with the shared authenticated storageState unless marked otherwise
+  - `@unauthenticated` — uses a clean BrowserContext with no cookies or storage
+  - `@authenticated` — optional intent tag for readability or selective runs; it does not currently change hook behavior on its own
+- `@fresh` — only runs in `e2e:full` mode (requires uninitialized instance)
+- `@skip` — excluded from all runs
+
+Keep scenarios short and declarative. Each step should describe **what** the user does, not **how** the UI works.
+
+### Step definition conventions
+
+```typescript
+import { When, Then } from '@cucumber/cucumber'
+import { expect } from '@playwright/test'
+import type { DifyWorld } from '../../support/world'
+
+When('I open the datasets page', async function (this: DifyWorld) {
+  await this.getPage().goto('/datasets')
+})
+```
+
+Rules:
+
+- Always type `this` as `DifyWorld` for proper context access
+- Use `async function` (not arrow functions — Cucumber binds `this`)
+- One step = one user-visible action or one assertion
+- Keep steps stateless across scenarios; use `DifyWorld` properties for in-scenario state
+
+### Locator priority
+
+Follow the Playwright recommended locator strategy, in order of preference:
+
+| Priority | Locator            | Example                                   | When to use                               |
+| -------- | ------------------ | ----------------------------------------- | ----------------------------------------- |
+| 1        | `getByRole`        | `getByRole('button', { name: 'Create' })` | Default choice — accessible and resilient |
+| 2        | `getByLabel`       | `getByLabel('App name')`                  | Form inputs with visible labels           |
+| 3        | `getByPlaceholder` | `getByPlaceholder('Enter name')`          | Inputs without visible labels             |
+| 4        | `getByText`        | `getByText('Welcome')`                    | Static text content                       |
+| 5        | `getByTestId`      | `getByTestId('workflow-canvas')`          | Only when no semantic locator works       |
+
+Avoid raw CSS/XPath selectors. They break when the DOM structure changes.
+
+### Assertions
+
+Use `@playwright/test` `expect` — it auto-waits and retries until the condition is met or the timeout expires:
+
+```typescript
+// URL assertion
+await expect(page).toHaveURL(/\/datasets\/[a-f0-9-]+\/documents/)
+
+// Element visibility
+await expect(page.getByRole('button', { name: 'Save' })).toBeVisible()
+
+// Element state
+await expect(page.getByRole('button', { name: 'Submit' })).toBeEnabled()
+
+// Negation
+await expect(page.getByText('Loading')).not.toBeVisible()
+```
+
+Do not use manual `waitForTimeout` or polling loops. If you need a longer wait for a specific assertion, pass `{ timeout: 30_000 }` to the assertion.
+
+### Cucumber expressions
+
+Use Cucumber expression parameter types to extract values from Gherkin steps:
+
+| Type       | Pattern       | Example step                       |
+| ---------- | ------------- | ---------------------------------- |
+| `{string}` | Quoted string | `I select the "Workflow" app type` |
+| `{int}`    | Integer       | `I should see {int} items`         |
+| `{float}`  | Decimal       | `the progress is {float} percent`  |
+| `{word}`   | Single word   | `I click the {word} tab`           |
+
+Prefer `{string}` for UI labels, names, and text content — it maps naturally to Gherkin's quoted values.
+
+### Scoping locators
+
+When the page has multiple similar elements, scope locators to a container:
+
+```typescript
+When('I fill in the app name in the dialog', async function (this: DifyWorld) {
+  const dialog = this.getPage().getByRole('dialog')
+  await dialog.getByPlaceholder('Give your app a name').fill('My App')
+})
+```
+
+### Failure diagnostics
+
+The `After` hook automatically captures on failure:
+
+- Full-page screenshot (PNG)
+- Page HTML dump
+- Console errors and page errors
+
+Artifacts are saved to `cucumber-report/artifacts/` and attached to the HTML report. No extra code needed in step definitions.
+
+## Reusing existing steps
+
+Before writing a new step definition, inspect the existing step definition files first. Reuse a matching step when the wording and behavior already fit, and only add a new step when the scenario needs a genuinely new user action or assertion. Steps in `common/` are designed for broad reuse across all features.
+
+Or browse the step definition files directly:
+
+- `features/step-definitions/common/` — auth guards and navigation assertions shared by all features
+- `features/step-definitions/<capability>/` — domain-specific steps scoped to a single feature area

e2e/features/auth/sign-out.feature

@@ -6,3 +6,13 @@ Feature: Sign out
     And I open the account menu
     And I sign out
     Then I should be on the sign-in page
+
+  Scenario: Redirect back to sign-in when reopening the apps console after signing out
+    Given I am signed in as the default E2E admin
+    When I open the apps console
+    And I open the account menu
+    And I sign out
+    Then I should be on the sign-in page
+    When I open the apps console
+    Then I should be redirected to the signin page
+    And I should see the "Sign in" button

e2e/features/support/hooks.ts

@@ -3,7 +3,7 @@ import { chromium, type Browser } from '@playwright/test'
 import { mkdir, writeFile } from 'node:fs/promises'
 import path from 'node:path'
 import { fileURLToPath } from 'node:url'
-import { ensureAuthenticatedState } from '../../fixtures/auth'
+import { AUTH_BOOTSTRAP_TIMEOUT_MS, ensureAuthenticatedState } from '../../fixtures/auth'
 import { baseURL, cucumberHeadless, cucumberSlowMo } from '../../test-env'
 import type { DifyWorld } from './world'
 
@@ -31,7 +31,7 @@ const writeArtifact = async (
   return artifactPath
 }
 
-BeforeAll(async () => {
+BeforeAll({ timeout: AUTH_BOOTSTRAP_TIMEOUT_MS }, async () => {
   await mkdir(artifactsDir, { recursive: true })
 
   browser = await chromium.launch({

e2e/fixtures/auth.ts

@@ -12,7 +12,7 @@ export type AuthSessionMetadata = {
   usedInitPassword: boolean
 }
 
-const WAIT_TIMEOUT_MS = 120_000
+export const AUTH_BOOTSTRAP_TIMEOUT_MS = 120_000
 const e2eRoot = fileURLToPath(new URL('..', import.meta.url))
 
 export const authDir = path.join(e2eRoot, '.auth')
@@ -39,40 +39,54 @@ const escapeRegex = (value: string) => value.replaceAll(/[.*+?^${}()|[\]\\]/g, '
 
 const appURL = (baseURL: string, pathname: string) => new URL(pathname, baseURL).toString()
 
-const waitForPageState = async (page: Page) => {
+type AuthPageState = 'install' | 'login' | 'init'
+
+const getRemainingTimeout = (deadline: number) => Math.max(deadline - Date.now(), 1)
+
+const waitForPageState = async (page: Page, deadline: number): Promise<AuthPageState> => {
   const installHeading = page.getByRole('heading', { name: 'Setting up an admin account' })
   const signInButton = page.getByRole('button', { name: 'Sign in' })
   const initPasswordField = page.getByLabel('Admin initialization password')
 
-  const deadline = Date.now() + WAIT_TIMEOUT_MS
+  try {
-
+    return await Promise.any<AuthPageState>([
-  while (Date.now() < deadline) {
+      installHeading
-    if (await installHeading.isVisible().catch(() => false)) return 'install' as const
+        .waitFor({ state: 'visible', timeout: getRemainingTimeout(deadline) })
-    if (await signInButton.isVisible().catch(() => false)) return 'login' as const
+        .then(() => 'install'),
-    if (await initPasswordField.isVisible().catch(() => false)) return 'init' as const
+      signInButton
-
+        .waitFor({ state: 'visible', timeout: getRemainingTimeout(deadline) })
-    await page.waitForTimeout(1_000)
+        .then(() => 'login'),
-  }
+      initPasswordField
-
+        .waitFor({ state: 'visible', timeout: getRemainingTimeout(deadline) })
+        .then(() => 'init'),
+    ])
+  } catch {
     throw new Error(`Unable to determine auth page state for ${page.url()}`)
+  }
 }
 
-const completeInitPasswordIfNeeded = async (page: Page) => {
+const completeInitPasswordIfNeeded = async (page: Page, deadline: number) => {
   const initPasswordField = page.getByLabel('Admin initialization password')
-  if (!(await initPasswordField.isVisible({ timeout: 3_000 }).catch(() => false))) return false
+
+  const needsInitPassword = await initPasswordField
+    .waitFor({ state: 'visible', timeout: Math.min(getRemainingTimeout(deadline), 3_000) })
+    .then(() => true)
+    .catch(() => false)
+
+  if (!needsInitPassword) return false
 
   await initPasswordField.fill(initPassword)
   await page.getByRole('button', { name: 'Validate' }).click()
   await expect(page.getByRole('heading', { name: 'Setting up an admin account' })).toBeVisible({
-    timeout: WAIT_TIMEOUT_MS,
+    timeout: getRemainingTimeout(deadline),
   })
 
   return true
 }
 
-const completeInstall = async (page: Page, baseURL: string) => {
+const completeInstall = async (page: Page, baseURL: string, deadline: number) => {
   await expect(page.getByRole('heading', { name: 'Setting up an admin account' })).toBeVisible({
-    timeout: WAIT_TIMEOUT_MS,
+    timeout: getRemainingTimeout(deadline),
   })
 
   await page.getByLabel('Email address').fill(adminCredentials.email)
@@ -81,13 +95,13 @@ const completeInstall = async (page: Page, baseURL: string) => {
   await page.getByRole('button', { name: 'Set up' }).click()
 
   await expect(page).toHaveURL(new RegExp(`^${escapeRegex(baseURL)}/apps(?:\\?.*)?$`), {
-    timeout: WAIT_TIMEOUT_MS,
+    timeout: getRemainingTimeout(deadline),
   })
 }
 
-const completeLogin = async (page: Page, baseURL: string) => {
+const completeLogin = async (page: Page, baseURL: string, deadline: number) => {
   await expect(page.getByRole('button', { name: 'Sign in' })).toBeVisible({
-    timeout: WAIT_TIMEOUT_MS,
+    timeout: getRemainingTimeout(deadline),
   })
 
   await page.getByLabel('Email address').fill(adminCredentials.email)
@@ -95,12 +109,13 @@ const completeLogin = async (page: Page, baseURL: string) => {
   await page.getByRole('button', { name: 'Sign in' }).click()
 
   await expect(page).toHaveURL(new RegExp(`^${escapeRegex(baseURL)}/apps(?:\\?.*)?$`), {
-    timeout: WAIT_TIMEOUT_MS,
+    timeout: getRemainingTimeout(deadline),
   })
 }
 
 export const ensureAuthenticatedState = async (browser: Browser, configuredBaseURL?: string) => {
   const baseURL = resolveBaseURL(configuredBaseURL)
+  const deadline = Date.now() + AUTH_BOOTSTRAP_TIMEOUT_MS
 
   await mkdir(authDir, { recursive: true })
 
@@ -111,25 +126,28 @@ export const ensureAuthenticatedState = async (browser: Browser, configuredBaseU
   const page = await context.newPage()
 
   try {
-    await page.goto(appURL(baseURL, '/install'), { waitUntil: 'networkidle' })
+    await page.goto(appURL(baseURL, '/install'), {
+      timeout: getRemainingTimeout(deadline),
+      waitUntil: 'domcontentloaded',
+    })
 
-    let usedInitPassword = await completeInitPasswordIfNeeded(page)
+    let usedInitPassword = await completeInitPasswordIfNeeded(page, deadline)
-    let pageState = await waitForPageState(page)
+    let pageState = await waitForPageState(page, deadline)
 
     while (pageState === 'init') {
-      const completedInitPassword = await completeInitPasswordIfNeeded(page)
+      const completedInitPassword = await completeInitPasswordIfNeeded(page, deadline)
       if (!completedInitPassword)
         throw new Error(`Unable to validate initialization password for ${page.url()}`)
 
       usedInitPassword = true
-      pageState = await waitForPageState(page)
+      pageState = await waitForPageState(page, deadline)
     }
 
-    if (pageState === 'install') await completeInstall(page, baseURL)
+    if (pageState === 'install') await completeInstall(page, baseURL, deadline)
-    else await completeLogin(page, baseURL)
+    else await completeLogin(page, baseURL, deadline)
 
     await expect(page.getByRole('button', { name: 'Create from Blank' })).toBeVisible({
-      timeout: WAIT_TIMEOUT_MS,
+      timeout: getRemainingTimeout(deadline),
     })
 
     await context.storageState({ path: authStatePath })

e2e/scripts/common.ts

@@ -1,4 +1,5 @@
 import { spawn, type ChildProcess } from 'node:child_process'
+import { createHash } from 'node:crypto'
 import { access, copyFile, readFile, writeFile } from 'node:fs/promises'
 import net from 'node:net'
 import path from 'node:path'
@@ -38,6 +39,10 @@ export const middlewareEnvExampleFile = path.join(dockerDir, 'middleware.env.exa
 export const webEnvLocalFile = path.join(webDir, '.env.local')
 export const webEnvExampleFile = path.join(webDir, '.env.example')
 export const apiEnvExampleFile = path.join(apiDir, 'tests', 'integration_tests', '.env.example')
+export const e2eWebEnvOverrides = {
+  NEXT_PUBLIC_API_PREFIX: 'http://127.0.0.1:5001/console/api',
+  NEXT_PUBLIC_PUBLIC_API_PREFIX: 'http://127.0.0.1:5001/api',
+} satisfies Record<string, string>
 
 const formatCommand = (command: string, args: string[]) => [command, ...args].join(' ')
 
@@ -166,13 +171,16 @@ export const ensureLineInFile = async (filePath: string, line: string) => {
   await writeFile(filePath, `${normalizedContent}${line}\n`, 'utf8')
 }
 
-export const ensureWebEnvLocal = async () => {
+export const getWebEnvLocalHash = async () => {
-  await ensureFileExists(webEnvLocalFile, webEnvExampleFile)
+  const fileContent = await readFile(webEnvLocalFile, 'utf8').catch(() => '')
-
+  return createHash('sha256')
-  const fileContent = await readFile(webEnvLocalFile, 'utf8')
+    .update(
-  const nextContent = fileContent.replaceAll('http://localhost:5001', 'http://127.0.0.1:5001')
+      JSON.stringify({
-
+        envLocal: fileContent,
-  if (nextContent !== fileContent) await writeFile(webEnvLocalFile, nextContent, 'utf8')
+        overrides: e2eWebEnvOverrides,
+      }),
+    )
+    .digest('hex')
 }
 
 export const readSimpleDotenv = async (filePath: string) => {

e2e/scripts/setup.ts

@@ -1,14 +1,15 @@
-import { access, mkdir, rm } from 'node:fs/promises'
+import { access, mkdir, readFile, rm, writeFile } from 'node:fs/promises'
 import path from 'node:path'
 import { waitForUrl } from '../support/process'
 import {
   apiDir,
   apiEnvExampleFile,
   dockerDir,
+  e2eWebEnvOverrides,
   e2eDir,
   ensureFileExists,
   ensureLineInFile,
-  ensureWebEnvLocal,
+  getWebEnvLocalHash,
   isMainModule,
   isTcpPortReachable,
   middlewareComposeFile,
@@ -23,6 +24,7 @@ import {
 } from './common'
 
 const buildIdPath = path.join(webDir, '.next', 'BUILD_ID')
+const webBuildEnvStampPath = path.join(webDir, '.next', 'e2e-web-env.sha256')
 
 const middlewareDataPaths = [
   path.join(dockerDir, 'volumes', 'db', 'data'),
@@ -110,27 +112,47 @@ const waitForDependency = async ({
 }
 
 export const ensureWebBuild = async () => {
-  await ensureWebEnvLocal()
+  const envHash = await getWebEnvLocalHash()
+  const buildEnv = {
+    ...e2eWebEnvOverrides,
+  }
 
   if (process.env.E2E_FORCE_WEB_BUILD === '1') {
     await runCommandOrThrow({
       command: 'pnpm',
       args: ['run', 'build'],
       cwd: webDir,
+      env: buildEnv,
     })
+    await writeFile(webBuildEnvStampPath, `${envHash}\n`, 'utf8')
     return
   }
 
   try {
-    await access(buildIdPath)
+    const [buildExists, previousEnvHash] = await Promise.all([
+      access(buildIdPath)
+        .then(() => true)
+        .catch(() => false),
+      readFile(webBuildEnvStampPath, 'utf8')
+        .then((value) => value.trim())
+        .catch(() => ''),
+    ])
+
+    if (buildExists && previousEnvHash === envHash) {
       console.log('Reusing existing web build artifact.')
+      return
+    }
   } catch {
+    // Fall through to rebuild when the existing build cannot be verified.
+  }
+
   await runCommandOrThrow({
     command: 'pnpm',
     args: ['run', 'build'],
     cwd: webDir,
+    env: buildEnv,
   })
-  }
+  await writeFile(webBuildEnvStampPath, `${envHash}\n`, 'utf8')
 }
 
 export const startWeb = async () => {
@@ -141,6 +163,7 @@ export const startWeb = async () => {
     args: ['run', 'start'],
     cwd: webDir,
     env: {
+      ...e2eWebEnvOverrides,
       HOSTNAME: '127.0.0.1',
       PORT: '3000',
     },
@@ -152,14 +175,25 @@ export const startApi = async () => {
 
   await runCommandOrThrow({
     command: 'uv',
-    args: ['run', '--project', '.', 'flask', 'upgrade-db'],
+    args: ['run', '--project', '.', '--no-sync', 'flask', 'upgrade-db'],
     cwd: apiDir,
     env,
   })
 
   await runForegroundProcess({
     command: 'uv',
-    args: ['run', '--project', '.', 'flask', 'run', '--host', '127.0.0.1', '--port', '5001'],
+    args: [
+      'run',
+      '--project',
+      '.',
+      '--no-sync',
+      'flask',
+      'run',
+      '--host',
+      '127.0.0.1',
+      '--port',
+      '5001',
+    ],
     cwd: apiDir,
     env,
   })