Plan: E2E Test Hardening

On this page

Status

COMPLETE

Context

The E2E test suite (83 Playwright tests across 15 spec files) has several fragility patterns that cause false passes or flaky failures. Identified during review after the Rules Cache Race Fix revealed that one E2E failure (cases.spec.ts:19) was caused by a test assuming seeded data always appears on page 1 of results.

Issues (Prioritized)

P0: .catch(() ⇒ false) Silences Real Failures

20+ occurrences across cases.spec.ts, intake.spec.ts, placement.spec.ts, exchange.spec.ts, error-paths.spec.ts.

// Anti-pattern: silently passes when element is missing
if (await element.isVisible({ timeout: 3000 }).catch(() => false)) {
  // do something
}
// else: test does NOTHING — masks broken features

Fix strategy per occurrence:

  • Element MUST exist → Remove the catch, use await expect(element).toBeVisible()

  • Element is genuinely conditional (state-dependent) → Assert the expected alternative in the else branch

  • Test depends on prior test state → Skip with test.skip() or restructure to be self-contained

P1: Pagination-Blind List Assertions

Tests use .data-table td/a with hasText to find items in tables. If test-created data pushes seeded items off page 1, assertions fail.

Fix: Add a findInTable(page, text) helper that paginates through all pages until it finds the text or exhausts all pages.

P2: networkidle Waits

5 occurrences of waitForLoadState('networkidle') which Playwright docs warn against.

Fix: Replace with waitForURL(pattern) (already present in most cases) or waitForLoadState('domcontentloaded').

P3: Overly Broad Flash Assertions

intake.spec.ts:206 — /assessed|submitted|created|error|failed/i matches literally any outcome.

Fix: Narrow to specific expected outcomes.

Files Modified

File Change

tests/e2e/lib/helpers.ts

Add findInTable(page, text) pagination-aware helper

tests/e2e/pages/cases-list.page.ts

expectCaseInTable() uses findInTable

tests/e2e/pages/rules-list.page.ts

expectRuleSetInTable() uses findInTable

tests/e2e/pages/referral-list.page.ts

expectReferralInTable() uses findInTable

tests/e2e/specs/cases.spec.ts

Remove 11 .catch(() ⇒ false) anti-patterns, remove networkidle waits

tests/e2e/specs/intake.spec.ts

Remove 4 catches, narrow overly-broad flash regex, use mandated+immediate to guarantee screening

tests/e2e/specs/placement.spec.ts

Remove 4 catches, tighten inner conditionals

tests/e2e/specs/exchange.spec.ts

Document 1 genuinely-conditional catch

tests/e2e/specs/error-paths.spec.ts

Replace catch with direct form-page assertion, replace networkidle

What We’re NOT Changing (And Why)

  • data-testid attributes: Would touch 39 templates for marginal benefit; CSS selectors are stable because we control the HTML

  • Seed data references: Deterministic and type-checked via TypeScript manifest; changes caught at compile time

  • Keycloak login selectors: Keycloak controls the login page; #username, #password, #kc-login are stable

  • Test execution order: Sequential (1 worker) by design

  • Page object CSS selectors: .data-table, .kv-key, .badge, .flash-banner__text are our own design system classes

Verification

  1. cargo xtask dev restart — clean seed data

  2. cargo xtask e2e — all 83 tests pass

  3. Run E2E 3x to confirm stability

  4. Manually verify that breaking a feature causes the previously-silent test to actually fail

Edit this page · latest