ADR-015: Axe-Core as a Blocking Pre-Push Accessibility Gate

On this page

Status

Accepted (2026-04-19)

Context

Child welfare information systems fall under Section 508 of the Rehabilitation Act (29 U.S.C. § 794d) — federal and state agencies procuring CRAIG must be able to certify WCAG 2.1 Level AA + Section 508 Chapter 5 (Software) conformance. The CCWIS Final Rule (45 CFR § 1355.52) does not independently mandate Section 508 but ACF’s separate procurement guidance does, and most state contracts name it explicitly.

Before 2026-04-19, the CRAIG roadmap marked accessibility as "partial": axe-core spec infrastructure was in place in tests/e2e/specs/accessibility-audit.spec.ts, but the spec logged violations without failing. Developers could (and did) ship regressions because the gate was cosmetic.

The Accessibility 508 plan completion MR (!104) had three goals:

Make the existing spec a blocking gate so the declared conformance is continuously verified, not a one-time snapshot.
Publish a VPAT 2.5 declaring "Supports" conformance for every applicable criterion.
Document the gate as a durable governance decision so future "the test is slow, let’s turn it off" pressure has a documented answer.

This ADR is goal #3.

Decision

CRAIG’s Playwright accessibility-audit.spec.ts is a blocking pre-push gate. Every push that reaches GitLab has already had axe-core run against:

4 public pages (anonymous role)
19 caseworker pages (authenticated via auth/caseworker.json storage state)
18 admin pages (authenticated via auth/admin.json storage state)

Total: 41 pages audited on every push, with zero WCAG 2.1 Level A + AA + Section 508 violations required for the push to succeed.

The audit spec configuration:

const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'section508'])
    .analyze();

expect(
    results.violations,
    `${name} has ${results.violations.length} accessibility violation(s)`,
).toEqual([]);

The ACCESSIBILITY_AUDIT=1 env-var opt-in that previously gated the spec is removed — the accessibility Playwright project now runs unconditionally under cargo xtask e2e.

Rationale

Continuous vs. point-in-time — a VPAT is worthless if the next MR ships a violation nobody catches until an external audit. A blocking CI gate means the VPAT claim is always true for main.
Policy has a cost — making the gate blocking adds roughly 40 seconds to every pre-push run. That cost is unambiguously worth it compared to the alternative of silent regressions (and the liability exposure in a system processing child welfare data).
Shift-left — catching a color-contrast or missing-aria-label regression on the developer’s machine takes seconds to fix; catching it in user testing takes weeks. Pre-push is the cheapest place to enforce.
No vendor lock-in to axe-core — axe-core is MIT-licensed, installable via npm, runs purely on the rendered DOM. If the library ever goes away we swap it out; the governance decision (audit every page, block on A/AA/Section 508 violations) survives the tooling change.
Documented 41-page floor — any new page added to the craig-web nav MUST also be added to tests/e2e/specs/accessibility-audit.spec.ts. A missing audit is caught by reviewer attention, not tooling; this ADR is the "why this is required" citation.

Consequences

Per-push runtime on a warm devstack: ~40 seconds for the accessibility project alone (part of the broader ~50-second E2E run).
Stale seed references in the audit spec fail hard rather than silently skipping — the cookie-lifecycle completion MR uncovered and fixed home0_douglas → home0_floyd, a regression that had been invisible under the gated run.
Every new craig-web page needs a corresponding entry in the audit spec. The authors' checklist in the project’s delivery protocol includes this.
The VPAT 2.5 document cites the gate as the conformance-maintenance mechanism. If the gate is ever disabled, the VPAT must be updated in the same MR or the claim becomes false.

Enforcement

Pre-push hook: cargo xtask e2e includes the accessibility Playwright project and fails on any violation.
CI: does not re-run E2E (CI carries scans, the no-devstack ci-tests subset, policy jobs, and promotion — never the devstack-dependent battery, per the pre-push-based quality model). The local gate is the only gate.
Reviewer expectation: an MR that adds a new craig-web template without adding a matching audit entry should be rejected during review.

Follow-up

When the constituent and provider portals land (Phase 11), extend the audit spec to cover those pages — the 41-page floor becomes a 50+-page floor without further governance change; this ADR already covers the intent.
Consider adding VPAT 2.6 once the 2.6 template ships (expected 2027).
Review the list of pages annually to ensure coverage matches the live nav — drift is possible.

Copy

View