Plan: Narrow the Authz Fail-Open Fallback (#786)

On this page

Status

Step Description Status

0

True up + commit the plan & nav link as the branch’s first commit; apply the scoped plan label to #786; amend #786’s AC (fixture tightening, degraded-boot fix, healthy-Denied no longer overridden)

Done (2026-07-18) — plan committed; Plan::Narrow-Authz-Fallback applied; AC amended (10 items)

1

craig-authz + craig-bootstrap: AuthzError::PolicyLoad; lazy_load fault/absent split; degraded boot keeps the REAL source (empty-InMemoryRulesetSource substitution retired); typed seams check_typed + auto_scope_list_typed

Done (2026-07-18) — 9 new engine tests + policy_load_maps_to_internal; craig-authz 123 green

2

craig-security: [lib] split (craig-cases precedent); RED fail-closed tests; retire both fail-open behaviors (3 helpers, 40 direct sites, 10 api files); skip-comment two-class sweep

Done (2026-07-18) — lib+RED 14869307; retire flips 5 tests GREEN; 37/3/40 sweep counts verified; 49 endpoints gain the 500 row; coverage gate clean

3

craig-rules: narrow all THREE admit sites via the pure decision fn allow_bootstrap_fallback(err, claims, action); audit every admit via notify_fallback_admit (5 handlers gain the Extension<RulesEngine> extractor)

Done (2026-07-18) — exhaustive 6×3×6 matrix + envelope tests; shared list_authz_gate_with_bootstrap extracted (40-line lint); 9 endpoints gain the 500 row

4

Policy fixtures: replace the wildcard service-caller row pairs with explicit single-action rows (rule_set → read+list; rule_evaluation → create only — live relay; read/list have no service consumer); healthy-engine service-mutation-denied test

Done (2026-07-18) — 4 fixtures tightened (JSON round-trip byte-stable); 4 black-box pins green against reseeded devstack (deny create/delete; read/list + evaluate relay survive)

5

Contracts & docs: docstrings, ADR-024 + ADR-018, implementation-guide + ADR-003 event catalogs, shared-crates.adoc, api-page regens, ONE consolidated CHANGELOG security entry, .claude/CLAUDE.md N/A

Done (2026-07-18) — ADR-024 + ADR-018 amendments; event catalogs updated; shared-crates 4 spots; api/craig-rules + api/craig-security regenerated (other pages carry PRE-EXISTING drift — excluded, issue filed in Step 6); CHANGELOG entry with the Pre-1.0 callout

6

Follow-ups + reconciliation: #873 amend/split (never close), #908 shape post, evaluate-relay actor-forwarding issue, file anything discovered

Done (2026-07-18) — #873 coverage map + AC scoped to the e2e half (open); #908 shape posted; filed #1060 (relay actor-forwarding), #1061 (api-page drift regen), #1062 (fleet wildcard-row audit); all related to #786; MR !1007

Issue: #786
Branch: feature/786-narrow-authz-fallback

Provenance: user-decided forks — 2026-07-17: (a) RETIRE the security-side fallbacks; (b) wire the fallback-admit audit in this MR. 2026-07-18: (c) audit via the engine notify pattern; (d) fixture rows tightened in this MR; (e) the degraded-boot empty-source substitution is fixed fleet-wide in this MR (real source + empty cache; genuine self-heal). Review rounds (independent contextless lenses per round): round 1 = 21 findings (5 P1); round 2 = 20 (2 P1: PolicyMissing over-breadth via lazy_load; rule_evaluation tightening would break three live relay flows); round 3 = 11 (1 P1: the empty-source substitution re-opens the masquerade → fork e); round 4 = 6 (0 P1: boot_degraded return-shape + self-heal-channel honesty, rule_evaluation list row dropped as dead config); round 5 = 6 (0 P1: utoipa/api-page regen per the #784 convention, shared-crates craig-bootstrap bullet, boot_degraded fallibility); round 6 = 1 P3 (response-code enumeration), security+conventions auditor returned ZERO findings — converged.

Context

Two services carry authz fallbacks that admit callers on engine errors: craig-security fails open through three helpers; craig-rules admits at three distinct sites; and one healthy-path exposure rides along in the seeded policies themselves.

Error model. AuthzError (crates/craig-authz/src/error.rs) has five variants: PolicyMissing, PolicyDenied, and three runtime faults (EngineEvaluation, WorkerUnavailable, MalformedOutput). All five collapse at the ApiError boundary, so a caller matching Err(_) cannot tell "not seeded" from "denied" from "broken". Three deeper defects:

  • PolicyMissing is over-broad at the source — ZenAuthzEngine::lazy_load (engine.rs:316-333) flattens a refresh_entry fault (source fetch failure OR JDM compile failure, engine.rs:201-233) into the same None as "ruleset genuinely absent", and check (engine.rs:392-404) mints PolicyMissing from that None. Left unfixed, a DB blip during a cold-cache miss would still trigger the "narrowed" fallback. refresh_entry already distinguishes the two (Err vs Ok with the source returning None), so the split is a refactor, not new machinery.

  • Degraded boot silently swaps in an empty source that "attests" universal absence — on a double bulk-fetch failure, boot_authz_engine boots the engine with Arc::new(InMemoryRulesetSource::new()) (crates/craig-bootstrap/src/authz.rs:121-128), whose get_by_name returns Ok(None) unconditionally (crates/craig-authz/src/source.rs:118-120). Post-split that would classify as source-attested PolicyMissing for every ruleset, forever — and the helper’s own comment ("Subscriber + TTL-refresh will populate the cache", authz.rs:79-81,108-111) is false today for TWO independent reasons: both refresh paths run against the same empty source, AND the TTL task can never populate an empty cache regardless of source (refresh_stale_entries iterates only names already cached, engine.rs:246-259).

  • On the LIST path the engine can’t signal PolicyMissing at all: auto_scope_list returns Ok(ListScope::Denied) on miss (engine.rs:448-457) — "not seeded" and "denied" are the same value.

Security side (services/craig-security/src/api/mod.rs) — three helpers, all fail-open for admins:

  • authz_check_or_admin_fallback (:81-102) — admits any admin on any error; 37 direct call sites (verified uniform: skeleton_ref shape, Extension(Jurisdiction), plain .await?), including delete_partner, issue_key (mints a plaintext partner API key), approve_signer_key (ADR-018 transition).

  • authz_list_or_admin_fallback (:167-194) — same admit for list scope AND overrides a healthy engine’s Denied to All for admins; 4 direct sites (signer_keys.rs:177, detection.rs:40, partners.rs:435, mod.rs:136 inside the wrapper).

  • authz_list_or_forbid (:130-141) — wraps the list fallback into Denied→403; 9 handler sites.

Total: 40 direct + 9 wrapper sites across 10 api files. craig-security is not the policy source of truth; policies are seeded and coverage-gated; the helpers' own doc-comment names Plan E (archived complete) as the retirement condition. Decision (fork a): retire.

Rules side (services/craig-rules/src/api.rs) — craig-rules IS the policy source of truth, so a real cold-start chicken-and-egg exists. THREE admit sites:

  • rules_authz_or_admin_fallback (:63- region) — admits admin OR any service caller on ANY error, including Create/Update/Delete.

  • list_rule_sets (:198-203) + list_evaluations (:860-864) — inline allow_bootstrap_fallback admits on Ok(ListScope::Denied) from a HEALTHY engine — the same live-override pattern being retired on the security side, and unauditable today.

Decision: narrow all three to genuine bootstrap (PolicyMissing via the typed seams), scope service callers to read shapes, audit every admit.

Healthy-path exposure (fork d). Each of the four seeded fixtures carries two service-caller rows with an empty (wildcard) i_action cell — one whose own description says "full read/list" (rulesets/georgia/georgia-authz-rule_set.json:151-163) and a second fully-wildcard duplicate (:164-175); same twin rows in georgia-authz-rule_evaluation.json and both Texas twins. So a bare service token can Create/Update/Delete authz rulesets through the _healthy engine in every reference deployment. The material exposure is on rule_set (service CRUD of rulesets). The rule_evaluation tightening is defense-in-depth: the only reachable rule_evaluation actions are evaluate (Create, api.rs:761 — evaluates a named ruleset against caller-supplied input; no stored PII read, no cross-entity mutation) and list_evaluations (List, api.rs:855). Decision: tighten in this MR — but the rule_evaluation rows must keep create: three live flows (safety-assessment submit services/craig-cases/src/api/investigations/safety.rs:143-152, person-match auto-link services/craig-cases/src/matching/mod.rs:216, report conversion services/craig-cases/src/api/reports/conversion.rs:201) relay only the raw service bearer from cases to POST /v1/rules/evaluate, which craig-rules gates as RuleEvaluation x Create with evaluation_is_service = true (no actor header on that hop) — the wildcard row is today their only admitting row. Residual service-create capability is tracked by the Step-6 actor-forwarding follow-up.

Neither fallback emits an audit event today. #908 (open) owns the craig-authz library audit-event ADR; this plan’s admit event is a craig-rules service event through the service’s own notify pattern — no #908 prejudice.

Scope

In: AuthzError::PolicyLoad + lazy_load fault/absent split · degraded-boot fix (real source + empty cache, fleet-wide) · typed seams (check + list) · retire the three security helpers' fail-open behavior · narrow all three rules admits · fixture-row tightening (with the create carve-out) + healthy-engine denial test · fallback-admit audit via the engine notify pattern · security [lib] target (test enabler) · red-before-green fail-closed matrices · docstring/ADR/CHANGELOG/canonical-page truth-up.

Out (tracked):

  • craig-authz library audit-event pipeline (cache_miss/cache_refreshed + emission ADR) → #908 (Step 6 posts the event shape there).

  • #873’s Playwright half (mutation-authz-audit.spec.ts) → stays on #873 (amended) or splits to a fresh issue — never closed by this MR.

  • Actor-header forwarding on the cases→rules evaluate relay (so evaluation authz can be actor-scoped and the rule_evaluation create service row can narrow further) → new issue, Step 6.

  • Rules eval-thread wedge/health → #1046.

Design

D1 — Honest error classes: PolicyLoad + the lazy_load split, then typed seams

New variant AuthzError::PolicyLoad(String) (error.rs): lazy-refresh fetch/compile failure. Maps to ApiError::internal in the existing From impl (+ a mapping unit test beside the five existing ones).

lazy_load split — signature becomes Result<Option<CachedRuleset>, AuthzError>:

  • cache hit → Ok(Some);

  • miss → refresh_entry: Ok → re-read cache (Some → Ok(Some); still absent → Ok(None) — the configured RulesetSource itself attested absence via get_by_name → None);

  • Err(e) → keep the warn!, return Err(PolicyLoad(…​)).

All three callers keep their current miss behavior for Ok(None) and propagate Err as-is: check (:392) → PolicyMissing; auto_scope_list (:448) → Ok(Denied) on the untyped path; resolve_field_permission (:477) → Ok(FieldPermission::None) — and truth-up its trait doc-comment ("resolves to FieldPermission::None (not an error)", engine.rs:73-77), which post-split holds only for genuine absence. This makes PolicyMissing precise — "the policy source attests this ruleset does not exist" — the only trigger the narrowed fallback may admit on.

Behavior change on the untyped paths (fail-closed both before and after, now honestly classed — pre-1.0 CHANGELOG callout): a refresh fault during a cache miss surfaces as PolicyLoad → 500, where before it was PolicyMissing → 403 (check), a silent Ok(Denied) (auto_scope_list), or a silent Ok(FieldPermission::None) (resolve_field_permission — no callers outside craig-authz today).

Typed seams — two provided (default) methods on AuthzEngine:

/// Like [`check`], but surfaces the typed [`AuthzError`] so a bootstrap
/// caller (craig-rules only) can distinguish "policy not seeded"
/// (`PolicyMissing`) from a real deny or a runtime fault.
///
/// Default: delegate to `check`, reporting any error as an opaque deny.
/// ONLY `PolicyMissing` carries meaning to callers of this method; the
/// sentinel below never admits anything. Applies only to impls that
/// don't model bootstrap (the test doubles) — `ZenAuthzEngine` overrides
/// with real classes.
async fn check_typed(&self, claims: &Claims, resource: ResourceRef<'_>, action: Action)
    -> Result<(), AuthzError>
{
    self.check(claims, resource, action).await.map_err(|_| AuthzError::PolicyDenied {
        ruleset_name: "unknown (opaque via check() default)".into(),
        reason: "error class unavailable through the untyped check() path".into(),
    })
}

/// Like [`auto_scope_list`], but a cache miss surfaces as
/// `Err(PolicyMissing)` instead of `Ok(ListScope::Denied)` — on the list
/// path those are otherwise the same value, which is exactly the
/// ambiguity the bootstrap fallback needs resolved. Default mirrors
/// `check_typed`: delegate, healthy scopes pass through verbatim, any
/// error becomes the same opaque non-admitting sentinel.
async fn auto_scope_list_typed(&self, claims: &Claims, rt: ResourceType, action: Action, jurisdiction: &str)
    -> Result<ListScope, AuthzError>
{
    self.auto_scope_list(claims, rt, action, jurisdiction).await.map_err(|_| {
        AuthzError::PolicyDenied {
            ruleset_name: "unknown (opaque via auto_scope_list() default)".into(),
            reason: "error class unavailable through the untyped list path".into(),
        }
    })
}

ZenAuthzEngine overrides both, factored typed-primary: check = check_typed(…​).await.map_err(ApiError::from); auto_scope_list = typed variant with Err(PolicyMissing) collapsed back to Ok(Denied) (untyped callers keep today’s miss shape). The typed list override: source-attested absence → Err(PolicyMissing); healthy deny → Ok(Denied); faults (incl. PolicyLoad) → their own Err. The four test doubles (MinimalEngine crates/craig-authz/tests/field_permission.rs:388, AllowAll/DenyAll/FixedScope services/craig-cases/tests/api/keyed_harness.rs:43/63/86) inherit the defaults — zero churn; the defaults can never yield PolicyMissing (provided-default precedent: resolve_field_permission, engine.rs:81-89).

Tests (Step 1): construct via ZenAuthzEngine::boot (engine.rs:127 — there is no new; in-crate precedent: tests/field_permission.rs:197-198, tests/engine_unit.rs:116-117). InMemoryRulesetSource can only return Ok — add a small failing-stub RulesetSource in the test for the fault cases. Matrix: get_by_name errors → check_typed yields PolicyLoad (never PolicyMissing); returns None → PolicyMissing; healthy deny → PolicyDenied. List path: absent → typed Err(PolicyMissing) while untyped auto_scope_list still yields Ok(Denied); fault → PolicyLoad through both. Doubles' defaults never yield PolicyMissing.

D2 — Degraded boot keeps the REAL source (fork e, fleet-wide)

Retire the empty-InMemoryRulesetSource substitution in boot_authz_engine (crates/craig-bootstrap/src/authz.rs:121-128). New constructor ZenAuthzEngine::boot_degraded(source, jurisdiction, service_name, policy_ttl) → Result<(Self, Option<CoverageWarning>)> (fallible exactly like boot — it too must call spawn_eval_thread()?, engine.rs:177, eval_thread.rs:40) — no initial bulk fetch, empty cache, the real source retained; the warning is computed by the existing compute_coverage over the (empty) cache, i.e. the existing CoverageWarning::Missing shape — no new variant (a second variant would break the five irrefutable let CoverageWarning::Missing {..} destructures in crates/craig-authz/tests/engine_unit.rs:125/153/674/724/750). Degraded-ness is surfaced where it matters instead: boot_authz_engine’s double-failure arm, when `require_full_coverage = true, bails inside the arm with a degraded-specific message (bulk fetch failed twice; the source is unreachable — fix source availability; provisioning rulesets won’t help), so the operator isn’t misdirected by the generic "provision missing rulesets" remedy at authz.rs:135-139; with the flag off it proceeds via boot_degraded(spec.authz_source, …​) and the standard coverage warn!. The InMemoryRulesetSource import leaves craig-bootstrap (the type stays in craig-authz as a test source).

Truth-up the helper’s doc-comment + warn! to credit ONLY the true self-heal channels: per-request lazy_load (the always-on channel) and the invalidation subscriber (on ruleset.changed events), plus the post-boot warmup within its bounded window (500ms grace + 20 attempts x 1s, crates/craig-authz/src/invalidation.rs:163-202, authz.rs:165-166). The TTL refresh task must NOT be credited — refresh_stale_entries iterates only names already in the cache (engine.rs:246-259), so it can never populate an empty one (a false runtime guarantee in an ops comment is exactly the J5 class this fork exists to eliminate).

Consequences (CHANGELOG): a degraded-booted service now self-heals (previously: never — it ran on the empty source until restart); during the outage window per-request authz yields PolicyLoad → 500 (honest fault) instead of PolicyMissing-shaped 403s from an empty source that claimed universal absence; and the degraded window’s load profile changes from zero upstream traffic to one real fetch per authz-check miss — accepted deliberately: no coalescing/backoff added (same class as today’s partial-miss lazy_load behavior; each fetch bounded by the shared HTTP client’s 5s connect / 30s total timeouts, and on the rules side it is a local DB query; the recovery herd is bounded — first success populates the cache and it stops). No admitting service can ever run on an empty-attesting source — the D6 invariant needs no degraded-boot carve-out.

Tests: craig-authz unit — boot_degraded yields an engine with an empty cache and the real source (a flip-able stub: fails while "down", then serves rulesets); a check during the outage yields PolicyLoad; after the stub recovers, refresh_entry/lazy_load populates and the same check passes. craig-bootstrap’s boot_authz_engine change is a mechanical arm swap (its integration coverage stays per-service, as today — authz.rs:189-198).

D3 — Retire the security-side fail-open (three helpers, two fates)

  • authz_check_or_admin_fallback: DELETE. All 37 sites repoint mechanically to authz.check(&claims, skeleton_ref(rt, id, jurisdiction), action).await?.

  • authz_list_or_admin_fallback: DELETE. Its 3 handler sites (signer_keys.rs:177, detection.rs:40, partners.rs:435) each use the scope only for an immediate matches!(scope, Denied) → 403 check — repoint to plain authz.auto_scope_list(…​).await? keeping that shape verbatim (do not collapse them into authz_list_or_forbid: that would flip their skip-comment sweep class). Denied stays Denied even for admins — the healthy-engine override dies (seeded policies admit admins today, so no practical change; a test pins the verdict’s authority).

  • authz_list_or_forbid: KEEP the fn (fail-closed Denied→403 convenience; 9 handlers) but reimplement its body over plain auto_scope_list.

An engine fault now returns 500 (deny) instead of admitting an admin. Skip-comment sweep, two classes: direct .check(/.auto_scope_list( handlers → delete the // authz: skip — gated by … comment (the coverage gate pattern-matches the body; a retained comment = stale allowlist entry); the 9 wrapper handlers → comment updated to name the fail-closed wrapper. cargo xtask validate-authz-coverage clean after.

Cold-boot posture (scoped honestly): the seed gate (craig-seed: service_completed_successfully) holds only craig-web closed — craig-security itself is reachable pre-seed on a genuine first start (direct API + intake partner traffic). In that window service callers already fail closed today; the change is admins going silent-admit → 403/500, which IS the fail-closed contract this issue demands. Web-mediated traffic can’t race the seed. Stated in the CHANGELOG entry.

D4 — Narrow all THREE rules-side admits, around ONE pure decision fn

The decision is pure and sees the error class — allow_bootstrap_fallback(err: &AuthzError, claims: &Claims, action: Action) → bool, true only when err is PolicyMissing(_) AND (admin → any action | service caller → Action::Read | Action::List). Everything async composes around it:

  • rules_authz_or_admin_fallback(authz, engine, claims, rt, id, jurisdiction, action) → check_typed; Ok)) passes; Err(e) → if allow_bootstrap_fallback(&e, &claims, action) { notify_fallback_admit (D5), Ok(( } else propagate e as its real ApiError. Service mutations are NEVER fallback-admitted (the docstring’s bulk-fetch cache-warm justification covers Read/List only).

  • list_rule_sets + list_evaluations → switch to auto_scope_list_typed; the inline allow_bootstrap_fallback-on-Denied dies. Ok(scope) is used verbatim — healthy Denied keeps today’s 403 shape (api.rs:201-203, :860-864), no longer overridden for bootstrap principals; Err(e) → if allow_bootstrap_fallback(&e, claims, Action::List) { treat as All + D5 audit } else propagate.

  • Bootstrap-window relay note: during genuine PolicyMissing, the cases→rules evaluate relay (service Create) is no longer admitted — accepted: web-mediated flows are seed-gated (C4-gate), so the window is unreachable for them, and fail-closed is the contract for anything else.

D5 — Audit every admit, via the engine notify pattern (fork c)

New RulesEngine::notify_fallback_admit(…​) mirroring notify_cache_invalidated (services/craig-rules/src/engine.rs:343-363) exactly: one-shot outbox staging through the engine’s own db (begin → stage_event → commit; the OutboxWorker drains asynchronously — the publisher field is not on this path), fire-and-forget — every failure warn!`s and NEVER blocks the admit. Latency honesty: the staging is awaited inline like the pattern it mirrors, so with the DB down each admit stalls up to the pool’s 5s `acquire_timeout before the warn! — accepted (single attempt, no retry loop, bootstrap-window only, bounded peer-service storm). The event_outbox table is created by boot migrations, so it exists pre-seed.

Event: rules.authz_fallback_admit, payload {sub, is_service, resource_type, resource_id, action, jurisdiction, reason: "bootstrap_policy_missing"} — resource_id is Option<Uuid> — pass through the id the wrapper already receives: the path id where the handler has one (update/delete/import/get/export — forensically attributable mutations), Uuid::nil() where the existing gate already passes nil (create_rule_set api.rs:336-343, evaluate), absent on the two list sites. A craig-rules service event (envelope builder lives beside the other envelope construction in craig-rules, not in craig-authz::audit; #908 untouched — Step 6 posts the shape there).

The fallback fns receive &RulesEngine. Five of the ten reaching handlers don’t yet extract it — add Extension(engine): Extension<RulesEngine> to get_rule_set, get_rule_set_by_name, export_rule_set, list_rule_sets, list_evaluations (routes() already layers the extension at api.rs:137; RulesEngine is Clone). The other five (create_rule_set, update_rule_set, delete_rule_set, import_rule_set, evaluate) already hold it.

Test decomposition (honest about seams): RulesEngine is async-constructed against a live DB, so the admit property is NOT unit-tested through the wrapper — it is pinned on the pure fn: allow_bootstrap_fallback is exhaustively table-driven over ALL SIX AuthzError variants x {admin, service, plain user} x ALL SIX actions {Read, List, Create, Update, Delete, Approve} (import=Update and delete are the pins that matter most; non-PolicyMissing — incl. PolicyLoad — never admits, BY the matrix). The envelope PAYLOAD is built by a pure build_fallback_admit_envelope — unit-tested. The staging line is a verbatim instance of the proven notify_cache_invalidated pattern (staging path covered by craig-mq’s own suites); no fake-Publisher seam is invented, and no devstack test deletes shared policy state to force PolicyMissing (would break concurrently-running suites — deliberately rejected).

D6 — Fail-closed is the invariant

After this change: on the security side NO error class admits anyone; on the rules side NOTHING but source-attested PolicyMissing admits (per the D4 table — faults can no longer masquerade as missing policy, and no engine can run on an empty-attesting source); a healthy engine’s Denied is never overridden anywhere; and the seeded policies no longer wildcard service callers. Asserted directly, red-before-green.

Steps

Step 0 — Plan + issue lockstep

True up this plan + the nav Active entry; commit as the branch’s first commit. Apply Plan::Narrow-Authz-Fallback to #786 (auto-drops Plan::NEEDED). Amend #786’s description/AC to cover the review-added surfaces: fixture-row tightening, the degraded-boot fix, healthy-Denied no longer overridden (issues are self-contained; the close-out checks criteria).

Step 1 — craig-authz + craig-bootstrap: honest classes + degraded boot

Per D1 + D2: PolicyLoad variant + From arm (+ mapping test), lazy_load refactor (3 callers + the resolve_field_permission doc truth-up), boot_degraded + the boot_authz_engine arm swap + comment truth-up, both provided methods, both ZenAuthzEngine overrides, untyped delegations. The D1 + D2 test matrices.

Step 2 — Security: lib split, RED tests, retire

  1. [lib] split on the realized craig-cases service-crate precedent (NOT the Cargo.toml:393 "public API surface" warn-set — that note targets shared crates): create src/lib.rs carrying the crate-level attrs currently on main.rs, with the deny(unused_crate_dependencies) reason reworded to the per-lib-target thin-bin wording (services/craig-cases/src/lib.rs:17-23 is the verbatim model); move the module tree + build_router + spawn_workers + subscriber/DLQ handlers into the lib behind pub async fn run(); main.rs becomes the thin delegator (mirror services/craig-cases/src/main.rs). Expose the assembly seam the tests need — api::routes + DeploymentConfig + a router-assembly path accepting a caller-supplied Arc<dyn AuthzEngine> (attach_standard_extensions inputs); services/craig-cases/tests/api/keyed_harness.rs:43-90 is the working doubles-based precedent (devstack Postgres/RMQ backed). sqlx::migrate! is manifest-relative — unaffected. Budget-neutral (B-series counts src/** paths target-independently). Verify boot behavior unchanged.

  2. RED commit: in-process fail-closed tests (fault-injecting authz double → assert the CURRENT fallback ADMITS an admin — these assert today’s defect) for delete_partner, issue_key, approve_signer_key, plus the Ok(ListScope::Denied)-overridden-for-admin list regression.

  3. Retire commit per D3 (3 helpers, 40 direct sites, wrapper reimplemented, 10 files, skip-comment sweep). The RED tests flip to their fail-closed (green) assertions in the same commit; the commit message explains the flips — deliberate red-before-green: defect-asserting tests inverted to the fail-closed contract (this is what J2 and the never-weaken rule key on); the red/green run pair is recorded in the MR. Same commit: utoipa response truth-up on every security endpoint whose response-class set changes (engine faults now surface 500 where the helpers flattened to 403) — the api-page regen lands in Step 5.

  4. cargo xtask validate-authz-coverage clean.

Step 3 — Rules: narrow + audit

Per D4/D5: typed-seam adoption at all three sites, the pure allow_bootstrap_fallback(err, claims, action), notify_fallback_admit + envelope builder, the five Extension<RulesEngine> additions, docstring truth-up. utoipa response truth-up: fault propagation adds a 500 response class on the 9 rules endpoints that don’t document one today (evaluate alone already documents 500+503; create/update also carry a 409 — leave those as-is) — annotate in this commit; the api-page regen lands in Step 5. Inline unit tests (bin crate): the exhaustive pure-fn matrix (6 error variants x 3 principals x 6 actions); healthy Denied on lists keeps the 403 shape (no bootstrap override); envelope-builder payload assertions (incl. resource_id).

Step 4 — Policy fixtures (fork d)

Replace each fixture’s TWO wildcard service-caller rows with explicit single-action rows (caseworker-row precedent; the corpus has no multi-action i_action syntax):

  • *-authz-rule_set.json (georgia + texas): service → read + list rows (named consumers: every peer’s cache-warm bulk fetch — get_by_name/list_by_prefix via craig-rules-client).

  • -authz-rule_evaluation.json (georgia + texas): service → *create row ONLY — load-bearing for the three live cases→rules relay flows (Context). No read row (no Read-shaped rule_evaluation endpoint exists) and no list row either: no production service calls GET /v1/rules/evaluations with a bare service bearer (only the test-lib client references the route), evaluation rows carry caller-supplied input payloads (case-PII-adjacent), and bootstrap-window listing doesn’t need the fixture row (the D4 fallback admits service List on PolicyMissing). Dead config is dropped, not kept.

  • Update the rows' _description cells to match.

Add the healthy-engine denial test: a bare service token (local mint_client_credentials_token helper — replicate the precedent at services/craig-cases/tests/api/ncands_export.rs:27-41; craig-test-lib has no service-token client) attempting create_rule_set via devstack HTTP (black-box, craig-rules tests/) → 403. Local pickup: cargo xtask dev reseed (or dev refresh) per local-dev.adoc’s rulesets decision table — the seed content-fingerprint fails loud anyway; don’t duplicate the page. The e2e battery exercises the relay flows against the tightened fixtures.

Step 5 — Contracts & docs

Docstrings on every touched fallback/helper match the code. ADR-024 (policy-engine design: typed methods on the trait, PolicyLoad, the degraded-boot posture, fail-closed) primary + ADR-018 note (signer-key approval now hard-fail-closed). Canonical pages per the doc-update checklist: implementation-guide.adoc craig-rules "RabbitMQ Events" Publishes table = `rules.authz_fallback_admit` payload (the #978/#979-realized home for new service events — NOT data-model-rules.adoc, which has no events section); *ADR-003* craig-rules routing-key catalog += same; `shared-crates.adoc` — `AuthzEngine` surface (two typed methods), `AuthzError` (PolicyLoad), boot_degraded beside boot (:50), fix the line-70 "auto_scope_list miss → ListScope::Denied" summary to note the typed path + fault split, AND the craig-bootstrap bullet (:205-209) that still describes the retired "fall back to empty InMemoryRulesetSource`" sequence. Regenerate `docs/modules/ROOT/pages/api/craig-rules.adoc + api/craig-security.adoc from each branch ApiDoc::openapi() (the #784-realized convention — fbc224ee/d626ec0c: utoipa truth-up in the code commit, page regen in the contracts commit). ONE consolidated CHANGELOG security entry (repo’s issue-scoped heading shape) covering fallback retirement + policy-row tightening + the new audit event + the error-class refinements + the degraded-boot fix, with a "Pre-1.0 breaking changes:" callout: admin-on-engine-fault 200→500/403 (security); engine-fault-on-cached-policy now 500 for ALL principals on the security side (the deleted helpers flattened every error to 403 for non-admins — this includes the intake S2S hot paths partners::verify + signer_keys::lookup_by_kid; fail-closed both ways, wire class change); healthy-engine Denied no longer overridden (security lists: admins; rules lists: bootstrap principals) — plain 403 in all cases; service-caller rule-set mutations now 403 (fixtures); refresh-fault class change — PolicyMissing 403 / silent Denied / silent FieldPermission::None → PolicyLoad 500; degraded boot now self-heals and 500s (honest fault) instead of 403ing from a fake-empty source, at one real fetch per miss during the window. .claude/CLAUDE.md explicitly N/A.

Step 6 — Follow-ups + reconciliation

  1. #873: comment the exact coverage map — Rust fail-closed halves land here; the Playwright mutation-authz-audit.spec.ts half does NOT — amend the AC or split to a fresh test(e2e): issue; /relate; do not close.

  2. #908: post the rules.authz_fallback_admit shape for coordination with the library-event ADR.

  3. File the cases→rules evaluate relay actor-forwarding issue (so evaluation authz can be actor-scoped and the rule_evaluation create service row can narrow further) — full issue-quality bar.

  4. File anything else discovered.

Verification

  1. cargo nextest run -p craig-authz -p craig-bootstrap -p craig-rules -p craig-security green; the security RED/GREEN pair recorded in the MR.

  2. cargo clippy --workspace --all-targets --locked — -D warnings.

  3. Grep sweeps scoped to services/ crates/ (archived plans + this plan legitimately mention the names): authz_check_or_admin_fallback + authz_list_or_admin_fallback = zero hits incl. comments; InMemoryRulesetSource absent from craig-bootstrap; no Err(_) ⇒ admit in the surviving rules fallback; no direct-call handler retains an // authz: skip comment. Separately, in rulesets/: no empty-i_action service rows remain in the four fixtures.

  4. cargo xtask validate-authz-coverage clean.

  5. Full pre-push battery (devstack reseed picks up the fixtures; the service-mutation-denied test proves them live; e2e proves the relay flows survived).

Delivery

Commit order: Step 0 plan → Step 1 classes+boot+seams → Step 2 lib+RED → Step 2 retire (GREEN) → Step 3 narrow+audit → Step 4 fixtures (+test) → Step 5 docs (Closes #786 in this commit + the MR description). Every commit runs the full pre-commit token gate AND the J1–J8 subagent pass. Battery push → open MR (Relates to #908, Relates to #873) → archive mechanics as the branch’s LAST commit once the IID is known (the #784-realized convention — archive commit sits directly below the merge commit: file → plans/archive/, row in plans/archive.adoc, Active nav line removed, Status Done (YYYY-MM-DD) — MR !N) → merge per standing policy → close-out (SHAs bare, files, checked criteria incl. the amended AC, Step-6 actions) → &62 update → branch delete + prune → Plan Completion Audit.

Edit this page · latest