Plan: Make craig-cases Field Encryption Work End-to-End (+ Activate in Devstack)

Already-authored fix routes all three convert return paths through decrypted_referral_response. Add: update the stale OpenAPI/doc 409 → 200 (conversion.rs:247), and cover the fresh, sequential-replay, and concurrent AlreadyExists racer arms. Its encrypted-at-rest proof completes at C7. Deps: none.

Both handlers thread Option<Extension<FieldEncryptor>> + Extension<EncryptionMode> and call decrypt_report_pii after the authz check, before consuming children/adults. get_person_suggestions already ordered existence → authz → content, so it only gained the decrypt; create_person_link was re-ordered — the report fetch + authz move into the handler and the bounds-check (hoisted to a testable check_jsonb_index) runs post-authz on the decrypted array, so no report content is inspected pre-authz.

As-built testing decision: rather than the keyless-passthrough pattern (referrals_encryption.rs, whose encrypted teeth wait for C7), C2 proves the fix with genuine HTTP-level encrypted teeth now. That required making craig-cases a lib+bin crate (the craig-intake-keyring model — api::routes
RulesEngineClient + ScreeningPolicyClient exposed) so a test can assemble the real router in-process with a test-controlled key. tests/api/report_persons_encryption.rs drives it via tower::oneshot with an allow/deny AuthzEngine double + injected caseworker Claims: it asserts encrypted-at-rest (raw-DB {"v":..} / is_encrypted), the decrypt round-trip (suggestions source, manual-link valid index), and the authz-before-content ordering (deny → 403, not a content 400). No-devstack unit tests pin the decode/bounds contract. The reusable in-process harness is a candidate to lift into craig-test-lib under C6 (#986).

Deps: none; must precede C7. Devstack encryption stays off until C7.

Substring search on narrative + reporter names (store/reports.rs:121) is impossible on randomized ciphertext. Resolved (2026-07-13) as a generic capability-based search abstraction rather than a per-field fix: search capability = f(encryption_scheme, logical_type), resolved through one FieldId registry every boundary consults; reports search repoints to plaintext admin_unit + reporter_type. Decision + threat model + crate rejection in ADR-049; decomposed into sub-epic &67 (MR0 #983 + MR1–MR7 #1015–#1021) governed by Plan: Capability-Based Encrypted Search (C3). Deps: none for the framework; MR2 (the C7 gate) needs C6 merged. Must precede C7. Complete (2026-07-13): all eight MRs merged (!970–!977); the C7 dependency on C3 is satisfied. As a rider, MR4 migrated the SSN blind index to a per-field HKDF domain, so C7’s activation reseed derives ssn_hmac under the new domain automatically.

Independent review of the single-MR floor found it too large and resting on unverified guard/lifecycle assumptions, so C4 is re-decomposed into focused MRs, with a determinism prerequisite landing first. Fork resolutions (2026-07-11): the seed identity is the full ADR §D2 recipe (seed
families + active-bundle content/version + generator digest + FORMAT_VERSION_V1 + non-SQL input fingerprints); markers are per-phase (rules/SQL/attachments, so a non-fatal non-SQL failure re-runs rather than being skipped forever); the completion gate is the compose service_completed_successfully on craig-web (xtask verify_seed checks a possibly-still-running one-shot, and three CI jobs use raw docker compose up with no seed check).

Not in C4-floor: the lineage INSERT → C5 (needs the key + craig-crypto in the seeder); the KCV-staleness wipe + cases boot/write-path verifier → C7 (recording the KCV while the key is inactive can’t detect the plaintext→encrypted transition). The auto_refresh marker-before-reseed ordering (non-bounced path) is #1005; CRAIG_FAMILIES 12-vs-9 is folded into the identity (an explicit input); the rules-API non-fatal branch is #1008 (C4-floor takes the attachment half). Deps: prerequisite for C5, C7.

As-built. craig-crypto = { workspace = true } (proptest already dev-dep; Cargo.lock committed). Person.ssn_hmac: Option<String> (constructed None) + a persons SqlRow column. A non-destructive encrypt_seed_data projection encrypts the covered columns (persons ssn_last_four; referral reporter_first_name/last_name/phone — not reporter_relation) and derives ssn_hmac = enc.hmac(ssn_last_four) from the plaintext first. The cases txn emits a plain generated INSERT INTO crypto_key_lineage (CryptoLineage; real KCV + encrypt_str(CANARY_PLAINTEXT) when keyed, deterministic kcv:none/canary:none sentinels keyless; matches the as-built migration incl. kcv_version).

D2 identity (the crux): write_output builds all artifacts in memory then writes; the _seed_identity content digest is over the plaintext renders (cases with a sentinel lineage) and compute_identity folds in the real enc.kcv() when keyed (KCV_SENTINEL keyless) — so the identity is stable for a fixed (data, key) yet key-bound. Randomized ciphertext + the real lineage go to the cases output file only (never the identity). compute_identity gained a kcv: &str param (14 callers); it now binds craig_crypto::FORMAT_VERSION_V1 directly (dropped the local dup).

Fail-closed load (main.rs): CRAIG_SEED__ENCRYPTION_MODE accepts only absent (⇒optional), optional, required — any other value is fatal; a malformed key is fatal in both modes (never silent plaintext); only an absent key is tolerated (optional). Keyless devstack until C7.

D7: re-blessed CASES_SHA256 only (ssn_hmac column + sentinel lineage; other 5 byte-identical)
ledger line; a round-trip proptest (in-src); a semantic encrypted-render test that decrypts every covered value from the real write_output render + asserts lineage/ordering; a leak test (covered plaintext absent, reporter_relation present); a real-binary key/mode matrix (absent / valid / required-without-key / malformed / invalid-mode, inherited env stripped). Deps: C4a (crypto surface); keyless→plaintext until C7, so it needs neither C4d’s key file nor the deferred C4-gate.

As-built. The gate is cargo xtask verify-seed --expect {keyless,keyed} (a required value-enum, not a --expect-keyless flag). Asserting the intended mode is the crux: an auto-detecting gate (read the stored KCV, assert whatever it implies) is circular — a silent C7 key-mount failure emits plaintext + the sentinel KCV, which an auto gate passes vacuously. So C6 runs --expect keyless; C7 flips both call sites to --expect keyed, and a plaintext seed then fails the keyed assertions.

Command shape. run (CLI) takes the devstack lifecycle lock, resolves the cases DSN (explicit CRAIG_CASES__DATABASE_URL override — used by CI — else reconcile ports + the live Postgres port, no 5432 fallback), and calls the lock-free verify_inner; validate’s devstack phase calls `verify_inner directly (it already holds no lock). verify_inner = the completion poll + fetch covered rows + the expected-mode assertion. The pure assert_keyless / assert_keyed take fetched rows and are unit-tested with synthetic fixtures both ways (keyed rows fail keyless and vice-versa; wrong key fails; a tampered ssn_hmac fails; all-NULL / empty can’t pass vacuously) — so the keyed branch is exercised now, not left dormant until C7.

Completion poll (bug fix). devstack::verify_seed read only {{.ExitCode}} via a status-ignoring compose helper — a running one-shot reports exit 0, so a still-running seed read as a clean success, and a dead daemon collapsed to Ok(""). Now a checked compose capture polls {{.State}}|{{.ExitCode}} with a timeout through a pure classify_seed (running → keep polling; exited-0 → ok; exited-nonzero / unexpected-state → fail; empty → missing → fail). Both callers (bring_up, auto_refresh) inherit the fix.

Assertions. Keyless: every present covered value is plaintext, ssn_hmac NULL, lineage KCV = the sentinel, rows non-empty + at least one covered value present. Keyed: every present covered value is a CGEF envelope that decrypts, ssn_hmac == hmac(decrypt(ssn_last_four)), lineage KCV = the loaded key’s KCV. The keyed branch loads the key via a hardened xtask::field_key::load_field_key (rejects symlink/non-regular/loose-perms, requires exactly one CRAIG_FIELD_ENCRYPTION_KEY= assignment, and is TOCTOU-safe via a stat→open→fstat inode (dev, ino) re-check — the O_NOFOLLOW intent without a libc/rustix dep; reused by provisioning’s validate_existing_key_file).

Coverage beyond the seed. A throwaway-per-test craig_cases DB (created + migrated + dropped on Drop, WITH (FORCE)) hosts an in-process keyed router that returns its FieldEncryptor — so the C2 consumer tests (rewired onto it) and a new API-created-report at-rest test (4 TEXT: narrative
reporter_{first,last}_name + reporter_phone; 3 JSONB {"v":..}: children/adults/raw_submission) run under a real key without polluting the shared DB. A decrypt-through-service round-trip test reads seeded persons/referrals back through the live cases API and asserts the covered fields equal the deterministic craig_seed::generate(seed=42, families=12, georgia) oracle (mode-independent: green keyless now, and the decrypt-through proof at C7). KCV_SENTINEL moved to its canonical home in craig-crypto (value unchanged ⇒ byte-identity + golden vector stable).

from_env boundary. Absent / valid / malformed-base64 are already covered by C5’s real-binary matrix; C6 adds the two genuine gaps — valid-base64-wrong-length and non-Unicode — to that subprocess matrix (both fatal in each mode).

Wiring. validate’s devstack phase runs `verify_inner(Keyless, dsn) after ensure_ready
reconcile, before the mutating integration tests; the pentest CI job (allow_failure, main/schedule) runs verify-seed --expect keyless as a non-blocking backstop. The encryption-asserting cases light up with C7.

Deferred → C7 (#987): the negative restart-cases-under-wrong-key ⇒ refuse-boot test — it needs C7’s boot verifier to exist, so it can neither land nor pass here (its #986 AC moved to #987 during the AC-refine). Deps: C2, C5.

Mount the shared key into cases + seed (coordinated recreate), set both required, full-volume down -v reseed; add the craig-cases boot verify-only (recompute KCV + decrypt canary, refuse on mismatch). Run the full pre-push battery incl. E2E + performance; prove #969/#413 on encrypted data. Activation correctness is the D6 round-trip (cases boots empty-first, then reads the seeded data under the shared key). Deps: C2, C3, C4a + C4-floor + C4d (crypto surface + volume floor + key file), C5, C6 — not the deferred C4-gate (CI hardening is independent of activation).

As-built (2026-07-13). Compose mounts devstack/field-key.env into cases + seed with both required; the D3 host-KCV staleness deferred here from C4 landed as a new .devstack/field-key.kcv marker whose absence/mismatch escalates to VolumeAction::Wipe (the migration-change mechanism) — so the first post-C7 dev start IS the coordinated activation reseed, no manual step. The boot verifier is craig_cases::boot_verify (verify_key_lineage, run before the authz engine/workers): scheme metadata → KCV → canary, plus the lineage-absent branch’s registry-driven ciphertext scan (a craig_crypto::ENVELOPE_B64_PREFIX SQL prefilter confirmed by exact is_encrypted). The #986-inherited negative landed as the boot_verify test matrix over the C6 throwaway-DB harness ("restart under key B" ≡ key-A lineage + key-B encryptor, which a fresh stack cannot exercise). Both verify-seed sites flipped to --expect keyed; the CI backstop stays red-but-allow_failure until C4-gate’s variable-first delivery (compose now fail-closes on the missing env_file in CI by design).

ADR-048 carries the as-built key lifecycle; C8 adds a cross-reference from ADR-020 to it (and the C3 search decision to its own ADR), then updates CHANGELOG.adoc, Configuration Reference, Local Development / Known Issues & Lessons Learned (devstack now encrypted; D1 one-time dev clean; key-change⇒reseed), and .claude/CLAUDE.md status. Deps: with C7.

As-built (2026-07-14). ADR-020 gained the epic-&66 amendment (key lifecycle → ADR-048; boot verify-only pre-empting the read-time misconfiguration 500s; the seed-determinism render pass; the ADR-049 search decision; keyless dev loop ended). Configuration Reference gained the craig-cases ENCRYPTION_MODE + CRAIG_FIELD_ENCRYPTION_KEY rows (the key is deliberately un-prefixed — shared with craig-seed) and the craig-seed section was un-staled. Local Development's key section flipped dormant → live, with the field-key KCV row added to the staleness table. Known Issues & Lessons Learned records the markers-absent boot-refusal edge (cold-start path skips the staleness check; remedy cargo xtask dev reseed) and the red-but-allow_failure CI devstack jobs pending C4-gate. The C7 note that local-dev’s verify-seed passage and quality-gates were already corrected in C7’s review sweep held — C8 found the section-level dormant framing still stale and fixed that.