ADR-057: Closed-Cohort SHINES Conversion Import — Staging Ledger, Manifest-Verified Finalize, Machine Payment Cutover

On this page

Status

Accepted (2026-07-25). The #1071 program (epic &70 child (e)) lands as two MRs: MR-A, the conversion substrate (merged 3d7df6c0 — the staging-ledger migration, the proof-typed import_agreement_history rework, the generator BeforeCutover rule on all three generation paths, the deploy-first audit-parser arms, and four BLOCKING invariants) and MR-B, the surface (LANDED with this record: the staging/finalize API + engine, the SubsidyImportSettings consent knob, the explicit-admin ruleset, the xtask operator tool, seed family F, and the web read scope; the endpoint contracts are on the craig-financial API page). Program plan: Closed-cohort SHINES conversion import (the external review’s 10 stop-ships + G1–G17 and their dispositions are indexed there). Closes #1116 (the imported-pending state becomes unrepresentable); #1115 is deliberately unchanged (below-floor backpay is orthogonal — the cutover only BOUNDS payment).

Amends (named): ADR-052 (D6’s import contract is COMPLETED: proof-typed, open-pending heads refused, review slots materialized at import, the cutover written with the identity insert) and ADR-054 (the "only import_agreement_history remains stamp-based / outside the matrix" sentences gain their precision update — the ACT of importing is now inside the matrix at StateOffice; `ApprovalStamp`s remain only on the HISTORICAL rows the import carries).

Context

RCS/ERCS (UAS 553) closed to new enrollment 2014-01-01; ERSG/ENRSG closed 2014-08-01 (GA 22.8). Existing recipients continue under 22.9 renewal reviews, so CRAIG must carry the cohorts for payment continuity and future ACF-199 reporting WITHOUT new enrollment. The #1070 tagged create already refuses all four closed tokens with the typed closed-program problem naming this import path. The pre-#1071 store primitive import_agreement_history existed but was proof-less, event-less, and API-less — and the external review showed that wiring it per-record would let money go live before a conversion batch is verified, with no machine cutover, no cohort-legality check, no immutable replay evidence, and no correction path.

Decision

D1 — A staging LEDGER, not staged-flags

Conversion imports land in subsidy_import_batches + subsidy_import_records (each record carries its canonical payload + hash); the live ledger is untouched until finalize. The alternative — staged-flags on live rows — would force "AND NOT staged" predicates into the generator, sweep, reads, and invariants, every miss a money bug. A separate ledger means abort is harmless row deletion, dry-run is free, pre-finalize correction is supersede-and-restage, and the canonical hash lives where lifecycle mutations can never touch it.

D2/D3 — Closed cohorts only, with a store-validated qualifying date

The conversion path takes rcs/ercs/ersg/enrsg ONLY (the MR-B wire enum makes open programs unrepresentable; the store refuses them independently). This kills the witness-bypass class outright — nothing the native ERR/SG/NRSG flows govern can be minted here — makes cohort legality checkable, and shrinks the exclusivity surface. A deployment converting OPEN-program recipients re-enrolls them natively (real witnesses, real F8, correct 22.9 anchors). Cohort legality is store-validated (D3): the qualifying date = the first ACTIVE interval’s start (never-active chains qualify on signed_at), STRICTLY before the closure date (2014-01-01 rcs/ercs, 2014-08-01 ersg/enrsg; constants live in subsidy/policy.rscohort_closed_since returns None for open programs, which doubles as the open-program refusal signal); ClosedCohortNotQualified is the typed refusal and subsidy_closed_cohort_qualifying_date the blocking invariant. source_system is a vocabulary ('shines'), never caller freetext.

D4 — Canonical-equivalence replay evidence

Replay identity is canonical equivalence — a deterministic re-serialization of the typed wire DTO, NOT "byte-identical" — and the evidence is IMMUTABLE: the record row’s canonical_payload + canonical_hash never change, while live rows mutate (transitions append intervals, amendments append terms, assignment events touch denorms — comparing a resend against them would falsely 409 honest replays after any lifecycle action). Post-finalize replays compare against the record row in O(1) and answer with the ORIGINAL facts (batch id, cutover month, persisted warnings — never re-evaluations). The normative replay table (per live record for the external_reference):

Live record for the reference Resend IDENTICAL Resend DIFFERENT

none (aborted / superseded / rejected-freed)

201 staged fresh

201 staged fresh

staged, same batch

200 already_staged (original outcome)

200 restaged (old row superseded — pre-finalize correction)

staged, DIFFERENT live batch

409 BATCH_CONFLICT

409 BATCH_CONFLICT

rejected (live batch)

201 re-staged fresh

201 re-staged fresh

materialized

200 already_imported — ORIGINAL agreement_id/batch_id/cutover/warnings

409 IMPORT_CONFLICT (post-money corrections = native amendments + #1028)

A race loser on the live-reference partial unique (subsidy_import_records_live_reference — one live claim per source record across ALL batches; superseded/rejected rows free the key) aborts its transaction, re-reads the winner in a FRESH transaction, and answers per the table — never a 500.

D5 — The persisted machine cutover, on all three generation paths

Every materialized agreement is born with subsidy_agreements.payment_cutover_month, written by the materializing identity insert and CHECK-shaped (present IFF imported — subsidy_agreements_cutover_iff_imported; a month start — subsidy_agreements_cutover_month_start; the store additionally refuses a non-month-start typed rather than dying on the DDL CHECK). The generator’s single month judgment gains SkipReason::BeforeCutover immediately after the #1070 BeforePaymentsBegin arm — and like it, BEFORE the active-as-of rule, because an imported agreement is ACTIVE at month start for months its legacy system already paid; activity never overrides the cutover. The rule is RECONCILING (a stray pre-cutover undisbursed row is VOIDED by the same machinery that voids pre-boundary guardianship months) and COUNTED (skipped_before_cutover on the report, the completion event, and telemetry). Because the scheduled, MANUAL, and reconcile-drain paths all share expected_month_row, the cutover bounds every generation path — SHINES owns pre-cutover months as machine truth, not runbook hope.

D6 — 22.9 posture: refuse-beyond-grace, never auto-suspend

An ACTIVE head with any review anchor overdue beyond the sweep’s exact 30-day grace is a stage/finalize BLOCKER (OVERDUE_ANCHOR_BLOCKS_ACTIVE — fix upstream in SHINES and re-export); within-grace overdue is a warning (native parity — the sweep itself would not act yet); suspended heads warn only. Auto-suspending at finalize was rejected for three reasons: (1) it would mint a leg-1 enforcement act OUTSIDE the sweep’s knob-consent model (authorize_system — the knob IS the recorded authority); (2) it would append a CRAIG-fabricated interval, making the landed chain differ from the canonical payload that is this rework’s replay evidence; and (3) it would enforce on a deployment whose suspend knob may be OFF. The compliant inbound shapes are "active, anchors at most grace-deep overdue" or "suspended as recorded" — an export showing active-beyond-grace means the SOURCE is out of compliance at snapshot. Payment safety is total either way: post-finalize the worst importable state is within-grace overdue — a state native policy pays too. The normative warning semantics (assessment date A = business today at stage, RE-evaluated at finalize — the finalize values persist; anchors from the OPEN terms revision; due = A is not overdue; grace = the sweep’s strict due < A − 30 days; suspension age = the sweep’s inclusive from ≤ A − 3 months, chrono clamped month-add with a chrono-vs-SQL parity pin):

Token Predicate Heads Class

overdue_renewal_anchor / overdue_paper_anchor

due < A, within grace

active, suspended

warning

OVERDUE_ANCHOR_BLOCKS_ACTIVE

due < A − 30d (strict)

active

blocker

(same predicate, suspended head)

due < A − 30d

suspended

warning (money-safe; the post-finalize sweep owns it)

aged_suspension

from ≤ A − 3mo (inclusive)

suspended

warning

terminal heads

no evaluation

D7 — Term coverage + the persisted payment assessment

Every ACTIVE interval must be priceable: with contiguous term chains this is exactly terms[0].effective_from ≤ the earliest Active interval start (vacuous for never-active chains) — refused typed as TermsCoverageGap, enforced again by the blocking invariant subsidy_terms_cover_active. The MR-B staging surface additionally persists a per-record payment assessment (payable_at_cutover + typed blockers) so reconciliation reads facts, not re-derivations.

D8 — Family taxonomy, gate-matches-index, the ALL lock order

The program families are {err}, {sg, nrsg, ersg, enrsg} (the enhanced closed variants are rate classifications of the SAME guardianship enrollment), and {rcs, ercs} — encoded once in craig_reference::SubsidyProgramFamily / program_family. The #1070 guardianship one-open index is RECREATED over all four family tokens and a NEW relative-care one-open index covers rcs/ercs (safe pre-1.0: import is the only path that can mint enhanced/relative-care rows, so no existing data can collide). The store gates READ THE FAMILY DEFINITION (SubsidyProgramFamily::Guardianship.programs() + the four-token SQL list) so the gate can never drift from the recreated index. Locking (the A-review widening — an agent-discovered race): the plan’s per-family sorted locks were insufficient for the ANY-program ActiveSibling gate, so a NON-terminal import locks ALL THREE families in the one canonical SubsidyProgramFamily::ALL order (terminal imports lock only their own family — coverage is family-scoped); create_guardianship_agreement was repointed to the same shared lock_family_history, which is what keeps create-vs-import deadlock-free. Non-terminal imports are refused while ANY other open agreement exists for the child (ActiveSiblingAgreement); family-wide coverage overlap and the full predecessor battery (same child, same family, terminal, closed before the new coverage) apply; classify() maps both family uniques + the one-successor unique to typed 409s, never 500s.

D9 — The wire drops denorms; S2S witnesses; store-stamped time

No assigned_worker_sub, supervisor_sub, placement_id, created_by*, imported_at, or import_batch_id field exists on the import DTO — the wire cannot assert what the server derives. The child and every party person are S2S-verified (plus a case-participant check when case_id is present); ≥ 1 caregiver party is required on non-terminal heads (terminal history may be partial); imported_at is STORE-STAMPED at insert; identity attribution comes from the proof + trusted claims, never the payload.

D10 — Open-pending heads are unrepresentable (closes #1116)

import_agreement_history refuses a chain whose head projection is pending with the typed ImportPendingHead — an authority proof does not make an unactivatable production state safe, and the old store seam ("import accepts pending for tests") made API/store invariants diverge. The ~10 pending-fixture test call sites migrated to a documented test-only raw-SQL builder in craig-test-lib (honest about being a fixture); the BFF/API #1116 guards remain as BELT only. #1116 closes: the state cannot exist through any production path.

D11 — The event pair; no external reference in any event

One financial.subsidy_agreement_imported per materialization — {agreement_id, child_id, program, head_status, reviews_materialized, import_batch_id, cutover_month, created_by, approval_level}, where created_by is the FINALIZING operator (the audit sink’s actor chain reads it) — plus one counters-only financial.subsidy_import_batch_finalized per finalized batch. external_reference (an unclassified source-system identifier) rides NO event: the DB is its only surface. The craig-security parser arms + the ADR-003 catalog entries landed with MR-A, deploy-first; the producers landed with the MR-B finalize engine.

ApprovalAction::ImportHistory sits at StateOffice — the FIRST named state-floor action — minted once per finalize, passed INTO the store, and re-checked in-transaction (covers(ImportHistory)). The split is act-vs-content: the ACT of importing is matrix-proofed; the HISTORICAL ApprovalStamp`s the records carry stay outside the matrix (no session can re-authorize approvals recorded years ago in SHINES) but are hygiene-checked (nonblank actor; `approved_at ≤ snapshot_date and ≥ 1990-01-01; the stamp vocabulary). The MR-B surface adds a NEW ruleset georgia-authz-subsidy_import.json with an EXPLICIT operation-discriminated admin allow, a leading service DENY, and default deny (Texas ships no allow rows), and the CRAIG_FINANCIALSUBSIDY_IMPORTENABLED knob (default OFF) gates ALL import writes including finalize; GETs stay readable knob-off for post-conversion audit, and the lifecycle of already-materialized rows is grandfathered (generator, transitions, reviews, sweep never consult the knob).

Review provenance: a named baseline, not a fabricated chain

SHINES stays authoritative for pre-cutover review history; batch.snapshot_date is the NAMED conversion baseline; the imported anchors ARE the baseline’s next-dues; cycle-1 slots materialize at import for active AND suspended heads (suspended recovery needs live slots — enforced by the blocking invariant subsidy_active_missing_review_slot, which also requires the paper slot for non-ERR programs whose head terms carry a paper anchor). Review HISTORY is deliberately not imported: completed-review rows would force fabricating resulting_term_id linkages (composite FKs to same-agreement terms) that were never recorded — an honest named baseline beats a fabricated chain. Docs therefore say "full status/terms history; review history begins at the conversion baseline", never "full historical chain".

The batch state machine + finalize (normative)

staging → finalizing → finalized; staging → aborted; a finalize pass that ends with fresh rejections loops finalizing → staging (already-materialized rows stay live — individually valid and cutover-bounded). Abort is forbidden from finalizing. Concurrency: a session advisory lease subsidy-import-finalize:<batch_id> on a detached connection (the subsidy-gen pattern); a second finalize gets 409.

Finalize: (0) lease; (1) one transaction — batch FOR UPDATE: finalized → 200 replay of the stored report; staging → the manifest gate (expected count == staged + materialized, checksum matches the sorted (external_reference, canonical_hash) pairs, zero live rejected; else 409 MANIFEST_MISMATCH carrying the four numbers) → finalizing; finalizing → resume. (2) Mint the ImportHistory proof once. (3) Keyset loop over staged records (chunk 100), ONE TRANSACTION PER RECORD: re-run the full validation battery against in-transaction truth (family locks here) → import_agreement_history (identity + provenance
payment_cutover_month + store-stamped imported_at, parties, chains, slot materialization) → flip the record materialized → stage the imported event. A validation failure rolls back, then a FRESH transaction marks the record rejected + blockers, and the loop continues; an infrastructure error rolls back and the record stays staged. (4) Zero staged + zero new rejections → batch finalized + the summary event + the full report; new rejections → back to staging + the rejection manifest; staged remaining → stays finalizing, a re-POST resumes. Crash-resume is exactly-once: per-record transactions + the live-reference unique + the same-transaction record flip; the batch id never changes.

Consequences

  • Deploy ordering is parser-first: the MR-A audit-parser arms must be live everywhere before any producer can emit (the ADR-003 catalog entries landed with them); MR-B deploys knob-off inert. The conversion runbook (deployment guide) sequences the window: enable → stage → reconcile → finalize → disable → verify 403-on-writes.

  • The money boundary is bilateral but CRAIG enforces only its side: SHINES pays every month strictly before cutover_month, CRAIG every month from it — BeforeCutover is CRAIG’s machine enforcement; the SHINES side is the inter-system agreement (⁂ #1073).

  • RESOLVED as-built (the review’s carried MEDIUM): MR-B widened the native ERR create — create_agreement_active takes the ALL-families locks and refuses beside ANY open agreement (typed ActiveSiblingAgreement, firing BEFORE the coverage/lineage refusals — a recorded precedence change with re-pinned tests). create_guardianship_agreement is DELIBERATELY exempt: the #1070 ERR→SG flow signs the A&A pre-transfer, so a pending SG must mint beside the open ERR (the witnessed activation’s own sibling gate owns the money exclusivity) — disposition documented on the gate and pinned by the cross-family pendency test.

  • #1115 is unchanged: the cutover BOUNDS payment; it does not create a backpay path for months below the generator’s −12-month floor.

  • Four BLOCKING invariants guard the substrate (subsidy_active_missing_review_slot, subsidy_terms_cover_active, subsidy_closed_cohort_qualifying_date, subsidy_cutover_iff_imported — the last also catches the money consequence: a live payment before the cutover).

  • Post-finalize corrections after money moved are a DESIGNED 409 (IMPORT_CONFLICT): they go through native amendments/transitions + #1028 adjustments, never a corrected re-export.

  • The operator tool lives in cargo xtask import-subsidy-history (MR-B): device-code OAuth as the responsible state-office human — NO service-principal mode — throttled, resumable, rejection-manifest output; the tool NEVER triggers generation (that authority stays with the ops runbook).

Review record

The program was reshaped by an external review (2026-07-24: 10 stop-ships + G1–G17, authoritative) before implementation; every finding’s disposition is indexed in the program plan. MR-A’s two commits each shipped behind a fresh-context J1–J8 review; the recorded execution deviations (the ALL-families lock widening, the typed non-month-start cutover refusal, the gate-reads-the-family-definition hardening, the DuplicateParty error) are carried in the plan’s Execution notes and folded into D5/D8 above as-built.

Edit this page · latest