Plan: Monthly Subsidy Payment Generator (#1068)
On this page
Status
| Step | Description | Status |
|---|---|---|
0 |
Plan committed (this file, nav → Active, MR-A first commit); GitLab: enabler fixes #1088/#1089/#1090 + follow-ups #1091 (PaymentType-enum wart) / #1092 (business timezone) filed; #1068 AC trued-up (+ |
Done (2026-07-20) — plan commit 1e13652f; issues + label + AC true-up landed |
1 (MR-A) |
Pre-existing-bug enablers, one justified multi-issue MR: #1088 BFF open-token payment/claim views; #1089 set-based eligibility UPDATE (both fields); #1090 payment-event resource-id extraction |
Done (2026-07-20) — MR !1017 merged (impl 42012e08, merge 559f0562); #1088/#1089/#1090 closed |
2 (MR-B / D1) |
Schema: payments subsidy origin — agreement_id + term_id provenance FK, nullable trio + |
Done (2026-07-20) — d92f6ef5; migration verified live on devstack |
3 (MR-B / D2) |
Exclusivity + correction protocol: child/month advisory lock in BOTH writers, in-tx re-evaluation, cross-program fail-closed guard, jurisdiction gate, reconcile-then-generate (void+regenerate undisbursed / freeze disbursed), conflict-free-insert derivation compare, terms-coverage hardening, |
Done (2026-07-20) — 4bb5622c (+ e3d8ee74 carries the in-tx re-eval/guards inside the generator); deviation: the 23505 arm became an |
4 (MR-B / D3) |
Generator core in |
Done (2026-07-20) — e3d8ee74; deviation: keyset is by child_id alone (children are the page unit — same never-splits-a-child guarantee the (child_id, id) ordering targeted) |
5 (MR-B / D4) |
Read-surface acceptance, seed collections + current-month history, full test matrix (conc/replay/fault/happy/sad/evil per taxonomy), unit-month presentation sweep, ADR-053 + docs, api regen |
Done (2026-07-20) — seed d868015a, matrix e3db0707, BFF/contracts 344eea57, docs 43929c86 + the verification-fixes commit. Deviations recorded: separate SeedSubsidyPayment + parametric |
6 |
Close-out: epic &70 tick (b), plan → Archive, follow-up reconciliation |
Done (2026-07-20) — MR !1018 merged (merge 82c9c8a0); #1068 closed with the SHA record; epic &70 child (b) ticked; plan moved to Archive; follow-ups reconciled (#1069 durable handoff, #1072/#1073 confirmations, #1091/#1092) |
Issues: #1068 (MR-B, Closes) + #1088/#1089/#1090 (MR-A, Closes-listed) · related #1072 (UAS), #1073 (DFCS confirmations), #1091/#1092 (follow-ups)
Branches: MR-A feature/subsidy-generator-enablers (this plan = first commit, #786 precedent); MR-B feature/1068-subsidy-generator
Provenance: exploration fact sheet 2026-07-20 (scheduler/payment-path/idempotency/exclusivity/month/config/test surveys). User-decided forks: payments table with nullable origin; unit-month rows. Review history: internal R1 (2P1/4P2/4P3) + R2 (0P1/1P2/4P3) fixed; external review R1 (~30 findings — cross-writer atomic exclusivity, correction protocol as ADR-052’s named consumer, capability choice, pre-existing-bug split, structural never-IV-E, lease/batching/attribution/clock, seed enumeration, presentation sweep, doc inventory) restructured this plan; internal R3 on the rework (1P1/4P2/5P3 — eligibility both-fields UPDATE, void primitive + event reshape, keyset-by-child, lease mechanics, report counter, per-child tx, constant/env homes, current-month seed, unscoped-ref shape) fixed in place.
Context
The ADR-052 ledger (foundation, merged 2026-07-20) holds active kinship-subsidy agreements; the
only payment writer is placement.activated’s first per-diem period
(`services/craig-financial/src/main.rs:345-441). Quarterly federal expenditure reporting (#161)
needs the ongoing MONTHLY subsidy streams to exist. GA 22.8 forbids subsidy + foster-care
per-diem for the same child in the same month; TANF interplay is recorded-only pending #1073.
Three pre-existing bugs block or entangle the work and land first as MR-A: the BFF’s closed
PaymentType enum already fails on live kinship rows (#1088), the eligibility handler’s
500-row window leaves eligible payments stale (#1089), and payment audit rows can carry a CASE
uuid as their resource id (#1090).
MR-A — enabler fixes (#1088 + #1089 + #1090, one justified multi-issue MR)
-
#1088:
PaymentView.payment_type+ClaimView.payment_type(services/craig-web/src/routes/financial/{payments,claims}.rs) →String, rendered verbatim (humanizetakesDisplay). BFF tests pin open-token rendering (kinship + program tokens). The fleet-wide enum disposition is #1091. -
#1089:
handle_eligibility_evaluated(main.rs:486-542) → ONE set-based UPDATE keeping BOTH written fields:UPDATE payments SET ive_eligible = $3, ffp_rate = $4, updated_at = now() WHERE case_id = $1 AND child_id = $2 AND status IN ('pending','approved')—ive_eligibletakes the payload value (legitimately false sometimes); droppingffp_ratewould miscompute the GENERATEDffp_amount; noNOT ive_eligiblefilter (changed rates must update). Only a count is logged today (rows_affectedcovers it). Test: a >500-payment case updates every eligible row, both fields. MR-B appendsAND agreement_id IS NULLhere. -
#1090: security’s resource-id extractor (
services/craig-security/src/lib.rs:326-336) gains event-family-specific extraction:financial.payment_*events resolvepayment_idFIRST (omitted keys tolerated). Tests: create/approve/issue/clear audit rows carryresource_id == payment_id, with and withoutcase_idpresent.
MR-B — the generator (Closes #1068)
D1 — Schema (<14-digit>_subsidy_payment_origin.sql)
-
payments.agreement_id UUID NULL REFERENCES subsidy_agreements(id) ON DELETE RESTRICT
payments.term_id UUID NULLwith table-level compositeFOREIGN KEY (agreement_id, term_id) REFERENCES subsidy_agreement_terms (agreement_id, id) ON DELETE NO ACTION— pricing provenance (which revision priced the row; mismatch detection). -
Trio
placement_id/foster_home_id/case_id→ nullable; CHECKpayments_origin_exclusive: placement rows keep the NOT-NULL trio + NULL agreement/term; subsidy rows have agreement_id + term_id NOT NULL + NULL placement/home (case optional — post-case SG/RCS episodes still pay). Pre-1.0 breaking nullability → CHANGELOGChanged. -
Subsidy shape CHECKs (conditional on
agreement_id IS NOT NULL):EXTRACT(DAY FROM period_start) = 1;period_end = (period_start + INTERVAL '1 month' - INTERVAL '1 day')::date;day_count = 1 AND daily_rate = gross_amount(unit-month);payments_subsidy_never_ive:NOT ive_eligible AND ffp_rate = 0(TANF-funded per 22.8: 542 TANF-MOE, 552/553 TANF; enhanced-cohort funding confirmation on #1073’s list) — the STRUCTURAL never-IV-E guarantee. -
Idempotency: partial UNIQUE
payments_one_subsidy_per_agreement_month (agreement_id, period_start) WHERE agreement_id IS NOT NULL AND status <> 'voided'— voided rows free the key (regeneration path). No separate agreement index (the unique leads on agreement_id). -
Paymentmodel + contractsPaymentResponse: +agreement_id/term_id: Option<Uuid>, trio →Option(test-lib lifecycle clients rely on the mirror). Payment LIST/count gainagreement_id+child_idfilters and an id tie-break on the current single-column sort. -
CreatePaymentParamsgainsagreement_id,term_id,assigned_worker_sub(no assignment field exists today; the placement caller passesNone, preserving NULL-then-backfill). -
Events:
publish_payment_created+ the lifecycle payloads (events.rs:43) OMITcase_idwhenNone(conditional key insert —json!would emit an explicit null that stops the audit fallback #1090 fixed).
D2 — Exclusivity + correction protocol (ADR-053 core)
Child/month payment lock: BOTH writers take
pg_advisory_xact_lock(hashtext('pay-excl:' || child_id)) inside their write tx before their
existence checks.
-
Generator: ONE tx PER CHILD covering reconcile-then-generate for all that child’s agreements (the xact lock spans the pass): lock → re-evaluate INSIDE the tx (TOCTOU kill) —
agreement_for_update(new lib read,SELECT … FOR UPDATE),status_as_of(month_start)== Active,terms_as_of(month_start)(captureterm_id), per-diem overlap check (EXISTS … WHERE child_id = $1 AND agreement_id IS NULL AND status <> 'voided' AND period_start ⇐ $3 AND period_end >= $2) — THEN insert +publish_payment_createdstaged same-tx → commit. Lock ordering is acyclic vs the #1067 history lock (distinct keyspaces; generator never takes the history lock;transition_statustakes agreement-row then history — never pay-excl). -
handle_placement_activatedgains the same lock + the reciprocal check: on subsidy-row overlap, per-diem STILL wins (a foster family is never starved; the 22.8 handoff is #1069’s flow) — WARN + REPORT-ONLY invariantsubsidy_perdiem_same_month_overlap(the transitional handoff month is legal until the correction voids it; the invariant is the operator queue). -
Cross-program guard: candidates grouped by child; >1 active program for a child+month → generate NOTHING for that child (
skipped_multi_program+ WARN) — fail-closed pending #1073’s coexistence confirmation; report-only invariantmultiple_subsidy_payments_same_child_monthwatches the data. -
Jurisdiction gate: subsidy payment policy is GEORGIA-DEFINED ONLY — the scheduler spawns only when
settings.jurisdiction == "georgia"; the endpoint returns 400 typedUNSUPPORTED_JURISDICTION(new craig-common constant) elsewhere. TX generation = future policy work (ADR-053). -
Correction protocol (ADR-052’s named deferred consumer, now defined):
-
Reconcile (per child, same tx): every UNDISBURSED (pending/approved) subsidy row of the month whose re-evaluated derivation no longer matches (agreement no longer active-on-month-start — backdated transition; or
terms_as_ofyields a different term_id/amount — backdated revision) → VOID via NEW store fnvoid_subsidy_payments_for_agreement_month(conn, agreement_id, month_start) → Vec<Uuid>(the existing set-based void is placement-scoped and can never match subsidy rows), then regenerate (voided-exclusive key permits it).publish_payments_voidedis RESHAPED: +cause: &'static str, scope becomes{payment_ids, count, cause, placement_id: Option, agreement_id: Option, month: Option}(placement caller:placement_ended+ placement_id; reconciler:subsidy_reconciliation+ agreement/month). ADR-003 + implementation-guide rows; security parsing verified (additive keys; reporting reads only payment_ids/count — confirm at implementation). -
DISBURSED (issued/cleared) rows are NEVER auto-corrected: mismatch → report-only invariant
subsidy_disbursed_derivation_mismatch+ operator remediation (payment_adjustments / #1028). Freeze rule in ADR-053. -
A 23505 on insert is NOT blind-counted: reload the row, compare derivation (term_id, gross) — match →
already_existed; mismatch →derivation_mismatch+ WARN (reconciled next pass if undisbursed, else invariant).
-
-
Terms-coverage hardening (store): native
create_agreementrejectsterms.effective_from > initial_status_effective_from— with the guard, generator-timeterms_as_of == Noneon an active agreement is a hard per-agreement error (errors+ WARN), not a quiet skip. Import path untouched.
D3 — Generator home, lease, scheduler, trigger
-
Core in the LIB:
craig_financial::subsidy::generator(#1067 lib-module precedent; tests reach it). Ownsinsert_subsidy_payment(its own subsidy-shaped INSERT incl. term_id/origin — distinct from the bin’s placement-shapedcreate_payment; B8-watched) andgenerate_for_month(pool, month_start, scope, initiator)withGenerateScope { All, Child(Uuid) }(test-determinism seam) andInitiator { Scheduled, Manual { sub, name } }(manual runs stamped with the CALLER, not "system"). -
GenerationReport { month, run_id, generated, already_existed, derivation_mismatch, skipped_exclusivity, skipped_multi_program, skipped_not_active, errors, voided_stale }(contracts; Serialize+Deserialize+ToSchema). Per-agreement errors continue the batch (errors++); endpoint returns 200 with the report (WARN logs keyed run_id + agreement_id); fatal candidate-query error → 500. Onefinancial.subsidy_generation_completedevent (run_id, month, initiator kind, counts — no PII) staged in its own tx after the batch; ADR-003 + implementation-guide + security parsing rows. -
Run lease:
pg_try_advisory_lock(hashtext('subsidy-gen'))as a SESSION lock on a connection the run HOLDS for its duration (dedicatedpool.acquire()guard or standalonePgConnection); session locks release only via same-connectionpg_advisory_unlockor connection close — unlock-or-close on ALL paths (a still-locked pooled connection would silently 409 every future run), and connection-drop-releases IS the crash-safety property. Second concurrent run → endpoint409 GENERATION_ALREADY_RUNNING(newproblem_typesconstant) / scheduler skips the tick. Batching: keyset ordered(child_id, id), page boundaries only BETWEEN children (the cross-program guard needs each child whole). -
Scheduler (bin
spawn_workers): typed settingssubsidy_generator_interval_seconds, serde-default 0 = disabled (explicit deployment consent); devstack/CI enablement =CRAIG_FINANCIAL__SUBSIDY_GENERATOR_INTERVAL_SECONDS: 3600in docker-compose.yml’s craig-financial block AND docker-compose.scale.yml (no separate CI compose exists). Loop = detection.rs shape BUT the FIRST tick runs immediately under the lease. Business clock: UTC month boundaries (documented v1 approximation; #1092 owns the timezone decision); injectednow: fn() → NaiveDateseam for tests. -
Trigger
POST /v1/financial/subsidy-payments/generatein NEWapi/subsidy_generation.rs(B1 + authz-coverage/evil-corpus gate visibility): authz =authz.check(claims, ResourceRef { resource_type: SubsidyAgreement, resource_id: Uuid::nil(), assigned_worker_sub: None, supervisor_sub: None, attrs: {} }, Action::Approve)— the collection-appropriate capability: GA admin+supervisor; TX ADMIN-ONLY (regional + service DENIED by the subsidy_agreement ruleset’s leading rows). Request{ month: Option<String> }via ONE typedYearMonthgarde-custom parser (contracts; proptests: never-panics + roundtrip
canonical-form); malformed → 422VALIDATION_FAILED, valid-but-out-of-range → 400INVALID_DATE; bound[current − 12 months, current]. Registered routes + ApiDoc + evil corpus. -
Metrics:
craig_common::metricsfamily (generated/voided_stale/derivation_mismatch/skipped_*/errors) — jws-worker precedent.
D4 — Read surface, seed, tests, docs
-
Read-surface acceptance (ADR-053): generated rows are FINANCIAL artifacts under Payment policy (exposure identical to per-diem rows); the agreement LEDGER stays behind SubsidyAgreement policy; the TX-regional/service breadth of Payment policy is the pre-existing documented hazard (noted on #1086).
-
Seed: FOUR collections with SqlRow impls rendered BEFORE payments (sql.rs order):
SeedSubsidyAgreement,SeedSubsidyParty(caregiver ≠ child — blocking invariants),SeedSubsidyTerms,SeedSubsidyInterval(pending→active chains); + 3 months of unit-month history per agreement ENDING AT AND INCLUDING the current month (the immediate first tick landsalready_existedno-ops on every boot — no count contamination); statuses PENDING only (claiming aggregation untouched);SeedPayment.payment_type→String(old tokens byte-identical); manifest.rsOptionrendering fix; dedicated fixture ids. Churn: sql_byte_identity re-bless, payments COLUMNS, rate-consistency test scoped to placement-origin + a new subsidy↔term assertion, downstream snapshots enumerated at implementation. -
Tests — taxonomy-placed (conc/fault/invariant/property in their named dirs for
cargo xtask reliability):-
conc: TRUE cross-writer race generator-vs-placement (both orders, committed data, no double-pay); generator-vs-generator lease (409/skip); backdated-terminate-vs-tick convergence.
-
replay/fault: idempotent re-run; payment+outbox both-or-neither; one event per winner; per-agreement failure continues; 23505 derivation match+mismatch; voided-then-regenerate.
-
happy/sad: exact monthly net incl. leap-February + December→January; active-on-first-day boundary; suspended/terminated/pending generate nothing; per-diem skip; multi-program skip-all; post-case NULL-case FULL lifecycle (approve→issue→clear + adjustments
claiming + outbox + audit ids) — the "unchanged paths" proof; lists incl. new filters; eligibility event leaves rows non-IV-E (Rust) + DB CHECK rejection probe (structural); jurisdiction gate. -
API: GA/TX role matrices (GA supervisor 200; TX supervisor 403; TX admin 200; regional 403 both; pure service 403; service+stray-roles 403); month-bound edges; exact 422-vs-400 pins; evil corpus asserting no payment/outbox mutation + no reflection + malformed JSON.
-
Constraint probes: every new CHECK both directions, composite FK cross-agreement rejection, voided-exclusion unique (void frees key), FK RESTRICT, positive inserts for BOTH origins.
-
Invariants: 2 report-only sweeps + blocking agreement↔payment child-consistency, registered with metadata; gate run fails on unexpected SKIPs.
-
BFF: open-token rendering, nullable-origin views, unit-month display, claims views.
-
-
Unit-month presentation: origin-aware BFF payment-detail labels ("Monthly amount"; day-count hidden for subsidy rows); contracts doc-comments de-daily-fied; api-docs
data-model-financial document the convention;day_count_gross_consistentremediation text gains the subsidy note. -
Docs: ADR-053 (+nav) — shared-table nullable origins, unit-month accounting, correction/freeze protocol, child/month locking, funding CHECK, capability choice, read-surface acceptance, jurisdiction gate, UTC-clock approximation; ADR-003
implementation-guide (2 events + voided reshape); ADR-052 pointer line; shared-crates (contracts); cross-service-reconciliation.adoc ("every payment has a case and placement" now false — reword; from_queries already NULL-filter, verify); state-machines void note; configuration-reference (knob + PAYMENT_PERIOD backfill); runbook guidance (disable/recover/partial-error/metrics) in developer-guide; CHANGELOG (Changed + Added).
Verification
-
Units:
cargo nextest run -p craig-financial -p craig-financial-contracts -p craig-common -p craig-test-lib -p craig-web -p craig-security -p craig-seed -p xtask. -
cargo xtask dev restart→cargo xtask dev reseed→ devstack suites--run-ignored=all(financial + security audit tests) + BFF e2e financial specs. -
Invariant gate green (new blocking sweep asserted; zero unexpected SKIPs); migration validator diffed against a pre-change capture (zero NEW lines for the new file); api regen with content assertions (routes + nullable schemas); reconciliation zero-SKIP.
-
Full battery per MR → J1–J8 per MR → MR-A then MR-B (
Closes #1068) → merge per standing procedure → close-outs + epic &70 tick (b) → Status flips per step → archive at close.