ADR-058: Retention Enforcement — Transport Sweeps and Archive-Then-Prune

event_outbox, event_inbox, exchange_send_jobs, settled subsidy_reconcile_queue rows. These are NOT records of the system of truth — the domain facts live in domain tables and the audit trail.

The window invariant (supersedes ADR-022’s "matching" wording): the two tables are pruned asynchronously by different service databases, so equal windows fail at the boundary — a replayed outbox row can arrive after its consumer dedup record was pruned and re-execute side effects. The invariant is inbox ≥ outbox + (max prune cadence
clock skew); the shipped 30/31 split’s 1-day margin dominates both, and ServiceSettings::load() rejects inbox < outbox + 1. Window knobs (CRAIG_<SVC>__EVENT_{OUTBOX,INBOX}_RETENTION_DAYS; 0 disables that table’s sweep) move fleet-wide only — one service raising its outbox window above a peer’s inbox window re-opens replay-past-dedup.

Mechanics (craig-mq::retention): hourly + service-name-jittered prune inside every OutboxWorker, AFTER the drain; outbox-before-inbox (a partial failure leaves the dedup-safe direction); batched id-keyed deletes (5,000/batch, yield + cancellation between batches); a 1,000,000-row cycle cap that stops-and-warns instead of trusting a possibly-jumped clock; single-pruner-per-DB via pg_try_advisory_lock on a DETACHED connection whose drop closes the socket (an async unlock cannot run in Drop; a pooled lock strands on an idle connection and retention silently self-disables — the subsidy-import-finalize detached-lease precedent). The two service-local sweeps reuse the same loop with their own lock keys.

The replay boundary is mechanical: POST /v1/admin/events/replay refuses from older than now() - outbox_window with a typed 400 naming the boundary. The replay window IS the retention window; application-layer idempotency must hold across it (ADR-022).

Monitoring split: the four *_retention_overrun invariants are devstack/CI drift tools (report-only, thresholds = default windows + 48 h grace). Production observability is the event_rows_pruned_total{table} counter family — deleting audit-relevant rows silently is not acceptable.

audit_log, dead_letter_audit (craig-security), rule_evaluations (craig-rules). Never hard-deleted while unarchived. The accepted design (MR-B; the program plan §B1–B5 carries the unit detail):

The engine’s object I/O carries NO explicit timeouts — its puts/deletes are bare awaits and craig-store sets no ClientOptions/RetryConfig, so the load-bearing bounds are object_store’s own defaults, verified against the vendored 0.13.2 source: 30 s per request, 5 s connect (`ClientOptions::default, client/mod.rs), 10 retries under a 180 s retry budget (RetryConfig::default, client/retry.rs). This is what bounds "stalled upload delays shutdown" to ≈ the 180 s retry budget rather than forever (the audit F33 claim, refuted by these defaults); the timeout-less Local backend is already refused at boot (D9). Maintenance rule: an object_store upgrade must re-verify both Default impls — if upstream loosens them, pin explicit ClientOptions/RetryConfig in craig-store rather than silently inheriting the change. The related session-budget consequence of the engine’s detached advisory-lease connection (outside pool accounting) is recorded once, in the deployment guide § Database connection budget (#1160 — not duplicated here per #1168).

Store::put_create’s `If-None-Match: * is honored by AWS S3 / MinIO / R2 but PERMANENTLY ignored by Garage (CRDT design, no consensus — upstream documents it as a known limitation, not roadmap; the committed canary crates/craig-store/tests/garage_put_create.rs pins the observed overwrite). Since the devstack backend IS Garage, the conditional put alone cannot carry this ADR’s "archive objects are immutable once written". The archiver’s object writes now go through craig_retention::persist_one_object — the §U digest-verified create (craig_store::store_blob_create): precheck the key, REFUSE a foreign occupant with the typed ArchiveError::ForeignObject BEFORE any write (the batch fails; the prune is structurally downstream of the write returning Ok, so nothing hot is ever deleted against an unverified archive), converge on an identical occupant with a warn! (anomalous — archive keys are minted under fresh UUIDv7 `archive_id`s), and re-verify on a conditional-put refusal from honoring backends. The advisory lease already excludes concurrent leased writers, so the precheck’s read-then-write window only faces the misdirected-writer class the refusal exists to catch. On honoring backends the native conditional put remains the first line; the digest verify is the guarantee everywhere else.

The "archive-then-prune requires INSERT-ONLY sources" precondition above is restated: insert-only is the degenerate case (a row is at its fixpoint from birth). What the protocol actually requires is that NO DESIGNED WRITE PATH can touch a row once it is archive-ELIGIBLE — the select and the delete observe the row at different instants, and a mutation between them would be silently lost from the archive. A source qualifies under FOUR preconditions:

Detectability boundary, stated plainly: precondition 4 protects only columns the fixpoint predicate covers. A column that may legitimately change while the row is otherwise terminal — ssa_screening_runs.invalidated_at, which can stamp a COMPLETED run — is protected SOLELY by a writer-side in-WHERE refusal (the fork-2 freshness no-op: NOT (status = 'completed' AND as_of < CURRENT_DATE - make_interval(months ⇒ $n)), DB-side and monotone, never app-side check-then-write). Onboarding a source with such a column REQUIRES that refusal to exist and to be named.

Onboarded under this admission (#1466, ADR-065 § Amendment #1479 D8–D11): ssa_screening_runs (fixpoint = completed AND past the 12-month quiescence horizon; the SQL bound binds SCREENING_FRESHNESS_MONTHS as a parameter — never a second literal) and ssa_screening_members (fixpoint = the terminal statuses, CHECK-paired completed_at). Their archive SPECS + floors land with E3; the exchange service’s hot_window_days is load()-validated ≥ 365 d from B5 (per-ENGINE, not per-table — recorded limitation). Zero craig-retention engine changes: the spec.rs doc contract was rewritten in the same MR as this amendment (the J5 rule — the doc asserted insert-only while this admission is the new truth). Rejected: event-sourcing the runs tables (rewrites B3+B4) and a current+history split (quietly re-classifies runs/members to transport).

The specs + floors owed by the #1466 amendment landed: craig-exchange runs the shared engine over three FK-deletion-ordered specs (ssa_transport_handoffs → ssa_screening_members → ssa_screening_runs; the two tables WITH dependents carry NOT EXISTS order guards in BOTH select and delete — handoffs, dependent-free, need none) against its own archive_ledger (the craig-rules shape; cargo xtask archive-fetch craig-exchange resolves it). Bookkeep is ledger-only — no fleet event (nothing downstream consumes SSA archival facts; recorded deviation from the rules shape). Two admission notes this amendment adds to the doctrine:

As-built detail: ADR-065 § Amendment #1566. Purge stays refused (D14, #1480); the exchange ledger carries no legal_hold column until purge tooling exists to honor it.

rule_set_snapshots + rule_set_promotion_previews (craig-rules) are audit-class by nature but deliberately NOT onboarded to the archive-then-prune tier: growth is human-admin-mutation-rate × KB-scale content (unlike per-evaluation rule_evaluations), pin resolution and dispute replay require hot presence, and both are insert-only — satisfying this ADR’s precondition if that call is ever revisited. The D14 posture (no destruction until DFCS names a schedule, #1073) applies regardless.