Plan: Contested-Environment Component Testing
On this page
- Status
- Context
- Corrected core mechanisms (the ten blockers)
- Scope
- Surface taxonomy — full-codebase coverage map
- Design — the units by phase
- Registry design (
contested-surfaces.toml, repo root, CODEOWNERS-guarded) - Coverage schedule (two-path, every surface, named location)
- Fault stage (required mode, no silent skips)
- Budget methodology
- The never-easier checklist (created in C1; applied in every J-review)
- Decided-now (no "at implementation" residue)
- Corrected narrative (v2 claims that were false)
- Verification
- Risk register
- Documentation Updates
Status
Program of record for epic &83 (anchor #1493). Plan v3 — a full rebuild after a maintainer stop-ship review found ten blocker-class defects in v2. Every correction is code-grounded (see Corrected core mechanisms (the ten blockers)). Honest size: ~97 weight across 25 units (v2’s 66/17 silently deferred docs-first sequencing, per-service fleet sweeps, two IN classes, and a closure phase).
Epic: &83
Anchor: #1493 (plan-authoring)
Branches: one per unit (docs/… for C0–C1, feature/…/test/…/chore/… per unit thereafter)
Scale: ~97 weight / 25 units (canonical 1/2/3/5/8)
Critical path: C0 → C1 → C6 → C12 → C13/C14 → C23 → C24
Context
Battery failures have repeatedly cost afternoons of forensics because nothing separates "the component mishandles adversity" from "the test/fixture/envelope is bad" (2026-08-17: three burned cycles on docker-devtools daemon contention). Maintainer direction: adversity becomes a first-class, DETERMINISTIC test input — retries, races, broker disconnects, bursty/duplicate/ out-of-order delivery, pool exhaustion — forced by the test, never a quiet-box assumption or perf-at-scale. Two more maintainer requirements: cover this test shape across the WHOLE codebase, and give new code an easy integration mechanism so future surfaces land with contested legs by construction.
Second purpose — the arbitration instrument. A green per-component contested contract + a flaky battery ⇒ the defect is in the test/fixture/envelope (or a capacity attribution); a red contract ⇒ a real component defect WITH a reproducible case. This gives the no-environmental-blame rule an objective backstop and should directly reduce battery failures.
|
Standing constraint (maintainer, 2026-08-17): tests never get easier. No added serialization
beyond semantically-required ordering; no envelope widening; no assertion loosening; tests stay
exposed to ambient load. "The application needs to be able to handle a contested environment.
Life is never 0% system usage." Every J-review in this program runs the The never-easier checklist (created in C1; applied in every J-review) checklist
(zero |
This is plan v3 — a full rebuild after a maintainer stop-ship review found ten blocker-class
defects in v2 (transactionally-impossible trigger audit, async-in-Drop, wrong injection
boundary, an unviable feature graph, schedule-coupling from killing live consumers, unscheduled
"whole-codebase" coverage, a scanner blind to the 33-field BFF macro family, false-pass/
false-fail ratchets, silent-green skips, a production durability bypass). Every correction below
is code-grounded.
Ratified forks (maintainer, 2026-08-18)
-
F1 posture — in-battery REQUIRED (no silent skips; see Fault stage (required mode, no silent skips)). L10 (cluster) out-of-band; the e2e leg (C22) is its own serial stage.
-
F2 fault hooks — feature-gated via an exact acyclic graph + a release-artifact gate (a dev-dependency cannot enable a parent’s feature; see [feature-graph]).
-
F7 — keep the cipher injector (now C8).
-
Sequencing — epic &83 to COMPLETION (C24 merged) before #1466 (B5) starts.
-
Engineering-decided — the ratchet is a maintained typed REGISTRY (not grep markers); enforcement starts report-only, promotes to blocking per class at C23; the #1404 boundary = this program ships the pool-contention harness + report-only characterization, #1404 keeps the production typed-degradation design.
Corrected core mechanisms (the ten blockers)
M1 — PgFaultArmer cadence/audit survive rollback. A trigger that increments a counter and
inserts an audit row then RAISE`s loses both to the rollback (the `inbox_tx.rs:439 precedent
leaves no trace). Fix: cadence = one non-transactional SEQUENCE per armed fault (nextval()
persists across rollback) — IF nextval('craig_fault_seq_{id}') = N THEN RAISE; audit = the Rust
side reads last_value/is_called between phases. PgFaultArmer does NOT implement
FaultInjector (it can’t honor recorded_attempts()); it returns its own typed PgFaultReport {
invocations, injected, passed }. Statement classes decided now: event_outbox.published_at,
event_inbox.processed_at, exchange_send_jobs.status, upload_attempts.status. Cleanup =
explicit disarm().await + scratch-DB disposal backstop (DDL dies with the per-test database);
never Drop-dependent.
M2 — no async assert in Drop. ScenarioGuard holds a sync FnOnce and suppresses under
panic. Armers with external (DB) oracles expose async fn finish(self) → Report /
async fn assert_consumed(self), called in the test body; Drop only panics on a forgotten
finished flag (never reads the DB). The sync ScenarioGuard stays for in-process injectors
(cipher/object-store/publisher) whose counters are process-local.
M3 — publish-fault boundary split (PlacementPostUpdate was unimplementable). Placement stages
events IN the SQL tx (events.rs:3); the handler’s Publisher extractor is unused
(placements.rs:669); only the outbox worker later publishes. So: (a) a StageFaultInjector hook
inside craig_mq::stage_event (outbox.rs:86) — failure rolls back domain UPDATE + outbox row
together (atomicity leg); (b) OutboxPreSendInjector — the F2 hook on Publisher before
basic_publish (publisher.rs:141), armed during drain — committed row, pending outbox, later
exactly-once publish (the semantics PlacementPostUpdate wanted, at the real boundary).
CrashPoint::PlacementPostUpdate retired → OutboxPreSend + StageFault; the third cell (publish
ok, stamp fails) is M1’s event_outbox.published_at trigger.
M4 — feature graph (acyclic opt-in + release gate). Host features
craig-mq/fault-injection, craig-crypto/fault-injection, craig-store/test-util gate the
hook FIELDS (a field must live in the host struct). Rules: craig-test-lib’s normal deps NEVER
enable them; it has its own default-off forwarding feature; xtask depends on test-lib WITHOUT the
feature (lint-guarded); host integration tests self-dev-dep with the feature (resolver-v2
test-only activation); cross-crate consumers dev-dep test-lib with the feature. Release-artifact
gate (new blocking lint): cargo tree -e normal,build over the Dockerfile -p list + xtask must
show the features absent. Honest claim: "absent from release artifacts and every normal-dep graph,
machine-enforced" — not "impossible" (they compile under --all-features).
M5 — U2 never kills live devstack consumers. consumer_connection returns the FIRST attachment
(races other tests under 8-way nextest); depth is not an oracle. Fix: each disconnect leg creates
a unique vhost (craig-test-{run}-{test}), starts ITS OWN consumer there, identifies the
connection as "the only one in my vhost", and asserts on event_inbox.processed_at + handler
effect rows in its scratch DB. Barriers staged: attach → probe → domain oracle → close by exact
name → reconnect barrier → probe-2 → exactly-once. Per-service handler-contract legs move to
service crates (a generic craig-mq test can’t import private handlers).
M6 — two-path whole-codebase coverage is scheduled (see Coverage schedule (two-path, every surface, named location)): both legs (graceful AMQP close + TCP reset) per subscribe variant at the library layer (C10), plus per-callsite handler legs (C13/C14) and all 8 outbox round-trips (C14). The registry’s two-path rule makes any residue a blocking failure at C23.
M7 — S2S census = maintained typed registry. syn can’t expand bff_client! (33 fields at
clients/mod.rs:201); sinks like CasesForwarderSink are invisible. The registry
(contested-surfaces.toml) is truth; scanners catch omissions by reading syntactically-exact
things: bff_client! macro-call args, build_shared_client consumers, and explicit
// contested-surface: <id> markers for bespoke types.
M8 — ratchet = registry + nextest-validated test IDs + registration inventory. .subscribe( is
type-unresolved; heartbeat literals miss dynamic names and collapse shared helpers. Fix:
surface→test-ID mapping in the registry, validated against cargo nextest list --message-format
json (a named test must compile/exist); MQ census = AST walk of direct subscribe-family calls
under services/ + craig-authz, excluding #[cfg(test)]/tests/ callsites; worker census =
supervisor watch*( REGISTRATIONS, not heartbeat text. Report-only at C3; exact-marker classes
promote to blocking at C23.
M9 — required-mode fault stage, no silent skips. The fault_layer_available() boolean probe is
DELETED. Tests are #[ignore = "requires devstack+fault"]; cargo xtask fault-preflight (a
validate step before nextest) hard-fails when the fault layer is absent; ensure_ready gains
fault-profile health (devstack.rs:129 only checks craig-web today); executed-fault accounting
(test-results/fault/*.json → fault-report.json) asserts every armed fault fired and every IN
class has >0 executed faults — a green battery with zero executed faults is a FAILURE.
M10 — no durability bypass. Store::from_parts is #[cfg(any(test, feature = "test-support"))]
and takes the typed StoreBackend enum (never a backend_is_local: bool); is_local_backend()
keeps deriving from the typed enum that retention’s boot guard trusts (boot.rs:37).
from_config untouched.
Scope
In scope:
-
Deterministic, forced adversity for every IN-class production surface (see Surface taxonomy — full-codebase coverage map): transport disconnects (graceful + TCP-reset), confirm timeouts, burst/duplicate/out-of-order floods, commit-boundary crash windows, cipher/object-store faults, S2S/partner/auth-plane/CLI fault triples, worker crash-and-resume, DB-pool contention (harness + report-only), degraded-UX e2e.
-
A maintained typed registry (
contested-surfaces.toml) + census scanners so new surfaces land covered by construction; report-only first, promoted to blocking per class at C23. -
A required-mode fault stage with executed-fault accounting (a zero-fault green battery fails).
Out of scope (recorded, with reasons in Surface taxonomy — full-codebase coverage map):
-
SDKs (already ratcheted: retry-same-id pins + shared test-vectors).
-
Tool-time surfaces (craig-cli ops / craig-seed / xtask migrate,import,archive-fetch / mock-server — no production request path).
-
The composition engine (pure/deterministic, property-covered; its runtime rides classes 1/3/4).
-
The production typed-degradation design itself (#1404) — this program ships only C21’s harness
report-only characterization; the C21 enforcement promotion threshold = #1404’s design merged. -
Cluster L10 legs run out-of-band (weekly CI + on-demand), not in the per-push battery.
Surface taxonomy — full-codebase coverage map
| # | Class | Enumerated by | IN/OUT | Units |
|---|---|---|---|---|
1 |
S2S HTTP clients |
registry: 33 |
IN |
C15 |
2 |
External partners |
11 craig-partner-* over transport (audit is types-only → OUT) + SHINES + SSA/SOLQ |
IN |
C16 |
3 |
MQ surfaces |
AST: subscribe/idempotent/exclusive/dlq ×N + |
IN |
C10, C12, C13, C14 |
4 |
Background workers |
supervisor |
IN |
C19 (+ drain legs) |
5 |
DB pool |
bootstrap pool configs |
crate-battery authoritative; harness IN (C21), per-service ratchet OUT (recorded) |
C5, C11, C21 |
6 |
Crypto / keyring |
|
IN |
C8 |
7 |
Browser / BFF |
e2e flow matrix (C22) |
IN |
C22 |
8 |
Auth-plane |
OIDC discovery / JWKS / introspection / JWS verify (already workers) |
IN (new) |
C17 |
9 |
CLI |
authenticated mutating client ( |
IN (moved from OUT) |
C18 |
10 |
SDKs |
retry-same-id pins + shared test-vectors |
already ratcheted (recorded) |
— |
11 |
Tool-time |
craig-cli ops / craig-seed / xtask migrate,import,archive-fetch / mock-server |
OUT (no production request path — recorded) |
— |
12 |
Composition engine |
pure / deterministic |
OUT (property-covered); runtime rides classes 1/3/4 |
— |
Design — the units by phase
Phase A — governance (docs land FIRST)
C0 (w2, deps none): plan docs MR alone. This page (nav Active, plan-lint clean), the epic &83 body + child list, tracker repairs (#1466: drop stale blocked-by-#1479, encode the "starts when C24 merges" gate; #1493 supersede note; #1404 boundary). Merges before any code.
C1 (w3, deps C0): ADR + runbook MR. Two-layer doctrine + different-recovery-paths rule; the seam
doctrine incl. the M4 feature graph; the crash-point contract replacement (M3); RabbitDownInjector
retirement; PgFaultArmer trait divergence (M1); U6 redacted-500 affirmation; toxiproxy image
repo/tag/DIGEST + healthcheck + SBOM/license path + digest-refresh owner + control-plane security
posture (loopback-only, isolated network, non-root, limits); OUT-class justifications; the
The never-easier checklist (created in C1; applied in every J-review) checklist (CREATED here — it does not exist yet); the arbitration-ladder runbook
no-environmental-blame wiring. No hook/armer unit starts before C1 merges.
C2 (w2, deps C0, C1): issue-filing step. ~25 self-contained child issues (mechanism, files,
oracles, acceptance criteria, weight, blocks-links), the macro-feedback escalation
(tests/fault_injection/ vs tests/fault/ template drift), tracker links.
Phase B — foundations
C3 (w3, deps C1): registry + census scanners (report-only) + fault recorder + preflight.
contested-surfaces.toml (Registry design (contested-surfaces.toml, repo root, CODEOWNERS-guarded)), the AST/macro/marker census cross-check, the
executed-fault JSON recorder, cargo xtask fault-preflight.
C4 (w5, deps C1): toxiproxy sidecar. Compose profile fault, digest pin, loopback-only control
binding, PRE-PUBLISHED port range (64 ports) + cross-process lease allocator (fault_port_leases
in the test-plane DB, TTL + dead-run sweep), run+test-scoped proxy names, hand-rolled reqwest
client (rabbitmq_mgmt.rs shape), ProxyGuard explicit destroy().await + stale-only sweep,
ensure_ready fault-health.
C5 (w3, deps C1): pg_terminate helper + DB-disconnect legs. Target pg_backend_pid() capture
+ admin-PID exclusion + advisory-lock in-phase barrier + same-role privilege spec
reconnect/recovery asserts.
C6 (w5, deps C1): PgFaultArmer (M1 + M2: sequences, finish().await, DSN hard guard, 4
statement classes, retire the Faulty*Store`s, outbox/inbox crash-cell consumers, `restart.rs
wiring update).
C7 (w3, deps C1): feature graph + StageFaultInjector + OutboxPreSendInjector + release-artifact
gate + RabbitDown retirement (M3 + M4).
C8 (w3, deps C1, C7): CipherErrorInjector. zeroize-skip hook, operation-enum-only exposure
(Encrypt\|Decrypt\|Hmac; kcv excluded), full decrypt-path scope (persons_search,
persons_federal_export, reports/summary, ssn_promotion, referrals); PINS the existing redacted-500
contract (non-500 degradation is an API change, OUT).
C9 (w3, deps C1, C7): ObjectStore injector + typed test-gated from_parts + request_timeout
config (M10; default 30s, error class stays StoreError::ObjectStore).
Phase C — transport + library legs
C10 (w5, deps C4): craig-mq transport suite. Per-variant graceful-close + TCP-reset legs on
test-owned vhosts (M5/M6); confirm-timeout with a channel-established-first phase barrier (arm the
downstream timeout toxic AFTER a clean confirm, else you get PublishError::Amqp not
ConfirmTimeout); the AMQP slicer leg (moved here from v2’s U8).
C11 (w3, deps C4, C9): DB / object-store L2 legs. Separated pool-acquire vs connection vs in-query legs; ambiguous-put readback/digest/orphan oracle; the 5s cliff restated.
C12 (w5, deps C6): burst driver + inbox permutation floods. Staged attach/deliver/nack/re-attach
(no live-consumer race, per M5); per-handler ordering contract
(commutative\|revision-gated\|buffered\|strictly-ordered) declared in the registry; seeded
permutations print CRAIG_BURST_SEED + the replay command.
Phase D — fleet sweeps
C13 (w5, deps C6, C12): inbox handler-contract legs, tranche 1 (cases, exchange, financial, reporting).
C14 (w5, deps C6, C12): tranche 2 (placement, security + DLQ, rules ×2, composition, authz invalidation) + 8 per-service outbox staged-event→drain→consume round-trips.
C15 (w5, deps C3): S2S client matrix. Deliverable #1 = the exact method-level existing-vs-missing
matrix (financial already has the hanging-cases test main.rs:1591; reporting’s test fails at
token-mint not the cases hop — isolate them; ScreeningPolicyClient has fresh-cache short-circuit,
NOT stale-fallback — pin actual behavior; financial API-path typed-503 vs MQ-path retry/DLQ;
build_shared_client already gives 30s/5s — no "add if absent"). Per-request deadlines decided
per-client in the matrix (financial CASES_S2S_DEADLINE precedent only where a latency contract
requires).
C16 (w3, deps C3): partner fault-triple. 9 mechanical caps-clones + 1 ssa-solq-shaped delta (audit excluded, types-only).
C17 (w3, deps C3): auth-plane fault legs. OIDC discovery / JWKS / introspection / JWS via wiremock.
C18 (w2, deps C3): CLI contested legs. Timeout, ambiguous response, no-unsafe-replay.
C19 (w5, deps C6; kill legs trail C20): worker crash sweep. Three workers
(composition-invalidation, subsidy-generator, review-sweep). Crash levers = M1 triggers
docker kill (CancellationToken is graceful drain, a SEPARATE leg). Oracles: cache re-clears
idempotently; zero money duplication on generator replay; sweep resumes from durable lease/cursor.
Phase E — heavy rigs
C20 (w5, deps C12, OUT-OF-BAND): MultiReplicaCluster redesign. Dedicated
docker-compose.cluster.yml (ONE service + minimal own infra, NO fixed host-port publishes,
project craig-cluster-{run}, scratch vhost/DB, healthcheck readiness, resource limits);
stage-owned teardown + signal handler + stale-project sweep; artifacts to test-results/cluster/;
kill = docker kill (OS crash) vs mgmt close (reconnect), both legs; cargo xtask cluster-tests
+ weekly CI job + on-demand label.
C21 (w3, deps C1): pool-contention harness + report-only characterization. Pinned M=4 / acquire
5s / K=8 / Barrier start; report test-results/pool-contention.json; promotion threshold =
#1404’s design merged.
C22 (w8, deps C4, C1): degraded-UX e2e. cargo xtask e2e --degraded with the SAME devstack
lifecycle lock + durable dirty marker + ensure_ready auto-reconciliation + signal-aware cleanup
+ base --force-recreate restore + post-restore health checks; the docker-compose.fault.yml
overlay; the 8-row flow matrix (page/action × failed dependency × down\|flapping × expected UI ×
retry affordance × no-data-loss oracle).
Phase F — ratchet + closure
C23 (w5, deps C10, C12–C19): registry promotion to blocking. nextest-list test-existence
validation, bidirectional drift, --bless (stubs as report-only, never covered), exact-marker
classes → blocking, S2S/browser stay report-only with recorded promotion criteria.
C24 (w3, deps C20, C22, C23): program-verification gate. The blocking command/job matrix: final
no-skip validate with the fault profile REQUIRED, every-armed-fault-fired assertion,
panic/process-death cleanup self-test, production-DSN-refusal test, loopback-only runtime
verification, artifact retention. Merging C24 is the #1466 (B5) start trigger.
Dependency graph
Blocks-links (the C2 issue-filing step encodes these):
C0 → C1 → C2
C1 → {C3, C4, C5, C6, C7, C21}
C7 → {C8, C9}
C4 → {C10, C11, C22}
C9 → C11
C6 → C12
C12 → {C13, C14, C20}
C3 → {C15, C16, C17, C18}
C6 → C19 ; C20 → C19 (kill legs)
{C10, C13, C14, C15, C16, C17, C18, C19} → C23
{C20, C22, C23} → C24
Critical path: C0 → C1 → C6 → C12 → C13/C14 → C23 → C24.
Registry design (contested-surfaces.toml, repo root, CODEOWNERS-guarded)
Per-surface entry: stable dotted id; class
(mq-consumer\|outbox-worker\|worker\|s2s-client\|partner-crate\|crypto\|object-store\|pool\|
browser-flow\|cli\|auth-plane); crate; anchor {file, call, nth}; ordering (inbox handlers
only); levers; tests [{id, leg, via}]; oracle; status
(covered\|report-only\|skip — skip requires reason/owner/review_by).
Validation:
-
anchors resolve via AST or fail (stale);
-
census ⊆ registry or fail (new uncovered surface);
-
every test id ∈
nextest list(+ Playwright manifest); -
every mq-consumer references BOTH a graceful-close and a tcp-reset leg;
-
skip needs reason/owner/unexpired review_by;
-
--blesswrites report-only stubs, nevercovered.
Coverage schedule (two-path, every surface, named location)
Library layer C10 discharges both legs per variant (subscribe/idempotent/exclusive/dlq ×
{graceful, reset}). Per callsite: cases/exchange/financial/reporting inbox → C13 handler legs;
placement/security(+dlq)/rules(×2)/composition/authz → C14; 8 × spawn_outbox_worker → C10 drain
transport (pre-send hook + confirm-timeout) + C14 per-service round-trip. The registry’s rule 4 is
the machine check that nothing IN-class is a baselined exception at C24.
Fault stage (required mode, no silent skips)
required is the default in validate + the contested CI job (an absent layer ⇒ hard fail before
nextest); optional only for explicitly-invoked raw dev runs + the no-devstack CI job.
fault-preflight: toxiproxy /version healthy, loopback binding verified via docker port,
fault services up, lease table reachable. Executed-fault accounting asserts every armed fault fired
and every IN class has >0 executed faults.
Budget methodology
Measured baseline first: pin the 3-run median of cargo xtask validate before C4 merges (the v2
"+3–6 min" claim is withdrawn as unsupported). Ceilings: contested craig-mq ≤4 min at 8-way;
per-leg ≤90s (setup ceiling, never assertion widening); fault-preflight ≤20s; e2e --degraded
its own serial stage ≤10 min; cluster-tests out-of-band (no battery budget). A ceiling breach is a
blocking finding — split or fix the fixture; widening needs a maintainer-approved amendment.
The never-easier checklist (created in C1; applied in every J-review)
-
Zero
.config/nextest.tomlserialization/envelope hunks — unless a test is semantically order-dependent AND the ordering is called out in the review. -
No timeout/envelope widening to make a leg pass; a setup ceiling (≤90s) is a fixture ceiling, never an assertion-widening budget.
-
No assertion loosening, no
[ignore]to dodge a real failure (the M9[ignore]is fault-layer gating, enforced byfault-preflight, not a skip). -
Tests stay exposed to ambient load — a leg must not quiesce the box to pass.
-
A ceiling breach or an added serialization is a blocking review finding, split or fix the fixture; a genuine exception needs a maintainer-approved amendment recorded here.
Decided-now (no "at implementation" residue)
Trigger statement classes (M1); wiremock for in-crate S2S/auth/CLI, mock-server for e2e + the
partner triple; object-store both legs + request_timeout(30s); cipher ops Encrypt\|Decrypt\|Hmac
(kcv out); the C21 matrix (M=4 / K=8 / 5s / Barrier + JSON report + #1404 promotion threshold);
C22’s 8-row flow matrix; C19 crash points + money-duplication/lease-resume/idempotent-cache
oracles; per-request deadlines per-client in the C15 matrix; reporting token-mint isolation; U6
redacted-500 affirmed; toxiproxy digest/SBOM/refresh-owner as C1 acceptance criteria.
Corrected narrative (v2 claims that were false)
"Three of four injectors need zero production code" → only PgFaultArmer is zero-production
(object-store/publisher/cipher add feature-gated code). "All four finished" → three built, one
(RabbitDown) retired, PublishInTx split+renamed. The keyring seam is private same-crate
cfg(test) — arming-style precedent only, not a public cross-crate feature API (authorized by C1
on its merits). MultiReplicaCluster is NOT unused (smoke-tested today). C16 is 9 clones + 1
delta, not 11. C19 covers 3 workers. "Feature-unification impossible" → "absent from release
artifacts, machine-enforced". ~14 clients → 33 BFF fields + bespoke + domain + rules-client
sinks.
Verification
Every unit: its own solo-sequenced battery + the The never-easier checklist (created in C1; applied in every J-review) checklist in each J-review (zero
nextest.toml serialization/envelope hunks unless semantically required and called out). C10
proves the two-paths rule live. C12 oracles reviewed for hidden ordering assumptions. C23 baselines
start exact (every surface enumerated, never blanket-blessed). C24 verifies a WORKING SYSTEM
(no-skip validate with fault required, every armed fault fired, panic-cleanup self-test, DSN
refusal, loopback runtime check, artifact retention) — not merged-MR counting. Program exit = C24
merged (the B5 trigger).
Risk register
-
Trigger/DDL cleanup — explicit
disarm()+ scratch-DB backstop. -
async-cleanup-in-
Drop—finish()contract;Dropdetects-only. -
Cross-suite proxy deletion — run-scoped names + lease-aware stale-only sweep.
-
Port-pool exhaustion/abandonment — bounded pool, fail-loud with holders, TTL reclaim.
-
Control-plane exposure — loopback + isolation + non-root + preflight runtime check.
-
Feature leakage into release artifacts — graph rules + cargo-tree gate.
-
Stranded devstack overlay — lifecycle lock + dirty marker + auto-reconciliation + signal cleanup + post-restore health.
-
Abandoned compose projects — label+age sweep, stage-owned teardown.
-
Non-reproducible burst failures — mandatory seed + replay command + retained artifacts.
-
Secondary-
Dropmasking — armers assert only via explicitfinish(). -
Early real-bug triage budget; wall-time creep vs ceilings.
-
toxiproxy outside cargo-deny — rides the pinned-image SBOM path.
Documentation Updates
-
docs/modules/ROOT/nav.adoc— this plan under Plans → Active (C0). -
Epic &83 body — clickable plan link + child list (C0 link; C2 fills the child issues).
-
ADR-067 (two-layer fault-injection doctrine) + the contested-environment arbitration runbook (
operations/contested-environment.adoc) (C1). -
testing-reference.adoc— the never-easier checklist + a pointer to the arbitration ladder (C1). -
CHANGELOG.adoc== Unreleased— per unit as tooling/CLI surfaces land.