Plan: Reports List Summary Projection (pre-1.0 wire fork)

On this page

Revision history. v1 was internally reviewed (17 findings folded: repo-plan lifecycle, dual-executor compat shim, label swap, issue amendment, typed-error layering, structure). v2 was externally REJECTED 2026-07-27 (~29 findings: compile-blocking error layering, production-seam binding, ciphertext typestate, runtime atomicity, benchmark rigor, mandatory consumer tests). This v3 folds every finding and was user-approved 2026-07-27. Rejected designs are recorded in Rejected options.

Status

Unit Description Status

U0

GitLab choreography (#1158 description amended with the steer; label Plan::NEEDED → Plan::RPT-SUMMARY; follow-up #1176 filed + related for the detail-GET received_ip_hash question) + this plan committed as the first commit on feature/1158-report-summary-list.

Done (2026-07-27) — #1176 filed/related; plan + nav first commit

U1

craig-search: validated Projection newtype (SearchError::EmptyProjection; duplicate refusal), SearchPlan::list_sql(projection), ONE hard-forked execute_search(pool, plan, projection, limit, offset) — all 5 call sites updated (4× None; reports None-temporary until U3). craig-common SearchError → ApiError arm + mapping test. sqlx-free list_sql test suite.

Done (2026-07-27) — commit 16c0c847; red-proof: a projection-ignored mutation fails the declared-order test

U2

craig-cases-fields: reports::SUMMARY_COLUMNS (6 entries) + structural pins — encrypted-subset is exactly ["narrative"] (the structural decrypt-count assertion), heavy/identifying columns excluded, subset-of-ALL exactly-once.

Done (2026-07-27) — commit 6fa075d6; red-proof: widening with RAW_SUBMISSION fails both pins

U3

craig-cases: ReportSummaryRow (FromRow only), api/reports/summary.rs (SUMMARY_PROJECTION static seam, consuming decrypt_into_summaries, prefix-160/wire-161 preview, fetch_summary_page), handler flip to PageResponse<ReportSummary>, OpenAPI registration + 6-key contract test, 6 unit tests incl. proptest.

Done (2026-07-27) — commit 986b88e6; 9 unit tests (incl. the ciphertext-preview seam, keyless-Required fail-closed, wider-walk RowMissingColumn); red-proof: a byte-truncate mutation fails the multibyte test

U4

ONE consolidated keyed-harness integration test: full-featured fixture (multibyte >160-char narrative, names/phone, non-empty children/adults/raw_submission, ip_hash path), list = exactly the 6 summary keys with a true decrypted preview, detail GET = all eight list-removed values round-tripping.

Done (2026-07-27) — commit 0cff8593; passed live against devstack (also pins partner_id/received_request_id + a received_at list↔detail cross-check)

U5

Consumers, same MR: web queue ReportSummaryView + template preview; chain ReportRow narrative removal + #[expect(dead_code)] removal + MANDATORY deterministic serde/timeline unit test; CLI column swap + table-render contract test; intake-review e2e preview assertion.

Done (2026-07-27) — commit 25160f04

U6

Docs + benchmark: CHANGELOG (fork entry + stop-the-world deploy procedure), ADR-049 amendment, shared-crates.adoc §craig-search, api/craig-cases.adoc regen with verification, thresholded 500-report benchmark (latency + bytes).

Done (2026-07-27) — benchmark PASS decisively (protocol as specified but hyperfine was unavailable on the box: identical parameters — 3 warmup
15 measured runs, mean/min/max/σ — via a Python driver; both legs under identical ambient load): per_page=500 mean 398.0 ms → 113.9 ms (−71.4%), head max 124.5 < base min 383.2 (non-overlapping); bytes 3,348,543 → 179,546 (5.4%, threshold ≤20%); per_page=100 mean 89.3 ms → 28.8 ms (−67.8%). api page regen verified (md5 ca03fda3… → fe24e93a…, page shows the summary description)

Ship

Battery push, MR Closes #1158, merge, closing comment + AC walk, epic &73 tick, branch cleanup, then the SEPARATE docs-only direct-to-main close-out commit (Status → Done, archive move, archive-index row citing the real MR).

Done (2026-07-27) — MR !1085 (merge caba5342); full battery green incl. 300 e2e (one pre-push abort at the fn-name-and gate → the 2750ef2c rename); #1158 closed with the AC walk; epic &73 ticked; this archive commit completes the plan

Context

External audit F23 (2026-07-26), verified in-tree:

  • execute_search’s only projection is `SELECT * (crates/craig-search/src/exec.rs:79-86).

  • The Report row model carries 4 encrypted TEXT + 3 encrypted JSONB columns (services/craig-cases/src/store/models.rs:77-101), including multi-KB raw_submission.

  • list_reports decrypts every row of the page (api/reports/crud.rs → api/encryption/report.rs over reports::ALL): up to 7 AES-256-GCM-SIV decrypts per row in an all-fields row (4–7 in Required mode), × up to 500 rows/page, reachable by any authenticated operator (realm-flat scope).

  • #1139 (!1061) fixed exactly this cost on the create-echo path; the list path was left as-is.

Precedent SubsidyAgreementSummary is post-fetch mapping; this plan is a SQL-level projection — heavy columns are never fetched at all.

User steer (2026-07-27): option (a) — summary-only list, pre-1.0 hard fork, no ?projection param. Full shape stays on detail GET and on the create echo (the creator already holds the submitted data — the fork governs collection-GET semantics only).

Summary shape (6 wire keys): id, received_at, reporter_type, reporter_relation, admin_unit, narrative_preview. partner_id + received_request_id are removed too (no inventoried consumer uses them — least-data); received_ip_hash is removed from the bulk-list surface (detail GET still returns it realm-flat — follow-up issue #1176, not absorbed here).

Consumer inventory (drives U5)

Consumer Where Uses from list items Break mode if unfixed

Web queue

services/craig-web/src/routes/intake/reports.rs:55-71
templates/intake/reports.html:52

id, received_at, reporter_type, admin_unit, narrative|truncate(120)

LOUD: serde fails → error page

Chain timeline

services/craig-web/src/routes/cases/chain.rs:52-64 (#[expect(dead_code)] on the struct BECAUSE narrative is unused)

id, received_at, admin_unit, reporter_type

SILENT: parse fail → unwrap_or_default empties every chain (chain.rs:251); the e2e checks only heading/banner

CLI

crates/craig-cli/src/cmd/intake.rs:111-137

the above + reporter names (encrypted)

blank cells, no crash

Typed client / k6 / SDKs / mock

—

untyped Value / data[0].id / none

none

No existing test asserts heavy fields in list items. The seed contains zero reports — every fixture is test-created.

Design decisions

# Fork Choice

D1

Validated Projection newtype (solves error layering + empty edge + trust posture in one)

New craig_search::Projection: wraps &'static [&'static FieldSpec]; Projection::new(cols) → Result<Self, SearchError> refuses empty (SearchError::EmptyProjection, added in crates/craig-search/src/error.rs) and duplicate column names. Constructed ONCE per entity at static-init — validation happens where SearchError already maps to ApiError (the exhaustive match at crates/craig-common/src/error.rs:623 gains the arm, server-fault class, + a mapping test). Rendering is then INFALLIBLE: SearchPlan::list_sql(&self, projection: Option<&Projection>) → String; execute_search keeps returning sqlx::Error only. Trust posture stated honestly: shape-validated by construction; column provenance is registry literals BY CONVENTION (FieldSpec is publicly constructible; SearchPlan carries no registry identity) — the U2 subset-of-ALL test pins provenance for the one instance.

D2

Executor: ONE hard-forked signature

execute_search(pool, plan, projection: Option<&Projection>, limit, offset) — the existing fn gains the param; all 5 call sites updated in U1: persons, cases, referrals, investigations pass None; reports passes None in U1 (temporary, explicit) and flips to the projection in U3 — keeps every unit compile-green. No _projected twin (a second name kept so call sites don’t move is itself a compat shim). Count query, #1138 try_join, #1159 bound LIMIT/OFFSET untouched.

D3

Seam binding: production CANNOT pass None

summary.rs owns static SUMMARY_PROJECTION: LazyLock<Projection> (built from craig_cases_fields::reports::SUMMARY_COLUMNS; a unit test asserts construction succeeds). fetch_summary_page hard-wires Some(&SUMMARY_PROJECTION) into execute_search AND passes the SAME static’s fields to the decrypt walk — one binding drives both SELECT and decrypt. The SQL-pin unit test renders from the SAME static, so test and production share the seam.

D4

Ciphertext typestate: one consuming operation

decrypt_row mutates Strings in place (crates/craig-search/src/row.rs:322) — a summarize-before-decrypt bug would emit truncated ciphertext. Therefore the ONLY public path is consuming: decrypt_into_summaries(rows: Vec<ReportSummaryRow>, enc, mode) → Result<Vec<ReportSummary>, ApiError> — per row: decrypt (1 op) → in-place Unicode-safe truncate (char_indices boundary + String::truncate, then push '…') → move into the DTO. summarize is private; no plural decrypt-then-map two-pass (also kills the 500-plaintext residency + copy). Row doc says "storage-form narrative" (Optional mode may hold plaintext; post-decrypt it holds plaintext).

D5

Two-shape rule

ReportSummaryRow (store/models.rs): FromRow only, NO Serialize. Wire ReportSummary (api/reports/summary.rs): Serialize + ToSchema, 6 fields, no narrative field.

D6

Preview contract

NARRATIVE_PREVIEW_PREFIX_CHARS = 160 = the CONTENT-prefix limit; wire max = 161 chars ('…' appended iff cut) — const name, schema doc, tests, CHANGELOG all state prefix-160/wire-161.

D7

Decrypt-count AC

Structural (U2: encrypted-subset(SUMMARY_COLUMNS) == exactly ["narrative"]; unfetched cannot be decrypted) + seam binding (D3) + the U6 benchmark with pass/fail thresholds.

D8

Docs of record

ADR-049 amendment (read-projection leg); shared-crates.adoc §craig-search (executor signature + SearchError inventory + Projection); encryption-rows module doc updated; api/craig-cases.adoc regenerated in-MR with verification (D10).

D9

Deployment reality (NOT one-MR atomicity)

Both skew orders break, and cases scales horizontally (deployment-guide:961, docker-compose.scale.yml). Pre-1.0 choice: a stop-the-world procedure documented in CHANGELOG + MR: stop craig-web + ALL craig-cases replicas → deploy both images → start; rollback = redeploy the previous image PAIR (also both-or-neither); CLI upgraded with the deploy (stale CLI degrades to blank cells, no crash). No transition-compat shapes built.

D10

OpenAPI truth

The generated page shows codes/descriptions, not property tables → add an ApiDoc::openapi() contract test asserting the list response references ReportSummary and its schema has EXACTLY the 6 keys. Regen procedure: rebuild head (dev reload) → cargo xtask api-docs --service craig-cases → verify the page content actually changed (the fetcher silently skips unreachable services).

Units (single MR; each unit compile-green)

U0 — choreography + repo plan

First commit, token-gated, J-reviewed with the docs-only carve-out noted.

  1. Amend #1158’s description with the steer (self-contained issue BEFORE implementation).

  2. Label swap Plan::NEEDED → Plan::RPT-SUMMARY.

  3. Author this plan + nav Active entry; cargo xtask plan-lint.

  4. File the follow-up issue for the detail-GET received_ip_hash question (/relate to #1158) — filed as #1176.

U1 — craig-search: Projection + list_sql + hard-forked executor

Files: crates/craig-search/src/{plan.rs,exec.rs,error.rs,lib.rs}; crates/craig-common/src/error.rs (:623 match arm + mapping test); the 5 call sites (4× None + reports None-temporary).

Tests (NEW crates/craig-search/tests/list_sql.rs, sqlx-free; suffixed for the coverage matrix):

  • @axis: happy list_sql_without_projection_renders_legacy_select_star_happy — exact legacy string incl. $n+1/$n+2 (refactor-equivalence pin; a one-time text change would merely re-prepare once — this is NOT a cache-risk claim).

  • @axis: happy list_sql_with_projection_renders_columns_in_declared_order_happy.

  • @axis: sad projection_rejects_empty_and_duplicate_columns_sad — Err(EmptyProjection) / duplicate refusal; never SELECT *.

  • (craig-common) @axis: sad search_error_empty_projection_maps_to_server_fault_sad.

U2 — craig-cases-fields: SUMMARY_COLUMNS

pub const SUMMARY_COLUMNS: &[&FieldSpec] = &[&ID, &RECEIVED_AT, &REPORTER_TYPE, &REPORTER_RELATION, &ADMIN_UNIT, &NARRATIVE]; (consts exist under these names, crates/craig-cases-fields/src/reports.rs:57-159).

Tests: @axis: happy encrypted-subset == exactly ["narrative"] (the structural decrypt-count assertion); @axis: sad excludes ip_hash/partner_id/received_request_id/names/phone/children/adults/raw_submission; non-empty; subset-of-ALL exactly-once.

U3 — craig-cases: row, DTO, consuming decrypt, handler, OpenAPI

  • store/models.rs: ReportSummaryRow (FromRow only; doc: "storage-form narrative — deliberately not Serialize").

  • api/encryption/rows.rs: impl_encryptable_row!(store::models::ReportSummaryRow { "narrative" ⇒ TextRequired(narrative), } hmac: none); + module-doc list updated.

  • NEW api/reports/summary.rs: SUMMARY_PROJECTION static (D3), ReportSummary, the D4/D6 truncate, private summarize, decrypt_into_summaries, fetch_summary_page(app, plan, enc, mode, limit, offset) → Result<(Vec<ReportSummary>, i64), ApiError>.

  • api/reports/mod.rs: pub(crate) mod summary; + re-export ReportSummary; update the "four submodules" module doc.

  • crud.rs::list_reports: reports call flips from the U1 temporary None to fetch_summary_page; PageResponse<ReportSummary>; utoipa body + #1158 doc paragraph. get_report/create/store/migrations UNCHANGED (create-echo scope per the steer).

  • api/mod.rs: register reports::ReportSummary.

Unit tests (matrix suffixes):

  • @axis: happy narrative_preview_short_and_exact_160_pass_through_happy

  • @axis: happy narrative_preview_truncates_multibyte_and_appends_ellipsis_happy (200×'é' → 160×'é'+'…' = 161 chars)

  • @axis: happy narrative_preview_empty_input_stays_empty_happy

  • @axis: evil proptest narrative_preview_prefix_property_never_panics_evil (∀ String: no panic; ≤161 chars; ≤160-char input ⇒ identity; else strip '…' leaves a 160-char prefix)

  • @axis: sad summary_projection_static_builds_and_sql_omits_heavy_columns_sad — SUMMARY_PROJECTION constructs; the REAL ReportsSearch plan rendered with it pins the exact SELECT id, received_at, reporter_type, reporter_relation, admin_unit, narrative FROM reports prefix; excluded columns absent.

  • @axis: happy openapi_list_response_schema_has_exactly_the_six_summary_keys_happy (D10 contract test on ApiDoc::openapi()).

U4 — ONE consolidated integration test (keyed harness, devstack)

NEW services/craig-cases/tests/api/report_summary_list.rs + alphabetical mod. ONE #[ignore = "requires devstack"] test (single harness build — don’t triple the expensive setup): report_list_summary_projection_and_detail_full_shape_happy (@axis: happy), fixture = create-report with >160-char MULTIBYTE narrative, names + phone set, non-empty children/adults/raw_submission, non-empty received_ip_hash path, EncryptionMode::Required. Phases:

  1. List: item key set is EXACTLY the 6 summary keys; narrative_preview == submitted-plaintext first-160 + '…' (proves the one decrypt ran — cannot be a ciphertext echo); plaintext fields round-trip.

  2. Detail GET body["report"]: ALL EIGHT list-removed values present and round-tripping (ip_hash, first/last/phone, full narrative, children, adults, raw_submission) — the fork boundary pinned on both sides.

U5 — consumers (same MR) + their deterministic tests

  • Web queue: ReportSummaryView { id, received_at, reporter_type, admin_unit, narrative_preview }; templates/intake/reports.html:52 → {{ r.narrative_preview }}; ReportView unchanged (detail envelope).

  • Chain: delete narrative from ReportRow and its #[expect(dead_code)]
    Default arm     (the expectation becomes unfulfilled under -D warnings once all remaining fields are used). Mandatory deterministic unit test (chain.rs tests mod): deserialize a new-shape list-item JSON (no narrative key) into ReportRow + assert the timeline builder emits the Report row — the SILENT-OK degrade path is otherwise untestable by e2e (seed has no reports; the spec checks only heading/banner). No conditional e2e hardening.

  • CLI: columns → id, received_at, admin_unit, reporter_type, reporter_relation + comment (names are detail-only by design). Table-render contract test in the CLI suite: table mode asserts reporter_relation header present and reporter-name headers absent (JSON-mode tests ignore the column list — they cannot catch this).

  • Web e2e preview assert: extend intake-review.spec.ts — submit a distinctive long multibyte narrative via the API fixture, assert the QUEUE renders the truncated preview text (blank/untruncated/wrong preview currently passes).

U6 — docs + benchmark (final implementation commit)

  • CHANGELOG === Changed: the fork entry — 6-key summary list (ONE decrypt/row, down from up-to-7 incl. raw_submission); partner_id/received_request_id/ received_ip_hash removed from the LIST surface; detail GET + create echo unchanged; prefix-160/wire-161 preview; the D9 stop-the-world deploy + rollback procedure.

  • ADR-049 amendment; shared-crates.adoc §craig-search; api/craig-cases.adoc regen per D10 with verification.

  • Benchmark protocol (recorded in the MR; pass/fail bearing):

    • Dataset: reseed, then create EXACTLY 500 reports via the typed client (narrative 2,000 chars; raw_submission ≈4 KB; 2 children + 2 adults each; one unique admin_unit value for isolation). Same dataset rebuilt identically per revision.

    • Revisions: merge-base SHA vs head SHA; per revision: dev reload, auth once, 3 warmup requests.

    • Measure per_page=100 AND 500: hyperfine -w 3 -r 15 (report mean/min/max/σ)
      downloaded bytes via curl -H 'Accept-Encoding: identity' -w '%{size_download}' (Content-Length is encoding-dependent), idle host.

    • Thresholds: PASS iff head max < base min at per_page=500 (non-overlapping — the JSONB removal should dominate noise) AND head bytes ≤ 20% of base bytes; any latency overlap → investigate before merging, never close on a regression.

Gates & risks

Gates: fmt; clippy -D warnings; quality-budgets (no new serde_json::Value; extraction keeps list_reports in budget); plan-lint; full battery + e2e. Pipeline cancellation ONLY per the documented red-CI infra-failure exception after a green local battery.

Risk Mitigation

U1 error layering

Solved by construction: validation at Projection::new (SearchError, mapped at common error.rs:623 + test); rendering infallible; executor stays sqlx::Error

Production regresses to SELECT *

D3 seam: fetch_summary_page hard-wires the static; the SQL pin renders from the SAME static

Summarize-before-decrypt emits ciphertext

D4 consuming typestate — no public summarize; single-pass

chain SILENT-OK masks the break

Mandatory deterministic serde/timeline unit test (U5)

Deploy skew (both orders break; cases scales horizontally)

D9 documented stop-the-world + image-pair rollback; no compat shapes

Residual exposure (honest)

The list still decrypts up to 500 narratives/page into previews for realm-flat operators — a large reduction, NOT elimination; the per-page cap is #1170’s; the detail-surface ip_hash question is #1176

Empty/duplicate projection

Typed constructor + U2 pins

Future ciphertext leak via refactor

Wire type lacks the field; U4 exact-key-set pin; D4 typestate

Pipeline

  1. U0 commit (plan + nav; token + J-review, docs-only carve-out stated).

  2. U1→U6 compile-green; gates; each substantive commit gets its own token
    staged-diff J-review    .

  3. Battery push (background, harness); git ls-remote verify.

  4. MR Closes #1158 (Summary/Changes/Test Plan + benchmark tables + deploy procedure + CLI UX note).

  5. Merge per the documented procedure (green local battery is the gate; cancel/force only under the recorded red-CI exception); J-record MR note; verify merge SHA.

  6. Closing comment (SHAs + AC walk); epic &73 tick; branch cleanup.

  7. Post-merge close-out commit (docs-only, direct-to-main per git-workflow): plan Status → Done, archive move + archive-index row citing the REAL MR number (ADR-030: Done = shipped).

Rejected options

  • (b) ?projection=summary|full param — still a default-shape change, keeps the heavy path reachable in bulk, and adds a permanent query-param surface for a pre-1.0 system that can simply fork.

  • (c) keep shape, decrypt lazily/stream — leaves multi-KB raw_submission on the wire for a list nobody renders it in; complexity without the least-data win.

  • Dual executor (execute_search_projected twin) — a second fn name kept so call sites don’t move is itself a backwards-compat shim; pre-1.0 forks the ONE signature.

  • Fallible list_sql rendering — pushes Result through every call site for a validation that can happen once at Projection::new; rendering from a shape-validated value is infallible by construction.

  • Separate summarize-then-decrypt passes — row.rs:322 decrypts in place; a public summarize on undecrypted rows would truncate ciphertext; the consuming single-pass makes the wrong order unrepresentable.

Edit this page · latest