Plan: Capability-Based Encrypted Search (C3)
On this page
- Status
- Context
- Scope
- Design
- 1. Registry +
capability(scheme, logical_type)— the single source - 2. Predicate/Relation AST + fail-closed planner + non-degradable policy channel
- 3. Typed filters +
ScopeConstraints+ authz-override - 4. BlindIndexSpec + domain-separated crypto + DOB
- 5. Cross-boundary single source (5 write shapes + seeder + verify-seed + trigram)
- 6. Crate boundaries
- 7. Search semantics (M9)
- 1. Registry +
- Test strategy
- MR sequence
- Follow-on issues to file (deferred)
- Reference anchors
Status
Sub-epic &67 (C3, under &66). Each MR is its own committed unit; this plan is the living spec for all of them (update Design/Scope on any deviation so the plan↔code diff stays zero).
| Step | Description | Status |
|---|---|---|
MR0 (#983) |
|
Done (2026-07-13) — !970 |
MR1 (#1015) |
|
Done (2026-07-13) — !971 |
MR2 (#1016) |
|
Done (2026-07-13) — !972 |
MR3 (#1017) |
|
Done (2026-07-13) — !973 |
MR4 (#1018) |
|
Done (2026-07-13) — !974 |
MR5 (#1019) |
|
Done (2026-07-13) — !975 |
MR6 (#1020) |
|
Done (2026-07-13) — !976 |
MR7 (#1021) |
|
Done (2026-07-13) — !977 |
Epic: &67 (under &66)
Issues: #983 (MR0), #1015–#1021 (MR1–MR7)
ADR: ADR-049
Branch: each MR on its own feature/… branch (MR0: feature/c3-mr0-adr-epic)
Context
CRAIG field-encrypts PII (ADR-020), but the
reports list endpoint ILIKE`s three columns (`narrative, reporter_first_name,
reporter_last_name) that encrypt_report_pii encrypts to opaque CGEF ciphertext — so under
EncryptionMode=Required report substring search silently returns nothing, the contract
(crates/craig-cases-contracts/src/reports.rs:46) advertises a capability it can’t deliver, and
the list/count WHERE clauses are hand-duplicated (store/reports.rs:120-142 vs :151-168) so
they can drift. The encryption↔searchability coupling is ad hoc across three hand-made instances
(referrals correct, persons blind-index, reports broken), and CRAIG is going multi-jurisdiction
(ADR-032).
Outcome: a field’s search capability is a pure function of (encryption_scheme,
logical_type), resolved through ONE registry of canonical FieldId`s that every boundary
(search, sort, write, decrypt, seeder, verify-seed, trigram) consults. A write-`Opaque field is
structurally unsearchable (enforced at const-eval, not by convention); an unsupported active
filter fails closed; authz/policy constraints are a separate, non-degradable channel; the
contract advertises real capabilities. Built on CRAIG’s own crypto (no new dep). Config-ready
for per-jurisdiction override, but that wiring is deferred (user scope, 2026-07-13). This is the
last blocker before C7 activation.
The decision, threat model, and crate rejection are recorded in ADR-049.
Scope
Fork decisions baked in (2026-07-13).
In (v1):
-
New leaf crate
crates/craig-search(generic framework) + acrates/craig-cases-fieldsleaf (the cases registry instance). Retrofit all five cases search entities: reports (fix), referrals, persons, cases, investigations. -
Single source of truth reaches everywhere: the registry drives the cases service write (5 shapes) + decrypt + search + trigram, and
tools/craig-seed+xtask verify-seedfor the entities they seed (persons + referrals only —SeedDatahas no reports), so their covered-column sets are registry-derived, not hand-maintained. Reports are not seeded today; adding report seeding (and thus registry-driven report encryption in the seeder) is a separate follow-on if ever wanted — there is no existing reports-seeding drift. -
SSN blind index migrated to a per-field HKDF domain now (one reseed, folds into C7).
-
Reports search repointed to plaintext
admin_unit+reporter_type; contract/OpenAPI corrected. -
ADR-049 + docs.
Out (deferred — config-ready, filed as follow-on issues):
-
Per-jurisdiction override wiring (
BundleContribution.search_schemes+ craig-cases bundle deps). The registry + fail-safe defaults support it additively (zero rework later). -
/search-capabilitiesendpoint (v1 honesty = corrected doc-comments). -
ADR-041 DOB hardening (encrypt DOB +
dob_hmac+birth_year): C3 models DOB asPlaintext{Date}(behaviour-preserving) withBlindIndexSpecDate-ready, so the hardening is a later additive registry flip. ORE/SSE; per-field AAD (ADR-048 follow-up).
Design
1. Registry + capability(scheme, logical_type) — the single source
One registry per entity maps a canonical FieldId (a per-entity enum, 1:1 with a physical column
— unknown field = compile error) to a FieldSpec. Predicates, sort, write, decrypt, seeder,
verify-seed, trigram all reference `FieldId`s resolved through it.
// craig-search (generic)
enum LogicalType { Uuid, Text, Date, Timestamp, Int, Bool, Jsonb }
enum OpaqueKind { Text, Jsonb }
enum FieldScheme {
Plaintext,
Opaque { at_rest: OpaqueKind },
BlindIndex { at_rest: OpaqueKind, spec: BlindIndexSpec, sibling_hmac: &'static str },
}
struct FieldSpec { column: &'static str, logical_type: LogicalType, scheme: FieldScheme }
// `sibling_hmac` is the physical _hmac column name — a registry `&'static str` literal, same
// trust model as `column` (verified by `registry_is_well_formed`, not a compile check, exactly
// like `column`). The "unknown field = compile error" guarantee applies to the per-entity
// `FieldId` ENUM used in predicates/sort/descriptors — physical column names are always literals.
struct Capability { substring: bool, equality: bool, ordering: bool }
const fn capability(scheme: &FieldScheme, ty: LogicalType) -> Capability;
Capability table (the whole policy on one screen):
| scheme \ type | Text | Uuid/Int/Timestamp | Date | Bool | Jsonb |
|---|---|---|---|---|---|
Plaintext |
substr+eq+ord |
eq+ord (no substr) |
eq+ord |
eq |
none |
BlindIndex |
eq |
eq |
eq |
eq |
eq iff canonicalizable |
Opaque |
none |
none |
none |
none |
none |
Rules: substring ⇒ Plaintext AND Text; ordering ⇒ Plaintext AND orderable type (not Jsonb;
UUIDv7 ok); equality per type. Predicate/sort/relation constructors are const fn that assert!
the required capability against capability(spec.scheme, spec.logical_type) → an illegal combo is
a const-eval build failure (edition 2024). Plus a mandatory registry_is_well_formed unit test
(every predicate/sort/relation capability-legal; every sibling_hmac exists + is the right type).
The executor uses the descriptor’s own table — never a caller-passed table arg.
2. Predicate/Relation AST + fail-closed planner + non-degradable policy channel
No free SQL — only registry &'static str (table/column), planner $N, and fixed
keyword/operator literals reach where_sql; user values reach only Bind.
enum UserPredicate { // NO "drop" variant
Equality { field: FieldId, param: ParamId },
Substring { target: SubstringTarget, param: ParamId }, // Plaintext+Text only
Relation { rel: RelationSpec, param: ParamId }, // replaces free-form Raw
}
enum SubstringTarget { Field(FieldId), Concat { fields: &'static [FieldId], sep: &'static str } }
struct RelationSpec { local_col: FieldId, via_table: &'static str,
via_select_col: &'static str, via_filter_cols: &'static [&'static str] }
// → {local_col} IN (SELECT {via_select_col} FROM {via_table} WHERE {f0}=$N [OR {fi}=$N ...])
RelationSpec covers every real join filter (reports→report_persons;
referrals/investigations→allegations victim_id OR perpetrator_id, one bind; cases→case_household)
— no {n}-string hack.
Two disjoint channels, no silent drop:
fn plan(descriptor, filters: &Entity::Filters, scope: &ScopeConstraints,
sort_by: Option<&str>, sort_dir: Option<&str>, rt: &SearchRuntime)
-> Result<SearchPlan, SearchError>; // { table, where_sql, binds, order_by, order_dir }
-
User filters never silently drop. §1 makes an unsupported search structurally impossible at build time (static v1). For the deferred runtime-override case, an active filter whose runtime scheme/type can’t support the op →
Err(SearchError::UnsupportedFilter)(fail-closed), never skipped. Absent optional filters contribute a bound NULL-guard($N::CAST IS NULL OR <clause>)(binds.len() stable). -
Policy constraints are a separate, always-ANDed, non-degradable category.
ScopeConstraintscompile to unconditional equality on their own columns (assigned_worker = $N,supervisor = $N) with a concrete bind (no NULL-guard, cannot no-op, cannot be dropped). A policy field that can’t be enforced is a hardErr, never a widened result. -
WHERE TRUEbase case — planner seedswhere_sql="TRUE"and appends ` AND (<clause>); never a bare `WHERE. -
Single plan feeds list AND count (kills the drift): one
execute_search::<T>(pool, &plan, paging) → (Vec<T>, i64)builds both fromplan.where_sql
plan.binds, readingplan.table.
3. Typed filters + ScopeConstraints + authz-override
Kill the string-keyed request. User filters = the existing typed Query DTOs
(ReportsQuery/ReferralQuery/CaseQuery/PersonSearchQuery) — missing/dup/unknown/type-mismatch
keys are impossible (typed Option<Uuid|String|NaiveDate>); each descriptor maps ParamId → DTO
accessor, bound with the FieldId’s CAST. Policy = a distinct type:
enum ScopeConstraints { None, Worker(Uuid), Supervisor(Uuid) }
// Entity-aware: the descriptor declares which scope columns it supports. Custom→Err needs a
// Result; lives in craig-cases (it maps craig_authz::ListScope, which the craig-search leaf
// must not depend on).
fn resolve_scope(desc: &EntityDescriptor, scope: ListScope)
-> Result<ControlFlow<PageEmpty, ScopeConstraints>, ApiError>;
// All → Continue(None); AssignedWorker(s) → Continue(Worker(s)); Denied → Break(PageEmpty);
// Custom(_) → Err(500). AssignedSupervisor(s): Continue(Supervisor(s)) for an entity WITH a
// supervisor column (cases), but Break(PageEmpty) for one WITHOUT (investigations —
// preserving `api/investigations/crud.rs:139-143`'s supervisor-scope-N/A empty page).
Handler resolves scope before the plan; under Worker(sub) it forwards the scope column from
the policy channel and never forwards the user ?worker= (a hostile ?worker=other can’t widen
scope — the two live in different types/channels). Mirrors today’s api/cases/crud.rs:222-234,
made structural. Tests: AssignedWorker + hostile ?worker= → only owner’s rows + scoped
total; AssignedSupervisor (cases narrows; investigations → empty); Denied→empty; Custom→500 (guard).
4. BlindIndexSpec + domain-separated crypto + DOB
enum Canon { AsIs, Utf8Nfc, IsoDate } // Date → "%Y-%m-%d" bytes
enum Codec { Base64 }
struct BlindIndexSpec { field_domain: &'static str, version: u32, canon: Canon, codec: Codec }
craig-crypto change (versioned): add pub const BLIND_INDEX_SCHEME_V1: u32 = 1; and
pub fn hmac_domain(&self, field_domain: &str, version: u32, canonical: &[u8]) → Result<String, CryptoError>
with info = b"craig-crypto/blind-index/" || field_domain || b"/v" || version. Takes bytes (Date
support) and gives per-field HKDF domain separation (no cross-column correlation vs the single
global BLIND_INDEX_INFO_V1, craig-crypto/src/lib.rs:172). SSN migrates to
field_domain="ssn", version=1, and MR4 co-locates every SSN blind-index site in one MR —
service write, service query, the seeder (tools/craig-seed/src/encrypt.rs), AND verify-seed’s
recomputation (xtask/src/cmd/verify_seed.rs, the 4th site the original "three sites" wording
missed) — so any reseed after MR4 is self-consistent (devstack reseeds routinely; no prod data
pre-1.0; C7’s down -v reseed also picks it up). All four derive the index through ONE shared
function, craig_search::FieldSpec::blind_index(enc, value), reading the registry’s
BlindIndexSpec (canonicalize + domain-separated HMAC) — so the ("ssn", 1, AsIs) tuple lives in
exactly one place (the registry) and write/query/seed can never drift. MR6’s broader seeder
unification then finds SSN already domain-correct. DOB: modeled Plaintext{Date} in C3 (behaviour-preserving, store/persons.rs:138);
BlindIndexSpec is Date-ready so ADR-041’s DOB hardening (dob_hmac + birth_year FieldId
reseed) is a later additive flip.
5. Cross-boundary single source (5 write shapes + seeder + verify-seed + trigram)
The craig-cases-fields leaf (deps: craig-search core + craig-crypto; NO axum/sqlx) holds the
FieldId enums + FieldSpec tables (the capability source of truth), consumed by craig-cases,
tools/craig-seed, and xtask.
Descriptor home (decided MR2, 2026-07-13). A per-entity SearchEntity impl binds
type Filters to the service’s wire DTO (e.g. ReportsQuery), which lives in
craig-cases-contracts — a crate that pulls sqlx. Making craig-cases-fields depend on it would
break that crate’s deliberate sqlx-free invariant (it is also consumed by the seeder + xtask).
So the SearchEntity descriptors live in the craig-cases service (api::<entity>::search), next
to the DTO + sqlx they bind — using the real DTO directly (a parallel filter struct would be a
drift surface, the very thing ADR-049 removes); likewise the EncryptableRow write descriptors live
with their row types in the service. craig-cases-fields stays the pure capability registry both
consult. The reports descriptor landed in MR2 as api::reports::search::ReportsSearch; the other
entities' descriptors follow in their MRs.
|
Write path is registry-driven via a per-shape accessor trait (Rust can’t index a field by name).
As built in MR6, the generic machinery lives in craig_search::row (core, DB-free):
enum FieldRef<'a> { Text(&'a mut Option<String>), TextRequired(&'a mut String), Jsonb(&'a mut serde_json::Value) }
trait EncryptableRow { fn field_mut(&mut self, column: &'static str) -> Option<FieldRef<'_>>;
fn hmac_slot_mut(&mut self, sibling_col: &'static str) -> Option<&mut Option<String>>; }
// Keying FINALIZED (MR6): BOTH accessors key by registry column literals — the walk iterates
// `encrypted_fields(registry::ALL)` and asks the shape for each spec's `column`. User-facing
// predicate/sort refs stay compile-checked FieldSpec consts; a registry-encrypted column the
// shape does not expose is a hard SearchError::RowMissingColumn (fail closed + loud), which is
// the cross-boundary alarm. Shapes are declared via the `impl_encryptable_row!` macro — a pure
// column→field table with no room for per-shape logic.
TextRequired covers reports.narrative (the one non-Option encrypted TEXT). Generic
craig_search::encrypt_row(row, registry::ALL, enc: Option<&FieldEncryptor>, mode: SearchMode)
walks the encrypted specs; Text/TextRequired→encrypt_str, Jsonb→the {"v": ct} envelope
(moved from the service into craig_search::row); for BlindIndex, set sibling
encryptor-conditionally via the shared FieldSpec::blind_index (None keyless — and an absent
base CLEARS a stale sibling) THEN encrypt the base. decrypt_row is the mirror walk. The ADR-020
fail-closed matrix is preserved in SearchError terms (MissingKeyRequired keyless-Required,
CiphertextWithoutKey for stored ciphertext with no key) — the service’s encrypt_field/
decrypt_field primitives are RETIRED (the wrappers in api/encryption/{report,referral,person}
now fix the entity + map SearchError→ApiError; error details changed accordingly, disclosed in
the CHANGELOG). Five shapes implement it: CreateReportParams, CreateReferralParams,
CreateReferralFromReportParams, persons create + update (MR6’s encrypt_person_pii unifies
the two former hand-written SSN blocks), plus the read models (Report/Referral/Person) for
decrypt_row. The seeder’s model::{Person,Referral} declare their own tables and
encrypt_seed_data runs the same walk; verify-seed derives its covered surface
(CoveredEntity: encrypted bases + blind-index siblings) from encrypted_fields() and builds its
SELECTs + assertions generically. The persons trigram query names its columns via the name
Concat FieldSpecs. The false invariant at api/encryption/mod.rs (encryption "only for fields
never full-text searched" / "narratives are NOT encrypted") is retired — the module doc now
points at the capability model.
6. Crate boundaries
-
craig-search(generic): depscraig-crypto,uuid,chrono,serde,thiserror;proptest(dev);sqlxoptional (only theexecute_searchglue). Nocraig-commondep (it drags axum+sqlx,craig-common/Cargo.toml:11,31); carries its ownenum SearchMode { Optional, Required }(handler mapsEncryptionMode→SearchMode). -
SearchError(own thiserror):Crypto,UnsupportedFilter{field,op},UnenforceablePolicy,MissingKeyRequired. MapCryptoError→SearchErrorinternally. -
ApiError mapping:
impl From<SearchError> for ApiErrorincraig-commonbehind asearchfeature (optionalcraig-searchdep) — the orphan rule forbids it in craig-cases; this mirrors the existingFrom<StageError>/craig-mq-errorprecedent (ADR-047;error.rs:493). No cycle (craig-searchdoesn’t depcraig-common).Crypto/internal→500 (ADR-020);UnsupportedFilter→400 (generic detail, not an oracle). -
sqlx features the glue needs:
["runtime-tokio","postgres","uuid","chrono"](already in workspaceCargo.toml:106);craig-casesenablescraig-search/sqlx. CI builds both configs:cargo check -p craig-search(core, DB-free security boundary) +--features sqlx.
7. Search semantics (M9)
Escape LIKE metacharacters in bound substring terms (%/_/\ via an ESCAPE '\' clause) so
search=% matches a literal %, not everything. Persons name search uses
Substring::Concat([first_name,last_name], " ") → (first_name || ' ' || last_name) ILIKE … to
hit the actual GIN trigram index (migrations/20260426193817_report_persons.sql:31), not
per-column ILIKEs.
Test strategy
-
Prereq: C6 (
services/craig-cases/tests/api/keyed_harness.rs) must be merged to main (merged 2026-07-13 as !969). -
Harness upgrade: add
serve()binding the assembledRouterto127.0.0.1:0(tokio::TcpListener+axum::serve) exposingbase_url(), so the typed HTTP clients work (oneshot can’t serve them); keep oneshot for raw at-rest reads. AddAssignedWorker/AssignedSupervisorauthz doubles + non-vacuous fixtures (seed two workers; assert the scoped list returns exactly the owner’s rows AND scopedtotal). -
Planner proptest (DB-free — the security boundary): injection corpus (
%_\';,sort_by="id;DROP…") never inwhere_sql, only inbinds; Opaque-never-in-SQL (structural); capability legality of every descriptor; NULL-guard bind-count stability; blind-index matrix (Some/None×Optional/Required,None,Required→Err); list-WHERE==count-WHERE; LIKE-escape. -
Integration (keyed harness,
#[ignore="requires devstack"],// @axis:tagged): reportssearch=<admin_unit substr>returns the row under Required+key;search=<narrative substr>returns nothing and does not 500; referrals parity; persons exact-SSN via blind index + name substring on the trigram index; cases scope fixtures (§3). -
Full battery per MR:
cargo xtask validate+ quality-budgets +cargo doc+ a fresh J1–J8 subagent per staged diff. Each MR is review-first (security-adjacent), held for user review.
MR sequence
-
MR0 (#983) — ADR-049 + committed sub-epic plan (nav-linked); file the sub-epic + children.
-
MR1 (#1015) — framework:
craig-search(registry/FieldId/capability/AST/planner/SearchError/ BlindIndexSpec/SearchMode) +craig-crypto::hmac_domain+BLIND_INDEX_SCHEME_V1+craig-commonsearch-featureFrom<SearchError>+craig-cases-fieldsskeleton. Proptest only; both build configs green. Blocks all. -
MR2 (#1016) — reports = C7 activation gate: reports onto the abstraction, search repointed, contract/OpenAPI honesty, integration tests via the ephemeral-listener harness. The only MR that gates C7; MR3-6 must not block activation. Deps: MR1 + C6-merged.
-
MR3 (#1017) — referrals retrofit. Deps MR1.
-
MR4 (#1018) — persons retrofit: name
Concaton the trigram index, DOBPlaintext{Date},ssn_last_fourBlindIndex through the planner (delete theapi/persons.rsmanual swap), SSN domain migration co-located across service write + service query + the seeder + verify-seed’s recomputation (the 4th site) — all via one sharedFieldSpec::blind_indexreading the registry spec, so a post-MR4 reseed is self-consistent,None,Required→Errhardening. Deps MR1. -
MR5 (#1019) — cases + investigations retrofit:
ScopeConstraintspolicy channel; addOption<Extension<FieldEncryptor>>+Extension<EncryptionMode>tolist_cases/list_investigations(EncryptionMode always layered,lib.rs:268). Deps MR1. -
MR6 (#1020) — write-path + cross-boundary single source:
encrypt_row+encrypt_person_pii(both persons sites); seeder + verify-seed consumeregistry.encrypted_fields()for persons+referrals (registry-derived covered set, no hand-lists; SSN already domain-correct from MR4); cross-boundary tests; retireencryption/mod.rs:6-10. Deps MR2-5. -
MR7 (#1021) — as-built docs: finalize ADR-049; fix stale contracts (
cases.rs:37,reports.rs:46,referrals.rs:47) + crate inventory. Deps MR2-6. As-built deviation: the reports + cases contract doc-comments were corrected in their own MRs (MR2/MR5), leaving only the referrals one here, and the committed API-reference pages are NOT regenerated in this MR —cargo xtask api-docsscrapes the running devstack containers (not source) and carries a pre-existing 7-service drift (~1,100 lines), so a regen here would sweep unrelated drift into a C3 MR against stale images. The OpenAPI source (the utoipa doc-comments) is corrected across MR2/MR5/MR7; the generated-page refresh + a from-source generator decision is #1022.
MR3-5 parallelizable after MR1; MR2 independently shippable + is the activation gate.
Follow-on issues to file (deferred)
-
Per-jurisdiction
BundleContribution.search_schemesoverride + craig-cases bundle wiring
fail-closedresolve()(refuse Opaque→Plaintext without a migration marker). -
/v1/cases/_meta/search-capabilitiesendpoint. -
ADR-041 DOB hardening (encrypt DOB +
dob_hmac+birth_year) — additive registry flip. -
Report seeding (the seeder has no
reportstoday):SeedData.reports+ generation
registry-driven report encryption in the seeder + verify-seed report assertions — only if report fixtures are wanted.
Reference anchors
| Thing | Location |
|---|---|
Reports bug + list/count WHERE drift |
|
referrals correct precedent |
|
persons blind-index swap + name-search trigram gap |
|
authz ListScope→SQL (override/Denied/Custom→500) |
|
five write shapes |
|
seeder/verify-seed drift (no reports enc) |
|
crypto primitives + single-domain hmac |
|
craig-common drags axum+sqlx; StageError From precedent |
|
leaf-error precedent |
|
PageRequest/PageResponse |
|
validation limits + garde/IntoParams DTOs |
|
OpenAPI 3-place registration |
|
C6 keyed harness (prereq) + typed clients |
|
stale invariant to retire |
|
ADRs to cross-ref |