Plan J: Env-var Documentation Coverage + Docker-compose DRY

On this page

Status

Step Description Status

1

Plan filing — body lands in the docs-only Plan D refresh MR alongside Plans D/G/H/I/K. nav.adoc + CHANGELOG. No code changes.

Done (2026-05-15 via !307)

2

F-038 implementation: expand .env.example to cover the env vars currently set in docker-compose.yml but missing from operator docs. Organize into sections: Required (must be set per deployment), Recommended (sane defaults exist but operators should review for production), Advanced (introspection cache knobs etc.). Document each with a one-line comment + the production-pattern value (not the devstack value, where they differ — e.g. CRAIG_CASES__ENCRYPTION_MODE=required not optional).
+ Shipped 2026-05-25. Audit at branch-time anchor: .env.example 74 vars, docker-compose.yml 131 CRAIG_ vars, net 57 missing (5 of which — CRAIG_E2E_BASE_URL, CRAIG_INTAKE_URL, CRAIG_INTAKE_STANDALONE_URL, CRAIG_FAMILIES, CRAIG_SEED — are devstack / test-harness only and correctly excluded per cross-cutting invariant #1). The remaining 52 operator-tunable vars are documented, plus 41 supplemental entries (per-service _FILE mount variants per Plan E § Step 14, file-mounted peer JWKS, field-encryption master key, log-level + body-limit + statement-timeout tunables for all services) — total 166 vars covered. File restructured into the Required / Recommended / Advanced sections from the Step 2 example template. Secrets carry <generated-at-{idp-setup,deployment,s3-provisioning,captcha-provisioning}> placeholders; devstack defaults are NOT mirrored. _FILE variants documented as the production-recommended deployment shape; inline-secret variants annotated as devstack-only.

Done (2026-05-25)

3

F-039 implementation: docker-compose YAML anchors. Define top-level extension fields and replace duplicated config with anchor references. Verify effective config unchanged via docker compose config diff (before vs after).
+ Shipped 2026-05-25. Implemented as 5 scalar anchors (x-oidc-issuer-default, x-oidc-internal-url, x-jurisdiction, x-admin-unit-label, x-rules-engine-url) + 1 mapping anchor (x-garage-credentials). Deviation from plan body draft: the original x-oidc-defaults and x-jurisdiction-defaults mapping anchors (intended for <<: anchor merge) cannot produce config-equivalent output because each service consumes its OIDC URL / jurisdiction under a service-prefixed env-var name (CRAIG_CASESOIDC_ISSUER, CRAIG_CASESJURISDICTION, …) — a single mapping anchor would yield unprefixed OIDC_ISSUER / JURISDICTION keys that the config crate’s CRAIG_<SVC> prefix wouldn’t pick up. Scalar anchors centralize the literals on the value-side instead, which IS config-equivalent across per-service-prefixed keys. The x-garage-credentials mapping anchor works as designed because CRAIG_STORE is a single shared prefix (not per-service-prefixed) — merged into 4 service environment blocks via <<: garage_credentials. Anchor counts (verified): OIDC issuer 10× (rules / cases / placement / exchange / financial / reporting / security / intake / intake-standalone / web — plan claimed 7×, actual 10×), OIDC internal-url 10×, jurisdiction 9× (every CRAIG service except web — plan claimed 9×, confirmed), admin-unit-label 7× (plan claimed 9×, actual 7× — intake doesn’t set it), rules-engine-url 7×, garage-credentials 4× (cases / exchange / reporting / security — confirmed). Verification: docker compose --profile identity-multibackend config --format json rendered before/after, structural keys {services, networks, volumes, configs, secrets} extracted via jq, sorted-key diff returns 0 (empty). The plain --format yaml | sort diff shows only the new x- root keys (echoed by docker compose config as opaque extension data; harmless per compose-spec semantics). Cross-cutting invariant #3 honored.

Done (2026-05-25)

4

Plan completion audit + archive.

Done (2026-05-25) — Audit verified all 3 prior steps via the plan-completion-audit subagent per .claude/docs/delivery-protocol.md. File:line evidence cross-checked: 3 tier headers in .env.example (REQUIRED / RECOMMENDED / ADVANCED), 6 top-level x-* anchors in docker-compose.yml (oidc-issuer-default / oidc-internal-url / jurisdiction / admin-unit-label / rules-engine-url / garage-credentials), <<: *garage_credentials mapping merge at 4 sites, *oidc_issuer_default scalar anchor at 10 sites. Plan body archived via cargo xtask docs plan-archive (Plan K F-048 tooling). nav.adoc Active entry removed; archive.adoc § Infrastructure & DevOps row added; CLAUDE.md sibling-plans row reclassifies Plan J as complete + archived. Closes #434.

Epic: &32 (epic: Env-var Documentation Coverage + Docker-compose DRY (Plan J))
Issues: #432 (Step 2) · #433 (Step 3) · #434 (Step 4)
Branch prefix: chore/env-compose-
*Milestone: TBD

Context

Two configuration-coherence smells surfaced in the 2026-05-15 post-Plan-C/F audit:

  • F-038 (P1): docker-compose.yml sets 114 env vars across the 9 CRAIG services; .env.example documents 72. 42 env vars are set in compose but missing from .env.example. Notably:

    • All cross-service URLs (CRAIG_REPORTINGPLACEMENT_URL, CRAIG_REPORTINGCASES_URL, CRAIG_REPORTING__FINANCIAL_URL, etc.)

    • All client credentials (CRAIG_*CLIENT_ID, CRAIG_*CLIENT_SECRET)

    • Intake-specific knobs (CRAIG_INTAKECAPTCHA_SECRET, CRAIG_INTAKEPUBLIC_RATE_LIMIT, CRAIG_INTAKE__MODE)

    • CRAIG_STORE__* (S3 backend) vars for the 4 services that store objects (cases, exchange, reporting, security)

      A new operator copying .env.example gets a deployment that boots in degraded mode (missing service URLs) without warning.

  • F-039 (P2): docker-compose.yml has substantial config repetition:

    • OIDC_ISSUER: ${OIDC_ISSUER:-http://host.docker.internal:8180/realms/craig} — duplicated 7×

    • OIDC_INTERNAL_URL: http://keycloak:8080/realms/craig — duplicated 7×

    • JURISDICTION: georgia + ADMIN_UNIT_LABEL: County — duplicated 9× (every service)

    • CRAIG_STORE__S3_* credentials block — duplicated 4× (cases, exchange, reporting, security)

      YAML anchors collapse these to declare-once, merge-in-N-places.

Cross-cutting invariants

  1. F-038 distinguishes secrets from config. Secrets (CRAIG_*CLIENT_SECRET, CRAIG_*JWS_SIGNER_KEY, S3 credentials) get placeholder values in .env.example, not the devstack defaults. Comment policy: # REQUIRED: generated by <method>; see <doc-link>. Devstack-only values like password go in docker-compose.yml, never in .env.example. FILE env-var pattern: for secrets that are file-mounted in production (per Plan E’s CRAIG<SVC>__SIGNING_JWK_FILE + CRAIG_PEER_JWKS_JSON_FILE), document the _FILE variant in .env.example rather than the literal-secret variant — the file-mount pattern is the production deployment recommendation and the literal-secret variant is devstack-only.

  2. Direct docker compose config invocation is a documented exception. .claude/docs/coding-conventions.md says "NEVER run docker compose directly — use cargo xtask dev." That rule covers runtime commands (up, down, start, restart); docker compose config is a config-render command — pure-textual; doesn’t start containers. The Step 3 verification uses it for diff-checking effective YAML, not for orchestration. Document the carve-out in the Step 3 MR body.

  3. F-039 must be config-equivalent. Before/after verification: docker compose --profile identity-multibackend config | sort > /tmp/{before,after}.yml && diff /tmp/before.yml /tmp/after.yml. Any diff = bug. (Empty diff = the YAML anchor merge produced equivalent effective config.)

  4. YAML anchor depth is bounded. Anchors are top-level x-* extension fields; no nested anchor declarations. Keeps readability OK and matches docker-compose’s documented anchor support.

Scope

In scope (2 findings):

  • F-038 .env.example coverage expansion

  • F-039 docker-compose YAML-anchor DRY

Out of scope:

  • xtask env-coverage lint to auto-detect drift between .env.example and actual env-var consumption — not yet filed. NOTE: an earlier draft of this plan said "fold into Plan K as F-046" but Plan K F-046 is plan-lifecycle doctrine, NOT env-coverage. If implementing the env-coverage lint becomes worth a finding, file it as a fresh finding (next free F-NNN) — likely as part of Plan H (since it’s a workspace-lint expansion) or as a small Plan M if scope warrants. Tracked as a documented TODO for now

  • Strum DTO conversion — Plan D

  • DRY refactors in code — Plan G

  • Idiomatic Rust — Plan H

  • Module decomposition — Plan I

  • Canopy xtask backports — Plan K

Steps

Step 2: F-038 .env.example coverage expansion

Files:

  • .env.example — grow from ~72 documented env vars to ~114. Section structure:

    # ═══════════════════════════════════════════════════════════════
# REQUIRED: per-deployment overrides
# ═══════════════════════════════════════════════════════════════

# OIDC issuer URL — typically your IdP's realm URL
OIDC_ISSUER=https://idp.example.com/realms/your-org

# Encryption mode — set to 'required' in production, 'optional' for devstack only
CRAIG_CASES__ENCRYPTION_MODE=required
# (similar for craig-placement, craig-financial, craig-exchange)

# Service-to-service URLs — internal hostnames; set per deployment topology
CRAIG_REPORTING__CASES_URL=http://craig-cases:8002
CRAIG_REPORTING__PLACEMENT_URL=http://craig-placement:8003
# (similar for all cross-service deps)

# Per-service OAuth2 client credentials — generated at IdP setup
CRAIG_CASES__CLIENT_ID=craig-cases
CRAIG_CASES__CLIENT_SECRET=<generated-at-idp-setup>
# (similar for every service)

# S3-compatible object store (Garage in devstack, S3/MinIO in production)
CRAIG_STORE__S3_ENDPOINT=https://s3.example.com
CRAIG_STORE__S3_BUCKET=craig-prod-attachments
CRAIG_STORE__S3_ACCESS_KEY=<generated-at-s3-provisioning>
CRAIG_STORE__S3_SECRET_KEY=<generated-at-s3-provisioning>

# ═══════════════════════════════════════════════════════════════
# RECOMMENDED: sane defaults exist; review for production
# ═══════════════════════════════════════════════════════════════

# Postgres statement timeout (per-service); default 30s
CRAIG_CASES__DB_STATEMENT_TIMEOUT_MS=30000

# Trusted proxies (CIDR list); default empty — set to your load-balancer subnet
CRAIG_INTAKE__TRUSTED_PROXIES=10.0.0.0/8

# ═══════════════════════════════════════════════════════════════
# ADVANCED: tune only if you understand the implications
# ═══════════════════════════════════════════════════════════════

# Introspection cache TTL ceiling (seconds); default 60
# Per Plan F (introspection-validation-mode.adoc), this is the upper bound
# for cached token claims; the effective TTL is min(token.exp - now, this).
CRAIG_<SVC>__INTROSPECTION_CACHE_TTL_MAX_SECONDS=60

# (full list — populate during audit)

Branch: chore/env-compose-step2-env-example-expansion

MR title: chore(.env.example): document 42 missing env vars; organize by Required / Recommended / Advanced [Step 2 of env-compose]

Verification:

  1. .env.example covers every env var found in docker-compose.yml (verify via diff — see derivation algorithm below)

  2. No secrets in .env.example (only placeholder values)

  3. Sections are coherent — a new operator can read top-to-bottom and configure a minimal production deployment

Branch-time baseline re-anchor: before opening the MR, run:

# Current .env.example documented env vars
docs_count=$(grep -cE '^[A-Z][A-Z0-9_]*=' .env.example)
# Current docker-compose.yml env vars (per service `environment:` block)
compose_vars=$(awk '/^[ ]+environment:/,/^[ ]+[a-z]+:/' docker-compose.yml | grep -E '^      [A-Z]' | sed 's/:.*//' | sort -u | wc -l)
echo "docs: $docs_count   compose: $compose_vars   delta: $((compose_vars - docs_count))"

The 2026-05-15 baseline (72 documented, 42 missing) is point-in-time. Plan F !304 + Plan C !299 + Plan B !280 may have already landed .env.example entries — re-anchor before claiming "+42".

The 42 missing env vars — appendix table: enumerate during Step 2 audit as | Var | Section (R/R/A) | Production-pattern value | Source line in docker-compose.yml |. Append to plan body before Step 2 ships to capture the audit deliverable in-plan.

Step 3: F-039 docker-compose YAML anchors

Files:

  • docker-compose.yml — define anchors at the top:

    # Top-level extension fields for shared env-var blocks (compose v2 anchors)
x-oidc-defaults: &oidc_defaults
  OIDC_ISSUER: ${OIDC_ISSUER:-http://host.docker.internal:8180/realms/craig}
  OIDC_INTERNAL_URL: http://keycloak:8080/realms/craig

x-jurisdiction-defaults: &jurisdiction_defaults
  JURISDICTION: georgia
  ADMIN_UNIT_LABEL: County

x-garage-credentials: &garage_credentials
  CRAIG_STORE__S3_ENDPOINT: http://garage:3900
  CRAIG_STORE__S3_BUCKET: craig-devstack
  CRAIG_STORE__S3_ACCESS_KEY: ${GARAGE_KEY_ID}
  CRAIG_STORE__S3_SECRET_KEY: ${GARAGE_KEY_SECRET}

services:
  craig-cases:
    environment:
      <<: [*oidc_defaults, *jurisdiction_defaults, *garage_credentials]
      # ... craig-cases-specific env vars ...

  # similar for craig-placement, craig-exchange, etc.

Branch: chore/env-compose-step3-yaml-anchors

MR title: chore(docker-compose): DRY up env-var blocks via YAML anchors [Step 3 of env-compose]

Verification:

  1. docker compose --profile identity-multibackend config | sort > /tmp/before.yml (before MR)

  2. Apply MR

  3. docker compose --profile identity-multibackend config | sort > /tmp/after.yml

  4. diff /tmp/before.yml /tmp/after.yml returns empty — proves no effective config change

  5. Devstack boots clean: cargo xtask dev start --force

  6. Multibackend profile boots clean: cargo xtask dev multibackend-up

Step 4: Plan completion audit + archive

Mirror Plan B Step 8 / Plan C Step 18 / Plan F Step 6 pattern.

Files Touched

File Step Change

.env.example

2

EDIT (~42 new entries; sectioned)

docker-compose.yml

3

EDIT (YAML anchors; effective config unchanged)

Verification

After every step: docker compose config diff is empty (Step 3) + cargo xtask dev start --force boots clean.

Risks

Risk Mitigation

YAML anchor depth reduces compose readability for new contributors

Counter-argument: 845-line compose with 42 duplications is already hard to scan; anchors make the intent clearer once a one-line "this is an anchor merge" comment is added at each merge site

.env.example growth invites vestigial entries (someone adds a knob, never removes when deleted)

Future-state: cargo xtask env-coverage (a Plan K follow-up — F-046) auto-detects drift between .env.example and actual env::var consumption. Not in this plan’s scope.

Docker Compose’s YAML anchor merge has surprising behavior (e.g. list merges)

Use the explicit <<: [*a, *b] sequence form (compose v2.27+ documents this); pin compose version in CI

.env.example placeholder values inadvertently get committed as real secrets

Use unambiguous placeholders like <generated-at-idp-setup>; CI grep for the <…​> pattern as a pre-commit gate (optional follow-up)

docker-compose.scale.yml (multi-replica scale file) may have its own duplications that F-039 doesn’t touch

Audit the scale file alongside the main compose for anchor opportunities; if it has the same OIDC / JURISDICTION / S3 cred duplications, include in F-039’s scope. Otherwise document as in-scope-but-no-changes-needed

After this plan lands

  • .env.example documents every operator-tunable env var, sectioned by Required / Recommended / Advanced

  • docker-compose.yml declares OIDC, jurisdiction, and S3-credentials blocks once via YAML anchors; effective config unchanged

  • New-operator onboarding: a copy of .env.example + per-deployment overrides is sufficient to configure a production CRAIG deployment without reading source code

Edit this page · latest