Plan K: Canopy xtask + Hook Backports
On this page
- Status
- Context
- Cross-cutting invariants
- Scope
- Steps
- Step 2: F-040 + F-041 (xtask lock + --remove-orphans)
- Step 3: F-042 reconcile_ports_env drift detection
- Step 4: F-043
cargo xtask dev refresh - Step 5: F-044
cargo xtask coverage - Step 6: F-045
cargo xtask migrate snapshot|rollback - Step 7: F-046 plan-lifecycle doctrine (ADR + pre-commit wording)
- Step 8: F-047
cargo xtask docs plan-lint - Step 9: F-048
cargo xtask docs plan-archive - Step 10: F-049
cargo xtask e2e --no-refreshflag - Step 11: Plan completion audit + archive
- Files Touched
- Verification
- Risks
- After this plan lands
Status
| Step | Description | Status |
|---|---|---|
1 |
Plan filing — body lands in the docs-only Plan D refresh MR alongside Plans D/G/H/I/J. nav.adoc + CHANGELOG. No code changes. |
Done (2026-05-15 via !307) |
2 |
F-040 + F-041 (combined — both compose-level): port |
Done (2026-05-15) |
3 |
F-042 implementation: port Scope substantiated 2026-05-25 (ADR-030 §3): Plan body’s "~30 lines" undercounted — actual port is ~500 LOC because canopy’s |
Done (2026-05-25) — Shipped. +501/-134 in |
4 |
F-043 implementation: |
Done (2026-05-25) — Shipped. +159 LOC in Scope notes (ADR-030 §3): SOPS branch (canopy’s |
5 |
F-044 implementation: |
Done (2026-05-25) — Shipped. |
6 |
F-045 implementation: |
Done (2026-05-25) — Shipped. |
7 |
F-046 implementation: plan-lifecycle doctrine. New CRAIG ADR-030 mirroring canopy ADR-013 — canonical Status vocabulary ( |
Done (2026-05-16) |
8 |
F-047 implementation: |
Done (2026-05-16) |
9 |
F-048 implementation: |
Done (2026-05-16) |
10 |
F-049 implementation: |
Done (2026-05-25) — Shipped. |
11 |
Plan completion audit + archive. |
Done (2026-05-25) — Audit verified all 10 prior steps via the plan-completion-audit subagent per |
Epic: &33 (epic: Canopy xtask Backports (Plan K))
Issues: #435 (Step 2) · #436 (Step 3) · #437 (Step 4) · #438 (Step 5) · #439 (Step 6) · #454 (Step 7 — F-046 plan-lifecycle ADR + Q4/Q6) · #455 (Step 8 — F-047 plan-lint) · #456 (Step 9 — F-048 plan-archive) · #457 (Step 10 — F-049 e2e --no-refresh) · #440 (Step 11 — plan completion)
Branch prefix: feat/canopy-backport-
*Milestone: TBD
Context
Canopy (sibling project, same author) has shipped six xtask patterns that solve real CRAIG pain points. Substantiated 2026-05-15 by side-by-side comparison of the two xtask trees:
-
F-040 (P0): Concurrent
cargo xtask dev startinvocations can race; CRAIG has no lock. Canopy’sacquire_lock(~30 lines) is a small, high-value port that prevents the race entirely. -
F-041 (P0): CRAIG’s
docker compose upinvocations omit--remove-orphans. When old containers from prior runs still exist with prior-run port allocations, Docker Compose reuses them and binds to new ports — exactly the "port already allocated" leak symptom we’ve hit repeatedly (memoryfeedback_devstack_port_collision). Canopy passes--remove-orphanseverywhere. -
F-042 (P1): Canopy’s
reconcile_ports_env(file:412-495) reads.ports.env, queriesdocker compose portfor each service, rewrites the file when drift detected. CRAIG has no equivalent — operators using stale.ports.envget confusing failures (e.g. tests connect to a port from a previous devstack run that’s no longer bound). -
F-043 (P1): Canopy has
cargo xtask dev refresh(a third lifecycle command alongsidereloadandrestart). It reads SHA-256 staleness markers + conditionally rebuilds only what changed. Reuses.ports.envwhen no rebuild needed — sidesteps the port-reassignment cascade. -
F-044 (P2): Canopy enforces a workspace-wide line-coverage floor via
cargo xtask coverage. CRAIG measures coverage in CI but doesn’t gate locally. -
F-045 (P2): Canopy’s
cargo xtask migrate snapshot/rollbacklets developers safely test migrations locally; CRAIG has no equivalent. -
F-046 (P1): Canopy ADR-013 (plan-lifecycle-and-status-vocabulary, Accepted 2026-04-23) defines a canonical Status vocabulary, plan archival rule, and reframes precommit Q4 + Q6 against the "plans are living specs, not backlogs/errata dumps" doctrine. CRAIG’s pre-commit Q4 still says "document why you deviated in an errata section" and Q6 still says "document it in the plan in a 'potential improvements' section" — both actively push the anti-patterns canopy ADR-013 enumerates. CRAIG’s ADR-013 slot is taken (web-session-strategy), so file as the next available number.
-
F-047 (P1): Without enforcement, F-046’s vocabulary will drift the same way canopy’s did pre-ADR (canopy measured 610 "Complete" / 191 "✓" / 85 "Done" / 45 "complete" / 0 "COMPLETE" across 95 plans before the lint landed).
cargo xtask docs plan-lintis the enforcement; ships soft-fail at first to grandfather existing drift. -
F-048 (P2):
cargo xtask docs plan-archiveis the operator-driven companion — scans for all-Done Status tables +git mv`s to `archive/. Today CRAIG plans get archived ad-hoc via manualgit mv; canopy’s command makes it idempotent. -
F-049 (P3): Canopy’s pre-push uses
cargo xtask e2e --no-refresh; CRAIG’sxtask/src/cmd/e2e.rshas no such flag. The motivation per canopy’s pre-push header: "prevents ensure_ready from wiping the just-loaded seed if markers drift during a long local session." Small ergonomics fix.
Together these close CRAIG’s most-painful infra gap (port collision is a recurring frustration) + add two genuinely useful developer-ergonomics commands (coverage + migrate) + close the plan-lifecycle drift that the canopy ADR-013 retrospective catches before it becomes a 95-plan cleanup.
Cross-cutting invariants
-
Verbatim port where possible. Canopy code is the reference; CRAIG adapts paths/project-names only. Each Step’s MR cites the canopy source file in the body with a one-line "ported from canopy:<path>:<lines>" attribution.
-
Step 4 is additive, not replacement. CRAIG already has
reload(full rebuild + restart) +restart(cold teardown + cold build).refreshslots in as the smartest-of-three. Document precedence in.claude/docs/local-dev.mdas a decision table ("I changed X → run Y"). -
Step 5 starts soft.
cargo xtask coveragemeasures and prints; CI gate added only after baseline written to a tracked file (docs/modules/ROOT/pages/test-coverage-scorecard.adocalready exists; promote it to a contract). Avoids a flag-day "coverage CI broken" event. -
Step 6 schema-version-aware. Manifest captures
SELECT version FROM _sqlx_migrationsper DB at snapshot time; rollback warns + bails if current schema is ahead of snapshot’s schema (data restore against newer schema = silent corruption). Per-DB granularity via--db <name>. -
Lock TTL handling. If xtask crashes mid-operation the lock stays held until manual rm; canopy’s
acquire_lockpolls 60s before timing out with a clear error message. Port that retry+timeout logic verbatim; don’t roll your own. -
Doctrine-before-enforcement (Step 7 → Step 8). F-046’s ADR lands before F-047’s lint — the lint cites the ADR as its source of truth. Promote the lint to blocking only after the existing-drift cleanup MR has landed (separate follow-up).
-
F-046 wording is verbatim from canopy ADR-013 §3. The Q4 + Q6 replacement text in canopy’s pre-commit hook (lines 83 + 85-86) is the authoritative form. CRAIG’s port adapts only the file:line example in the meta-protocol’s bullet (a) to a CRAIG path.
Scope
In scope (10 findings):
-
F-040 xtask exclusive lock
-
F-041
--remove-orphansin all compose calls -
F-042
.ports.envreconciliation -
F-043
cargo xtask dev refresh -
F-044
cargo xtask coverage -
F-045
cargo xtask migrate snapshot/rollback -
F-046 plan-lifecycle doctrine (new ADR + pre-commit Q4/Q6 reframe)
-
F-047
cargo xtask docs plan-lint -
F-048
cargo xtask docs plan-archive -
F-049
cargo xtask e2e --no-refreshflag + pre-push wire-up
Out of scope:
-
Canopy’s
secretsSOPS workflow — separate decision (CRAIG hasn’t picked a secrets management approach; defer) -
Canopy’s
validate-in-network— useful, but defer until F-042 lands (port reconciliation may obviate the need) -
Pre-commit AI meta-protocol port — shipped 2026-05-15 via branch
chore/precommit-ai-verification-protocol(verbatim from canopy.githooks/pre-commit:44-78). Lives in CRAIG’s.githooks/pre-committoday; out of scope here
Steps
Step 2: F-040 + F-041 (xtask lock + --remove-orphans)
Files:
-
xtask/src/docker.rs— addpub fn acquire_lock(project: &str) → Result<LockGuard>mirroringcanopy/xtask/src/docker.rs:610-637. File lock atenv::temp_dir().join(format!("{project}-xtask.lock"));try_lock_exclusive()with 60s polling timeout. -
xtask/src/devstack.rs— callacquire_lockat the start ofbring_up,restart,reload,reseed. TheLockGuarddrops at end-of-function, releasing the lock. -
xtask/src/docker.rs::compose— extend signature to accept extra args via the existingargs: &[&str]parameter; ensure call sites inxtask/src/devstack.rs(lines 282-293, 343-345, 159-160, 168-169) pass--remove-orphanson everyupanddown.
Branch: feat/canopy-backport-step2-xtask-lock-and-remove-orphans
MR title: feat(xtask): exclusive lock + --remove-orphans in all compose calls [Step 2 of canopy-backports]
Verification:
-
Open two terminals;
cargo xtask dev startin one, immediately run same in the other → second one polls + reports "another xtask process is running" rather than racing -
cargo xtask dev start --forcetwice in succession; no "port is already allocated" errors (was the recurring failure mode) -
cargo nextest run --workspace— no regression
Step 3: F-042 reconcile_ports_env drift detection
Files:
-
xtask/src/docker.rs— portreconcile_ports_env()from canopy:412-495. Logs each port drift with before/after values. -
xtask/src/cmd/{test,e2e,validate}.rs— callreconcile_ports_envat the start of each command (before tests run that rely on.ports.env). -
xtask/src/devstack.rs::ensure_ready— call after composeupto capture freshly-bound ports.
Branch: feat/canopy-backport-step3-port-reconciliation
MR title: feat(xtask): reconcile .ports.env against actual docker compose port bindings [Step 3 of canopy-backports]
Verification:
-
Manually bind a port to something other than the expected service; run reconcile; see the drift logged +
.ports.envrewritten -
Run
cargo xtask dev start --forcetwice;.ports.envis consistent withdocker compose portoutput both times -
cargo xtask testagainst a stale.ports.envtriggers reconciliation transparently
Step 4: F-043 cargo xtask dev refresh
Files:
-
xtask/src/devstack_guard.rs— extend with SHA-256 staleness markers per category (source / deps / Dockerfile / compose / migrations / rulesets / seed). Mirror canopy:devstack_guard.rs:544-644. -
xtask/src/cmd/dev.rs— addRefreshaction to theDevCommandenum + match arm. Callsdevstack_guard::auto_refresh(). -
.claude/docs/local-dev.md— new "When to use reload vs restart vs refresh" decision table.
Branch: feat/canopy-backport-step4-dev-refresh
MR title: feat(xtask): cargo xtask dev refresh — smart minimum rebuild [Step 4 of canopy-backports]
Verification:
-
Edit a
services//src/.rsfile; runcargo xtask dev refresh→ rebuilds + reloads only the affected service -
Edit
docker-compose.yml; run refresh → re-runs full lifecycle (because compose changed) -
No edits since last
dev start; run refresh → reports "already up to date"
Step 5: F-044 cargo xtask coverage
Files:
-
xtask/src/cmd/coverage.rs(NEW) — port canopy:cmd/coverage.rs. Wrapscargo llvm-cov --workspace --json; parses output; checks against floor. -
xtask/src/cmd/mod.rs— register subcommand. -
xtask/Cargo.toml— no new deps (cargo-llvm-cov is invoked as a subcommand, not a library dep) -
.gitlab-ci.yml— addcargo install cargo-llvm-covto the CI image build OR install pre-baked -
docs/modules/ROOT/pages/test-coverage-scorecard.adoc— promote to contract: list per-crate baselines
Branch: feat/canopy-backport-step5-coverage
MR title: feat(xtask, ci): cargo xtask coverage — line-coverage measurement + (soft) gate [Step 5 of canopy-backports]
Verification:
-
cargo xtask coverageproduces per-crate line-coverage summary -
CI runs the command and reports results; soft-fails only if
--fail-under-lines <pct>flag is set -
docs/…/test-coverage-scorecard.adoclists current per-crate baselines
Step 6: F-045 cargo xtask migrate snapshot|rollback
Files:
-
xtask/src/cmd/migrate.rs(NEW) — port canopy:cmd/migrate.rs.snapshotenumerates the 8 CRAIG service DBs (craig_rules,craig_cases,craig_placement,craig_exchange,craig_financial,craig_reporting,craig_security,craig_intake… wait, intake is stateless per ADR-017; verify the actual DB list at branch time). For each DB:pg_dump -F c -h localhost -p $CRAIG_PORT_POSTGRES_5432 -U craig <db> > .devstack/snapshots/<ts>/<db>.dump. Manifest captures git HEAD SHA +SELECT version FROM _sqlx_migrationsper DB. -
xtask/src/cmd/mod.rs— register subcommand. -
.devstack/snapshots/— added to.gitignore. -
.claude/docs/local-dev.md— document the snapshot/rollback workflow.
Branch: feat/canopy-backport-step6-migrate
MR title: feat(xtask): cargo xtask migrate snapshot/rollback — DB lifecycle safety net [Step 6 of canopy-backports]
Verification:
-
cargo xtask migrate snapshotcreates.devstack/snapshots/<ts>/{<each-db>.dump, manifest.json} -
Modify schema (e.g. add a column via psql);
cargo xtask migrate rollback --snapshot <ts>restores -
Schema-version-ahead detection: snapshot at version A, advance schema to B, attempt rollback to A → warns + bails
Step 7: F-046 plan-lifecycle doctrine (ADR + pre-commit wording)
Files:
-
docs/modules/ROOT/pages/adrs/adr-030-plan-lifecycle-and-status-vocabulary.adoc(NEW). CRAIG’s previous ADR slots: ADR-028 = Service Identity (Plan E), ADR-029 = Introspection-mode token validation (Plan F). Plan L’s ADR will be 031 if needed. Section structure (CRAIG convention): Status / Context / Decision / Consequences (4 sections per.claude/docs/coding-conventions.md:157). Canopy ADR-013 uses 5 (adds Enforcement); roll Enforcement into Decision §4 to fit CRAIG convention. Adapt canopy ADR-013 verbatim; substitute "canopy" → "CRAIG" + update example MR refs to CRAIG MRs. -
Q4 + Q6 replacement wording (inlined verbatim from canopy
.githooks/pre-commit:83,85-86):4. Have you deviated from the plan? If so: (a) UPDATE the plan's Design/Scope section to reflect what was built — plans are living specs, not immutable records; (b) file a GitLab issue for any follow-up work. Errata is for post-hoc corrections only (typos, citation errors), not a dumping ground for "I built it differently" (see ADR-030). 6. Can this feature be improved in any way? If so, FILE A GITLAB ISSUE and link it. Plans are specifications, not backlogs — do NOT add ideas to a plan's "Potential Improvements" section (see ADR-030).Q1-Q3, Q5, Q7-Q8 are unchanged from the current craig pre-commit hook.
-
Canonical Status vocabulary (inlined verbatim — the ADR’s Decision §1):
-
Not started— default state for rows added to a new plan -
In progress— step is being actively worked in an open MR -
Done (YYYY-MM-DD) — …— step shipped; date + freeform detail required. Optional MR reference (MR !N) -
Deferred (…)— step explicitly descoped; reason required in parentheses -
Blocked (…)— step cannot proceed; blocker (external partner, spec pending, dependent plan) required -
N/A— row added to the table for structural reasons but doesn’t applyCase-insensitive first-token match; whitespace-preserved. Anything else is a lint violation.
-
-
.githooks/pre-commit— apply the Q4 / Q6 wording above. -
docs/modules/ROOT/nav.adoc— register ADR-030.
Branch: feat/canopy-backport-step7-plan-lifecycle-adr
MR title: feat(adr, .githooks): plan-lifecycle doctrine — canonical Status vocabulary + Q4/Q6 reframe [Step 7 of canopy-backports]
Verification:
-
ADR renders cleanly (
cargo xtask check-docsfor Tier 2 docs) -
.githooks/pre-commitshows the new Q4/Q6 wording when the token-gate fires -
No retroactive plan migration this MR; F-047’s lint lands separately + soft-fails first
Step 8: F-047 cargo xtask docs plan-lint
Design note: this lint operates on AsciiDoc Status tables, NOT Rust source. syn::visit::Visit (used by Plan D F-024 silent-skip lint and Plan I F-054 struct-method-count lint) does NOT apply here. AsciiDoc table parsing approach:
-
Option A (regex; simpler): match the
|===block delimiters; per-row|\s*(\d+)\s*\|\s*(.?)\s*\|\s*(.?)\s*$shape. Brittle if any row uses|literally inside a cell (escape with\|per AsciiDoc rules). -
Option B (
pulldown-cmark-style AsciiDoc parser; sturdier): use theasciidoctorRust binding (if available) or a small hand-written parser. Higher-effort but handles cell escapes correctly.
Pick Option A initially (regex) since plan bodies don’t currently use escaped pipes. Promote to a parser-backed implementation if false-positives appear.
Files:
-
xtask/src/cmd/docs.rs(NEW or EDIT if exists) — port canopy’s plan-lint implementation. Iteratesdocs/modules/ROOT/pages/plans/*.adoc(excludingarchive/); parses the Status table; validates each row’s Status cell begins with a canonical token from the F-046 ADR (case-insensitive first-token match). -
xtask/src/cmd/validate.rs— wire into validate withallow_failure: trueinitial behavior (print violations but exit 0). Promote to blocking via a follow-up MR after existing-drift cleanup.
Branch: feat/canopy-backport-step8-plan-lint
MR title: feat(xtask): cargo xtask docs plan-lint — Status vocabulary enforcement (soft-fail) [Step 8 of canopy-backports]
Verification:
-
cargo xtask docs plan-lintruns against the full plans tree -
Output cites every plan with a non-canonical Status cell (e.g. bare "Complete", "✓", lowercase "done")
-
cargo xtask validateruns the lint but does not block on violations -
Follow-up issue filed for "promote plan-lint to blocking" after the cleanup MR
Step 9: F-048 cargo xtask docs plan-archive
Files:
-
xtask/src/cmd/docs.rs— addAction::Archivesubcommand. Scans plans for all-Done/N/A Status tables; for each candidate, runsgit mv <plan>.adoc archive/<plan>.adoc+ prints the move. Operator-driven; not run in CI. -
docs/modules/ROOT/pages/plans/archive/— directory may already exist; verify before first move.
Branch: feat/canopy-backport-step9-plan-archive
MR title: feat(xtask): cargo xtask docs plan-archive — idempotent plan archival [Step 9 of canopy-backports]
Verification:
-
cargo xtask docs plan-archive --dry-runlists candidates without moving -
cargo xtask docs plan-archiveperforms the moves;git statusshows renames -
xref updates to archived plans tracked manually (the command does not rewrite xrefs)
Step 10: F-049 cargo xtask e2e --no-refresh flag
Files:
-
xtask/src/cmd/e2e.rs— add--no-refreshflag to theE2eArgsstruct (clap). When set,ensure_readyskips its staleness-marker check + does not wipe-and-reseed. -
xtask/src/devstack_guard.rs::ensure_ready— accept askip_refresh: boolparameter; gate the marker-check branch on it. -
.githooks/pre-push— aftercargo xtask dev reseed, runcargo xtask e2e --no-refresh(so the reseed isn’t wiped). Update comment to mirror canopy’s pre-push header explaining the flag’s purpose.
Branch: feat/canopy-backport-step10-e2e-no-refresh
MR title: feat(xtask, .githooks): cargo xtask e2e --no-refresh + wire into pre-push [Step 10 of canopy-backports]
Verification:
-
cargo xtask e2e --no-refreshagainst a freshly-reseeded devstack: Playwright sees the seed fixtures (no wipe) -
cargo xtask e2e(default behavior) still re-runsensure_ready -
Pre-push end-to-end:
git pushtriggers the hook → reseed runs → e2e runs against the reseeded data
Files Touched
| File | Step | Change |
|---|---|---|
|
2,3 |
EDIT (acquire_lock, --remove-orphans, reconcile_ports_env) |
|
2,4 |
EDIT (lock integration, refresh action) |
|
4,10 |
EDIT (staleness markers; skip_refresh param) |
|
4 |
EDIT (Refresh action) |
|
5 |
NEW |
|
6 |
NEW |
|
8,9 |
NEW or EDIT (plan-lint + plan-archive) |
|
10 |
EDIT (--no-refresh flag) |
|
8 |
EDIT (wire plan-lint as soft-fail) |
|
5,6,8,9 |
EDIT (subcommand registration) |
|
7 |
NEW |
|
7 |
EDIT (register ADR) |
|
7 |
EDIT (Q4 + Q6 wording) |
|
10 |
EDIT (wire --no-refresh) |
|
5 |
EDIT (install cargo-llvm-cov) |
|
6 |
EDIT (snapshots dir) |
|
4,6 |
EDIT (decision tables) |
|
5 |
EDIT (per-crate baselines) |
Risks
| Risk | Mitigation |
|---|---|
F-043 |
Decision table in |
F-044 cargo-llvm-cov adds a heavyweight dep on |
Bake the install into the CI image before adopting the lint; verify in a CI dry-run |
F-045’s |
Mitigation: docker-shell |
F-040 lock TTL: if xtask crashes mid-operation the lock stays held until manual rm |
Port canopy’s 60s polling + clear-error-message; lock file lives in |
F-042 reconciliation rewrites |
Drift-detected log line is loud; consider an |
After this plan lands
-
Concurrent xtask invocations no longer race (F-040)
-
"Port already allocated" failure mode eliminated (F-041 prevents stale containers from holding ports)
-
.ports.envdrift auto-detected + corrected (F-042) -
cargo xtask dev refreshis the daily-driver: rebuilds only what changed (F-043) -
Workspace coverage measured + (soft) gated via
cargo xtask coverage(F-044) -
DB snapshot/rollback in developer flow via
cargo xtask migrate(F-045) -
Plan-lifecycle doctrine codified — canonical Status vocabulary, archival rule, pre-commit Q4/Q6 reframed against "plans are living specs, not backlogs/errata dumps" (F-046)
-
cargo xtask docs plan-lintenforces the vocabulary (soft-fail initially; promoted to blocking after cleanup MR) (F-047) -
cargo xtask docs plan-archivemakes plan archival idempotent (F-048) -
cargo xtask e2e --no-refreshprevents seed-wipe during long local sessions; pre-push uses it (F-049)