Test Coverage Gap Analysis — Road to 100%
On this page
- Status
- Context
- Current Test Inventory
- Prerequisites (Bugs Discovered During Analysis)
- Coverage Gaps by Priority
- What We’re NOT Testing (And Why)
- Implementation Order
- Step 1: Save Plan Document
- Step 2: Test Infrastructure — cargo-nextest + JUnit XML
- Step 3: Fix Reporting Bug (Prerequisite for Step 5)
- Step 4: P0 — Security-Critical Tests (~41 new tests)
- Step 5: P1 — Business Logic Tests (~16 new tests)
- Step 6: P2 — Shared Crate Tests (~10 new tests)
- Step 7: P2.5 — BFF Contract Tests (~4 new tests)
- Step 8: P3 — Service Endpoint Tests + Dead Code Removal (~5 new tests)
- Step 9: P4 — CLI Command Tests (~31 new tests)
- Step 10: Verify
- Estimated New Tests: ~107
- Phase 2: Post-Hardening Coverage Gaps
- Phase 2 Status
- P2-1: Placement Education Page E2E (Partial — 2/3 done)
- P2-2: Placement Health Page E2E (Partial — 2/3 done)
- P2-3: Security Changes and Alerts Pages E2E (Complete)
- P2-4: Reporting Dashboard CRUD Workflow E2E (Partial — 6 tests done, 1 missing)
- P2-5: Placement Document Upload/Download E2E (Not started — 0 coverage)
- P2-6: AFCARS Export Download E2E (Not started — 0 coverage)
- P2-7: i18n Language Switching E2E (superseded 2026-08-17: #1332 removed /set-locale + pinned worker negotiation to en; the switcher/es legs below are #1487 scope)
- P2-8: OTEL Telemetry Unit Tests (Partial — 3/4 done)
- P2-9: i18n Module Unit Test Expansion — post-Arc<str> refactor (Partial)
- P2-10: Idempotency Middleware Improvements
- P2-11: State Machine Serde Roundtrip Tests
- Phase 2 Estimated New Tests (refreshed 2026-04-21)
- Phase 2 Implementation Order
|
Phase 2 refreshed 2026-04-21 (MR !123). Each sub-item audited against current |
Status
PHASE 1 COMPLETE — PHASE 2 COMPLETE (2026-04-21)
|
Phase 1 Status: COMPLETE (March 2026) Implemented in Phase 2 Status: NOT STARTED (March 2026) Significant new functionality added after Phase 1 completion: placement education/health pages, security changes/alerts pages, reporting dashboard CRUD, document upload/download, AFCARS export, i18n, OTEL telemetry, and idempotency middleware. See Phase 2: Post-Hardening Coverage Gaps below. |
Context
CRAIG has 653+ tests across unit, integration, CLI, and E2E tiers.
Overall functional coverage is strong, but several crates and services have significant gaps — ranging from zero unit tests (craig-db, craig-api) to untested security-critical paths (JWT validation) to dead code masquerading as features (sibling placements).
This plan catalogs every gap and provides a prioritized implementation path to 100% coverage.
Current Test Inventory
| Tier | Count | Scope |
|---|---|---|
Unit tests (inline |
~200 |
Shared crates (104), validation, transitions, seed, SDK |
Integration tests ( |
~350 |
Service APIs (8 services), CLI commands (13 modules) |
E2E tests (Playwright) |
100 |
Web UI flows (17 spec files) |
Total |
~653 |
| Test count grew from ~497 to ~653 with the addition of craig-intake (60 tests), craig-intake-sdk (9 tests), 3 new E2E spec files (+17 tests), and miscellaneous additions across existing services. |
Prerequisites (Bugs Discovered During Analysis)
AFCARS/NCANDS lifecycle is broken — missing /review endpoint
The state machine defines validated → reviewed → approved → transmitted.
generate_afcars/generate_ncands creates submissions directly in validated status.
The approve_afcars handler validates reviewed → approved and rejects validated → approved (returns 400).
But there is no /review endpoint — no way to transition validated → reviewed.
Result: no submission can ever be approved or transmitted through the API.
Fix (prerequisite for Step 3): Add review_afcars (PUT /reporting/afcars/{id}/review) and review_ncands (PUT /reporting/ncands/{id}/review) endpoints in services/craig-reporting/src/api.rs.
These should validate the validated → reviewed transition, set reviewed_by/reviewed_at, and require admin or supervisor role.
The store function update_submission_status already accepts reviewed_by/reviewed_at parameters — just needs an API handler.
Coverage Gaps by Priority
P0: Security-Critical (must fix first)
1. JwksProvider::validate_token() — zero coverage
File: crates/craig-auth/src/jwks.rs
The JwksProvider struct has 5 methods, 0 tests:
#[derive(Clone)]
pub struct JwksProvider {
issuer: String,
fetch_url: String,
client: reqwest::Client,
keys: Arc<RwLock<Option<JwkSet>>>, // None until first refresh()
}
validate_token has 6 error branches in order:
-
"JWKS not loaded"—keysis stillNone -
decode_headererror — malformed JWT structure -
"token missing kid"— nokidin JWT header -
"no matching key for kid"—kidnot in cachedJwkSet -
DecodingKey::from_jwkerror — bad key material -
decode::<Claims>error — expired, wrong issuer, bad signature
Algorithm is hardcoded RS256. Issuer validation + expiry are both enabled.
Implementation approach: Add #[cfg(test)] mod tests to jwks.rs. Use jsonwebtoken (already a dependency, v10 with rust_crypto feature) + rsa crate (add as [dev-dependencies]) to:
-
Generate an RSA keypair at test time
-
Build a
JwkSetfrom the public key -
Construct a
JwksProviderviawith_fetch_url, then inject keys directly via*provider.keys.write().await = Some(jwks) -
Sign test JWTs with the private key using
jsonwebtoken::encode
// Test setup pattern:
use rsa::RsaPrivateKey;
use rsa::pkcs1::EncodeRsaPublicKey;
use jsonwebtoken::{encode, EncodingKey, Header, Algorithm};
use jsonwebtoken::jwk::{Jwk, JwkSet, CommonParameters, RsaKeyParameters, KeyAlgorithm};
async fn test_provider(issuer: &str) -> (JwksProvider, EncodingKey) {
let mut rng = rand::rng();
let private_key = RsaPrivateKey::new(&mut rng, 2048).unwrap();
let public_key = private_key.to_public_key();
// Build JWK from public key DER, set kid = "test-kid-1"
// Build JwkSet with that single JWK
let provider = JwksProvider::with_fetch_url(issuer, "http://unused");
*provider.keys.write().await = Some(jwk_set);
let encoding_key = EncodingKey::from_rsa_der(&private_key.to_pkcs1_der().unwrap().as_bytes());
(provider, encoding_key)
}
6 tests:
-
validate_valid_token— sign a JWT with correct issuer, valid exp, correct kid →Ok(Claims) -
validate_expired_token— setexpto past timestamp → Err containing "ExpiredSignature" -
validate_wrong_issuer— sign withiss: "wrong-issuer"→ Err containing "InvalidIssuer" -
validate_missing_kid— sign a JWT with nokidin header → Err containing "token missing kid" -
validate_unknown_kid— sign withkid: "nonexistent"→ Err containing "no matching key for kid" -
validate_jwks_not_loaded— create provider without injecting keys → Err containing "JWKS not loaded"
2. auth_middleware — untested at unit level
File: crates/craig-auth/src/middleware.rs
Current implementation:
pub async fn auth_middleware(
State(layer): State<AuthLayer>,
mut request: Request,
next: Next,
) -> Response {
let token = request.headers().get("authorization")
.and_then(|v| v.to_str().ok())
.and_then(|v| v.strip_prefix("Bearer "));
let Some(token) = token else {
return (StatusCode::UNAUTHORIZED, "missing or invalid authorization header").into_response();
};
match layer.provider.validate_token(token).await {
Ok(claims) => { request.extensions_mut().insert(claims); next.run(request).await }
Err(e) => { (StatusCode::UNAUTHORIZED, "invalid token").into_response() }
}
}
require_role() already has 2 unit tests (added during intake work). The auth_middleware handler itself remains untested.
|
3 tests (add to existing middleware.rs test module):
-
middleware_rejects_missing_auth_header— build aRequestwith no Authorization header, callauth_middleware→ assert 401 with body "missing or invalid authorization header" -
middleware_rejects_invalid_token— build aRequestwithAuthorization: Bearer garbage, create a realJwksProviderwith loaded keys, call middleware → assert 401 with body "invalid token" -
middleware_passes_valid_token— use the test helper from P0.1 to sign a valid JWT, build request with it, pass a simplenexthandler that returns 200 → assert 200 and thatClaimsextension is present
3. parse_event_type — 29 match arms + fallbacks, zero tests
File: services/craig-security/src/main.rs
Currently a private fn parse_event_type(event_type: &str) → (String, String) with 23 explicit match arms + 2 fallback branches:
| Event Type | Action | Resource Type |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The 6 intake.* events are missing from parse_event_type — they currently fall through to the generic fallback ("report_submitted", "intake"). Add explicit match arms for proper action/resource_type classification.
|
Implementation:
-
Add 6
intake.*match arms toparse_event_typeinmain.rs(alongside the code extraction below) -
Extract
parse_event_typeinto a new fileservices/craig-security/src/events.rs(or make itpub(crate)inmain.rs). NOTE:events.rsalready exists with publish functions — either rename it topublish.rsand useevents.rsfor parsing, or putparse_event_typein a newaudit.rsmodule -
Add
#[cfg(test)] mod testswith a test for every row in the table above (32 tests) -
Use
[test_case]or individual[test]functions (no async needed — pure function)
P1: Business Logic Gaps
4. Wildcard # event subscriber (craig-security)
File: services/craig-security/src/main.rs
The subscriber uses subscribe() (competing consumers), queue "craig-security.events", routing key ["#"].
handle_inbound_event extracts:
-
user_idfrom payload:created_by→approved_by→user_id→archived_by→"system" -
resource_idfrom payload:id→case_id→payment_id→placement_id→review_id→archive_id
Then calls store::audit::insert_audit_entry(…).
Integration test (add to services/craig-security/tests/api/audit.rs):
#[tokio::test]
async fn audit_trail_captures_inbound_events() {
// 1. Use harness.admin_cases_client() to create a case (triggers "case.created" event)
// 2. Sleep 2 seconds for the event subscriber to process
// 3. Use harness.admin_security_client() to query GET /v1/security/audit?action=create&resource_type=case
// 4. Assert at least one audit entry exists with action="create", resource_type="case"
// 5. Verify the resource_id matches the created case ID
}
| This test is inherently timing-dependent (event processing is async). Use a retry loop with 5s timeout instead of a fixed sleep. |
5. FFP allocation logic (craig-financial)
Files:
-
services/craig-financial/src/main.rs— event handlers -
services/craig-financial/src/store/payments.rs—create_payment,update_payment_ive,update_payment_status -
services/craig-financial/src/store/claims.rs—aggregate_issued_payments
Subscriber: queue "craig-financial.events", routing keys ["placement.created", "placement.ended", "eligibility.evaluated", "rules.evaluated"].
5 integration tests (add to services/craig-financial/tests/api/payments.rs):
// Test 1: payment_lifecycle_approve_issue_clear
// - Create a rate (POST /v1/financial/rates)
// - Create a case + placement via cases/placement services (or use seed data)
// - Calculate payment (POST /v1/financial/payments/calculate) → get payment ID
// - Approve (PUT /v1/financial/payments/{id}/approve) → assert 200, status="approved"
// - NOTE: issue and clear transitions may need direct store calls if no API endpoint exists
// Check if there's an issue_payment endpoint; if not, just test approve.
// Test 2: eligibility_evaluated_updates_ffp
// - Create a rate + calculate a payment (status="pending")
// - Publish an EventEnvelope with event_type="eligibility.evaluated"
// payload: { case_id, child_id, ive_eligible: true, ffp_rate: 0.75 }
// - Retry-poll GET /v1/financial/payments/{id} until ffp_rate/ffp_amount are updated (5s timeout)
// - Assert ive_eligible=true, ffp_rate=0.75, ffp_amount = net_amount * 0.75
// Test 3: placement_created_auto_generates_payment
// - Create a rate for jurisdiction/payment_type/age
// - Publish EventEnvelope with event_type="placement.created"
// payload: { case_id, child_id, placement_id, foster_home_id, placement_type, child_age }
// - Retry-poll GET /v1/financial/payments?case_id=... until a payment appears
// - Assert payment exists with correct placement_id, daily_rate, gross_amount
// Test 4: placement_ended_voids_pending_payments
// - Create a payment in pending status (via calculate)
// - Publish EventEnvelope with event_type="placement.ended"
// payload: { placement_id }
// - Retry-poll GET /v1/financial/payments/{id} until status="voided" (5s timeout)
// Test 5: claim_aggregation_with_issued_payments
// - Create a rate, calculate a payment, approve it
// - Advance to "issued" status (if API endpoint exists, or via direct store call)
// - Generate a claim (POST /v1/financial/claims/generate) for that period
// - Assert total_expenditure > 0, ive_eligible_amount reflects the payment
To publish events in tests, use craig_mq::connect() + Publisher::new() with the devstack RabbitMQ URL from TestConfig (#1202: the broad craig-test AMQP identity, e.g. amqp://craig-test:craig-test@localhost:5672) — not a per-service or operator account.
6. Reporting valid transitions — never succeed at API level
Prerequisite: Add review_afcars/review_ncands endpoints (see Prerequisites section above).
The store function update_submission_status already supports all the fields:
pub async fn update_submission_status(
db: &sqlx::PgPool, id: Uuid, status: &str,
reviewed_by: Option<&str>, reviewed_at: Option<DateTime<Utc>>,
approved_by: Option<&str>, approved_at: Option<DateTime<Utc>>,
transmitted_at: Option<DateTime<Utc>>,
) -> anyhow::Result<Option<AfcarsSubmission>>
4 integration tests (add to services/craig-reporting/tests/api/afcars.rs and ncands.rs):
// Test 1: afcars_full_lifecycle
// - Generate (POST /v1/reporting/afcars/generate) → status="validated"
// - Review (PUT /v1/reporting/afcars/{id}/review) → status="reviewed"
// - Approve (PUT /v1/reporting/afcars/{id}/approve) → status="approved"
// - Transmit (POST /v1/reporting/afcars/{id}/transmit) → status="transmitted"
// - Assert each step returns 200 and status field advances
// Test 2: ncands_full_lifecycle — same pattern for NCANDS
// Test 3: afcars_approve_from_validated_fails
// - Generate → status="validated"
// - Approve directly (skipping review) → assert 400
// Test 4: afcars_transmit_from_reviewed_fails
// - Generate, review → status="reviewed"
// - Transmit directly (skipping approve) → assert 400
Roles: generate = admin, review = admin/supervisor, approve = admin, transmit = admin.
7. Financial happy paths — only error cases tested
3 integration tests (add to existing files in services/craig-financial/tests/api/):
// payments.rs: approve_payment_success
// - Create rate, calculate payment, then approve → assert 200, status="approved", approved_by is set
// adjustments.rs: adjustment_lifecycle
// - Create rate, calculate payment
// - Create adjustment (POST /v1/financial/payments/{payment_id}/adjustments) → assert 200
// - Approve adjustment (PUT /v1/financial/adjustments/{id}/approve) → assert 200
// - Get payment → verify net_amount = gross_amount + adjustment
// claims.rs: submit_and_verify_claim
// - Generate claim → assert status="draft"
// - Submit claim (PUT /v1/financial/claims/{id}/submit) → assert status="submitted"
Test harness pattern (from existing tests):
let harness = TestHarness::new().await.unwrap();
let admin = harness.admin_financial_client().await.unwrap();
let supervisor = harness.supervisor_financial_client().await.unwrap();
// admin creates, supervisor approves (role-appropriate)
8. MQ subscriber retry/DLQ logic
File: crates/craig-mq/src/subscriber.rs
Three branches to test:
-
Handler success →
ack -
Handler failure + first delivery (
redelivered=false) →nack(requeue: true) -
Handler failure + redelivered (
redelivered=true) →nack(requeue: false)(DLQ) -
Deserialization failure →
nack(requeue: false)immediately
Assessment: Testing these requires a live RabbitMQ + publishing raw bytes (for deserialization failure) + a handler that intentionally fails. This is feasible but complex. Defer to after all other priorities — the logic is straightforward and unlikely to regress.
9. CaptchaVerifier — zero tests (craig-intake)
File: services/craig-intake/src/api/captcha.rs
CaptchaVerifier has a verify() method with 5 code paths:
-
Disabled mode (
secret == "disabled") → returnsOk(()) -
Token missing (
None) → returnsErr("captcha_token is required") -
Token empty (
"") → returnsErr("captcha_token cannot be empty") -
HTTP verification failure → returns
Err("CAPTCHA verification request failed: …") -
Verification response
success: false→ returnsErr("CAPTCHA verification failed")
3 unit tests (add #[cfg(test)] mod tests to captcha.rs):
#[tokio::test]
async fn disabled_captcha_accepts_any_token() {
let verifier = CaptchaVerifier::new("disabled".into(), "http://unused".into(), reqwest::Client::new());
assert!(verifier.verify(None).await.is_ok());
assert!(verifier.verify(Some("any-token")).await.is_ok());
}
#[tokio::test]
async fn enabled_captcha_rejects_missing_token() {
let verifier = CaptchaVerifier::new("real-secret".into(), "http://localhost:1".into(), reqwest::Client::new());
let result = verifier.verify(None).await;
assert!(result.is_err());
assert!(result.unwrap_err().contains("required"));
}
#[tokio::test]
async fn enabled_captcha_rejects_empty_token() {
let verifier = CaptchaVerifier::new("real-secret".into(), "http://localhost:1".into(), reqwest::Client::new());
let result = verifier.verify(Some("")).await;
assert!(result.is_err());
assert!(result.unwrap_err().contains("empty"));
}
Testing paths 4-5 (HTTP success/failure) requires a mock HTTP server (e.g. wiremock crate). Defer unless we add wiremock as a dev dependency. The 3 tests above cover the critical disabled-mode bypass and input validation.
|
10. API key expiry check — untested (craig-intake)
File: services/craig-intake/src/api/api_key_auth.rs (lines 57-61)
The middleware checks expires_at < chrono::Utc::now() and returns 401 "API key has expired". This code path has zero test coverage.
1 integration test (add to services/craig-intake/tests/api/api_keys.rs or partner.rs):
// expired_api_key_returns_401
// - Create an API key (POST /v1/intake/api-keys)
// - Directly UPDATE the key's expires_at to a past timestamp via SQL:
// UPDATE intake_api_keys SET expires_at = NOW() - INTERVAL '1 day' WHERE id = $1
// - Use the key to submit a report (POST /partner/v1/reports)
// - Assert 401 response with body containing "expired"
Requires direct SQL access via TestHarness to set past expiry (no API endpoint to set expiry). Use sqlx::query!("UPDATE …") against the intake database pool.
|
P2: Shared Crate Gaps
11. craig-db — zero tests
File: crates/craig-db/src/lib.rs
impl Default for DbPoolOpts {
fn default() -> Self {
Self {
max_connections: 10,
idle_timeout: Duration::from_secs(600),
}
}
}
1 unit test (add #[cfg(test)] mod tests to lib.rs):
#[test]
fn default_pool_opts() {
let opts = DbPoolOpts::default();
assert_eq!(opts.max_connections, 10);
assert_eq!(opts.idle_timeout, Duration::from_secs(600));
}
connect(), health_check(), run_migrations() are already exercised transitively by every integration test that creates a TestHarness. No additional integration tests needed.
12. craig-api — zero tests
File: crates/craig-api/src/lib.rs
impl Default for ServerOptions {
fn default() -> Self {
Self {
cors_origins: "*".into(),
body_limit: 2 * 1024 * 1024, // 2 MiB
}
}
}
build_cors is private — cannot be tested directly. Two options:
-
Change to
pub(crate)to enable unit tests (preferred) -
Test indirectly through
ApiServer::router()(heavier)
3 unit tests (add #[cfg(test)] mod tests to lib.rs, change build_cors to pub(crate)):
#[test]
fn default_server_options() {
let opts = ServerOptions::default();
assert_eq!(opts.cors_origins, "*");
assert_eq!(opts.body_limit, 2 * 1024 * 1024);
}
#[test]
fn build_cors_wildcard() {
let layer = build_cors("*");
// CorsLayer doesn't expose internals for assertion,
// but we can verify it doesn't panic on "*"
}
#[test]
fn build_cors_specific_origins() {
let layer = build_cors("http://a.com, http://b.com");
// Verify no panic; specific origin parsing works
}
/healthz endpoint is already tested transitively by every devstack health check.
13. craig-mq Publisher — zero direct tests
File: crates/craig-mq/src/publisher.rs
Publisher is exercised transitively by every service integration test that triggers events.
No additional direct tests needed — the value of testing Publisher::publish() in isolation is low when every service already publishes events during integration tests.
14. craig-store error conversion — 6 arms untested
File: crates/craig-store/src/error.rs
// All 6 conversion arms:
StoreError::NotFound(msg) → ApiError::NotFound(msg)
StoreError::TooLarge { size, limit } → ApiError::BadRequest("upload too large: {size} bytes exceeds limit of {limit} bytes")
StoreError::DisallowedContentType(ct) → ApiError::BadRequest("disallowed content type: {ct}")
StoreError::InvalidFilename(msg) → ApiError::BadRequest("invalid filename: {msg}")
StoreError::ObjectStore(e) → ApiError::Internal("object store error")
StoreError::Config(msg) → ApiError::Internal("store config error: {msg}")
6 unit tests (add #[cfg(test)] mod tests to error.rs):
use super::*;
#[test]
fn not_found_converts_to_api_not_found() {
let err: ApiError = StoreError::NotFound("gone".into()).into();
assert!(matches!(err, ApiError::NotFound(msg) if msg == "gone"));
}
#[test]
fn too_large_converts_to_bad_request() {
let err: ApiError = StoreError::TooLarge { size: 5000, limit: 1000 }.into();
assert!(matches!(err, ApiError::BadRequest(msg) if msg.contains("5000") && msg.contains("1000")));
}
#[test]
fn disallowed_content_type_converts_to_bad_request() {
let err: ApiError = StoreError::DisallowedContentType("text/html".into()).into();
assert!(matches!(err, ApiError::BadRequest(msg) if msg.contains("text/html")));
}
#[test]
fn invalid_filename_converts_to_bad_request() {
let err: ApiError = StoreError::InvalidFilename("../etc/passwd".into()).into();
assert!(matches!(err, ApiError::BadRequest(msg) if msg.contains("../etc/passwd")));
}
#[test]
fn object_store_error_converts_to_internal() {
let err: ApiError = StoreError::ObjectStore(
object_store::Error::NotFound { path: "x".into(), source: "".into().into() }
).into();
assert!(matches!(err, ApiError::Internal(_)));
}
#[test]
fn config_error_converts_to_internal() {
let err: ApiError = StoreError::Config("bad config".into()).into();
assert!(matches!(err, ApiError::Internal(msg) if msg.contains("bad config")));
}
ApiError variant matching depends on the actual enum definition — adjust the matches! patterns to match the real ApiError variants (may use status() method instead of pattern matching).
|
P2.5: craig-web BFF Contract Tests
craig-web has 66 route handlers with zero automated tests (see P5 below). Most are thin proxies where E2E coverage is sufficient. However, handlers that map form fields to cross-service API calls introduce a translation layer where field name and value mismatches hide — invisible to integration tests (which call APIs directly) and opaque to E2E tests (which only see Playwright timeouts when the server returns 500).
This was proven during Phase 12 development, where two BFF mapping bugs survived all integration tests and produced hours of fruitless Playwright debugging:
-
screen_out_public_report()sent{"screen_out_reason": …}but craig-intake expects{"reason": …}→ 422 -
Report detail template used priority values
standard/high/emergencybut craig-cases validatesimmediate/24_hour/72_hour→ 500
Both bugs were trivially diagnosed once curl was used against the API, but E2E tests gave zero signal about the root cause.
Scope
Test only BFF handlers that perform cross-service field mapping — not every route. Currently this means the report review actions in services/craig-web/src/routes/intake.rs:
-
claim_public_report— POST form → PUT JSON to craig-intake/v1/intake/reports/{id}/claim -
convert_public_report— POST form (priority) → PUT JSON to craig-intake/v1/intake/reports/{id}/convert -
screen_out_public_report— POST form (screen_out_reason) → PUT JSON to craig-intake/v1/intake/reports/{id}/screen-out -
submit_public_report_form— POST form (full report) → POST JSON to craig-intake/public/v1/reports
Future cross-service handlers (e.g., if craig-web gains direct calls to craig-cases, craig-financial, etc.) should be added to this list.
Implementation approach
These are HTTP-level integration tests — not unit tests, not browser tests. They POST form-encoded data to craig-web and assert the downstream API call succeeds (200/302) rather than fails (400/422/500).
Test file: services/craig-web/tests/bff_contracts.rs (or tests/bff/intake.rs if multiple modules emerge)
Prerequisites: Running devstack (craig-web + craig-intake + craig-cases + Keycloak).
Pattern:
use craig_test_lib::{TestHarness, devstack_available};
use reqwest::Client;
/// Obtain a session cookie by logging in via craig-web's OIDC flow.
/// Returns a cookie jar with a valid session.
async fn web_session(harness: &TestHarness) -> reqwest::Client {
// 1. GET /intake/reports to trigger OIDC redirect
// 2. Follow redirect to Keycloak, POST credentials (jane.doe/password)
// 3. Follow callback redirect back to craig-web
// 4. Client now has session cookie
// Alternative: reuse the auth/setup.ts pattern from E2E —
// POST directly to Keycloak token endpoint, then set session via craig-web
todo!()
}
#[tokio::test]
async fn convert_sends_valid_priority_to_intake_api() {
if !devstack_available().await { return; }
let harness = TestHarness::new().await.unwrap();
let client = web_session(&harness).await;
// 1. Submit a report via craig-intake public API
let intake = harness.intake_client();
let report = intake.submit_public_report(&json!({...})).await.unwrap();
let id = report["id"].as_str().unwrap();
// 2. Claim via craig-web form POST (mirrors what the browser does)
let resp = client.post(format!("http://localhost:8080/intake/reports/{id}/claim"))
.send().await.unwrap();
assert!(resp.status().is_redirection() || resp.status().is_success());
// 3. Convert via craig-web form POST with form-encoded priority
let resp = client.post(format!("http://localhost:8080/intake/reports/{id}/convert"))
.form(&[("priority", "24_hour")])
.send().await.unwrap();
// Should redirect to /intake/referrals/{referral_id}, NOT back to report with error flash
let location = resp.headers().get("location").unwrap().to_str().unwrap();
assert!(location.contains("/intake/referrals/"), "convert should redirect to referral, got: {location}");
}
4 tests
-
claim_sends_valid_request_to_intake_api— POST to/intake/reports/{id}/claim→ assert redirect back to report detail (not error flash) -
convert_sends_valid_priority_to_intake_api— POST withpriority=24_hour→ assert redirect to/intake/referrals/ -
screen_out_sends_valid_field_names_to_intake_api— POST withscreen_out_reason=…→ assert redirect back to report detail withscreened_outstatus -
submit_report_form_sends_valid_fields_to_intake_api— POST full form data to/report→ assert redirect to/report/confirmation/RPT-*
Session authentication challenge
The main complexity is obtaining a valid session cookie for craig-web (which uses OIDC via Keycloak, not Bearer tokens). Options:
-
Follow the full OIDC flow in reqwest (redirect to Keycloak → POST credentials → follow callback) — most realistic but fragile
-
Add a test-only backdoor (
/test/login?username=jane.doebehind a feature flag) — simpler but requires code changes -
Reuse Playwright’s
storageStateJSON — parse the session cookie fromauth/caseworker.jsonafter E2E setup runs
Option 1 is preferred for correctness. If it proves too brittle, fall back to option 3.
P3: Service Endpoint Gaps
15. update_nist_control endpoint (craig-security)
Route: PUT /v1/security/nist/{control_id} (path param is a String, e.g. "AC-2")
Request body:
UpdateNistControlRequest {
implementation_status: String, // required — "partial", "implemented", "not_applicable"
implementation_notes: Option<String>,
evidence_key: Option<String>,
last_assessed: Option<NaiveDate>,
assessed_by: Option<String>,
}
Requires admin role. Validates transition via transitions::validate_nist_transition.
Existing NIST tests (in tests/api/nist.rs): only list_nist_controls, list_nist_with_family_filter, caseworker_cannot_list_nist_controls.
3 integration tests (add to nist.rs):
// update_nist_control_planned_to_partial
// - List NIST controls, find one with status="planned"
// - PUT /v1/security/nist/{control_id} with implementation_status="partial"
// - Assert 200, verify status changed
// update_nist_control_partial_to_implemented
// - Continue from above control (now "partial")
// - PUT with implementation_status="implemented" and evidence_key="some-key"
// - Assert 200
// update_nist_control_invalid_transition
// - Find a "planned" control
// - PUT with implementation_status="implemented" (skipping "partial" — depends on state machine)
// - NOTE: Check if planned→implemented IS valid (it is per the state machine!)
// - Instead test: take a "partial" control, try to go back to "planned" → assert 400
16. audit_by_resource endpoint (craig-security)
Route: GET /v1/security/audit/resource/{resource_type}/{resource_id}
Requires supervisor or admin role. Returns { "data": […] }.
1 integration test (add to audit.rs):
// audit_by_resource_returns_entries
// - Create a case (via cases client) — this generates audit entries via events
// - Wait for event processing (retry loop)
// - GET /v1/security/audit/resource/case/{case_id}
// - Assert data array contains at least one entry
17. scheduled→overdue review transition (craig-security)
1 integration test (add to reviews.rs):
// update_review_to_overdue
// - Create a review with status="scheduled" (POST /v1/security/reviews)
// - PUT /v1/security/reviews/{id} with status="overdue"
// - Assert 200
18. Dead code removal: sibling_placements (craig-placement)
Files to modify:
-
Delete
services/craig-placement/src/store/sibling_placements.rs -
Remove
pub mod sibling_placements;fromservices/craig-placement/src/store/mod.rs -
Remove
SiblingPlacementfrom OpenAPI schemas inservices/craig-placement/src/api.rs(if referenced in#[derive(OpenApi)]) -
Remove
SiblingPlacementmodel fromservices/craig-placement/src/store/models.rs(if defined there)
Verify with cargo check -p craig-placement — no compilation errors.
P4: CLI Command Gaps
21. Ten untested CLI command modules
Test pattern (from services/craig-cli/tests/cli/rules.rs):
use craig_cli::client::ApiClient;
use craig_cli::cmd::<module>::<CmdEnum>;
use craig_cli::output::Format;
use craig_test_lib::{TestHarness, devstack_available};
async fn admin_<service>_client(harness: &TestHarness) -> ApiClient {
let token = harness.tokens.get_token("admin", "password").await.unwrap();
ApiClient::new(&harness.config.<service>_url, &token)
}
#[tokio::test]
async fn test_name() {
if !devstack_available().await { return; }
let harness = TestHarness::new().await.unwrap();
let client = admin_<service>_client(&harness).await;
let cmd = <CmdEnum>::<Variant> { field: value, ... };
craig_cli::cmd::<module>::run(&cmd, Format::Json, &client).await.expect("should succeed");
}
Config URLs: harness.config.financial_url (8005), harness.config.reporting_url (8006), harness.config.security_url (8007), harness.config.placement_url (8003).
Per-module test specifications:
tests/cli/financial.rs (6 subcommands → 4 tests)
// list_payments — ListPayments { case_id: None, ..., page: 1, per_page: 10 } → assert Ok
// list_rates — ListRates { jurisdiction: None, payment_type: None, page: 1, per_page: 10 } → assert Ok
// create_rate — CreateRate { jurisdiction: "georgia", payment_type: "foster_care", age_min: 0, age_max: 5, daily_rate: "25.00", effective_date: "2026-01-01", end_date: None } → assert Ok
// calculate_payment — CalculatePayment { case_id: <from seed>, child_id: <from seed>, ... } → may 404 if no rate; just assert no panic
tests/cli/claim.rs (4 subcommands → 3 tests)
// list_claims — List { claiming_period: None, ..., page: 1, per_page: 10 } → assert Ok
// generate_claim — Generate { claiming_period: "2026-Q1", payment_type: "foster_care", period_start: "2026-01-01", period_end: "2026-03-31", ccwis_operations_cost: "1000.00" } → assert Ok
// get_claim_not_found — Get { id: random_uuid } → assert error (404)
tests/cli/adjustment.rs (2 subcommands → 2 tests)
// create_adjustment_not_found — Create { payment_id: random_uuid, reason: "correction", amount: "10.00" } → assert error (404)
// approve_adjustment_not_found — Approve { id: random_uuid } → assert error (404)
tests/cli/reporting.rs (12 subcommands → 6 tests)
// quality_dashboard — QualityDashboard → assert Ok
// list_issues — ListIssues { source_service: None, ..., page: 1, per_page: 10 } → assert Ok
// generate_afcars — GenerateAfcars { reporting_period: "2026-Q1" } → assert Ok
// list_afcars — ListAfcars { status: None, page: 1, per_page: 10 } → assert Ok
// generate_ncands — GenerateNcands { reporting_year: 2026 } → assert Ok
// list_ncands — ListNcands { status: None, page: 1, per_page: 10 } → assert Ok
tests/cli/security.rs (6 subcommands → 4 tests)
// audit_log — Audit { user_id: None, service: None, action: None, ..., page: 1, per_page: 10 } → assert Ok
// list_reviews — ListReviews { review_type: None, status: None, page: 1, per_page: 10 } → assert Ok
// create_review — CreateReview { review_type: "annual", scheduled_date: "2026-06-01", reviewer: "admin" } → assert Ok
// update_review — use ID from create_review, UpdateReview { id, status: "in_progress", completed_date: None } → assert Ok
tests/cli/archive.rs (3 subcommands → 3 tests)
// list_archives — List { source_service: None, purge_eligible: None, page: 1, per_page: 10 } → assert Ok
// run_archive — Run → assert Ok
// run_purge — Purge → assert Ok
tests/cli/nist.rs (2 subcommands → 2 tests)
// list_nist_controls — List { control_family: None, implementation_status: None, page: 1, per_page: 10 } → assert Ok
// update_nist_control — find a control from list, Update { control_id, status: "partial", notes: Some("test"), last_assessed: None, assessed_by: None } → assert Ok (if control exists in seed)
tests/cli/kinship.rs (2 subcommands → 2 tests)
// list_kinship — List { case_id: <from seed> } → assert Ok
// create_kinship — Create { case_id: <from seed>, child_id: <from seed>, relative_name: "Test Relative", relationship: "aunt", evaluated: true, approved: Some(true), rejection_reason: None, evaluated_at: None } → assert Ok
tests/cli/intake.rs (7 subcommands → 5 tests)
// list_reports — ListReports { status: None, admin_unit: None, page: 1, per_page: 10 } → assert Ok
// Config: harness.config.intake_url (port 8008)
// get_report — submit a report via intake SDK first, then GetReport { id: <report_id> } → assert Ok
// claim_and_screen_out — submit report, ClaimReport { id }, ScreenOutReport { id, reason: "insufficient evidence" } → assert Ok
// create_and_list_api_keys — CreateKey { name: "test", organization: "Test Org", contact_email: "test@example.com", rate_limit_rpm: 60 } → assert Ok, ListKeys → assert Ok
// revoke_api_key — create key, then RevokeKey { id: <key_id> } → assert Ok
ConvertReport is hard to test in isolation (requires seed data + running craig-cases for cross-service referral creation). The claim+screen_out test covers the simpler workflow. If convert is needed, use seed report data and accept potential 500 if cases service state doesn’t match.
|
Also register all new test modules in tests/cli/mod.rs.
P5: craig-web (BFF) — Mostly E2E-Covered
82 route handlers with zero unit/integration tests. Most are thin proxies where E2E coverage is sufficient — do NOT add unit tests for simple pass-through handlers. However, handlers that perform cross-service field mapping (form fields → JSON API bodies) are covered by P2.5 contract tests, since E2E tests give zero diagnostic signal when these mappings are wrong (they just timeout).
What We’re NOT Testing (And Why)
| Item | Reason |
|---|---|
|
Test utility crate — tested transitively |
|
Thin BFF proxies; E2E coverage sufficient for pass-through handlers. Cross-service mapping handlers covered by P2.5 contract tests. |
|
Requires live Postgres; exercised by every integration test’s |
|
Exercised transitively by every service integration test that triggers events |
|
Requires live Keycloak; exercised transitively by integration tests |
|
Interactive/utility; not automatable |
MQ subscriber retry/DLQ logic |
Deferred — requires complex RabbitMQ test harness for low-risk code |
CAPTCHA HTTP verification (paths 4-5) |
Requires mock HTTP server ( |
Rate limiting (429 response) |
Requires timing-sensitive test that exceeds governor quota. Governor is well-tested upstream; middleware is 5 lines. Low risk. |
|
Requires running devstack; SDK types are tested via builder tests (9 tests). Client is exercised transitively by E2E tests. |
Event publication verification |
All 6 intake events are published but not verified at RabbitMQ level. Verified transitively by craig-security audit trail integration test (P1.4). |
Implementation Order
Step 1: Save Plan Document
Save this plan as docs/modules/ROOT/pages/plans/test-coverage-100.adoc and add nav entry. (Done)
Step 2: Test Infrastructure — cargo-nextest + JUnit XML
Adopt cargo-nextest as the test runner and save structured JUnit XML results to a gitignored test-results/ directory.
Output directory
test-results/
unit.xml # cargo nextest --lib
integration.xml # cargo nextest (full workspace)
e2e.xml # playwright --reporter=junit
Add test-results/ to .gitignore.
Install cargo-nextest
Add to CI image and local dev:
-
CI (
.gitlab-ci.yml): addcargo install cargo-nextest --lockedto.rust-jobbefore_script(or use the pre-built binary:curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C /usr/local/bin) -
Dockerfile: add
RUN cargo install cargo-nextest --lockedin the builder stage (for integration tests run inside containers) -
Local dev:
cargo install cargo-nextest --locked
Nextest config
Create .config/nextest.toml in the repo root:
[store]
dir = "test-results"
[profile.default]
fail-fast = false
slow-timeout = { period = "60s", terminate-after = 2 }
[profile.ci]
fail-fast = false
junit.path = "unit.xml"
[profile.ci-integration]
fail-fast = false
junit.path = "integration.xml"
Update devstack scripts
Update cargo xtask commands:
-
Replace
cargo test --workspace --libwithcargo nextest run --workspace --lib --profile ci -
Replace
cargo test --workspacewithcargo nextest run --workspace --profile ci-integration -
Results automatically written to
test-results/unit.xmlandtest-results/integration.xml
Playwright JUnit reporter
Update tests/e2e/playwright.config.ts — add junit reporter:
reporter: [
['list'],
['junit', { outputFile: '/app/results/e2e.xml' }],
],
Update docker-compose.yml for the craig-e2e service — mount the results directory:
volumes:
- ./test-results:/app/results
This writes test-results/e2e.xml on the host after each E2E run.
Update .gitlab-ci.yml
Add JUnit artifacts to test jobs:
# rust-test job:
artifacts:
when: always
reports:
junit: test-results/unit.xml
# integration-test job:
artifacts:
when: always
reports:
junit: test-results/integration.xml
# e2e-test job:
artifacts:
when: always
reports:
junit: test-results/e2e.xml
GitLab auto-parses these and shows test results in MR UI.
Step 3: Fix Reporting Bug (Prerequisite for Step 5)
-
Add
review_afcarsendpoint (PUT /reporting/afcars/{id}/review) toservices/craig-reporting/src/api.rs -
Add
review_ncandsendpoint (PUT /reporting/ncands/{id}/review) toservices/craig-reporting/src/api.rs -
Wire into router
-
Validate
validated → reviewedtransition -
Set
reviewed_by/reviewed_atfields
Step 4: P0 — Security-Critical Tests (~41 new tests)
-
Add
rsaas[dev-dependencies]incrates/craig-auth/Cargo.toml -
JwksProvider::validate_token()unit tests injwks.rs(6 tests) -
auth_middlewareunit tests inmiddleware.rs(3 tests —require_rolealready has 2 tests) -
Add 6
intake.*match arms toparse_event_typeinservices/craig-security/src/main.rs -
Extract
parse_event_typeto a testable module (e.g.audit.rs), add tests (32 tests — one per match arm including new intake arms + 2 fallback tests)
Step 5: P1 — Business Logic Tests (~16 new tests)
-
Security wildcard subscriber integration test (1 test in
audit.rs) -
Financial: approve payment happy path (1 test)
-
Financial: adjustment lifecycle (1 test)
-
Financial: claim submit (1 test)
-
Financial: eligibility.evaluated event updates FFP (1 test)
-
Financial: placement.created auto-generates payment (1 test)
-
Financial: placement.ended voids payments (1 test)
-
Financial: claim aggregation with issued payments (1 test)
-
Reporting: AFCARS full lifecycle (1 test)
-
Reporting: NCANDS full lifecycle (1 test)
-
Reporting: invalid transition assertions (2 tests)
-
Intake: CaptchaVerifier unit tests — disabled mode, missing token, empty token (3 tests in
captcha.rs) -
Intake: API key expiry integration test (1 test — requires direct SQL to set past expiry)
Step 6: P2 — Shared Crate Tests (~10 new tests)
-
craig-db:DbPoolOpts::default()(1 test) -
craig-api:ServerOptions::default()+build_cors(3 tests, makebuild_corspub(crate)) -
craig-store:From<StoreError> for ApiError(6 tests)
Step 7: P2.5 — BFF Contract Tests (~4 new tests)
-
Create
services/craig-web/tests/bff_contracts.rs -
Implement OIDC session authentication helper (reqwest follows Keycloak redirect flow)
-
claim_sends_valid_request_to_intake_api(1 test) -
convert_sends_valid_priority_to_intake_api(1 test) -
screen_out_sends_valid_field_names_to_intake_api(1 test) -
submit_report_form_sends_valid_fields_to_intake_api(1 test)
Step 8: P3 — Service Endpoint Tests + Dead Code Removal (~5 new tests)
-
update_nist_control(3 tests) -
audit_by_resource(1 test) -
scheduled→overduereview (1 test) -
Remove
sibling_placements.rsdead code (craig-placement) -
Remove
create_metric()dead code (craig-reporting) -
Remove
publish_breach_detected()dead code (craig-security)
Step 9: P4 — CLI Command Tests (~31 new tests)
-
financial.rs(4 tests) -
claim.rs(3 tests) -
adjustment.rs(2 tests) -
reporting.rs(6 tests) -
security.rs(4 tests) -
archive.rs(3 tests) -
nist.rs(2 tests) -
kinship.rs(2 tests) -
intake.rs(5 tests — list, get, claim+screen_out, create+list keys, revoke key) -
Register all new modules in
tests/cli/mod.rs
Step 10: Verify
-
cargo nextest run --workspace --lib --profile ci— all unit tests pass, results intest-results/unit.xml -
cargo xtask dev restart— fresh data (schema may have changed for review endpoints) -
cargo nextest run --workspace --profile ci-integration— all integration tests pass, results intest-results/integration.xml -
cargo xtask e2e— all 100+ E2E tests pass, results intest-results/e2e.xml -
Run full suite 3x to confirm no flakiness
-
Verify all three JUnit XML files are written and parseable
Estimated New Tests: ~107
| Priority | New Tests |
|---|---|
P0 Security |
~41 (6 + 3 + 32) |
P1 Business Logic |
~16 (12 original + 3 captcha + 1 key expiry) |
P2 Shared Crates |
~10 |
P2.5 BFF Contracts |
~4 |
P3 Service Endpoints |
~5 |
P4 CLI Commands |
~31 (26 original + 5 intake) |
Total |
~107 |
Plus the prerequisite reporting fix (2 new endpoints, no new tests counted separately — tested in P1) and 3 dead code removals (no tests needed).
This would bring the total from ~653 to ~760 Rust tests + 100 E2E tests = ~860 total tests with comprehensive coverage across all tiers.
Phase 2: Post-Hardening Coverage Gaps
Phase 1 brought the test count from ~653 to ~966. Since then, significant new functionality has been added that introduces new coverage gaps. This section catalogs them with specific test files and assertions.
Phase 2 Status
Status column reflects audit at 2026-04-21. Numbering aligns with the narrative sections below (P2-1 … P2-11).
| Step | Description | Status |
|---|---|---|
P2-1 |
E2E: Placement education page |
Complete (MR !128) — |
P2-2 |
E2E: Placement health page |
Complete (MR !128) — same file now has 4 tests: load/headers, data-when-seeded, pagination via |
P2-3 |
E2E: Security changes + alerts pages |
Complete (MR !123) — |
P2-4 |
E2E: Reporting dashboard CRUD workflow |
Complete (MR !128) — |
P2-5 |
E2E: Placement document upload/download |
Complete (MR !128) — new |
P2-6 |
E2E: AFCARS export download |
Complete (MR !128, closes #200) — added |
P2-7 |
E2E: i18n language switching |
Complete (MR !128) — |
P2-8 |
Unit: OTEL telemetry module |
Complete (MR !125) — |
P2-9 |
Unit: i18n module expansion (post-Arc<str> refactor) |
Complete (MR !125) — |
P2-10 |
Unit: Idempotency middleware function |
Complete (MR !125) — |
P2-11 |
Unit: State-machine serde roundtrip tests |
Complete (MR !127) — 32 new tests (2 per enum × 16 enums across 7 transitions.rs files). Each enum gets a |
Branch: feature/test-coverage-phase-2
P2-1: Placement Education Page E2E (Partial — 2/3 done)
Route: /placement/education
File: extend tests/e2e/specs/placement-records.spec.ts (existing — already has 2 education tests for load+headers and data-when-seeded)
Already covered: column headers (/school/i, /grade/i, /iep/i, /504/i), data rendering when seeded.
Missing (add 2 tests):
test('education table supports column sort', async ({ page }) => {
await page.goto('/placement/education');
// Click a sortable column header, assert ?sort_by=... in URL and
// that the sort indicator direction flipped.
});
test('education page shows empty state for child with no records', async ({ page }) => {
// Either pick a seeded child_id known to have zero education rows, or
// create a fresh child + navigate to ?child_id=<id>. Assert the
// "No education records" empty-state message.
});
P2-2: Placement Health Page E2E (Partial — 2/3 done)
Route: /placement/health
File: extend tests/e2e/specs/placement-records.spec.ts (existing — already has 2 health tests for load+headers and data-when-seeded)
Already covered: column headers (/type/i, /provider/i, /date/i), data rendering with /physical|immunization|dental|vision/i.
Missing (add 2 tests):
test('health table supports pagination when records exceed per-page limit', async ({ page }) => {
// Navigate to health page; if visible row count == per-page limit, click
// Next, assert page=2 in URL and different rows appear. If seed has fewer
// records than the per-page limit, use `test.skip()` with a clear reason.
});
test('health page shows empty state for child with no records', async ({ page }) => {
// Fresh child or a known seeded child_id with zero health rows. Assert
// the "No health records" empty-state copy.
});
P2-3: Security Changes and Alerts Pages E2E (Complete)
Covered by tests/e2e/specs/security-extra.spec.ts (5 tests):
-
major changes shows seeded worker_assignment change— assertsworker_assignment|org_restructure|system_upgradeappears in the table and at least one data row renders -
major changes columns include Type, Description, Scope— header text assertion -
security alerts page shows table with correct columns—severity,ruleheaders -
security alerts table renders with correct columns— structural (alerts may or may not be seeded, empty-safe) -
cross-nav links on all security pages navigate correctly— walks /security/{audit,reviews,archive,nist,changes,alerts} and asserts h1 + cross-nav presence
Nothing to add. Forbidden-for-caseworker coverage from the original plan is better placed in rbac.spec.ts (already covers role-based page access); not tracked here.
P2-4: Reporting Dashboard CRUD Workflow E2E (Partial — 6 tests done, 1 missing)
Covered across tests/e2e/specs/reporting.spec.ts (6 tests: stat tiles, issue nav, issues headers, AFCARS 2025-Q4 seeded row, NCANDS structure, nav link) and tests/e2e/specs/reporting-actions.spec.ts (6 tests: AFCARS generate, review button presence, NCANDS generate, issue resolve, stat-tile numbers, severity breakdown).
Missing (add 1 test) — the full validated → reviewed → approved lifecycle is not exercised end-to-end. Review button is shown but not clicked; no approval step is verified.
// Add to reporting-actions.spec.ts:
test('AFCARS submission lifecycle: generate → review → approve', async ({ page }) => {
await page.goto('/reporting/afcars');
const period = `E2E-L-${Date.now()}`;
await page.locator('input#reporting_period').fill(period);
await page.locator('button', { hasText: /^generate$/i }).click();
await page.waitForLoadState('networkidle');
const row = page.locator('.data-table tbody tr', { hasText: period });
await expect(row).toBeVisible();
// Expect status badge to be "validated" right after generate
await expect(row.locator('.badge', { hasText: /validated/i })).toBeVisible();
// Click Review → status becomes "reviewed"
await row.locator('button', { hasText: /^review$/i }).click();
await page.waitForLoadState('networkidle');
await expect(row.locator('.badge', { hasText: /reviewed/i })).toBeVisible();
// Click Approve → status becomes "approved"
await row.locator('button', { hasText: /^approve$/i }).click();
await page.waitForLoadState('networkidle');
await expect(row.locator('.badge', { hasText: /approved/i })).toBeVisible();
});
P2-5: Placement Document Upload/Download E2E (Not started — 0 coverage)
Routes: POST /placement/homes/:id/documents (upload), GET /placement/homes/:id/documents/:doc_id/download, POST /placement/homes/:id/documents/:doc_id/delete — all in services/craig-web/src/routes/placement/documents.rs.
File: tests/e2e/specs/placement-documents.spec.ts (new)
Note: the routes are scoped to foster homes, not child-level placements — adjust assertions accordingly. Also requires a fixture file in tests/e2e/fixtures/test-document.pdf (or an inline Buffer passed to page.setInputFiles).
// tests/e2e/specs/placement-documents.spec.ts
import { test, expect } from '@playwright/test';
import * as path from 'path';
test.describe('Placement Document Management', () => {
test('upload document to placement', async ({ page }) => {
// Navigate to placement detail page
// Click "Documents" tab
// Click "Upload" button
// Use page.setInputFiles to upload a test PDF
// Assert upload success message
// Assert document appears in list
});
test('download uploaded document', async ({ page }) => {
// Navigate to placement with existing document
// Click download link
// Assert download starts (use page.waitForEvent('download'))
});
test('delete document from placement', async ({ page }) => {
// Navigate to placement with existing document
// Click delete button on document
// Confirm deletion
// Assert document removed from list
});
test('upload rejects disallowed file types', async ({ page }) => {
// Try to upload an .exe file
// Assert error message about disallowed content type
});
});
4 tests. Requires seed data with a placement record. Needs a test fixture file (e.g., tests/e2e/fixtures/test-document.pdf).
P2-6: AFCARS Export Download E2E (Not started — 0 coverage)
Generation is covered (P2-4); the downstream /reporting/afcars/{id}/export download path is not exercised. Add one test alongside the P2-4 lifecycle test.
test('export approved AFCARS submission produces a downloadable file', async ({ page }) => {
await page.goto('/reporting/afcars');
// Pick a seeded or just-generated approved submission
const row = page.locator('.data-table tbody tr', { hasText: /approved/i }).first();
await expect(row).toBeVisible();
const downloadPromise = page.waitForEvent('download');
await row.locator('a, button', { hasText: /export|download/i }).click();
const download = await downloadPromise;
const path = await download.path();
expect(path).toBeTruthy();
// AFCARS export is tab-delimited; sanity-check non-empty
const fs = await import('node:fs/promises');
const bytes = await fs.readFile(path!);
expect(bytes.length).toBeGreaterThan(0);
});
P2-7: i18n Language Switching E2E (superseded 2026-08-17: #1332 removed /set-locale + pinned worker negotiation to en; the switcher/es legs below are #1487 scope)
Covered by tests/e2e/specs/i18n.spec.ts (3 tests): set-locale POST returns success and page still renders, English locale renders translated text on multiple pages, no raw FTL keys visible on any page.
Missing (add 2 tests) — the current specs don’t exercise the switcher UI and don’t verify actual Spanish content (because the es FTL files have only a handful of translations; most keys fall back to English):
test('language switcher UI sets the locale cookie and reloads localized content', async ({ page }) => {
await page.goto('/dashboard');
// Click the language switcher control (locator depends on header layout —
// `[data-locale-switcher] button[value="es"]` or similar). If the UI isn't
// surfaced yet, use `page.request.post('/set-locale?lang=es')` and verify
// the Set-Cookie response header, then reload to see the effect.
await page.reload();
// The `es` bundle translates dashboard-title → "Panel de Control" — assert
// that specific key lands as Spanish. (Other headings may still be English
// pending FTL-file expansion; don't overassert.)
await expect(page.locator('h1')).toContainText(/Panel de Control|Dashboard/);
});
test('locale cookie persists across navigation + reload', async ({ page }) => {
await page.request.post('/set-locale?lang=es');
await page.goto('/cases/');
await page.reload();
// Cookie survives a full reload; the `craig-locale` (or equivalent) cookie
// should be set. Assert either via page content or `page.context().cookies()`.
const cookies = await page.context().cookies();
expect(cookies.some((c) => c.value === 'es')).toBe(true);
});
P2-8: OTEL Telemetry Unit Tests (Partial — 3/4 done)
File: crates/craig-common/src/telemetry.rs
Already present (3 tests): metrics_registry_returns_some_or_none, inject_trace_headers_no_panic, telemetry_guard_default_drop_is_safe.
Missing (add 1 test) — env-filter-respecting init:
#[test]
fn init_respects_rust_log_env_filter() {
// SAFETY: modifying process env is safe here because tests in the
// telemetry module run serially (the TelemetryGuard is a global;
// parallel init would conflict anyway).
std::env::set_var("RUST_LOG", "debug");
let _guard = init("info", "test-service");
std::env::remove_var("RUST_LOG");
// No panic = success; the subscriber honoured RUST_LOG over the
// default-level argument.
}
crates/craig-api/src/otel.rs (the otel_propagation Axum middleware) is a thin wrapper around opentelemetry::global::get_text_map_propagator. Direct testing needs an Axum router mock for limited value; skip — it’s exercised transitively when OTEL is wired in devstack.
P2-9: i18n Module Unit Test Expansion — post-Arc<str> refactor (Partial)
File: services/craig-web/src/i18n.rs
Context: CQR April Step 10 (MR !114) rewrote the module to use nested HashMap<String, HashMap<String, Arc<str>>> storage. translate() now returns Arc<str>, not String; tests must compare via &*translate(…) deref. The original Phase 2 test list was written against the pre-refactor String signature — asserts have been adjusted below.
Already covered (6 tests):
- translate_english, translate_spanish, fallback_to_default, missing_key_returns_key, available_locales_sorted, translate_returns_shared_arc (the new ptr-eq assertion proving cloned `Arc<str>`s share storage)
Missing (add 6 tests):
#[test]
fn empty_locale_dir_produces_empty_i18n() {
let dir = TempDir::new().unwrap();
let i18n = I18n::load(dir.path(), "en");
assert!(i18n.available_locales.is_empty());
// Miss path still produces Arc<str>(key).
assert_eq!(&*i18n.translate("en", "any-key"), "any-key");
}
#[test]
fn nonexistent_locale_dir_produces_empty_i18n() {
let i18n = I18n::load(std::path::Path::new("/nonexistent/path"), "en");
assert!(i18n.available_locales.is_empty());
}
#[test]
fn translate_nonexistent_locale_falls_back_to_default() {
let dir = create_test_locales();
let i18n = I18n::load(dir.path(), "en");
// "fr" locale isn't loaded — falls back to "en" storage.
assert_eq!(&*i18n.translate("fr", "dashboard-title"), "Dashboard");
}
#[tokio::test]
async fn with_locale_sets_task_local_and_translate_current_reads_it() {
let dir = create_test_locales();
let i18n = I18n::load(dir.path(), "en");
let result = with_locale(i18n, "es".to_string(), async {
translate_current("dashboard-title")
})
.await;
assert_eq!(&*result, "Panel de Control");
}
#[tokio::test]
async fn translate_current_without_scope_returns_key_as_arc() {
// Outside `with_locale`, the task-local isn't set; translate_current
// allocates an Arc<str> from the key itself.
let result = translate_current("some-key");
assert_eq!(&*result, "some-key");
}
#[test]
fn current_locale_default_is_en_without_scope() {
// Outside `with_locale`, current_locale falls back to "en".
assert_eq!(current_locale(), "en");
}
Bringing i18n total from 6 to 12.
P2-10: Idempotency Middleware Improvements
File: crates/craig-api/src/idempotency.rs
The existing tests (5 tests) cover cache operations but NOT the middleware function itself. The cache_expired_entry test is conditionally skipped on freshly booted systems (checked_sub fails when system uptime < 24h).
// Add to existing #[cfg(test)] mod tests in idempotency.rs:
use axum::body::Body;
use axum::http::{Request, StatusCode};
use axum::middleware::{self, Next};
use axum::routing::post;
use axum::Router;
use tower::ServiceExt;
#[tokio::test]
async fn middleware_passes_through_get_requests() {
// GET requests should not be deduplicated
let cache = IdempotencyCache::new();
let app = Router::new()
.route("/test", axum::routing::get(|| async { "ok" }))
.layer(middleware::from_fn_with_state(cache.clone(), idempotency_middleware));
let req = Request::builder()
.method("GET")
.uri("/test")
.header(IDEMPOTENCY_KEY_HEADER, "key-1")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
// GET should not be cached
assert!(cache.entries.is_empty());
}
#[tokio::test]
async fn middleware_caches_post_with_idempotency_key() {
let cache = IdempotencyCache::new();
let app = Router::new()
.route("/test", post(|| async { "created" }))
.layer(middleware::from_fn_with_state(cache.clone(), idempotency_middleware));
let req = Request::builder()
.method("POST")
.uri("/test")
.header(IDEMPOTENCY_KEY_HEADER, "key-2")
.body(Body::empty())
.unwrap();
let resp = app.clone().oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
// Should have cached the response
assert_eq!(cache.entries.len(), 1);
}
#[tokio::test]
async fn middleware_replays_cached_response() {
let cache = IdempotencyCache::new();
let call_count = std::sync::Arc::new(std::sync::atomic::AtomicU32::new(0));
let count = call_count.clone();
let app = Router::new()
.route("/test", post(move || {
let c = count.clone();
async move {
c.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
"response"
}
}))
.layer(middleware::from_fn_with_state(cache.clone(), idempotency_middleware));
// First request
let req1 = Request::builder()
.method("POST")
.uri("/test")
.header(IDEMPOTENCY_KEY_HEADER, "key-3")
.body(Body::empty())
.unwrap();
let resp1 = app.clone().oneshot(req1).await.unwrap();
assert_eq!(resp1.status(), StatusCode::OK);
// Second request with same key
let req2 = Request::builder()
.method("POST")
.uri("/test")
.header(IDEMPOTENCY_KEY_HEADER, "key-3")
.body(Body::empty())
.unwrap();
let resp2 = app.oneshot(req2).await.unwrap();
assert_eq!(resp2.status(), StatusCode::OK);
// Should have replay header
assert_eq!(resp2.headers().get("x-idempotency-replay").unwrap(), "true");
// Handler should only have been called once
assert_eq!(call_count.load(std::sync::atomic::Ordering::SeqCst), 1);
}
#[tokio::test]
async fn middleware_passes_through_without_header() {
let cache = IdempotencyCache::new();
let app = Router::new()
.route("/test", post(|| async { "no-key" }))
.layer(middleware::from_fn_with_state(cache.clone(), idempotency_middleware));
let req = Request::builder()
.method("POST")
.uri("/test")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
// No caching without idempotency key
assert!(cache.entries.is_empty());
}
4 new tests, bringing idempotency total from 5 to 9.
P2-11: State Machine Serde Roundtrip Tests
Files: All 7 transitions.rs files across services
Each service has state machine enums with Serialize, Deserialize, Display, and EnumString derives. The existing tests cover can_transition_to() but NOT serialization/deserialization roundtrips. A serde regression would silently break API contracts.
Approach: Add roundtrip tests to each transitions.rs that verify:
-
Display(strum) → string →FromStrroundtrip -
serde_json::to_string→serde_json::from_strroundtrip -
Snake_case serialization matches expected API format
// Pattern for each enum (add to existing #[cfg(test)] mod tests):
#[test]
fn investigation_status_serde_roundtrip() {
for status in [InvestigationStatus::Open, InvestigationStatus::PendingReview, InvestigationStatus::Closed] {
// Display → FromStr roundtrip
let s = status.to_string();
let parsed: InvestigationStatus = s.parse().unwrap();
assert_eq!(parsed, status);
// serde_json roundtrip
let json = serde_json::to_string(&status).unwrap();
let deserialized: InvestigationStatus = serde_json::from_str(&json).unwrap();
assert_eq!(deserialized, status);
}
}
#[test]
fn investigation_status_snake_case_format() {
assert_eq!(InvestigationStatus::PendingReview.to_string(), "pending_review");
assert_eq!(serde_json::to_string(&InvestigationStatus::PendingReview).unwrap(), "\"pending_review\"");
}
Per-service test counts:
| Service | Enums | New Tests |
|---|---|---|
craig-cases |
InvestigationStatus, CaseStatus, CasePlanStatus, CaseStage, MilestoneStatus, ReferralStatus |
12 (2 per enum: roundtrip + format) |
craig-placement |
PlacementStatus |
2 |
craig-exchange |
TransactionStatus, IcpcStatus |
4 |
craig-financial |
PaymentStatus, ClaimStatus, AdjustmentStatus |
6 |
craig-reporting |
SubmissionStatus, QualityIssueStatus |
4 |
craig-security |
ReviewStatus, NistImplementationStatus |
4 |
craig-intake |
ReportStatus, ApiKeyStatus |
4 |
Total |
16 enums |
36 tests |
Phase 2 Estimated New Tests (refreshed 2026-04-21)
Delta after the audit — "New Tests" counts reflect what’s still missing, not what was originally proposed.
| Step | Category | New Tests | Status |
|---|---|---|---|
P2-1 |
E2E: Placement education |
0 |
Complete (MR !128) |
P2-2 |
E2E: Placement health |
0 |
Complete (MR !128) |
P2-3 |
E2E: Security changes/alerts |
0 |
Complete (MR !123) |
P2-4 |
E2E: Reporting dashboard CRUD |
0 |
Complete (MR !128) |
P2-5 |
E2E: Placement documents |
0 |
Complete (MR !128) |
P2-6 |
E2E: AFCARS export download |
0 |
Complete (MR !128, closes #200 — added UI Export + Download buttons + craig-web proxy routes) |
P2-7 |
E2E: i18n language switching |
0 |
Complete (MR !128) |
P2-8 |
Unit: OTEL telemetry |
0 |
Complete (MR !125) |
P2-9 |
Unit: i18n expansion (post-Arc<str>) |
0 |
Complete (MR !125) |
P2-10 |
Unit: Idempotency middleware |
0 |
Complete (MR !125) |
P2-11 |
Unit: State-machine serde roundtrip |
0 |
Complete (MR !127 — 32 new tests; plan had estimated 36, actual landed at 32 reflecting the current enum inventory) |
Total |
0 remaining — Phase 2 Complete. 55 tests shipped: 11 in MR !125 (P2-8/P2-9/P2-10) + 32 in MR !127 (P2-11) + 12 in MR !128 (P2-1/P2-2/P2-4/P2-5/P2-6/P2-7). P2-3 was already covered before the refresh. |
This would bring the workspace test total from ~1212 (verified 2026-04-17) + the Phase 1 additions already landed to ~1271 Rust + E2E combined.
Phase 2 Implementation Order
Step P2-A: Unit Tests (Steps P2-8 through P2-11)
No devstack dependency. Can be implemented and verified immediately:
-
OTEL telemetry tests (4 tests in
crates/craig-common/src/telemetry.rs) -
i18n expansion (8 tests in
services/craig-web/src/i18n.rs) -
Idempotency middleware tests (4 tests in
crates/craig-api/src/idempotency.rs) -
State machine serde roundtrip tests (36 tests across 7
transitions.rsfiles)
Verify: cargo nextest run --workspace --lib
Step P2-B: E2E Tests (Steps P2-1 through P2-7)
Requires running devstack with seed data:
-
tests/e2e/specs/placement-education.spec.ts(new, 3 tests) -
tests/e2e/specs/placement-health.spec.ts(new, 3 tests) -
tests/e2e/specs/security-pages.spec.ts(new, 5 tests) -
Update
tests/e2e/specs/reporting.spec.ts(+4 tests) -
tests/e2e/specs/placement-documents.spec.ts(new, 4 tests) -
tests/e2e/specs/i18n.spec.ts(new, 4 tests)
Verify: cargo xtask e2e