Plan: API Idempotency Audit & Implementation

On this page

Superseded (2026-07-31; the successor program completed 2026-08-02). The Idempotency-Key middleware this plan built was retired under ADR-062 and the Transactional Idempotency Claims program (epic &77): it records completion outside the domain transaction (a lease-based, unfenced 30-second claim — #1182) and is opt-in with zero senders. Under that program idempotency moves to an in-domain-transaction claim and this middleware is deleted (unit B2). This page is retained as the historical record of the 2026-03 implementation.

Status

Step Description Status

1

Plan file and nav entry

Done (2026-03-21) — MR !47

2

Audit all POST endpoints — classify as naturally idempotent, needs key, or stateless

Done (2026-03-21) — MR !47

3

Implement idempotency middleware in craig-api

Done (2026-03-21) — MR !47

4

Apply middleware to all services

Done (2026-03-21) — MR !47

5

Document idempotency guarantees in OpenAPI specs

N/A (superseded by the ADR-062 program — the middleware is retired)

6

Integration tests

Done (2026-03-21) — MR !47

7

Documentation, commit, push, MR

Done (2026-03-21) — MR !47

Issues: TBD
Branch: feature/api-idempotency

Context

The coding conventions state: "All API calls must be idempotent." Currently, 51 POST endpoints across 8 services lack retry safety. A network failure during a POST could result in duplicate records if the client retries.

11 endpoints have natural deduplication via database unique constraints (e.g., case_number, partner_name, control_id). 5 endpoints are stateless (evaluation, connectivity tests). The remaining ~35 endpoints need explicit Idempotency-Key header support.

Approach

Implement a shared Axum middleware layer in craig-api that:

  1. Extracts an optional Idempotency-Key header from the request

  2. If present: checks an in-memory cache for a previous response with that key

  3. If cached: returns the cached response immediately (replay)

  4. If not cached: processes the request normally, caches the response, returns it

  5. Cache TTL: 24 hours (configurable)

  6. Key scope: per-user (keyed by claims.sub + idempotency_key)

For endpoints with natural deduplication, the unique constraint provides retry safety without the header. The middleware is additive — endpoints work without the header, but clients that provide it get guaranteed exactly-once semantics.

Scope

In scope:

  • Shared IdempotencyLayer middleware in crates/craig-api

  • In-memory response cache (DashMap with TTL eviction)

  • Application to all 8 API services

  • OpenAPI documentation of Idempotency-Key header on all POST endpoints

  • Integration tests verifying duplicate request returns cached response

Out of scope:

  • Redis-backed cache (production enhancement — in-memory is sufficient for single-instance deployments)

  • Idempotency for PUT/DELETE (these are naturally idempotent by HTTP semantics)

  • craig-intake public endpoints (no auth, different security model)

  • craig-web BFF (not an API service)

Design

Middleware Implementation

File: crates/craig-api/src/idempotency.rs

use axum::{extract::Request, middleware::Next, response::Response};
use dashmap::DashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};

const CACHE_TTL: Duration = Duration::from_secs(86400); // 24 hours
const HEADER_NAME: &str = "idempotency-key";

#[derive(Clone)]
pub struct IdempotencyCache {
    cache: Arc<DashMap<String, CachedResponse>>,
}

struct CachedResponse {
    status: u16,
    headers: Vec<(String, String)>,
    body: bytes::Bytes,
    created_at: Instant,
}

impl IdempotencyCache {
    pub fn new() -> Self {
        Self { cache: Arc::new(DashMap::new()) }
    }
}

pub async fn idempotency_middleware(
    cache: axum::Extension<IdempotencyCache>,
    request: Request,
    next: Next,
) -> Response {
    // Only apply to POST requests
    if request.method() != axum::http::Method::POST {
        return next.run(request).await;
    }

    // Extract idempotency key from header
    let key = match request.headers().get(HEADER_NAME) {
        Some(v) => v.to_str().unwrap_or("").to_string(),
        None => return next.run(request).await, // No key = no dedup
    };

    // Scope by user (claims.sub) if available
    let user_id = request.extensions()
        .get::<craig_auth::Claims>()
        .map(|c| c.sub.clone())
        .unwrap_or_default();
    let cache_key = format!("{user_id}:{key}");

    // Check cache
    if let Some(cached) = cache.cache.get(&cache_key) {
        if cached.created_at.elapsed() < CACHE_TTL {
            // Return cached response (replay)
            return rebuild_response(&cached);
        }
        // Expired — remove and proceed
        drop(cached);
        cache.cache.remove(&cache_key);
    }

    // Process request
    let response = next.run(request).await;

    // Cache the response
    // ... (extract status, headers, body, store in cache)

    response
}

Endpoint Classification

Category Count Behavior

Naturally idempotent (unique constraints)

11

Retry returns 409 Conflict on duplicate — client knows it’s a retry

Needs Idempotency-Key

35

Without key: normal behavior. With key: cached response on retry

Stateless (no side effects)

5

Always safe to retry. Key header ignored

Dependencies

Add dashmap = "6" to crates/craig-api/Cargo.toml for the concurrent cache.

Verification

  1. Unit test: send POST with Idempotency-Key, verify response cached

  2. Unit test: send same POST again with same key, verify identical response returned

  3. Unit test: send POST without key, verify normal processing

  4. Integration test: create resource with key, retry, verify no duplicate

  5. Integration test: different key creates separate resource

Files Touched

File Change

crates/craig-api/src/idempotency.rs

New: middleware + cache

crates/craig-api/src/lib.rs

Add idempotency layer to middleware stack

crates/craig-api/Cargo.toml

Add dashmap dependency

services/*/src/main.rs

No changes needed (middleware applied via craig-api)

OpenAPI specs across all services

Add Idempotency-Key header parameter to POST endpoints

Documentation Updates

  • .claude/docs/services.md — note idempotency support per endpoint

  • CHANGELOG.adoc — entry under Unreleased

  • docs/modules/ROOT/pages/roadmap.adoc — tick idempotency items

  • OpenAPI specs — Idempotency-Key header documentation

Edit this page · latest