IdP Integration

On this page

Overview

CRAIG is IdP-neutral over the set of OIDC backends that implement the client_credentials grant. Any conforming issuer can authenticate CRAIG users and (since Plan E) issue per-service client_credentials tokens for service-to-service auth. The codebase contains no IdP-vendor-specific logic: endpoints are resolved at boot via the OIDC discovery doc (<issuer>/.well-known/openid-configuration), and inbound bearer-token validation runs via one of two strategies (ADR-029):

Local JWS validation: craig-auth::JwksProvider decodes the bearer (a JWS, JSON Web Signature, token) locally against the issuer’s JWKS (JSON Web Key Set). Fast (~10µs) and resilient to short IdP outages; the default path for JWS-issuing backends.
RFC 7662 introspection: craig-auth::IntrospectionClient POSTs the bearer to the issuer’s introspection endpoint and caches the claims (DashMap, SHA-256-hashed keys, configurable TTL). This is the required path for backends that issue JWE-encrypted (JSON Web Encryption) or opaque access tokens: Kanidm by default, ZITADEL’s opaque default, Keycloak lightweight tokens, and authentik’s encrypted mode.

Per-service env var CRAIG_<SVC>__TOKEN_VALIDATION_MODE selects the strategy: jws / introspect / auto (default). Auto dispatches on token shape: 3 segments → JWS, 5 segments → JWE-via-introspection, no dots → opaque-via-introspection. Plan F (docs/modules/ROOT/pages/plans/introspection-validation-mode.adoc) shipped the introspection path across 6 step MRs (!269–!274).

Plan E (service identity) made the client_credentials grant load-bearing for CRAIG: the craig-cases authz bootstrap, the craig-web BFF outbound path, and audit attribution all depend on per-service client_credentials tokens. Backends that do not support that grant cannot host CRAIG. Notably, Dex 2.45.1 does not implement the grant (compiled out of the server’s allSupportedGrants map; no config can enable it) and is therefore not a supported backend.

Tested backends (JWS or introspection-validating):

Backend Mode IaC adapter Notes

Backend	Mode	IaC adapter	Notes
Keycloak	JWS	`keycloak`	Devstack reference; production-tested with the full E2E suite green. Supports "lightweight access tokens" (opaque) per Plan F’s introspection path.
authentik	JWS	`authentik`	Verified end-to-end against Authentik 2026.2.2 (!266). Rendered blueprint imports cleanly; full `client_credentials` mint round-trip in CI. Encrypted-token mode supported per Plan F’s introspection path.
ZITADEL	JWS (with `ACCESS_TOKEN_TYPE_JWT` pin) or introspection	`zitadel`	Verified structurally against ZITADEL 4.15.0 (!268). Renderer pins JWT type on machine users. ZITADEL’s default opaque-token mode works under Plan F’s `TOKEN_VALIDATION_MODE=introspect`. Full mint needs operator Terraform bootstrap (`devstack/zitadel/README.md`).
Kanidm	Introspection only	`kanidm`	Verified structurally against Kanidm 1.10.1 (!272 / !273). `client_credentials` access tokens are unconditionally JWE-encrypted (`oauth2.rs:1861`); validation requires introspection. Renderer emits operator CLI bootstrap script. Set `CRAIG_<SVC>__TOKEN_VALIDATION_MODE=introspect` (or rely on the `Auto` default which detects the 5-segment JWE shape). Full mint needs operator-side `bootstrap.sh` after `idm_admin` password recovery (`devstack/kanidm/README.md`).

Keycloak

JWS

keycloak

Devstack reference; production-tested with the full E2E suite green. Supports "lightweight access tokens" (opaque) per Plan F’s introspection path.

authentik

JWS

authentik

Verified end-to-end against Authentik 2026.2.2 (!266). Rendered blueprint imports cleanly; full client_credentials mint round-trip in CI. Encrypted-token mode supported per Plan F’s introspection path.

ZITADEL

JWS (with ACCESS_TOKEN_TYPE_JWT pin) or introspection

zitadel

Verified structurally against ZITADEL 4.15.0 (!268). Renderer pins JWT type on machine users. ZITADEL’s default opaque-token mode works under Plan F’s TOKEN_VALIDATION_MODE=introspect. Full mint needs operator Terraform bootstrap (devstack/zitadel/README.md).

Kanidm

Introspection only

kanidm

Verified structurally against Kanidm 1.10.1 (!272 / !273). client_credentials access tokens are unconditionally JWE-encrypted (oauth2.rs:1861); validation requires introspection. Renderer emits operator CLI bootstrap script. Set CRAIG_<SVC>__TOKEN_VALIDATION_MODE=introspect (or rely on the Auto default which detects the 5-segment JWE shape). Full mint needs operator-side bootstrap.sh after idm_admin password recovery (devstack/kanidm/README.md).

To run the multi-backend integration tests:

cargo xtask dev multibackend-up
cargo nextest run -p craig-test-lib --test identity_multibackend
cargo xtask dev multibackend-down

Untested but expected to work given OIDC conformance:

Okta, Azure AD / Entra ID, ForgeRock, PingFederate, Auth0, custom OIDC providers.

For backends without a render adapter, the operator configures the backend in whatever tooling they already use (Terraform, Helm, Pulumi, the console); the contract below is the spec to satisfy.

Required IdP Configuration

Realm / Tenant

A single realm or tenant containing the CRAIG worker identities. CRAIG does not provision users — operators integrate with whatever directory the agency already uses (Active Directory, LDAP, SAML federation, social login).

Clients

CRAIG needs two OIDC clients in the realm:

Client Flow Use

Client	Flow	Use
`craig-ui`	Authorization Code + PKCE (Proof Key for Code Exchange)	Web UI (craig-web BFF). Public client; no client secret. Redirect URI: `https://<your-deployment>/auth/callback`.
`craig-api`	Resource Owner Password Credentials (transitional)	CLI clients, integration tests, and (pre-Plan-E) service-to-service auth. Public client with direct access grants (per the realm JSON). Plan E retires this in favor of per-service `client_credentials` clients (one per CRAIG service).

craig-ui

Authorization Code + PKCE (Proof Key for Code Exchange)

Web UI (craig-web BFF). Public client; no client secret. Redirect URI: https://<your-deployment>/auth/callback.

craig-api

Resource Owner Password Credentials (transitional)

CLI clients, integration tests, and (pre-Plan-E) service-to-service auth. Public client with direct access grants (per the realm JSON). Plan E retires this in favor of per-service client_credentials clients (one per CRAIG service).

Claims & Scopes

CRAIG expects the access token to carry:

Claim Notes

Claim	Notes
`sub`	Stable user UUID. Used as the worker identity for all data writes.
`preferred_username`	Human-readable username; rendered in the web UI.
`email`	Optional; populated into `worker_identities.email` on first sign-in.
`realm_access.roles` (or configured equivalent)	RBAC roles. Default path: `realm_access.roles` (Keycloak shape). `CRAIG_IDENTITY_SERVICE_ROLE_PREFIX` (default `service:`) selects which role entries mark a token as a service-caller. Per Plan E § Step 3 Errata: configurable `CRAIG_IDENTITY_ROLES_CLAIM_PATH` for non-Keycloak shapes (Okta `groups`, Azure AD `roles`, custom paths) is currently a deferred follow-up — backends emitting roles outside `realm_access.roles` need a future small claims-mapper change in `craig-auth::Claims`. Track via successor MR if a non-Keycloak deployer hits this.
`aud`	Multi-valued array of service names. CRAIG validates that the token’s audience contains the target service (`craig-cases`, `craig-rules`, etc.). See ADR-021 for full strictness rules.
`typ` (optional)	When present, must be `"Bearer"`. Absent values are tolerated for non-Keycloak issuers.

sub

Stable user UUID. Used as the worker identity for all data writes.

preferred_username

Human-readable username; rendered in the web UI.

email

Optional; populated into worker_identities.email on first sign-in.

realm_access.roles (or configured equivalent)

RBAC roles. Default path: realm_access.roles (Keycloak shape). CRAIG_IDENTITY_SERVICE_ROLE_PREFIX (default service:) selects which role entries mark a token as a service-caller. Per Plan E § Step 3 Errata: configurable CRAIG_IDENTITY_ROLES_CLAIM_PATH for non-Keycloak shapes (Okta groups, Azure AD roles, custom paths) is currently a deferred follow-up — backends emitting roles outside realm_access.roles need a future small claims-mapper change in craig-auth::Claims. Track via successor MR if a non-Keycloak deployer hits this.

aud

Multi-valued array of service names. CRAIG validates that the token’s audience contains the target service (craig-cases, craig-rules, etc.). See ADR-021 for full strictness rules.

typ (optional)

When present, must be "Bearer". Absent values are tolerated for non-Keycloak issuers.

CRAIG’s 9 RBAC roles: 6 operational roles — admin, supervisor, caseworker, eligibility_worker, icpc_coordinator, readonly — plus 3 office-authority roles — county_director, regional_director, state_office (the ADR-054 subsidy approval matrix axis; office principals also hold a base operational role). The operational roles are inputs to JDM decision tables (rulesets/<jurisdiction>-authz-<resource>.json), so an operator can extend or reshape the role model via ruleset edits without touching CRAIG source; the approval matrix authorizes on the office axis.

CRAIG-specific OAuth2 scopes

Per RFC 6749, scopes are space-separated identifiers in the scope claim. CRAIG enforces them via Claims::require_scope as an additive gate on top of role checks — RBAC decides what a principal can do, scope narrows which clients may do it. The catalogue is intentionally tight: most authorization is role-driven, and a new scope is added only when a single client should be the sole legitimate caller of an endpoint.

Scope Assigned to Enforced at

Scope	Assigned to	Enforced at
`partners.replay-check`	`craig-intake` (default client scope)	`services/craig-cases/src/api/jws_replay.rs::check_jws_replay` — guards the `/v1/cases/_internal/jws-replay-check` S2S dedup endpoint (#489). The role check beneath it (`require_caseworker_or_above`) is bypassed for any service caller per Plan E (ADR-028); this scope is what stops a different compromised service-account or a worker token from writing to the JWS replay ledger.

partners.replay-check

craig-intake (default client scope)

services/craig-cases/src/api/jws_replay.rs::check_jws_replay — guards the /v1/cases/_internal/jws-replay-check S2S dedup endpoint (#489). The role check beneath it (require_caseworker_or_above) is bypassed for any service caller per Plan E (ADR-028); this scope is what stops a different compromised service-account or a worker token from writing to the JWS replay ledger.

Operators replacing the devstack Keycloak realm with another IdP must mint partners.replay-check on the equivalent of craig-intake’s `client_credentials flow. Devstack provisions it via devstack/keycloak/craig-realm.json (clientScopes array + defaultClientScopes on the craig-intake client); see ADR-028 for the underlying identity model.

CRAIG’s standard scopes inherited from Keycloak (profile, email, roles, acr, basic, web-origins) are mirrored from the master realm via defaultDefaultClientScopes so the existing claim shape is preserved alongside the new catalogue.

Audience Mappers

Configure the realm to emit a multi-valued aud claim containing every CRAIG backend service:

craig-rules, craig-cases, craig-placement, craig-exchange,
craig-financial, craig-reporting, craig-security, craig-composition,
craig-intake, craig-web

This prevents cross-realm token reuse without burdening operators with per-service tokens (RBAC remains the authorization boundary; audience is defense-in-depth).

CRAIG-Side Configuration

Every CRAIG service reads these env vars:

Variable Value

Variable	Value
`CRAIG_<SVC>__OIDC_ISSUER`	Public issuer URL, e.g. `https://idp.your-deployment.gov/realms/craig`. Required.
`CRAIG_<SVC>__OIDC_INTERNAL_URL`	Internal URL for split-DNS deployments (Docker / k8s). When set, CRAIG fetches the discovery doc + JWKS via this URL and rewrites the doc’s `jwks_uri` + `token_endpoint` host portions accordingly. Browser-facing endpoints (authorization, end_session) keep the public URL. Optional — defaults to `OIDC_ISSUER`.

CRAIG_<SVC>__OIDC_ISSUER

Public issuer URL, e.g. https://idp.your-deployment.gov/realms/craig. Required.

CRAIG_<SVC>__OIDC_INTERNAL_URL

Internal URL for split-DNS deployments (Docker / k8s). When set, CRAIG fetches the discovery doc + JWKS via this URL and rewrites the doc’s jwks_uri + token_endpoint host portions accordingly. Browser-facing endpoints (authorization, end_session) keep the public URL. Optional — defaults to OIDC_ISSUER.

At boot, each service:

Builds an OidcDiscovery client pointing at the issuer (and internal URL if split-DNS).
Fetches <issuer>/.well-known/openid-configuration and caches the four endpoints CRAIG consumes: jwks_uri, token_endpoint, authorization_endpoint, end_session_endpoint.
Fetches the JWKS from jwks_uri and starts a background refresh task (default TTL 1h). Additionally (#981), a bearer whose kid misses the key cache triggers ONE on-demand refresh-and-revalidate, single-flight across concurrent requests and bounded by a 30s cooldown that also covers failed attempts. Normal IdP signing-key rotation is therefore picked up immediately instead of failing fresh tokens until the next periodic refresh. Every other rejection class (signature, exp, iss, aud, azp, typ, missing kid) never triggers IdP traffic, and a failed on-demand refresh still rejects the token (fail-closed).
Spawns a per-instance authz cache invalidation subscriber that listens for ruleset.changed.* events from craig-rules.

If the discovery doc fetch fails at boot, CRAIG retries with exponential backoff (250ms → 4s, ~7.75s budget). A post-boot warm-up task (spawn_post_boot_warmup) retries bulk_refresh 20× × 1s after axum starts serving so a slow IdP startup doesn’t permanently brick the service.

Introspection-mode tuning

Plan F (ADR-029) added five per-service knobs for operators running Kanidm / ZITADEL-opaque / authentik-encrypted / Keycloak-lightweight-token deployments where CRAIG_<SVC>__TOKEN_VALIDATION_MODE resolves to introspect. All carry sensible defaults; operators only override when tuning latency vs. revocation-staleness, sizing memory, or shifting availability posture.

Variable Default When to override

Variable	Default	When to override
`CRAIG_<SVC>__INTROSPECTION_ENDPOINT`	(auto-discovered)	Set explicitly when the IdP exposes a non-standard introspection path or runs on a separate host from the OIDC issuer.
`CRAIG_<SVC>__INTROSPECTION_AUTH_METHOD`	`client_secret_basic`	Switch to `client_secret_post` for IdPs that disallow Basic auth on the introspection endpoint, or `private_key_jwt` for ADR-026-style mTLS-equivalent deployments where the introspection client uses an asymmetric keypair instead of a shared secret.
`CRAIG_<SVC>__INTROSPECTION_CACHE_TTL_SECONDS`	`60`	Raise (e.g. `300`) for slow IdPs or to reduce introspection load on the issuer; lower (e.g. `10`) when revocation latency matters more than RTT amortization. Actual TTL is `min(claims.exp - now, this)` so short-lived tokens cap naturally.
`CRAIG_<SVC>__INTROSPECTION_CACHE_MAX_ENTRIES`	`10000`	Raise for high-cardinality bearer populations (many concurrent active sessions across the 9 API services); lower for memory-constrained pods. Hard cap that triggers a clear when exceeded — not an LRU.
`CRAIG_<SVC>__INTROSPECTION_SERVE_ON_OUTAGE`	`false`	Set `true` for resilience against IdP blips — expired cache entries continue serving up to 1× `INTROSPECTION_CACHE_TTL_SECONDS` past expiry while the introspection endpoint is unreachable (worst case 2× that knob after the last successful validation, since the TTL itself is capped by it). Keep `false` (fail-closed) when strict revocation semantics outweigh availability.

CRAIG_<SVC>__INTROSPECTION_ENDPOINT

(auto-discovered)

Set explicitly when the IdP exposes a non-standard introspection path or runs on a separate host from the OIDC issuer.

CRAIG_<SVC>__INTROSPECTION_AUTH_METHOD

client_secret_basic

Switch to client_secret_post for IdPs that disallow Basic auth on the introspection endpoint, or private_key_jwt for ADR-026-style mTLS-equivalent deployments where the introspection client uses an asymmetric keypair instead of a shared secret.

CRAIG_<SVC>__INTROSPECTION_CACHE_TTL_SECONDS

60

Raise (e.g. 300) for slow IdPs or to reduce introspection load on the issuer; lower (e.g. 10) when revocation latency matters more than RTT amortization. Actual TTL is min(claims.exp - now, this) so short-lived tokens cap naturally.

CRAIG_<SVC>__INTROSPECTION_CACHE_MAX_ENTRIES

10000

Raise for high-cardinality bearer populations (many concurrent active sessions across the 9 API services); lower for memory-constrained pods. Hard cap that triggers a clear when exceeded — not an LRU.

CRAIG_<SVC>__INTROSPECTION_SERVE_ON_OUTAGE

false

Set true for resilience against IdP blips — expired cache entries continue serving up to 1× INTROSPECTION_CACHE_TTL_SECONDS past expiry while the introspection endpoint is unreachable (worst case 2× that knob after the last successful validation, since the TTL itself is capped by it). Keep false (fail-closed) when strict revocation semantics outweigh availability.

All five knobs default to values appropriate for a Keycloak-style deployment with sub-second introspection latency + sub-10000 concurrent sessions. Kanidm + ZITADEL operators with higher-latency endpoints typically raise INTROSPECTION_CACHE_TTL_SECONDS to 120-300. See ADR-029 for the underlying cache-design rationale.

Common Deployment Topologies

Docker Compose (devstack)

The default docker-compose.yml ships Keycloak as a reference IdP. Two URLs differ:

Public issuer (in JWT iss claims): http://host.docker.internal:<ephemeral_port>/realms/craig
Internal fetch URL (Docker-network): http://keycloak:8080/realms/craig

The split-DNS rewriter in OidcDiscovery::refresh swaps the host portion of jwks_uri and token_endpoint from the public URL to the internal URL automatically. No operator action required.

Kubernetes

For k8s deployments behind an ingress:

Public issuer: https://idp.your-deployment.gov/realms/craig
Internal fetch URL (cluster DNS): http://keycloak.identity.svc.cluster.local:8080/realms/craig

Set CRAIG_<SVC>__OIDC_INTERNAL_URL to the internal URL on every CRAIG service deployment. CRAIG validates JWT iss claims against the public URL but reaches the IdP for discovery + JWKS via cluster DNS.

Production (external IdP)

If your agency already runs Okta, Azure AD/Entra, Keycloak, ForgeRock, or any other OIDC provider:

Create the craig-ui and craig-api clients per the Required IdP Configuration section above.
Configure the realm to emit the audience mappers.
Set CRAIG_<SVC>__OIDC_ISSUER on each service to your IdP’s public issuer URL.
(Optional) Set CRAIG_<SVC>__OIDC_INTERNAL_URL if your service-to-IdP path differs from the public URL.
Run cargo xtask identity verify --issuer <url> to conformance-check the IdP against CRAIG’s contract before going live.

No code changes in CRAIG itself.

Conformance Test

cargo xtask identity verify --issuer <url> (shipped in Plan E Step 12) exercises:

Fetch <issuer>/.well-known/openid-configuration and verify the four required endpoints (authorization_endpoint, token_endpoint, jwks_uri, optionally end_session_endpoint).
Fetch the JWKS and parse it; confirm at least one signing key is present.
(Optional, when --client-id/--client-secret or CRAIG_IDENTITY_TEST_CLIENT_ID/CRAIG_IDENTITY_TEST_CLIENT_SECRET are set) Attempt a client_credentials grant against a configured test service principal and validate the response token shape (non-empty access_token, token_type matches Bearer case-insensitively).

Each check reports pass/fail. Three outcome variants:

AllPass — every check ran and passed; exit 0.
Failures(…) — one or more contract failures; exit 1 with per-check diagnostics.
Inconclusive(…) — the issuer endpoint itself was unreachable (offline, wrong URL); exit 0 so the pre-push gate can downgrade to a warning rather than blocking offline contributors.

Read-only — no mutation. Safe in dev, CI, and production deploy gating. Wired into cargo xtask validate (the identity verify OIDC-conformance step) as a soft check that silently SKIPs when CRAIG_IDENTITY_ISSUER isn’t set; production deploy pipelines pass the env vars so the same invocation becomes a hard gate.

Reference IaC

cargo xtask identity render --backend <name> [--out <path>] [--production] (shipped in Plan E Step 13) emits IaC fragments for supported backends. Pure code generation — no network, no mutation.

--backend keycloak [--out realm.json] — Keycloak realm definition with 9 worker roles, 10 service:* realm roles (one per service, craig-web included), 11 OIDC clients (craig-ui, craig-api, 9 service-account), audience mappers per client. Feed to kc.sh import or merge into a Keycloak Operator CR / Helm chart / Terraform Keycloak provider.
--backend authentik [--out blueprint.yaml] — Authentik blueprint covering the same canonical shape: authentik_core.group for each role + authentik_providers_oauth2.oauth2provider + linked authentik_core.application per client. Audience-mapper wiring left as a documented operator merge step (Authentik handles aud via per-application property_mapping).
--backend zitadel [--out craig.tf] — ZITADEL Terraform HCL covering 1 zitadel_org + 1 zitadel_project + 9 worker zitadel_project_role + 9 service zitadel_project_role + 9 zitadel_machine_user (each with access_token_type = ACCESS_TOKEN_TYPE_JWT pinned for local JWS validation) + 9 zitadel_machine_user_secret + 2 zitadel_application_oidc (craig-ui + craig-api) + optional 7 zitadel_human_user + grants. Operator applies via the official zitadel/zitadel Terraform provider — bootstrap path documented in devstack/zitadel/README.md.
--backend kanidm [--out bootstrap.sh] — Kanidm CLI bootstrap script (set -euo pipefail bash) invoking kanidm group create for 9 worker roles + 9 service:<svc> roles; kanidm system oauth2 create <svc> for 9 service principals; update-scope-map idm_all_accounts openid for each (per the Phase 0 sandbox finding — Kanidm RS carries EntryClass::Account so the map covers the RS-as-principal client_credentials path); show-basic-secret to print each generated secret to stdout; 2 user-facing OAuth2 clients (craig-ui public+PKCE, craig-api confidential); optional 7 dev users via kanidm person create. No admin API client per ADR-026 — operators run the script against their Kanidm instance authenticated as idm_admin. Bootstrap path documented in devstack/kanidm/README.md. Kanidm requires Plan F’s introspection mode at runtime (CRAIG_<SVC>__TOKEN_VALIDATION_MODE=introspect).

--production toggles between devstack-friendly defaults (7 dev users + filled callback URIs) and production-clean output (no users, empty URIs with operator-hint comments).

Operators of unsupported backends (Okta, Entra, ForgeRock, PingFederate, custom) use the Required IdP Configuration spec above to configure their backend manually. The conformance test (Step 12) gives them the contract.

Service Identity Keypair Provisioning (Plan E § Step 14)

Beyond the OIDC client/role provisioning above, each CRAIG service signs X-Craig-Actor JWTs (the on-behalf-of header introduced in Plan E Steps 5–9) with its own ES256 keypair. Peer services verify those JWTs against a deployment-wide JWKS map. Production deployments provision these keypairs once + rotate them on a schedule.

Generating fresh keypairs

cargo xtask gen-actor-keys [--out <path>] [--kid-suffix <suffix>] (shipped in Plan E Step 14) generates 9 ES256 keypairs + the deployment-wide peer-JWKS map. Output matches the env-file shape crates/craig-auth::keypair_env expects.

# Devstack: regenerate the committed keys file
cargo xtask gen-actor-keys --out devstack/devstack-actor-keys.env

# Stdout (production secret-mounting pipeline reads from here)
cargo xtask gen-actor-keys --kid-suffix prod_2026Q2_1

Devstack: inline env vars

The committed devstack/devstack-actor-keys.env carries inline JWK + kid env vars per service plus a single CRAIG_PEER_JWKS_JSON array. docker-compose.yml env_file:-references it. Convenient for dev; never copy these values into production — the leading header in the generated file calls this out explicitly.

Production: file-mount env vars

Production deployments must NOT carry private JWKs in the process environment (visible via /proc/<pid>/environ, audit logs, container introspection). Plan E Step 14 adds file-path variants that read the JWK from a mounted-secret path instead:

CRAIG_<SVC>__SIGNING_JWK_FILE=/run/secrets/craig-<svc>/signing-jwk.json — path to the per-service private JWK
CRAIG_<SVC>__SIGNING_KID=<kid> — kid is non-secret; inline env is fine
CRAIG_PEER_JWKS_JSON_FILE=/run/secrets/craig-peer-jwks/peer-jwks.json — path to the deployment-wide peer-JWKS array

The loader (crates/craig-auth/src/keypair_env.rs::load_signing_keypair_from_env) tries inline env first, then file-path. When both are set, inline wins — but in production neither operator should be setting the inline variant.

Mount the secrets via Docker secrets, Kubernetes secretKeyRef, HashiCorp Vault Agent sidecar, or whatever your platform provides. The content written to the file is byte-identical to what cargo xtask gen-actor-keys emits for the inline form (one JSON JWK per file for the signing key; the JSON array for peer-JWKS).

Rotation procedure

ES256 keypairs should rotate on a cadence (industry baseline: annually; tighter for high-security deployments). The rotation flow exploits the peer-JWKS map’s ability to carry multiple entries per iss so callers can verify either old or new signatures during the cutover window:

Pre-rotation — generate new keys with a fresh kid suffix:

cargo xtask gen-actor-keys --kid-suffix rot_2026Q2 --out /tmp/new-actor-keys.env

Advertise — extend the deployment-wide CRAIG_PEER_JWKS_JSON (or its file-mounted variant) to include both the old + new entries per service. Every service redeploys; verifiers now accept signatures from either kid.
Rotate — rolling-restart each service with its new SIGNING_JWK + SIGNING_KID from /tmp/new-actor-keys.env. Each restarted service starts minting JWTs under the new kid; peers still verify old + new during the rollout.
Drain — wait at least one actor-JWT TTL (Plan E uses 10 minutes; bump if your deployment uses a longer TTL) past the last rolling-restart. All in-flight tokens minted under the old kid are now expired.
Retire — redeploy with the old kid removed from CRAIG_PEER_JWKS_JSON. The rotation is complete.
Audit — the audit_log.actor_service + actor_user_sub columns (Plan E Step 11) make it easy to confirm no requests are still riding the old kid after the drain window: SELECT DISTINCT actor_service FROM audit_log WHERE timestamp > <rotation-start>.

If a key is compromised, skip the drain window and remove the bad kid from peer-JWKS immediately — every service issuing tokens with that kid will fail verification on its next outbound call, which is the intended failure mode for an emergency rotation.

Copy

View