ADR-028: Service Identity + On-Behalf-Of (craig-identity contract)
On this page
Status
Accepted (2026-05-10). Plan E Step 3 ships the foundational Claims
extensions (is_service, service_id, require_service_caller,
actor field) that this ADR’s contract requires; Steps 4–10 wire
the runtime path (OidcServiceToken + ActorTokenIssuer + X-Craig-Actor
middleware + outbound flips).
Plan E plan body: Service Identity plan.
Revision 2026-05-12: the IdP-neutral claim is sharpened from "any
OIDC-compliant backend" to "any OIDC-compliant backend that implements
the client_credentials grant." Dex 2.45.1 does not — its server-level
allSupportedGrants map is compiled in and excludes client_credentials
(server/server.go), and no configuration can enable it. The Dex
render adapter, devstack profile entry, and integration test were
stripped in the successor MR to !266. See CHANGELOG Unreleased
Multi-Backend Identity
Integration Tests for the source-confirmed analysis. A Rust-aligned
successor backend (Kanidm primary; ZITADEL fallback) is queued for
sandbox validation.
Revision 2026-05-13: CRAIG’s bearer-token validation now spans
two strategies — local JWS validation (today’s path; this ADR’s
original assumption) and RFC 7662 introspection (Plan F, !269–!274;
codified in ADR-029).
This ADR’s outbound client_credentials model is unchanged —
OidcServiceToken still drives the per-service token flow Plan E
built. The actor-JWT-on-X-Craig-Actor model is also unchanged
(actor JWTs are CRAIG’s own ES256-signed tokens, validated against
the peer-JWKS map regardless of how the outer bearer was extracted).
What’s new: receiving services can validate the inbound bearer
either locally (JWS) or via the issuer’s introspection endpoint
(JWE / opaque tokens). Cross-cutting invariants — claim semantics
parity, actor JWT lifting symmetry, audit log column population —
hold across both strategies (see Plan F’s plan body for the
codified test matrix).
Context
Service-to-service calls in craig currently pass the worker’s JWT end-to-end. craig-web extracts the worker’s bearer from the inbound request and attaches it to outbound calls to all 8 backend services (services/craig-web/src/api_client.rs:28,44,61,72,98,114). craig-intake similarly forwards the worker’s bearer (or a static cases_forward_secret shared credential) to craig-cases + craig-security.
Three observed problems:
-
Audit log can’t distinguish caller services.
audit_log.user_id = claims.subrecords the worker but not which craig service made the call. "Did the worker invoke /cases directly via CLI, or via the web BFF?" cannot be answered without parsing application logs. -
KeycloakServiceTokenviolates ADR-026.services/craig-intake/src/api/service_token.rs::KeycloakServiceTokenis hardcoded to Keycloak (the type name itself) and uses ROPC (username + password) rather thanclient_credentials. ADR-026 just landed (2026-05-09) demanding IdP-neutrality. This is a directly-named anti-pattern. -
Worker JWT lifetime bounds service-to-service calls. The worker token expires when the worker session does (8h sliding for craig-web). Long-running orchestrator-style calls or deferred outbox publishes can outlive the token. Today this is masked because most calls are synchronous; the concern is real for any future flow that drifts toward longer-running.
The architectural endpoint is service identity — each craig-* service authenticates as itself to other craig-* services. Worker identity, when relevant for audit or authz decisions, travels alongside as an explicit "on-behalf-of" assertion, not as the credential itself.
Origin
This ADR derives from canopy ADR-019 (Service Identity and On-Behalf-Of, 2026-05-08), which itself cites craig ADR-011/021/026 verbatim for worker-auth patterns. Canopy and craig converge on the same model:
-
Worker auth: published JWT contract, OIDC discovery, IdP-neutral identity hydration (lazy from JWT claims)
-
Service auth: per-service
client_credentials+ actor JWT on-behalf-of header
ADR-028 adapts canopy’s design to craig’s call graph (smaller fan-out, dual-purpose endpoints) with one principal divergence: soft-cutover, not hard. Internal services continue to accept worker JWTs in addition to service-identity tokens, so direct CLI / integration-test callers don’t break.
Constraints
-
Per-jurisdiction, not multi-tenant SaaS. Each jurisdiction runs its own craig stack. Architectural simplicity per stack is what matters.
-
Operators choose any IAM backend. Keycloak / Okta / Auth0 / Azure AD-Entra / Authentik / ForgeRock / on-prem AD with ADFS or LDAP. craig must not lock anyone into a specific backend (ADR-026).
-
craig is not in the ops business. xtask is dev/CI tooling. Production identity-backend lifecycle (provisioning, secret rotation, access-review) is the deployer’s responsibility via their existing IaC. craig supplies contract, conformance test, and reference templates — nothing that mutates production infrastructure.
Options considered
-
Status quo: continue forwarding worker JWTs. Dismissed — fails the audit-attribution test the 2026-05-09 audit raised. Same anti-pattern canopy ADR-019 closed.
-
Single shared "craig-internal" service credential. Considered. Loses caller-service attribution: every receiver sees the same
azp. Blast radius on leak is the entire fleet rather than one service. Permission scoping is impossible at the IdP (one client = one set of permissions). Rejected for these reasons during plan-shaping (2026-05-09). -
Self-signed service JWTs with craig-signing as trust root. Builds a parallel auth system inside craig. Confuses craig-signing’s existing role (partner intake JWS per ADR-018) with auth identity. Forces every service to validate two kinds of tokens. Rejected.
-
Build a craig-identity proxy service that wraps backends. New craig-* service container that sits between craig services and the actual IAM backend. Adds latency, a SPOF, operational footprint. Most of what such a proxy would do is already done by OIDC discovery. Rejected.
-
Define craig-identity as an interface contract; backends fulfill it directly. No new service container. craig services depend on
CRAIG_IDENTITY_ISSUER(an OIDC issuer URL); operators point that at any compliant backend. Selected.
Decision
craig-identity is a contract, not a service container. Every craig-* service depends on the craig-identity contract URL — an OIDC issuer that satisfies the requirements below. No new service is introduced. Operators choose any compliant backend that supports the client_credentials grant (Keycloak by default in the dev stack; Authentik, Okta, Entra, ForgeRock, PingFederate, Auth0, or custom in production). craig ships the contract definition, a conformance test, and reference IaC templates for keycloak/authentik — but does not own production identity-backend lifecycle. (Dex was previously listed; see the 2026-05-12 revision note under Status.)
The contract has three layers:
-
OIDC Discovery as the integration point. craig services know one thing about an issuer: its discovery URL. They fetch
<issuer>/.well-known/openid-configurationat boot, cache the discoveredjwks_uri/authorization_endpoint/token_endpoint/end_session_endpoint, and refresh on TTL +kidcache miss. No code path constructs IdP-specific URLs. The 2026-05-09 external review surfaced that craig’s existingcrates/craig-auth/src/jwks.rs:81andservices/craig-web/src/auth.rs:198violated this — they built Keycloak realm paths directly. Plan E Step 2 corrects this convergence; from that step forward, swappingkeycloakfordex/authentik/oktais a one-lineCRAIG_IDENTITY_ISSUER=…config change. -
Per-service service-identity via
client_credentials. Each craig-* service authenticates as itself using its own client_id + client_secret. The bearer JWT’sazpclaim distinguishes which service called any given endpoint; per-service permission scoping at the IdP becomes possible; secret rotation has bounded blast-radius. -
Worker on-behalf-of via
X-Craig-Actor. Worker identity propagates as an explicit actor JWT signed by craig-signing’s existing keypair (distinctaud: craig-internal-actornamespace from partner JWS signing). Two JWKS validated per request when actor is present: craig-identity’s (bearer) and craig-signing’s (actor). Audit log records both:actor_service(which craig service made the call) +actor_user_sub(which worker is the principal).
Plan E (Plan E: craig-identity Contract + Service Identity + On-Behalf-Of) implements this in 15 steps. Plan E § D1 carries the contract spec, § D1.5 the OIDC discovery client design, § D2 the wire-shape examples, § D3 the trust topology, § D4–D10 the implementation details.
Consequences
Positive
-
Single integration point per service. Every craig-* service depends on one URL (
CRAIG_IDENTITY_ISSUER). No per-service operator-IdP integration. No per-service per-IdP claim-mapper hand-coding. -
Genuine backend pluggability. Services don’t know the backend type because they only consume OIDC discovery + JWKS + standard JWT claims. Keycloak, Dex, Authentik, Okta, Entra, ForgeRock, or a custom Rust binary all work as long as
xtask identity verifypasses. -
No new service container. Zero net deployment-surface increase. craig-auth (existing crate) gains client-side helpers; everything else is reference templates and a verifier.
-
Audit log gains caller-service.
audit_log.actor_service = claims.service_id()(always present on service tokens).actor_user_sub = claims.actor.as_ref().map(|a| &a.sub)(when X-Craig-Actor is present). -
Service token lifetime decoupled from worker session. Service tokens refresh on the service’s schedule. Long-running flows don’t fail mid-flight on session expiry.
-
KeycloakServiceTokenretired. The IdP-coupled type name + ROPC grant goes away. Replaced byOidcServiceTokenincrates/craig-auth.
Negative
-
Two JWKS to validate per request when X-Craig-Actor is present. Bearer validates against craig-identity JWKS; actor validates against craig-signing JWKS. Mitigated by caching: craig-auth caches verified JWTs for 30 s, with separate cache hits for bearer and actor tokens.
-
Operators must configure 9 service-account clients in their IAM backend. This is real work, but it’s a one-time per-stack cost (craig is per-jurisdiction, not multi-tenant SaaS) and
xtask identity renderprovides templates for the supported backends. -
Reference templates can drift from craig’s expectations. Mitigated by
xtask identity verify— operators run it post-provisioning to confirm their backend matches craig’s contract. -
Secrets rotation is the deployer’s job. craig-side caching survives a rotation event up to the cached service-token TTL (1h default); after that, services need a fresh client_credentials grant. Operators document their rotation procedure; craig doesn’t ship rotation tooling.
Mitigations
-
JWKS staleness: OIDC discovery + JWKS endpoints publish with sensible cache headers; craig-auth refreshes on
kidcache miss. -
Per-stack trust isolation: Each craig stack uses its own craig-identity issuer URL. Tokens from stack A don’t validate against stack B because the JWKS keys differ.
-
Soft-cutover continues to accept worker JWTs at internal endpoints — direct CLI/integration test callers don’t break. Future hardening (rejecting worker JWTs at internal-only endpoints) is a follow-up beyond Plan E.
Migration: soft-cutover (pre-1.0)
Three implementation phases per Plan E:
-
Phase A — Foundations (Steps 2–7):
ClaimsAPI extensions;OidcServiceToken(replacesKeycloakServiceToken);ActorTokenIssuer; auth middleware actor extraction; outbound helpers; bootstrap wiring;xtask identity verify. Worker JWTs still accepted at every endpoint. -
Phase B — Outbound flip (Steps 8–9): craig-web’s 6 client call sites + craig-intake’s flip from JWT pass-through to
with_service_identity+with_actor. End-to-end Playwright validates the BFF flow. -
Phase C — Audit + tooling (Steps 10–13): audit-log enrichment with
actor_service+actor_user_subcolumns;xtask identity renderfor keycloak/authentik/dex; devstack realm.json + secrets updates.
Worker-facing entry points (craig-web inbound, craig-intake partner-intake) continue to accept worker JWTs validated against craig-identity’s JWKS — same JWKS, different aud. That’s the steady state.
Out of scope
-
mTLS between services. Could layer on top of this design. Not a substitute.
-
Token-binding (RFC 8473). Not necessary at craig’s threat model.
-
Per-service token audience. Single
aud: craig-internal-serviceis sufficient. -
Replacing craig-signing. craig-signing keeps its existing role (partner JWS per ADR-018) plus a small extension (signed actor JWTs, distinct
audnamespace). -
Hard cutover (rejecting worker JWTs at internal endpoints). craig’s call graph is dual-purpose (BFF + direct CLI/API); soft-cutover is the right balance.
-
Service catalog, multi-tenancy, projects/domains, quotas, endpoint registry. OpenStack Keystone provides these; craig doesn’t need them.
-
Production provisioning tooling. craig ships dev provisioning + reference templates + a conformance test. Production lifecycle is the deployer’s responsibility.
Amendments
#1329 — the multi-hop relay-attribution contract (2026-08-09)
The Decision above is silent on what happens when a service that RECEIVED
an on-behalf-of pair must call a further service on the same logical
operation (BFF → craig-cases → craig-rules). The as-built contract,
implemented by #1060/!1270 (services/craig-cases/src/relay_auth.rs) and
codified here:
-
Verbatim forward, not re-mint. A relaying service forwards the inbound
Authorizationbearer ANDX-Craig-Actorheader UNCHANGED, because it is relaying someone else’s on-behalf-of context, not asserting its own. The actor JWT’s fleet-wideaud: craig-internal-actor(no per-service audience — see "Out of scope" above) is what makes the extra hop valid: the receiver verifies the actor JWT against the deployment-wide peer-JWKS registry keyed by the JWT’s OWN(iss, kid), independent of which service’s bearer carried it. -
Attribution semantics at hop N.
audit_log.actor_servicenames the FIRST hop’s bearer service (the relay forwards that bearer too) andrule_evaluations.evaluated_bynames the acting worker (claims.acting_worker().sub— the unit-pinnedevaluation_attributionhelper in craig-rules). The relay chain itself is not recorded; chain visibility (RFC 8693-styleact_forchains) is a deliberate non-goal today, tracked under the #1481 enforcement tracker (see the #1377 amendment below). -
No bearer↔actor issuer binding — accepted boundary. The middleware verifies the pair as two independent checks; nothing requires the actor JWT’s
issto equal the bearer’sservice_id(). Verbatim forwarding keeps pairs coherent in practice (the relay forwards both halves of the SAME inbound pair), and production services hold only their own signing key. Decided 2026-08-16: this is an ACCEPTED boundary, per the #1377 amendment below; the enforcement question (enforceiss == service_id, or add chain visibility instead) is tracked as #1481. -
TTL bounds the chain. The 10-minute actor-JWT TTL means verbatim forwarding only suits SYNCHRONOUS relay chains. Deferred work (outbox / inbox consumers) uses
RelayAuth::service— the service’s own bearer, NO actor by construction — and its audit rows correctly name the machine (the conversion auto-link consumer is the reference case). -
Caching boundaries stay service-context. A relay whose response feeds a CROSS-REQUEST cache must NOT forward per-worker credentials (one worker’s authz outcome would be served to another): craig-cases' screening-policy ruleset fetch (a
rule_setREAD writing norule_evaluationsrow) deliberately sends only the service bearer, per the hazard note inrelay_auth.rs. Converting such a site requires keying the cache by acting identity first. -
Fixture posture. The
rule_evaluationauthz fixtures (GA/TX 1.2.0) narrow the bare-servicecreaterow toclaims.service_id == 'craig-cases'— the relay’s system leg is the only legitimate bare-service evaluator (#1107 per-principal precedent). Therule_setread/list service rows stay UN-narrowed: they serve the fleet-wide authz cache-warm every service performs with its own bare token, and a principal IN-list there would need editing on every new service — a maintenance hazard with no attribution stake (reads write no audit evaluation rows).
As-built correction, same amendment: actor JWTs are signed with each
service’s OWN per-service ES256 key (CRAIG_<SVC>SIGNING_JWK /
SIGNING_KID, verified via the deployment-wide CRAIG_PEER_JWKS_JSON
registry) — the Decision’s "signed by craig-signing’s existing keypair"
wording describes the original design; the 2026-05-13 revision note above
already reflects the per-service model, and this amendment is that
correction’s permanent home.
#1377 — the actor-iss↔bearer binding: accepted boundary (2026-08-16)
The #1329 amendment above named the open question; this amendment closes it. Decision: the missing bearer↔actor issuer binding is an ACCEPTED boundary of the on-behalf-of design — recorded here with its threat model, not hardened today. Enforcement (or chain visibility instead) is the real work item #1481, blocked until the first multi-tenant or partner-adjacent deployment gives it a concrete threat to answer.
-
The gap, precisely. The middleware verifies the bearer and the actor JWT as two INDEPENDENT checks. The actor path is gated only on the bearer being a service principal (
!claims.is_service()skips it —crates/craig-auth/src/middleware.rs:215);verify_actor_tokennever receives the bearer’s claims (middleware.rs:218); the verified actor is lifted unconditionally (middleware.rs:225). Verification resolves the key by the actor JWT’s OWN(iss, kid)against the peer registry (crates/craig-auth/src/actor_verifier.rs:239-244), and the issuer check is self-referential —set_issuerreceives the sameissjust extracted from the token (actor_verifier.rs:250-251; the audience is the fleet-widecraig-internal-actor). Nothing binds the actor JWT’sissto the bearer’sservice_id(): a (craig-cases bearer, craig-web actor) pair verifies today. The characterization testcross_issuer_actor_pair_verifies_today_the_adr_028_accepted_boundary(inmiddleware.rs) pins this so the behavior stays visible, never accidental. -
Threat accepted, explicitly. An attacker holding a compromised low-privilege service’s bearer credential plus ANY registered peer’s actor JWT — captured in transit or coaxed, usable for its ≤10-minute TTL — evaluates downstream as the asserted worker. And because
audit_log.actor_servicenames the first-hop bearer service only (the #1329 attribution contract above), the relay chain such a pair claims to represent is invisible in audit: forensics cannot distinguish the legitimate relay from the mixed pair. -
Compensating controls (why this is tolerable today). (1) Production services hold ONLY their own ES256 signing key (
CRAIG_<SVC>__SIGNING_JWK) — minting another service’s actor JWT requires that service’s private key, so the mixed pair needs a CAPTURED token, not a minted one; the devstack’s everyone-has-everything key layout is dev-only. (2) The actor-JWT TTL ceiling is enforced server-side at 600 s + 30 s clock skew (actor_verifier.rs:271-275), bounding the replay window. (3) TheRelayAuthverbatim-forward practice (services/craig-cases/src/relay_auth.rs:27-34) forwards both halves of the SAME inbound pair, so legitimate traffic keeps pairs coherent — a mixed pair is anomalous, never routine. -
Revisit trigger. #1481 is the standing enforcement tracker (bind
iss == service_id(), or add chain visibility instead — alternatives, not a bundle). It is BLOCKED until the first multi-tenant or partner-adjacent deployment: today every service inside a stack’s trust boundary is first-party (one jurisdiction, one operator), and the binding would add a fail-closed coupling with no present attacker it defeats. -
act_for, honestly. The minted claim"act_for": service_name(crates/craig-auth/src/actor_token.rs:143) is a SCALAR duplicate ofiss: it is not present onClaims, is silently dropped at decode (actor_verifier.rs:255), and has ZERO consumers repo-wide. It is NOT an RFC 8693actchain and must not be read as chain evidence; an actual chain (each relay hop appending itself) is future design under #1481.
References
-
ADR-011 — IAM abstraction (worker-auth contract)
-
ADR-018 — Partner JWS uploads (existing craig-signing usage)
-
ADR-021 — JWT validation policy (per-service
audenforcement) -
ADR-026 — IdP-neutral identity layer (worker-side)
-
ADR-027 — Architectural principles (pluggability, configuration-as-data)
-
canopy ADR-019 (Service Identity and On-Behalf-Of) — sibling project’s parallel ADR; this ADR derives from it
-
RFC 8693 — OAuth 2.0 Token Exchange (semantic inspiration for X-Craig-Actor)