CAPTCHA Integration Testing

On this page

Status

Step Description Status

1

Save plan document and link in nav.adoc

Done (pre-ADR-030)

2

Configure devstack with Turnstile test keys

Not started

3

Add CaptchaVerifier unit tests with wiremock

Done (pre-ADR-030)

4

Add integration tests for CAPTCHA verification endpoint

Done (pre-ADR-030)

5

Add E2E tests for CAPTCHA-enabled form submission

Not started

6

Verification and documentation updates

Not started

Branch: feature/placement-picker

Context

The public intake form (/report) uses Cloudflare Turnstile for CAPTCHA verification. The CaptchaVerifier in services/craig-intake/src/api/captcha.rs handles token validation by posting to Cloudflare’s siteverify endpoint.

Currently, CAPTCHA is disabled in the devstack and test environment by setting secret = "disabled", which causes the verifier to skip validation entirely. This means:

  • No tests verify that CAPTCHA actually works when enabled

  • No tests verify that invalid tokens are rejected

  • The HTTP verification code path (posting to Cloudflare) has zero test coverage

  • A regression in CAPTCHA handling would not be caught until production

The existing unit tests (3 tests in captcha.rs) cover only the disabled-mode bypass and input validation (missing/empty token). The HTTP round-trip paths (success, failure) are untested.

Cloudflare provides official test site keys that always pass or always fail, enabling deterministic CAPTCHA testing without real browser challenges.

Current Implementation

// services/craig-intake/src/api/captcha.rs
pub struct CaptchaVerifier {
    enabled: bool,           // false when secret == "disabled"
    secret: String,
    verify_url: String,      // Cloudflare siteverify URL
    client: reqwest::Client,
}

impl CaptchaVerifier {
    pub async fn verify(&self, token: Option<&str>) -> Result<(), String> {
        if !self.enabled { return Ok(()); }           // Path 1: disabled mode
        let token = token.ok_or("captcha_token is required")?; // Path 2: missing
        if token.is_empty() { return Err(...); }      // Path 3: empty
        // Path 4: HTTP request failure
        // Path 5: Cloudflare returns success: false
        // Path 6: Cloudflare returns success: true (happy path)
    }
}

The form submission handler in services/craig-intake/src/api/public.rs calls captcha.verify(body.captcha_token.as_deref()) before processing the report.

Cloudflare Turnstile Test Keys

Cloudflare provides deterministic test credentials (documented at https://developers.cloudflare.com/turnstile/troubleshooting/testing/):

Purpose Values

Always passes

Site key: 1x00000000000000000000AA
Secret key: 1x0000000000000000000000000000000AA

Always fails

Site key: 2x00000000000000000000AB
Secret key: 2x0000000000000000000000000000000AB

Forces interactive challenge

Site key: 3x00000000000000000000FF
Secret key: (use always-pass secret)

These keys work against the real Cloudflare siteverify endpoint (https://challenges.cloudflare.com/turnstile/v0/siteverify), so no mock server is needed for the HTTP round-trip tests.

Scope

In scope:

  • Unit tests for CaptchaVerifier HTTP paths using wiremock mock server

  • Integration tests with Turnstile test keys against real Cloudflare endpoint

  • E2E tests with CAPTCHA widget rendered in the browser

  • Devstack configuration for test-mode CAPTCHA

Out of scope:

  • Testing interactive CAPTCHA challenges (requires real browser interaction with Cloudflare widget)

  • Load testing CAPTCHA verification

  • CAPTCHA analytics or metrics

Design

Test Strategy

Three testing tiers:

  1. Unit tests (wiremock) — test CaptchaVerifier in isolation with a mock HTTP server. Cover all 6 code paths including HTTP failure and JSON parse error. No network dependency.

  2. Integration tests (Turnstile test keys) — test the full intake API with CAPTCHA enabled, using Cloudflare’s always-pass and always-fail test secrets against the real siteverify endpoint. Requires internet access.

  3. E2E tests (Playwright) — test the web form with the Turnstile widget rendered, using the always-pass test site key. Verifies the full client-to-server CAPTCHA flow.

Configuration Approach

The devstack currently sets CRAIG_INTAKE__CAPTCHA_SECRET=disabled. For CAPTCHA-enabled testing:

  • Add a second docker-compose profile or override file for CAPTCHA-enabled mode

  • Or: integration tests configure their own CaptchaVerifier instance with test keys (no devstack change needed)

  • E2E tests need the web form to render the Turnstile widget — requires the test site key in craig-web config

Steps

Step 1: Save Plan and Link in nav.adoc

Files: docs/modules/ROOT/pages/plans/captcha-integration-testing.adoc, docs/modules/ROOT/nav.adoc

Create this plan file and add nav entry under Planned.

Step 2: Configure Devstack with Turnstile Test Keys

Files: docker-compose.yml (or equivalent xtask dev config), services/craig-intake/src/config.rs, services/craig-web/src/config.rs

Add optional CAPTCHA test configuration. The default devstack remains CAPTCHA-disabled for fast development. A separate profile enables it:

Option A — environment variable override:

# Run devstack with CAPTCHA enabled (test keys):
CRAIG_INTAKE__CAPTCHA_SECRET=1x0000000000000000000000000000000AA \
CRAIG_INTAKE__CAPTCHA_VERIFY_URL=https://challenges.cloudflare.com/turnstile/v0/siteverify \
CRAIG_WEB__TURNSTILE_SITE_KEY=1x00000000000000000000AA \
cargo xtask dev reload

Option B — docker-compose override file docker-compose.captcha.yml:

services:
  craig-intake:
    environment:
      CRAIG_INTAKE__CAPTCHA_SECRET: "1x0000000000000000000000000000000AA"
      CRAIG_INTAKE__CAPTCHA_VERIFY_URL: "https://challenges.cloudflare.com/turnstile/v0/siteverify"
  craig-web:
    environment:
      CRAIG_WEB__TURNSTILE_SITE_KEY: "1x00000000000000000000AA"

Verify that services/craig-web/templates/report/form.html (or equivalent) conditionally renders the Turnstile widget when the site key is configured. If the template always renders the widget, ensure it handles an empty/missing site key gracefully.

Step 3: Add CaptchaVerifier Unit Tests with wiremock

Files: services/craig-intake/src/api/captcha.rs, services/craig-intake/Cargo.toml

Add wiremock as a dev dependency:

[dev-dependencies]
wiremock = "0.6"

Add unit tests covering the HTTP round-trip paths that the existing tests skip:

// Add to existing #[cfg(test)] mod tests in captcha.rs:

use wiremock::{Mock, MockServer, ResponseTemplate};
use wiremock::matchers::{method, path};

async fn mock_captcha_verifier(mock_url: &str, secret: &str) -> CaptchaVerifier {
    CaptchaVerifier::new(secret.into(), mock_url.into(), reqwest::Client::new())
}

#[tokio::test]
async fn enabled_captcha_accepts_valid_token() {
    // Start a mock server that returns {"success": true}
    let server = MockServer::start().await;
    Mock::given(method("POST"))
        .and(path("/"))
        .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"success": true})))
        .mount(&server)
        .await;

    let verifier = mock_captcha_verifier(&server.uri(), "test-secret").await;
    assert!(verifier.verify(Some("valid-token")).await.is_ok());
}

#[tokio::test]
async fn enabled_captcha_rejects_failed_verification() {
    // Mock returns {"success": false}
    let server = MockServer::start().await;
    Mock::given(method("POST"))
        .and(path("/"))
        .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"success": false})))
        .mount(&server)
        .await;

    let verifier = mock_captcha_verifier(&server.uri(), "test-secret").await;
    let err = verifier.verify(Some("bad-token")).await.unwrap_err();
    assert!(err.contains("CAPTCHA verification failed"), "got: {err}");
}

#[tokio::test]
async fn enabled_captcha_handles_http_failure() {
    // Mock server that returns 500
    let server = MockServer::start().await;
    Mock::given(method("POST"))
        .and(path("/"))
        .respond_with(ResponseTemplate::new(500))
        .mount(&server)
        .await;

    let verifier = mock_captcha_verifier(&server.uri(), "test-secret").await;
    let err = verifier.verify(Some("any-token")).await.unwrap_err();
    assert!(err.contains("CAPTCHA") || err.contains("parse"), "got: {err}");
}

#[tokio::test]
async fn enabled_captcha_handles_malformed_json_response() {
    // Mock returns invalid JSON body
    let server = MockServer::start().await;
    Mock::given(method("POST"))
        .and(path("/"))
        .respond_with(ResponseTemplate::new(200).set_body_string("not json"))
        .mount(&server)
        .await;

    let verifier = mock_captcha_verifier(&server.uri(), "test-secret").await;
    let err = verifier.verify(Some("any-token")).await.unwrap_err();
    assert!(err.contains("parse"), "expected parse error, got: {err}");
}

#[tokio::test]
async fn enabled_captcha_sends_correct_form_fields() {
    // Verify the verifier sends secret and response fields
    let server = MockServer::start().await;
    Mock::given(method("POST"))
        .and(path("/"))
        .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"success": true})))
        .expect(1)  // Exactly one request
        .mount(&server)
        .await;

    let verifier = mock_captcha_verifier(&server.uri(), "my-secret-key").await;
    verifier.verify(Some("user-token")).await.unwrap();

    // wiremock automatically verifies the expected call count on drop
    // For field validation, use wiremock body matchers or inspect received requests:
    let received = server.received_requests().await.unwrap();
    let body = String::from_utf8(received[0].body.clone()).unwrap();
    assert!(body.contains("secret=my-secret-key"), "body: {body}");
    assert!(body.contains("response=user-token"), "body: {body}");
}

This adds 5 new unit tests, bringing the captcha.rs total from 3 to 8.

Step 4: Add Integration Tests with Real Turnstile Test Keys

Files: services/craig-intake/tests/api/captcha.rs (new file), services/craig-intake/tests/api/mod.rs

These tests use Cloudflare’s official test keys against the real siteverify endpoint. They require internet access and a running devstack.

// services/craig-intake/tests/api/captcha.rs

use craig_test_lib::{TestHarness, devstack_available};
use serde_json::json;

const ALWAYS_PASS_SECRET: &str = "1x0000000000000000000000000000000AA";
const ALWAYS_FAIL_SECRET: &str = "2x0000000000000000000000000000000AB";
const TURNSTILE_VERIFY_URL: &str = "https://challenges.cloudflare.com/turnstile/v0/siteverify";

#[tokio::test]
async fn turnstile_test_key_always_pass() {
    // Verify that Cloudflare's always-pass test key works
    let client = reqwest::Client::new();
    let resp = client
        .post(TURNSTILE_VERIFY_URL)
        .form(&[
            ("secret", ALWAYS_PASS_SECRET),
            ("response", "any-token-value"),
        ])
        .send()
        .await
        .expect("failed to reach Cloudflare");
    let body: serde_json::Value = resp.json().await.unwrap();
    assert_eq!(body["success"], true, "always-pass key should succeed: {body}");
}

#[tokio::test]
async fn turnstile_test_key_always_fail() {
    let client = reqwest::Client::new();
    let resp = client
        .post(TURNSTILE_VERIFY_URL)
        .form(&[
            ("secret", ALWAYS_FAIL_SECRET),
            ("response", "any-token-value"),
        ])
        .send()
        .await
        .expect("failed to reach Cloudflare");
    let body: serde_json::Value = resp.json().await.unwrap();
    assert_eq!(body["success"], false, "always-fail key should fail: {body}");
}

#[tokio::test]
async fn captcha_verifier_with_always_pass_key() {
    use craig_intake::api::captcha::CaptchaVerifier;

    let verifier = CaptchaVerifier::new(
        ALWAYS_PASS_SECRET.into(),
        TURNSTILE_VERIFY_URL.into(),
        reqwest::Client::new(),
    );
    assert!(verifier.verify(Some("test-token")).await.is_ok());
}

#[tokio::test]
async fn captcha_verifier_with_always_fail_key() {
    use craig_intake::api::captcha::CaptchaVerifier;

    let verifier = CaptchaVerifier::new(
        ALWAYS_FAIL_SECRET.into(),
        TURNSTILE_VERIFY_URL.into(),
        reqwest::Client::new(),
    );
    let err = verifier.verify(Some("test-token")).await.unwrap_err();
    assert!(err.contains("CAPTCHA verification failed"), "got: {err}");
}
These tests require CaptchaVerifier to be pub (it already is). If the captcha module is not re-exported from the crate root, add pub mod api; and pub use api::captcha; to services/craig-intake/src/lib.rs or make the module path accessible for integration tests.

Register the module in services/craig-intake/tests/api/mod.rs:

mod captcha;

Step 5: Add E2E Tests for CAPTCHA-Enabled Form Submission

Files: tests/e2e/specs/captcha.spec.ts

These tests run with the Turnstile test site key configured so the widget renders in the browser.

// tests/e2e/specs/captcha.spec.ts
import { test, expect } from '@playwright/test';

test.describe('CAPTCHA-Enabled Report Form', () => {
  // Skip if CAPTCHA is not configured in the test environment
  test.beforeEach(async ({ page }) => {
    await page.goto('/report');
    // Check if Turnstile widget is rendered
    const widget = page.locator('iframe[src*="challenges.cloudflare.com"]');
    if (!(await widget.count())) {
      test.skip(true, 'CAPTCHA not enabled in test environment');
    }
  });

  test('turnstile widget renders on report form', async ({ page }) => {
    await page.goto('/report');
    // Turnstile renders as an iframe
    const widget = page.locator('iframe[src*="challenges.cloudflare.com"]');
    await expect(widget).toBeVisible({ timeout: 10000 });
  });

  test('submit report with valid CAPTCHA token succeeds', async ({ page }) => {
    await page.goto('/report');

    // With the always-pass test site key, the widget auto-completes
    // Wait for the hidden input to be populated with a token
    await page.waitForFunction(() => {
      const input = document.querySelector('input[name="cf-turnstile-response"]') as HTMLInputElement;
      return input && input.value.length > 0;
    }, null, { timeout: 15000 });

    // Fill required fields and submit (same as public-report.spec.ts pattern)
    // Step 1: Reporter — click Next
    await page.locator('button', { hasText: 'Next' }).click();

    // Step 2: Incident
    await page.locator('#admin_unit').fill('Fulton');
    await page.locator('#concern_type').selectOption('neglect');
    await page.locator('#concern_description').fill('E2E CAPTCHA test report');
    await page.locator('button', { hasText: 'Next' }).click();

    // Steps 3-5: Skip optional steps
    await page.locator('button', { hasText: 'Next' }).click();
    await page.locator('button', { hasText: 'Next' }).click();
    await page.locator('button', { hasText: 'Next' }).click();

    // Step 6: Submit
    await page.locator('button[type="submit"]', { hasText: 'Submit Report' }).click();
    await page.waitForURL(/\/report\/confirmation\/RPT-/, { timeout: 15000 });
  });

  test('submit without CAPTCHA token is rejected', async ({ page }) => {
    // This test requires the always-fail site key configured
    // OR we can clear the turnstile response before submitting
    await page.goto('/report');

    // Fill form but clear the CAPTCHA token before submit
    await page.locator('button', { hasText: 'Next' }).click();
    await page.locator('#admin_unit').fill('Fulton');
    await page.locator('#concern_type').selectOption('neglect');
    await page.locator('#concern_description').fill('E2E CAPTCHA rejection test');
    await page.locator('button', { hasText: 'Next' }).click();
    await page.locator('button', { hasText: 'Next' }).click();
    await page.locator('button', { hasText: 'Next' }).click();
    await page.locator('button', { hasText: 'Next' }).click();

    // Clear the CAPTCHA token to simulate failure
    await page.evaluate(() => {
      const input = document.querySelector('input[name="cf-turnstile-response"]') as HTMLInputElement;
      if (input) input.value = '';
    });

    await page.locator('button[type="submit"]', { hasText: 'Submit Report' }).click();

    // Should show error, not redirect to confirmation
    await expect(page.locator('.alert-error, .error-message, [role="alert"]')).toBeVisible({ timeout: 5000 });
  });
});
E2E CAPTCHA tests are inherently environment-dependent. They should be tagged and run separately from the main E2E suite. Add a @captcha tag or use Playwright’s --grep to run them only when CAPTCHA is configured: 
# Run only CAPTCHA tests (requires CAPTCHA-enabled devstack):
npx playwright test --grep "CAPTCHA"

Step 6: Verification and Documentation Updates

  1. cargo nextest run -p craig-intake --lib — 8 unit tests pass (3 existing + 5 new wiremock tests)

  2. cargo nextest run -p craig-intake — integration tests pass (4 new Turnstile tests + existing)

  3. E2E CAPTCHA tests (when CAPTCHA-enabled devstack is running)

  4. Verify disabled-mode still works: default devstack with secret=disabled — all existing E2E tests pass unchanged

Files Touched

File Change

services/craig-intake/src/api/captcha.rs

Add 5 wiremock-based unit tests for HTTP round-trip paths

services/craig-intake/Cargo.toml

Add wiremock = "0.6" dev dependency

services/craig-intake/tests/api/captcha.rs

New file: 4 integration tests using Cloudflare Turnstile test keys

services/craig-intake/tests/api/mod.rs

Register mod captcha;

tests/e2e/specs/captcha.spec.ts

New file: 3 E2E tests for CAPTCHA-enabled form submission

docker-compose.captcha.yml (optional)

Docker-compose override for CAPTCHA-enabled devstack

Verification

  1. cargo nextest run --workspace --lib — unit tests pass (including new wiremock tests)

  2. cargo nextest run -p craig-intake — integration tests pass

  3. cargo xtask e2e — existing E2E tests pass (CAPTCHA disabled by default)

  4. With CAPTCHA-enabled devstack: npx playwright test --grep "CAPTCHA" — 3 E2E tests pass

  5. Verify Cloudflare test keys work by running integration tests (requires internet)

Documentation Updates

  • CHANGELOG.adoc — entry under == Unreleased

  • .claude/docs/testing.md — document CAPTCHA test configuration and how to run CAPTCHA-specific tests

  • .claude/docs/local-dev.md — document CAPTCHA-enabled devstack profile

Edit this page · latest