Plan: Data Breach Detection & Notification
On this page
Status
| Step | Description | Status |
|---|---|---|
1 |
Plan file and nav entry |
Done (2026-03-21) — MR !48 |
2 |
Detection rules engine (audit log analysis) |
Done (2026-03-21) — MR !48 |
3 |
Alert table and API endpoints |
Done (2026-03-21) — MR !48 |
4 |
RabbitMQ alert publishing |
Done (2026-03-21) — MR !48 |
5 |
Notification configuration (email/webhook) |
Done (2026-03-21) — MR !48 |
6 |
Web UI alert dashboard |
Done (2026-03-21) — MR !48 |
7 |
Integration tests |
Done (2026-03-21) — MR !48 |
8 |
Documentation, commit, push, MR |
Done (2026-03-21) — MR !48 |
Issues: TBD
Branch: feature/breach-detection
Context
craig-security has comprehensive audit logging — a wildcard RabbitMQ subscriber captures all events across all services into the audit_log table. However, there is zero detection logic, zero alerting, and zero notification. The system records everything but detects nothing.
Data breach detection is required by Phase 8 (roadmap.adoc:258) and is a critical gap identified in external review.
Approach
Build a detection layer on top of the existing audit infrastructure:
-
Detection rules: configurable thresholds that trigger alerts when audit patterns exceed limits
-
Alert table: persistent record of detected anomalies with severity and status
-
Alert events: RabbitMQ events for real-time notification
-
Notification config: webhook URL per alert type (email integration is out of scope — webhook to external email/Slack/PagerDuty service)
Scope
In scope:
-
Detection rules table with configurable thresholds
-
Alert table + CRUD endpoints on craig-security
-
Periodic audit log analysis (RabbitMQ scheduled check or cron-style)
-
Alert severity levels: info, warning, critical
-
Webhook notification on critical alerts
-
Web UI alert dashboard
-
CLI commands for alert management
Out of scope:
-
Direct email/SMS sending (use webhook to external notification service)
-
Machine learning anomaly detection (rule-based only)
-
Real-time stream processing (batch analysis of recent audit entries)
Design
Database
File: services/craig-security/migrations/YYYYMMDDHHMMSS_breach_detection.sql
CREATE TABLE detection_rules (
id UUID PRIMARY KEY DEFAULT uuidv7(),
rule_name TEXT NOT NULL UNIQUE,
description TEXT NOT NULL,
rule_type TEXT NOT NULL CHECK (rule_type IN (
'failed_auth', 'bulk_access', 'after_hours_access',
'privilege_escalation', 'data_export', 'account_lockout'
)),
threshold INTEGER NOT NULL, -- e.g., 5 failures
window_minutes INTEGER NOT NULL, -- e.g., within 10 minutes
severity TEXT NOT NULL CHECK (severity IN ('info', 'warning', 'critical')),
enabled BOOLEAN NOT NULL DEFAULT true,
notify_webhook TEXT, -- URL to POST alert JSON
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
active BOOLEAN NOT NULL DEFAULT true
);
CREATE TABLE security_alerts (
id UUID PRIMARY KEY DEFAULT uuidv7(),
rule_id UUID REFERENCES detection_rules(id),
rule_name TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('info', 'warning', 'critical')),
description TEXT NOT NULL,
details JSONB, -- audit entries that triggered the alert
acknowledged BOOLEAN NOT NULL DEFAULT false,
acknowledged_by TEXT,
acknowledged_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_alerts_severity ON security_alerts(severity) WHERE NOT acknowledged;
CREATE INDEX idx_alerts_created ON security_alerts(created_at DESC);
Detection Rules (Seeded Defaults)
| Rule | Description | Threshold | Window | Severity |
|---|---|---|---|---|
failed_auth_burst |
Multiple failed authentication attempts from same user |
5 failures |
10 min |
critical |
bulk_record_access |
Unusual volume of record access by single user |
100 reads |
60 min |
warning |
after_hours_access |
System access outside business hours (configurable) |
1 access |
N/A |
info |
privilege_escalation |
Unauthorized role access attempt |
3 attempts |
30 min |
critical |
bulk_data_export |
Large-scale data export or download |
50 downloads |
60 min |
warning |
account_lockout |
Repeated lockout events for same account |
3 lockouts |
24 hr |
critical |
API Endpoints
| Method | Path | Description | RBAC |
|---|---|---|---|
GET |
|
List alerts (paginated, filterable) |
admin |
GET |
|
Get alert detail |
admin |
PUT |
|
Acknowledge alert |
admin |
GET |
|
List detection rules |
admin |
POST |
|
Create rule |
admin |
PUT |
|
Update rule (enable/disable, change threshold) |
admin |
DELETE |
|
Soft-delete rule |
admin |
POST |
|
Manually trigger detection scan |
admin |
Detection Scan Logic
The detection scan queries the audit_log table for patterns matching each enabled rule:
-- Example: failed_auth_burst rule (threshold=5, window=10min)
SELECT user_id, COUNT(*) as count
FROM audit_log
WHERE action = 'auth_failed'
AND timestamp > now() - interval '10 minutes'
AND success = false
GROUP BY user_id
HAVING COUNT(*) >= 5
Each match creates a security_alert record and publishes security.alert.created to RabbitMQ.
If the rule has a notify_webhook, the alert JSON is POSTed to that URL.
Files Touched
| File | Change |
|---|---|
|
New migration (2 tables) |
|
New: CRUD for alerts |
|
New: CRUD for rules |
|
Add DetectionRule, SecurityAlert structs |
|
Add 8 endpoints |
|
Add alert event publishers |
|
New: scan logic (audit log queries) |
|
Add alerts subcommands |
|
New page |
|
Add alerts handler |
|
Seed 6 default detection rules |
Verification
-
Seed detection rules into devstack
-
Trigger a rule (e.g., 5+ failed auth attempts via invalid tokens)
-
Run detection scan manually via
POST /v1/security/detection/run -
Verify alert created with correct severity and details
-
Verify webhook notification sent (mock HTTP server)
-
Acknowledge alert via PUT, verify status change
-
All existing E2E tests pass
Documentation Updates
-
.claude/docs/services.md— add 8 endpoints, 2 tables, 2 events -
CHANGELOG.adoc— entry under Unreleased -
docs/modules/ROOT/pages/roadmap.adoc— tick breach detection item -
docs/modules/ROOT/pages/guide/admin.adoc— alert management section