Object Storage Adoption for Placement & Reporting
On this page
Status
| Step | Description | Status |
|---|---|---|
1 |
Database migration for placement documents table |
Done (pre-ADR-030) |
2 |
craig-placement document upload/download/list/delete endpoints |
Done (pre-ADR-030) |
3 |
craig-reporting AFCARS export generation and download endpoints |
Done (pre-ADR-030) |
4 |
craig-store bucket initialization for new buckets |
Done (Store init added to both services; local backend auto-creates dirs) |
5 |
Seed sample files for testing |
Not started |
6 |
Integration tests for all new endpoints |
Done (pre-ADR-030) |
Epic: &TBD
Issues: #TBD
Branch: feature/object-storage-adoption
Context
craig-placement needs photo and document upload capabilities for foster home management — inspection photos, license documents, and training certificates. craig-reporting needs to generate and store AFCARS/NCANDS flat files for federal submission. The craig-store crate already provides a thin wrapper around the object_store crate (v0.13) with put/get/delete/list operations, upload validation, and filename sanitization. craig-cases already uses this pattern for contact attachments (services/craig-cases/src/api/contact_attachments.rs), so this plan extends the same approach to placement and reporting.
The craig-store::Store supports two backends: Local (filesystem, used in tests) and S3 (Garage in devstack, any S3-compatible in production). Configuration is via CRAIG_<SERVICE>STORE* env vars.
Scope
In scope:
-
Document upload/download/list/delete endpoints on craig-placement for foster home documents
-
AFCARS flat file export generation and download on craig-reporting
-
Database migration for a
home_documentstable on craig-placement -
craig-store bucket initialization for
placement-documentsandreporting-exportspaths -
Seed sample files (test PDF, test image) for E2E testing
-
Integration tests for all new endpoints
Out of scope:
-
Virus scanning (future hardening task)
-
Thumbnail generation for images
-
Document versioning
-
Bulk download (ZIP archives)
Design
New API Endpoints
craig-placement
| Method | Path | Description |
|---|---|---|
POST |
|
Upload a document (multipart/form-data) |
GET |
|
List documents for a foster home |
GET |
|
Download a specific document |
DELETE |
|
Soft-delete a document |
Database Schema
New table on craig_placement database:
-- Migration: YYYYMMDDHHMMSS_add_home_documents.sql
CREATE TABLE home_documents (
id UUID PRIMARY KEY DEFAULT uuidv7(),
home_id UUID NOT NULL REFERENCES foster_homes(id),
filename VARCHAR(255) NOT NULL,
content_type VARCHAR(127) NOT NULL,
size_bytes BIGINT NOT NULL,
object_key VARCHAR(512) NOT NULL,
document_type VARCHAR(50) NOT NULL, -- 'inspection_photo', 'license', 'training_certificate', 'other'
uploaded_by UUID NOT NULL,
description TEXT,
active BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_home_documents_home_id ON home_documents(home_id) WHERE active = TRUE;
No new table needed for craig-reporting — the existing afcars_submissions table gains an export_object_key column:
-- Migration: YYYYMMDDHHMMSS_add_afcars_export_key.sql
ALTER TABLE afcars_submissions
ADD COLUMN export_object_key VARCHAR(512),
ADD COLUMN export_generated_at TIMESTAMPTZ;
Object Store Path Convention
Follow the pattern from craig-cases contact attachments:
-
Placement documents:
placement-documents/{home_id}/{doc_id}/{sanitized_filename} -
Reporting exports:
reporting-exports/afcars/{submission_id}/{filename}
Upload Handling Pattern
Follow the existing pattern from services/craig-cases/src/api/contact_attachments.rs:
// Pattern: multipart extraction → validation → store → DB record
pub async fn upload_document(
State(app): State<AppState>,
Extension(claims): Extension<Claims>,
Extension(object_store): Extension<Store>,
Path(home_id): Path<Uuid>,
mut multipart: axum::extract::Multipart,
) -> Result<Json<HomeDocument>, ApiError> {
claims.require_caseworker_or_above()?;
// Verify home exists
// Extract multipart fields (file, document_type, description)
// Validate upload (size, MIME type) using craig_store::validate_upload()
// Sanitize filename using craig_store::sanitize_filename()
// Build object_key: "placement-documents/{home_id}/{doc_id}/{filename}"
// Put to object store
// Insert DB record
// Return HomeDocument
}
Download Pattern
pub async fn download_document(
State(app): State<AppState>,
Extension(claims): Extension<Claims>,
Extension(object_store): Extension<Store>,
Path((home_id, doc_id)): Path<(Uuid, Uuid)>,
) -> Result<impl IntoResponse, ApiError> {
claims.require_caseworker_or_above()?;
// Look up document record from DB
// Verify it belongs to home_id
// Get bytes from object store
// Return with Content-Type and Content-Disposition headers
let headers = [
(header::CONTENT_TYPE, doc.content_type.clone()),
(header::CONTENT_DISPOSITION, format!("attachment; filename=\"{}\"", doc.filename)),
];
Ok((headers, Body::from(bytes)))
}
Steps
Step 1: Database Migration for Placement Documents
Files: services/craig-placement/migrations/YYYYMMDDHHMMSS_add_home_documents.sql
Create the home_documents table as specified in the Design section. Check existing migration timestamps to avoid collisions. Follow the pattern of other CRAIG migrations (UUID v7 default, active column, timestamps).
Step 2: craig-placement Document Endpoints
Files: services/craig-placement/src/api/home_documents.rs (NEW), services/craig-placement/src/api/mod.rs, services/craig-placement/src/store/home_documents.rs (NEW), services/craig-placement/src/store/models.rs, services/craig-placement/src/store/mod.rs, services/craig-placement/src/main.rs
Create the store layer:
services/craig-placement/src/store/models.rs — add HomeDocument struct:
#[derive(Debug, Clone, Serialize, Deserialize, sqlx::FromRow, utoipa::ToSchema)]
pub struct HomeDocument {
pub id: Uuid,
pub home_id: Uuid,
pub filename: String,
pub content_type: String,
pub size_bytes: i64,
pub object_key: String,
pub document_type: String,
pub uploaded_by: Uuid,
pub description: Option<String>,
pub active: bool,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
services/craig-placement/src/store/home_documents.rs — CRUD functions:
-
insert_document(pool, doc) → Result<HomeDocument> -
list_documents(pool, home_id) → Result<Vec<HomeDocument>>(WHERE active = TRUE) -
get_document(pool, doc_id) → Result<Option<HomeDocument>> -
delete_document(pool, doc_id) → Result<()>(SET active = FALSE)
Create the API layer following the pattern in services/craig-cases/src/api/contact_attachments.rs:
-
upload_document— POST multipart handler withcraig_store::validate_upload()andcraig_store::sanitize_filename() -
list_documents— GET returningJson<Vec<HomeDocument>> -
download_document— GET returning binary stream with Content-Type/Content-Disposition -
delete_document— DELETE soft-deleting DB record and removing from object store
Wire routes in services/craig-placement/src/api/mod.rs:
.route("/homes/:home_id/documents", post(home_documents::upload_document).get(home_documents::list_documents))
.route("/homes/:home_id/documents/:doc_id/download", get(home_documents::download_document))
.route("/homes/:home_id/documents/:doc_id", delete(home_documents::delete_document))
Add craig-store dependency to services/craig-placement/Cargo.toml. Initialize Store in main.rs and add as Extension.
Step 3: craig-reporting AFCARS Export Endpoints
Files: services/craig-reporting/migrations/YYYYMMDDHHMMSS_add_afcars_export_key.sql, services/craig-reporting/src/api/afcars.rs, services/craig-reporting/src/store/models.rs
Add the migration for export_object_key and export_generated_at columns.
Add two handlers to the existing AFCARS API module:
generate_export — POST /v1/reporting/afcars/{submission_id}/export:
-
Verify submission exists and is in
submittedstate -
Generate AFCARS flat file content (tab-delimited text per federal spec)
-
Store in object store at
reporting-exports/afcars/{submission_id}/afcars-{date}.txt -
Update
export_object_keyandexport_generated_aton the submission record -
Return updated submission
download_export — GET /v1/reporting/afcars/{submission_id}/export/download:
-
Look up submission, verify
export_object_keyis set -
Get bytes from object store
-
Return with
Content-Type: text/plainandContent-Disposition: attachment
Add craig-store dependency to services/craig-reporting/Cargo.toml if not already present. Initialize Store in main.rs.
Step 4: craig-store Bucket Initialization
Files: crates/craig-store/src/config.rs, docker-compose.yml, devstack/garage/ (if bucket creation scripts exist)
Ensure the S3 buckets (or path prefixes for local backend) are initialized:
-
For local backend:
Store::from_config()already callscreate_dir_all— no changes needed -
For S3 (Garage in devstack): add bucket creation for
placement-documentsandreporting-exportsto the Garage init script (or use path-style access within an existing bucket)
Add store configuration to docker-compose.yml environment blocks for craig-placement and craig-reporting:
# craig-placement
CRAIG_PLACEMENT__STORE__BACKEND: s3
CRAIG_PLACEMENT__STORE__BUCKET: craig-placement
CRAIG_PLACEMENT__STORE__S3_ENDPOINT: http://garage:3900
CRAIG_PLACEMENT__STORE__S3_REGION: garage
CRAIG_PLACEMENT__STORE__S3_ACCESS_KEY: ${GARAGE_ACCESS_KEY}
CRAIG_PLACEMENT__STORE__S3_SECRET_KEY: ${GARAGE_SECRET_KEY}
CRAIG_PLACEMENT__STORE__MAX_UPLOAD_BYTES: 10485760
# craig-reporting (same pattern, different bucket)
CRAIG_REPORTING__STORE__BUCKET: craig-reporting
Step 5: Seed Sample Files
Files: tests/fixtures/sample.pdf, tests/fixtures/sample.png, xtask/src/seed.rs (or seed command)
Create minimal test fixtures:
-
tests/fixtures/sample.pdf— 1-page PDF with "Test Document" text (generate with a simple PDF library or commit a pre-built minimal file) -
tests/fixtures/sample.png— 100x100 solid blue PNG (generate programmatically or commit a pre-built file)
Update the seed process (cargo xtask seed) to upload these fixtures:
-
After SQL seeding completes, upload
sample.pdfto the placement-documents path for a seeded foster home -
Upload
sample.pdfto the reporting-exports path as a sample AFCARS export -
Insert corresponding DB records in
home_documentsand updateexport_object_keyon a seeded AFCARS submission
Step 6: Integration Tests
Files: services/craig-placement/tests/home_documents.rs (NEW), services/craig-reporting/tests/afcars_export.rs (NEW)
Follow existing integration test patterns using TestHarness:
services/craig-placement/tests/home_documents.rs:
-
Upload a document to a foster home (multipart POST)
-
List documents — verify uploaded document appears
-
Download document — verify content matches uploaded data
-
Delete document — verify soft-deleted (GET still returns, LIST excludes)
-
Upload with invalid MIME type — verify 400 error
-
Upload exceeding size limit — verify 400 error
services/craig-reporting/tests/afcars_export.rs:
-
Create AFCARS submission, generate export — verify 200 and
export_object_keyset -
Download export — verify file content is valid flat file format
-
Download before generation — verify 404 or appropriate error
Each test file should have 4-6 test functions minimum. Use reqwest::multipart::Form and reqwest::multipart::Part for multipart uploads in tests (same pattern as services/craig-cases/tests/contact_attachments.rs).
Files Touched
| File | Change |
|---|---|
|
NEW: home_documents table |
|
NEW: upload, list, download, delete handlers |
|
Wire new routes |
|
NEW: CRUD queries |
|
Add HomeDocument struct |
|
Add home_documents module |
|
Initialize Store, add Extension |
|
Add craig-store dependency |
|
NEW: add export columns |
|
Add generate_export, download_export handlers |
|
Update AfcarsSubmission model |
|
Initialize Store, add Extension |
|
Add craig-store dependency (if needed) |
|
Add store config env vars for placement and reporting |
|
NEW: test PDF fixture |
|
NEW: test PNG fixture |
|
Upload seed files to object store |
|
NEW: integration tests |
|
NEW: integration tests |
Verification
-
cargo nextest run --workspace --lib— unit tests pass -
cargo xtask dev reload— services rebuild with new migrations -
cargo nextest run -p craig-placement— placement integration tests pass (including new document tests) -
cargo nextest run -p craig-reporting— reporting integration tests pass (including new export tests) -
cargo xtask seed— seed files uploaded without errors -
Manual verification: upload a file via curl, download it, verify content matches
Documentation Updates
-
.claude/docs/services.md— add new endpoints to craig-placement and craig-reporting tables -
CHANGELOG.adoc— entry under== Unreleased -
docs/modules/ROOT/pages/data-model-placement.adoc— add home_documents table -
docs/modules/ROOT/pages/data-model-reporting.adoc— note export columns on afcars_submissions