How it works

Architecture & guardrails

FleetAlert AI investigates fleet-maintenance alerts end to end: Claude reasons over telemetry and a knowledge base, proposes a fix, and then waits — structurally, not just by convention — for a human to confirm before anything executes.

Try the live demo or read the source and full decision record.

The investigation loop

Visitor (browser)HTTPSReact SPA — S3 + CloudFrontalert list · trace · confirm/reject · resetRESTAPI Gateway — HTTP API7 routes: list · investigate · status · confirm ·reject · audit · resetstart_executionRunInvestigation — Agent Loop λClaude tool-calling loop+ RAG: search_knowledge_basepropose_fix (whitelisted)WaitForConfirmationStep Functions task-token callback — paused hereconfirmedExecuteFix λre-checks whitelist + token, then runsResolvedreject → RAG retry (1×)no fix / not whitelisted2nd rejectionRouted to Supportno code path from modelreasoning straight to execution

Every alert runs through Step Functions, not client-side logic. RAG happens inside the Agent Loop itself — one of the model’s own tools queries the knowledge base mid-loop, rather than a separate retriever component sitting in front of it.

The five-minute version

01

Confirmation is structurally unskippable, not just prompted

The agent loop can only ever end in “awaiting confirmation” or “routed to support” — it never executes anything itself. Step Functions pauses the whole execution on a task-token callback (ADR-0001) that only a human confirm/reject action can resolve. There is no code path from the model’s own reasoning straight to a running fix.

02

A fixed whitelist, not model judgment, decides what can run

Three fix types — restart_sensor, schedule_service_visit, send_diagnostic_reset — are the entire set of actions the agent is allowed to execute (ADR-0002). Anything else it proposes is automatically routed to a human. The check runs twice: once when the fix is proposed, again immediately before ExecuteFix runs, as defense in depth against the first check ever being bypassed.

03

Rejecting a fix triggers another RAG pass, not a dead end

A rejected proposal used to just end the run. Now it loops back into the same Agent Loop Lambda with the rejected fix_id named in the model’s own prompt, so it searches the knowledge base for a genuinely different angle instead of repeating itself — enforced in code, not just prompted, since the finalize step refuses a repeated fix_id regardless of what the model does. Capped at one retry for this demo before it gives up and routes to support.

04

No free text ever reaches the model

Visitors pick from five pre-seeded alert scenarios; nothing they type reaches Claude (ADR-0003). This is a public demo making real, paid API calls with no login — that constraint caps the worst-case cost and removes prompt injection from strangers as a category of problem, not something to detect and block.

05

What's deferred is written down, not just skipped

Backpressure between an alert feed and the investigation queue is fully designed — SQS, a dead-letter queue, a concurrency-capped dispatcher — but not built, because there's no real alert traffic yet to design it against (ADR-0006). Same treatment for CORS before the frontend's domain existed (ADR-0007) and for Claude API-level retries, since the Step Functions retry already makes failures non-silent (ADR-0008).

06

A production bug that only showed up live

Rejecting a fix would flip the alert's status and then just hang forever. The cause: a Step Functions Catch clause's default behavior replaces the entire state input with its own error object, silently wiping the alert_id before the retry Lambda ever saw it — six retries, no error surfaced, nothing written back to the database. Fixed with an explicit ResultPath — the kind of failure mode that only shows up once guardrails are actually exercised end to end, not in a unit test.

07

The model’s tool output is untrusted input

Claude occasionally called the propose_fix tool without fields its own schema marks as required — once crashing the run with a KeyError, once storing a half-empty proposal that showed up in the UI as a bare title and two buttons. Every tool call is now validated, and a malformed one goes back to the model as an error to correct, inside the same bounded loop, instead of taking the investigation down or reaching a person half-finished.

Key decisions, from the record

ADR-0001
Step Functions task-token callback. Human confirmation pauses the whole execution natively, instead of a database flag needing its own bypass-proof token.
ADR-0002
Fixed action whitelist. Three hardcoded fix types the agent may execute; everything else is escalated. Checked once at proposal, again at execution.
ADR-0003
Pre-seeded demo, no free text. A public demo with real API calls needs a hard ceiling on cost and on what a stranger can get the model to do.
ADR-0004
GitHub OIDC trust policy vs. immutable subject IDs. A real deploy failure: GitHub's newer tokens embed hidden numeric IDs the standard trust-policy pattern doesn't expect. Found in AWS's raw audit log, not the (deliberately vague) error message.
ADR-0006
Backpressure, designed but not built. SQS + DLQ + a concurrency-capped dispatcher, specified in full — deferred because there's no real alert feed yet to build it against.
ADR-0007
CORS, deferred until the domain existed. The rule needs an exact origin to allow; written down with the exact config to add once the frontend had a real domain.
ADR-0008
No bespoke Claude API retry logic (yet). The Step Functions-level retry already turns a transient failure into a visible one; a targeted fix is a one-line change away if this ever becomes the real bottleneck.
ADR-0009
Custom domain in two applies. Attaching fleetalert.10finger.dev needs a DNS-validated certificate that lives outside Terraform, so the first apply requests it and a second attaches it — rather than one apply hanging on a record nobody has added yet.

By the numbers

3
Whitelisted, agent-executable fix types
6
Max agent loop iterations before forced escalation
1
Reject round that triggers a RAG retry, before routing to support
6×
Step Functions retries (2s, doubling) before a terminal failure
5
DynamoDB tables: machines, telemetry, alerts, knowledge base, audit log
8
Architecture decision records, including the ones not built yet

Built with an AI agent

Every guardrail, deferral, and fix lives in a numbered decision record — an ADR-first workflow run with Claude Code as a pair. The agent handled breadth: the Lambda handlers, the Step Functions definition, the Terraform for all of it, CI/CD via GitHub OIDC, and diagnosing a production bug from raw AWS error output pasted back into the chat.

The judgment calls stayed with a person — where the whitelist boundary sits, what a public demo is and isn’t allowed to do, and how many times a rejected fix gets to try again before a human takes over. The result is a working guardrailed agent system, with the reasoning behind every constraint written down next to the code that enforces it.