How to Build Human-in-the-Loop AI for High-Stakes Products
A security analyst is three steps into a threat-handling workflow. They've classified the endpoint, pulled the process tree, started attributing the activity to a user — then realize step one was wrong, the device was mi
A security analyst is three steps into a threat-handling workflow. They've classified the endpoint, pulled the process tree, started attributing the activity to a user — then realize step one was wrong, the device was misidentified, and everything downstream is built on it.
There is no back button. The workflow only moves forward, so they either finish a decision they know is wrong or abandon the case and start over, losing the notes and the reasoning trail with it.
That's the actual failure mode of human-in-the-loop AI, and it has nothing to do with the model. Human-in-the-loop is a workflow-engineering problem with four components: an explicitly accountable decision, a reversible review state machine with durable history, a deterministic data path that AI only overlays, and validated ingestion with provenance on every record. Confidence scoring is the easy part.
The standard advice — route low-confidence predictions to a human, feed corrections back — is a routing rule, not a system, and it assumes the model knows when it's uncertain. In the highest-stakes system I'll cite here, humans review 100% of activity with no threshold at all. The binding constraint is reviewer throughput and reversibility, not model uncertainty.
Developers already behave this way: in the 2025 Stack Overflow Developer Survey, 46% actively distrust the accuracy of AI output versus 33% who trust it, 66% name "almost right, but not quite" their top frustration, and 75.3% go ask another person when they don't trust an AI answer. Your reviewers will do the same. Build for it.
Start from what the human is accountable for
Design backward from the decision a person signs their name to, not forward from model output.
Get three answers in writing before any architecture:
- Consequence class. Irreversible (funds moved, a device quarantined, a dose recommended) or correctable (a record flagged, a ticket routed)? Irreversible decisions need a hold state and a second reviewer; correctable ones don't, and adding one burns throughput.
- Who is liable. Not "the team" — the role that answers in an audit. That role dictates what gets logged and for how long.
- What evidence they need at the decision. Not what exists somewhere in the system: what's on screen, in the same viewport, when they click approve.
If you're shipping into the EU, Article 14 of the AI Act reads like a design spec: the overseeing person must be able to "correctly interpret the high-risk AI system's output," to "decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output," to "interrupt the system through a 'stop' button," and to "remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)." Obligations for Annex III high-risk systems start applying on 2 December 2027, Annex I systems on 2 August 2028.
Read that as four features — interpretability at the point of decision, a disregard path, an override path, a safe-halt path — each one a state transition you implement.
The limitation: this is a product-scope decision engineering can't make alone. If nobody will name the accountable role, you're not building HITL, you're building a liability sponge with a UI.
Review is a state machine, not a queue
The failure mode isn't bad predictions. It's a workflow a reviewer can't back out of.
A queue gives you one direction: pull, decide, pop. A state machine gives you explicit states, legal transitions, and a persisted record of each one with actor, timestamp and prior state — which is what makes stepping backward safe rather than destructive.
CYDEF, a Canadian cybersecurity company whose desktop Agent detects abnormalities to prevent data breaches, is the case I keep returning to, because its experts review 100% of endpoint activity — no threshold, no sampling. As published on the case page, the problems were exactly the ones above: threat-processing workflows so long that security experts had no way to revisit previous steps, slow database request processing, Agent performance issues, and hundreds of gigabytes of incoming telemetry daily.
What the Devico team shipped, per that page: a redesigned threat-processing workflow preserving historical data in PostgreSQL and AWS Blob Storage, database indexation and caching, profiler-driven optimization, a memory-leak tracking service with alerting, Azure IoT Hub ingestion handling millions of messages, and device monitoring. The published outcomes on that page — 98% test coverage of the desktop agents, over 700 automated tests, over 1,500 test cases, from a team of five since 2020 on .NET/C#, Vue.js, Python 3, PostgreSQL, Azure, AWS, Docker and Playwright — come from the QA side of the engagement, not from the review workflow.
The transferable part is the shape: model review as states and legal transitions, and make the transition log the source of truth.
type ReviewState =
| 'ingested' // raw event landed, not yet enriched
| 'triaged' // deterministic rules + AI overlay attached
| 'in_review' // a named reviewer holds it
| 'escalated' // second reviewer required (irreversible class)
| 'resolved' // decision recorded
| 'reopened'; // resolved case pulled back
const LEGAL: Record = {
ingested: ['triaged'],
triaged: ['in_review'],
in_review: ['triaged', 'escalated', 'resolved'], // note the step-back
escalated: ['in_review', 'resolved'],
resolved: ['reopened'],
reopened: ['in_review'],
};
async function transition(caseId: string, to: ReviewState, actor: Actor) {
const current = await cases.lockForUpdate(caseId);
if (!LEGAL[current.state].includes(to)) {
throw new IllegalTransition(current.state, to);
}
// append-only: never UPDATE the case row's state without writing this first
await transitions.append({
caseId,
from: current.state,
to,
actorId: actor.id,
actorRole: actor.role,
at: new Date().toISOString(),
snapshot: current.decisionContext, // what the reviewer could see
});
await cases.setState(caseId, to);
}
Three things to look at. in_review → triaged is the step-back the analyst in the intro didn't have. The log is append-only, so "undo" is a new forward transition, not a mutation. And snapshot stores what the reviewer could see — six months later, "why did they approve this" is unanswerable without it.
Two rules follow. Keep active cases in a hot store and transition history in blob storage — active review is latency-sensitive and low-cardinality, history is high-volume, append-only and rarely read, and putting both in the same tables is how you get slow database requests under sustained ingest. And set a p95 budget for "reviewer clicks a case → decision context rendered," measured under ingest load: a review UI that stalls at peak telemetry teaches reviewers to click through without reading.
Telemetry volume sets reviewer economics
Reviewing 100% of anything is affordable only when the pipeline collapses raw volume into a small number of reviewable units — my reasoning, not a claim from the CYDEF page, which documents the volume and the redesign but not the ratio. If enrichment turns hundreds of gigabytes into a few hundred cases a day, a small expert team can genuinely review everything; if it turns them into fifty thousand alerts, you have a rubber-stamping machine.
73% of organizations named false positives their top detection challenge in the 2025 SANS Detection & Response Survey, as summarized by Stamus Networks, with "very frequent" false positives rising from 13% to 20% year over year — that's what an uncollapsed pipeline does to a review team.
So the throughput question comes before the model question: collapse ratio from raw events to reviewable units, sustainable decisions-per-hour per reviewer, staffed hours. If the product of the last two doesn't exceed the first, no amount of confidence scoring saves you.
The limitation: full-review models don't generalize. They suit high-consequence, low-decision-count domains — don't copy the pattern into a system making millions of decisions a day.
The AI is an overlay on a deterministic path
Put probabilistic output beside verifiable data, never in place of it, and scope who is allowed to act on it.
GazeHealth, a US mental-healthcare EMR and analytics platform built with Devico since 2019, has a useful shape: the AI/Big Data-powered treatment recommendation engine sits alongside an analytics tool of roughly 15 charts — including a custom D3 Sankey built from scratch and a hybrid timeline-chart for medicine intake and test results — with role- and company-based routing through AWS Cognito, on React/TypeScript/D3 over NestJS and PostgreSQL. The team was 6 full-stack developers and 2 AI data scientists: a 3:1 ratio that tells you where the work actually is. No outcome metrics are published for that platform and nothing on the case page says the recommendations are clinically validated, so treat it as an architecture reference, not an efficacy claim.
Why the overlay-not-replacement rule matters is the best-evidenced thing here. In a 2023 Radiology study of 27 radiologists reading mammograms, accuracy collapsed when the AI suggested an incorrect BI-RADS category — from 79.7% to 19.8% for inexperienced readers, and from 82.3% to 45.5% even for very experienced ones.
Training doesn't inoculate people either: in a 2025 randomized clinical trial of 44 physicians who had all completed 20 hours of AI-literacy training, diagnostic reasoning accuracy fell from 84.9% with an error-free LLM to 73.3% when the LLM's advice was flawed.
Implications for the review surface:
- Render deterministic facts first, the AI overlay second, visually distinct — never merged into one "recommendation" blob.
- Attach provenance to the suggestion: model version, input snapshot, generated-at timestamp.
- Scope override authority by role, enforced in code rather than a policy doc.
- Make declining a suggestion a first-class recorded action — you can't measure override rate if "ignored" and "never seen" look identical in the log.
Regulated domains add validation, auditability and explainability cost on top of all this. Price it into the estimate, not the retro.
A human can only be accountable for data they can trace
Review integrity is an ingestion problem: if the input isn't validated and attributable, an override is theater.
The Carbon Management Solution project — a German client working with Devico since 2023, built by 3 full-stack developers — is a clean example of the staged shape: as published, data moves through a serverless pipeline on AWS Lambda, SNS, SQS, DynamoDB and Cognito where "a certain service is responsible for a certain stage of the process," carrying it "from a simple CSV file to ready-to-use calculations," with AWS TypeScript CDK and SAM so distinct infrastructure could be created fast for white-labelled per-client deployments. No engineering metrics are published for it.
The stages are the point, because each is a place where a record either earns provenance or doesn't:
// upload -> validation -> processing -> calculation, one handler per stage
export const validate = async (evt: SQSEvent) => {
for (const record of evt.Records) {
const { batchId, tenantId, s3Key, checksum } = JSON.parse(record.body);
// idempotency: same batch + same bytes must never be processed twice
const seen = await ledger.get({ tenantId, batchId, checksum });
if (seen?.status === 'validated') continue;
const rows = await parseCsv(s3Key);
const { ok, rejected } = schema.check(rows);
await ledger.put({
tenantId, batchId, checksum,
status: 'validated',
acceptedCount: ok.length,
rejectedCount: rejected.length,
sourceKey: s3Key, // provenance: every downstream row points back here
validatedAt: Date.now(),
});
await sns.publish({ topic: PROCESS_TOPIC, message: { tenantId, batchId } });
}
};
The parts that make review meaningful:
- Idempotency on tenant + batch + checksum, so a retry or duplicate upload can't silently double a number a human later signs off on.
- Replay from any stage — the fix for a bad calculation is re-running that stage against the same validated rows, not asking users to re-upload.
- Per-tenant isolation, so one client's bad batch can't stall or contaminate another's queue.
- Provenance on every reviewed record — source file, row number, validation verdict, transform version — surfaced in the review UI, not just the database.
That last one is where most teams under-invest, and the security data reflects it: IBM's 2025 Cost of a Data Breach reportfound 13% of organizations reported breaches of AI models or applications, 97% of those lacked AI access controls, and 63% of breached organizations had no AI governance policy or were still writing one.
The limitation: serverless batch suits bursty ingest with per-batch latency budgets. If you need sub-second decisions, this shape is wrong — say so early.
You can't tune a loop you can't ship
Thresholds, escalation rules and reviewer UI need dozens of iterations, so release safety gates HITL harder than architecture does. You will get the first escalation rule wrong — trigger too broad and the seniors drown, too narrow and you miss what mattered. That's normal. What kills the project is a release process where fixing it takes six weeks.
MINT, an Italian AdTech platform working with Devico since 2019, is worth citing purely for topology and cadence: up to 40 engineers across 14 teams, each with at least one QA professional, teams split into units with their own sandboxes to divide areas of responsibility, a separate UAT environment, and several releases per day. Devico's engagement there was a dedicated QA team, a dedicated development team and staff augmentation, so MINT's own platform capabilities (16 channels, over 350 integrations) are the client's product, not an outcome of the loop described here.
What that topology buys a HITL system:
- Independent sandboxes — test a threshold or escalation change against realistic data without coordinating with every other team.
- A dedicated UAT environment — the accountable reviewers, not engineers, get to reject a workflow change before it hits production cases.
- Embedded QA per team — the review workflow stays under test, which matters because the state machine is business logic, not glue.
The 2025 DORA report, based on nearly 5,000 technology professionals, is blunt about the trade-off: it finds a positive relationship between AI adoption and throughput but notes "AI adoption does continue to have a negative relationship with software delivery stability," and that without strong automated testing, mature version control and fast feedback loops, more change volume just means more instability. In the same survey, 30% report little or no trust in AI-generated code.
The limitation is real: cadence without an evaluation harness ships regressions faster. Before you earn the right to release the loop daily, you need a replayable set of past cases with known-correct outcomes that every threshold change runs against.
When HITL is the wrong design
Four conditions where adding a human makes the system worse, not safer:
- Decision volume exceeds any staffable review capacity. If the throughput arithmetic doesn't close, you're designing a bottleneck and calling it oversight.
- The latency budget is shorter than human reaction time. Sub-second decisions can't hold a synchronous human. Move the human to policy, sampling and post-hoc audit instead.
- Reviewers can't be given enough context to add signal. If the evidence needed to judge a case won't fit on one screen in the time available, you get rubber-stamping — and the automation-bias numbers show what that does to accuracy.
- The human exists only to absorb liability. If nobody expects the reviewer to change outcomes, you've built an accountability shield. It will fail its first audit, and it deserves to.
What to measure
If you ship only one dashboard, make it this one:
- Override rate and its direction — disagreement with the overlay, split more-cautious versus less-cautious. Near zero means rubber-stamping, not agreement.
- Time-to-decision at p95, per decision type, under peak ingest.
- Queue age — the oldest unreviewed case, not the average, which hides starvation.
- Reversal rate after the fact — resolved cases reopened and flipped. Best proxy you have for review quality.
- Reviewer agreement on a shadow-reviewed sample. Low agreement means insufficient evidence on screen, not careless reviewers.
- Cost per reviewed unit, fully loaded — the number that says whether 100% review is sustainable.
- Threshold-change lead time, commit to production. If it's weeks, everything above is theoretical.
The build order, and what it costs
In order, because each step depends on the one before it:
- Name the accountable decision and the role that owns it.
- Build the reversible state machine with append-only, actor-stamped history.
- Put the deterministic path first and scope who may act on the AI overlay.
- Validate and attribute ingestion end to end, with idempotency and replay.
- Earn a release cadence fast enough to tune thresholds weekly, with an evaluation harness behind it.
The cost, honestly: HITL adds permanent staffing you can't automate away, latency you can't optimize below human reaction time, and an audit surface that grows with every transition you log. It's a recurring operational expense wearing an architecture diagram's clothes — and a review layer nobody trusts is worse than none, because it turns an obviously unsupervised system into one that looks supervised.
Reach for it when the decision is consequential, the volume is reviewable, and someone will genuinely be accountable. Skip it when any one of those isn't true, and put the engineering into constraining the automated path instead.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.