Dev.to AI 🤖 Ai 👁 0 📖 6 min read

System design for physical AI: an exception desk that gets better every week

By Muhammad Umar. This is the engineering summary of an open reference architecture, a self-improving AI system for carrier exception handling. The paper, its object model as JSON and the model register are free to reuse

By Muhammad Umar. This is the engineering summary of an open reference architecture, a self-improving AI system for carrier exception handling. The paper, its object model as JSON and the model register are free to reuse under CC BY 4.0: https://codeatoms.ai/carrier-exceptions-us/. DOI 10.5281/zenodo.24048585. The operator is described by class, never by name.

A third party logistics provider in the United States plans, tenders and tracks tens of thousands of truckload and less than truckload shipments a month for shippers, moved by hundreds of contracted carriers. When a truck breaks down on a multi stop load at night, the coordinator on shift needs one answer within minutes: given this shipment, this carrier, this consignee and this appointment, what actually works here, and who has to be told. The facts exist, but apart: the stops in the transportation management system, the live position in the tracking platform, the window in the scheduling tool, the carrier's record in the scorecards, and the fix that worked last time in an email thread nobody can find. When a coordinator leaves, which carriers answer at night leaves with them.

1. Read everything, write one thing

The design is a projection over the systems of record, never a migration. The transportation management system, the carrier tracking and visibility platform, the appointment scheduling tool, the claims system, the carrier scorecards and contracts, the customer order feeds and the email and chat threads stay exactly what they are and are read through their own surfaces. The design writes one thing anywhere: an approved appointment change request into the scheduling tool, and only after a coordinator approves it. Carrier messages and shipper notices go out through the same approval gate. No inference call leaves the operator's facility and no third party model service is called, because every shipment carries a shipper's commodities, values and consignees under contractual data terms.

2. One adapter family, four guarantees

Seven sources enter through a single integration adapter family: Debezium change capture on the transactional systems, governed pulls on the feeds and threads. Every record carries four things. Provenance: the source, the extraction time and that source's clock offset, recorded as a field rather than silently corrected, so a claim about which event came first stays honest. Ordering: a per stream sequence number beside the backbone offset, so a gap or a replay is visible. Scope: the shipper whose data terms govern the record, which is what permission aware retrieval enforces. Conformity: the record maps to the model's vocabulary before it enters, so the same carrier and consignee carrying different identifiers in each system are reconciled at the adapter, under governance, not in an agent's prompt.

Apache Kafka in KRaft mode carries ordering, delivery, buffering and replication inside the facility. Delivery is at least once and the projection is idempotent, so a replayed stream updates the same objects instead of duplicating them, and a parsing backlog or a serving restart loses nothing because the adapter replays from the retained log without reading the source again. Nothing is installed in the field: the trucks are seen through the tracking feeds the operator already owns.

3. Sixteen objects, one walk

The object model holds sixteen objects with thirteen typed links on Hyper Ontology v0.9: shipper, order, shipment, stop, appointment, carrier, driver, tracking event, exception, exception type, action, resolution, standing fix, claim, scorecard entry and exception coordinator. Shipment is anchored in the transportation management system, appointment in the scheduling tool, claim in the claims system; exception type, standing fix and coordinator are vocabularies the operator owns outright. One query starts at an exception on a late shipment, walks to the carrier's scorecard, sees whether that carrier is reachable at night, crosses to the next stop's appointment and its change history, and ends at the standing fixes and confirmed resolutions on that same lane, carrier and consignee. A document store can find threads that mention a load number; it cannot walk that path unless someone wrote the join in advance.

Three status vocabularies carry the human loop. An exception moves from New through Triaged, Recovery drafted and Approved to Resolved or Became claim; an action from Drafted through Awaiting approval and Approved to Sent, Acknowledged or Declined; a resolution from Proposed through Confirmed to Promoted to standing fix.

4. One node, sized from the weights up

The frontier tier serves GLM 5.3, 753 billion parameters, at FP8: 753 GB of weights, 904 GB with the 1.2 planning factor. One node of eight 141 GB HBM cards gives 1,128 GB, about 1,038 GB usable at 0.92, which holds the weights with about 280 GB of KV cache left for concurrent coordinator sessions. BF16 would need about 1,506 GB and two nodes over an interconnect, which is why FP8 on one node was chosen. The site tier runs Qwen3.8 27B at FP8 (about 31 GB) on one 48 GB PCIe card for years of archived threads and notes, a throughput job rather than a reasoning one, with Qwen3-Embedding 0.6B beside it for retrieval. The latency budget is stated in the desk's own beats: a draft must be waiting before the coordinator opens the queue, the cited conversational answer is bounded by that 280 GB ceiling, and archive parsing trails behind. If the wide area link fails, the desk loses only outbound notices, because no inference, retrieval or drafting ever left the building.

5. Better every week, without training

Triage, recovery and review agents draft; a named coordinator confirms the triage, approves the carrier message, the appointment change and the shipper notice, and the operations manager approves each standing fix. The approver is recorded on the action object, and confirmations are tested with known answer probes and a weekly sample, so ground truth is a decision and not a reflex. Every confirmed resolution becomes a record the next exception reads. No weights are trained, fine tuned or distilled, so the improvement is auditable and reversible when a shipper disputes an action, and it stays inside the estate under licences (GLM-5.3 bespoke, Apache-2.0 for both Qwen models) that let the operator hold its own copies.

Appendix A prices both tiers at about 606,000 US dollars to own over three years (577,000 to 665,000: node, card, support and power at national average prices). Renting the same hardware on the deepest three year commitment in AWS US East (N. Virginia) is about 644,000, so ownership costs about 94 percent of it, and at the top of the range slightly more; on demand it is about 1.71 million, against which ownership costs about a third. Renting is a price reference only, because every rented option would carry shippers' freight data and the threads out of the facility.

The rollout is two phases and 18 scope items (17, then 1), closed by gates, never dates, and traced against 21 requirements. Phase one is a pilot quarter on one regional lane group: baseline time to resolve and repeat exceptions before any draft reaches a coordinator, and pass the gate only when both move against that baseline. Phase two is the extension decision for the other lane groups, taken on the pilot's measured record.

Made with CodeNinja Praxis, the CodeNinja Atoms platform for designing physical AI systems. The object model imports into Hyper Ontology. Paper, ontology and register: https://codeatoms.ai/carrier-exceptions-us/

One of the CodeNinja Atoms worked examples of a self-improving AI system: systems of record read and never written, a governed ontology joined by typed links, open weights on hardware the operator owns, and memory that compounds from every confirmed outcome, the shape every physical AI design in the series is published in. More on self-improving systems and how they improve without training.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.