Who checks the AI software factory?
Picture a team where AI agents write the tests, write the code, review it and merge it. A person only says what to build, and accepts the result at the end. Teams are starting to build exactly this. One name for it is an
Picture a team where AI agents write the tests, write the code, review it and merge it. A person only says what to build, and accepts the result at the end. Teams are starting to build exactly this. One name for it is an AI software factory.
Here is the catch. Once the factory builds the product, the factory is the thing that must not be wrong. So who checks the factory?
I don't run a factory. I have one small board on which AI coding agents are set up to work unattended. It is built on four rules. In my view the same four belong under any factory, though I have not tested them under one.
Rule 1: the agents don't get the keys
On a computer, "root" is the account that can change anything. My agents don't have it. They work as an ordinary, non-root user, inside a harness that root owns.
They get four fixed admin commands and no more: two that only read, one that starts the existing deploy job early, and one that installs the pinned version of their own tool.
They also can't quietly change the plan. If an agent run writes to the work queue or to the loop's protocol, the run fails, and the agent has to propose the change instead. A run that adds, changes or deletes a ruling fails too.
So when an agent goes wrong, the damage is limited by design to what its user can write and what those four commands can do.
Rule 2: risky work goes through a person
Some jobs do need root, like changing a service's definition or repairing the host. Beyond those four commands, no agent does these.
Each one becomes a shell script. An independent session reads it line by line. That session did not write the script and has no stake in it passing. Then I run it by hand.
Is the review worth the wait? It found three kinds of mistake in those scripts: a link swapped in after the checks had run, an environment that the agent could set, and a gap between checking a script and running it. Each has its fix on record. The repository holds 104 operator scripts so far.
Rule 3: make a check fail before you trust it
A test that cannot fail looks like safety and protects nothing.
Here is one case. A test passed 14 of 14. It was running against a stub, a stand-in for the real thing, and the stub could not show the fault. On the real machine, the change it was testing never took effect.
So checks come with a control: a test that plants the defect and shows the check catches it. The same goes for tools. A script that reports zero problems should be believed only after it has been fed one input known to be bad and has said so. The repository holds 317 control scripts.
Rule 4: every rule says what it prevents
The agent sessions work under 47 numbered rules. Every rule carries a line that starts "Prevents" and says what went wrong before the rule existed.
One of them, reworded for this article: "A wait for every busy service exempts the service it runs in, and prints the full list it waits on. Prevents: a measurement job waited 46 minutes on its own service, then stopped with nothing run."
A rule without its story gets argued away the first time it is inconvenient. A rule with one gets followed, or it gets changed on purpose.
What is on record
The loop has four agent roles on timers: an orchestrator, an executor, a reviewer and an operator. Two judges sit beside them. They take work from a queue and act on written rulings. The agents' usage is capped: at 95% of the weekly allowance the loop pauses itself.
These counts are as of 2026-11-18. They are counts of different things, not a comparison. They say how much was written down, not how good it is, and I give no rate for them.
And it is not finished. The loop stopped falsely twice, on 2026-11-09 and 2026-11-17. The fix is built and not yet live. I make no uptime claim: the loop has stood still for days at a time.
What carries over
None of the four rules names a framework or a model. The loop is custom and uses no agent framework.
I plan to publish the loop so that its three pieces can sit under a factory built on any of them: the non-root harness, the reviewed path for root work, and the control rigs. A reference agent that applies the same controls on LangGraph, with a retrieval step and runs on Azure, is specified; its code comes next.
What this does not show
- It is one person's setup on one machine. Nothing here was measured against another way of working.
- The method costs time. Each root step waits for a review and for me.
- The loop's repository is private for now. The first project to be published under this discipline will be peddet-orin, a benchmark of a person detector on a Jetson board; the loop did not build it.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.




