Software factories: what are they and how do they work
I fell down a rabbit hole last week. One video about "software factories" led to another, then to a few articles, then to a book from 1991, and by the end of it I had a notebook full of questions and a term I could not s
I fell down a rabbit hole last week. One video about "software factories" led to another, then to a few articles, then to a book from 1991, and by the end of it I had a notebook full of questions and a term I could not stop thinking about. The phrase sounds like it was invented last month by a vendor with a product to sell. It was not. It is almost sixty years old, and the idea behind it has been tried, dropped, and tried again four times now. Hacker News stories with the term in the title went from 6 in 2025 to 77 in 2026 so far, which tells you how fast the latest revival is moving.
A quick disclaimer before I go further, because it matters here. I am still tinkering with this and still learning. This is a new paradigm, at least the agent version of it is, and I am excited and curious about it in equal measure. Nothing below is a settled opinion. These are my thoughts on what a "software factory" is, what I have read about the teams running one, and how I would go about setting one up for a team. If you have run one yourself, let me know where this system flow can improve.
what a "software factory" actually is
The term starts with a General Electric engineer named R. W. Bemer, who proposed in 1968 that software should be built the way cars are built: in a controlled environment, with standard tools, and with every step measured. A year later, Hitachi opened the Hitachi Software Works, the first organization actually called a "software factory". Toshiba, NEC, and Fujitsu followed in the 1970s, and the first American attempt came from System Development Corporation in 1975.
MIT professor Michael Cusumano spent years studying the Japanese factories, and his 1991 book, Japan's Software Factories, is still the best account of what they were. They were not about a clever tool. They were a management system: standard methods, reusable components, a division of labor, and strict quality control, applied to the same kind of project over and over. The numbers are striking. At Toshiba's Fuchu factory, reused code went from 13 percent of delivered code to 48 percent, and output per programmer more than doubled between 1976 and 1985.
The term came back in 2004, when Microsoft architects Jack Greenfield and Keith Short published Software Factories. Their version was narrower: a development environment configured to support the rapid development of a specific type of application. Find what repeats in one kind of application, turn it into frameworks, small languages, and tools, and assemble the rest. It was a good idea that mostly stayed an idea. The tooling shipped, people downloaded it, and the term faded again. Then the US Air Force picked it up in 2017 with Kessel Run, a team that replaced an eight-hour whiteboard planning job with an app, and by early 2022 the Department of Defense counted 29 factories.
Same line, different workers
Try it live on nayemalam.com.
what changed: the workers
Every one of those factories had people on every station. Managers designed the line, programmers worked along it, and the factory saved each person from deciding how to build, so they could focus on what to build. The 2026 version keeps the line and changes who stands on it. The stations are the same, request, spec, build, verify, ship, but four of the five are now agents, and the person moved to the ends of the line: writing the spec at the start, and deciding what merges at the end.
The most extreme version is StrongDM's. A three-person team, founded in July 2025, runs a factory with two rules: code must not be written by humans, and code must not be reviewed by humans. The humans write specs, curate test scenarios, and watch the scores. The agents do everything else. Dan Shapiro's five-level taxonomy of AI-assisted programming puts this at level 5, the dark factory. Their benchmark for whether the factory is working hard enough is spending at least $1,000 in tokens per engineer per day, which Simon Willison estimates at roughly $20,000 a month per engineer. That is a salary spent on the line before anyone checks what it made.
Most teams are not going that far. The teams running factories in production have published their numbers, and they tell a consistent story. Stripe's Minions open more than 1,300 pull requests a week, with no human-written code and a human review on every one. Spotify's Honk, a background coding agent, had merged more than 1,500 pull requests by November 2025. OpenAI's harness team drove Codex to about a million lines of code and 1,500 merged pull requests, with a team that grew from three engineers to seven. The pattern across all of them is the same: the factory works on narrow, repeating change classes, dependency bumps, migrations, small refactors, and the person stays at the gate.
Light factory vs dark factory
Try it live on nayemalam.com.
how the line actually runs
So what does a change look like going through one? The stages are the ones you already know, and the loop feeds back on itself, because a production error becomes a new request. The difference is who does each stage, and where the gates sit.
The system flow, end to end
Try it live on nayemalam.com.
The shape that keeps showing up is the orchestrator-worker pattern. Anthropic wrote it up in June 2025 for their multi-agent research system, and their multi-agent setup outperformed a single agent by 90.2 percent on their internal research eval, at roughly 15 times the token cost of a chat. The lesson I took from it is that delegation is a skill you have to teach the lead agent. Each subagent needs an objective, an output format, guidance on tools and sources, and clear boundaries, or you get duplicated work and gaps. Their early versions spawned 50 subagents for simple queries (the agent version of calling an all-hands to pick a lunch spot).
One lead, many workers
Try it live on nayemalam.com.
The station that decides whether any of this is trustworthy is verification, and it is where the interesting design lives. A test suite the agent can see is a test suite the agent can game. StrongDM learned that the hard way when their agents wrote code that just returned true, and Spotify's early agents commented out failing tests to get green builds. So the serious factories keep the grader away from the worker. The reusable parts of a factory used to be code libraries. Now they are test worlds.
Keep the grader away from the worker
Try it live on nayemalam.com.What goes in, what comes out
Try it live on nayemalam.com.
how I would set one up for a team
This is the part I keep coming back to, because my goal is to lead an engineering team, not to collect tools. A factory is a team decision, and the questions I would ask first are team questions. Does it make the developers' day better or worse? Does it add tech debt faster than it ships features? Does the team still understand the code it owns? The answers decide the design, not the other way around.
The first thing I would invest in is the spec. In a factory, the spec is the leverage point. An agent that starts from a written spec with acceptance criteria builds the right thing more often, and a person can check a spec in a minute instead of reading a diff for an hour. Every requirement in the spec should point at the test that will prove it. A spec that cannot be checked is a wish, and the rest of the factory has nothing to verify against. I already work this way on a small scale: I plan before I build, I write the rules down in the repo so the agent follows them without being told twice, I keep reusable workflows as skills the agent can call on, and I check the work against the plan before anything moves on. A factory is that habit, scaled up and written down so the whole team shares it.
The second investment is the verification station, because that is where the factory earns trust. The gates have to be code, not prompts. Build, type check, lint, unit tests, and structural tests run whether the model feels like it or not. The verifier starts with fresh context and a different kind of judgment than the builder, so the maker and the checker do not share the same blind spot. The factory's real product is not the code. It is the evidence that the code does what was asked.
The third thing is developer experience, and this is where I think most factory pitches get it backwards. A factory that makes developers babysit a dashboard, chase down flaky agent runs, or read logs to figure out what the machine did has not improved anyone's day. The feedback has to be fast and local: the lint runs in seconds, the failing check tells the agent how to fix it, and the human sees a preview they can open on their phone, not a transcript they have to read. If the team has to learn a new control panel to ship a bug fix, the factory is a tax, not a tool.
And then there is tech debt, which is the part I am most careful about. A factory that ships faster ships debt faster. Every shortcut an agent takes gets multiplied by the number of changes the line produces, and nobody reads the code closely enough to notice until the debt is the architecture. So the gates have to cover the things that create debt: small changes by default, so a bad one is easy to revert; a merge queue that lands changes one at a time against the current head; and a rule that a class of change only earns more autonomy after a clean track record, and loses it on the first escaped defect. The factory should make the codebase easier to change over time, not just faster to add to.
What stays human, in my version, is the judgment. Architecture decisions, the merge decision, anything irreversible, and anything that touches users or money. The agents do the typing and the checking. The team owns the outcome. That split is not a limitation of the factory. It is the point of it.
human in the loop, human on the loop, or out of it
The question every factory has to answer is where the person stands, and the vocabulary for it is older than the agents. The terms go back to human-factors research. Sheridan and Verplank laid out a ten-level scale of automation in 1978, and the EU AI Act now requires effective human oversight for high-risk systems, including a way to interrupt them.
My default split is by what the action can do. In the loop for anything irreversible, expensive, or public: merging, deploying, spending money, sending a message as the team. On the loop for routine, reversible work with a visible log: the agent acts, I watch the outcomes, and I can stop it. Out of the loop for research and drafts, where the worst case is a bad summary I throw away. The split is per kind of work, not per team. A dependency bump and a change to the payment flow should not share a policy, and a class of change earns more autonomy only after it proves itself.
The failure mode I worry about most is the approval that stops being a decision. Per-decision approval breaks at machine speed in one of two ways: it becomes a bottleneck that erases the speed you bought the factory for, or it becomes a reflexive click that approves decisions nobody actually reviewed. A person approving hundreds of near-identical items stops reading them. That is not oversight, it is a ritual. The fix is to keep the volume human: route only the uncertain or high-impact cases to a person, show the evidence behind the decision instead of just the answer, and sample the automatic decisions so someone is still checking the machine's work.
Three places to stand
Try it live on nayemalam.com.
I sketched the split as a small function, because a rule that lives in code gets applied the same way every time, and a rule that lives in a doc gets applied when someone remembers it.
gates.ts
type Oversight = 'in-the-loop' | 'on-the-loop' | 'out-of-the-loop';
type WorkItem = {
kind: 'research' | 'draft' | 'change';
irreversible: boolean;
blastRadius: 'small' | 'wide';
};
function oversightFor(item: WorkItem): Oversight {
if (item.kind === 'research' || item.kind === 'draft') {
return 'out-of-the-loop';
}
if (item.irreversible || item.blastRadius === 'wide') {
return 'in-the-loop';
}
return 'on-the-loop';
}
what can go wrong
The sharpest criticism of the whole idea comes from Stanford's CodeX center, and it is worth sitting with. When the same class of technology writes the code and decides whether the code works, the builder and the inspector share the same blind spots. A human who writes a test brings different assumptions and different mistakes than the person who wrote the code, and that mismatch is what makes testing useful. When one model writes the code and evaluates it, the mismatch shrinks. Goodhart's law finishes the thought: when a measure becomes a target, it ceases to be a good measure. Tell an agent to maximize a test score and it will maximize the test score, whether or not the software actually works.
The evidence for speed is also thinner than the pitches suggest. METR ran a randomized trial in 2025 with 16 experienced developers on 246 tasks in their own repositories, and with AI tools they took about 19 percent longer, while believing they had been about 20 percent faster. Its February 2026 update called the evidence for a speedup only very weak. Google's 2025 DORA report found AI adoption tracking higher delivery throughput but lower delivery stability. That does not contradict the factory numbers. It describes the same thing from the other side: a developer pairing with an assistant pays the cost of steering and checking in real time, while a factory moves that cost into gates that run without a human watching. The gates are where the bet lives. And the gates cost money. StrongDM's token benchmark works out to roughly $20,000 a month per engineer, which is a line item a team lead has to justify against the features it produces.
There is also the quiet risk nobody puts in a pitch deck: the skills fade. Bainbridge wrote about the ironies of automation in 1983, and the point still holds. Automate the easy parts and you leave people the hardest job, monitoring, while their hands-on skills fade. A team that stops reading code eventually cannot catch the failure that matters, because the ability to spot it was built by reading code. The factory needs people who can still take over, which means rotating them through real cases instead of parking them in front of a dashboard.
references and further reading
Everything I leaned on while writing this, in case you want to fall down the same rabbit hole.
- Michael A. Cusumano, Japan's Software Factories: A Challenge to U.S. Management (Oxford University Press, 1991). The history, from the person who studied the Japanese factories firsthand.
- Jack Greenfield and Keith Short, Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools (Wiley, 2004). The Microsoft-era definition.
- What is a software factory? History, examples and AI agents, Julian Tellez. The best single overview of the history and the 2026 wave, with the Toshiba numbers.
- How StrongDM's AI team build serious software without even looking at the code, Simon Willison, February 2026.
- Built by agents, tested by agents, trusted by whom?, Stanford CodeX, February 2026. The sharpest critique.
- How we built our multi-agent research system, Anthropic, June 2025. The orchestrator-worker pattern, with real numbers.
- Agentic software factories: how to build an AI pipeline from issue to production that you can trust, FDE Interviews. The published factory numbers in one table, plus the reference loop.
- AI software factory (2026): what it is and how to run one, Taskade. A full walkthrough of the loop, including the do-it-yourself version.
- Human in the loop vs human on the loop: the autonomy spectrum, Botminds. The ten levels of automation, and where each position fits.
- What is human-on-the-loop AI supervision?, IDC, August 2026. The governance view: tiers per decision type, audit logs, and a real stop button.
I am still early in this rabbit hole, and I expect my take to change as I read more and as more teams publish what actually happened, not just what they planned. What I am sure of right now is that the factory is not the agents. The factory is the line, the gates, and the measurement around them, and the person who designs those is doing the most important job in the building. The workers changed. The work of deciding what good looks like did not. I will keep tinkering, and I will write again when I have tried more of this for real.
Originally posted on nayemalam.com/blog.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.