90% of our AI coding agent pipeline is plain scripted code

Roughly 90% of the pipeline that ships our code is plain scripted code. The AI coding agent does one thing: the engineering work inside a single ticket's workspace, in response to what a person wrote in that ticket.

We didn't start there. The first version gave the agent more of the operational steps, and this post is about why we took them back.

I lead the software team at Oizom, where Envizom is our environmental monitoring platform. The pipeline below is our dev pipeline for Envizom.

The shape of it

A developer writes what they want in a ticket, in plain English, and drags it to Approved. Our orchestrator picks it up and gives the ticket its own stack: a branch in every repo named after the ticket, its own backend, drivers, simulator and frontend, a copy of the dev database, and a private preview URL with an HTTPS certificate issued on demand.

The agent works inside that stack and talks to the developer only through the ticket thread. When it's done it posts the preview link and moves the ticket to QAQC. The developer tests the preview, replies in the thread, and either sends it back or drags it on.

Approved ──► [scripts] provision stack ──► [agent] engineering work ──► QAQC
                                                    ▲                    │
                                                    └── feedback ────────┤
                                                                         ▼
Completed ◄── [scripts] commit, merge, PR, CI, deploy, health ◄── Git Push
     │                         │
     │                         └── unhealthy: roll back images, revert merges
     ▼
 stack torn down

One box in that diagram has the agent in it. Everything else is a script.

What the scripts do

Here is the list of jobs no agent touches:

• watching the board
• provisioning the per-ticket stack
• copying the database
• issuing certificates
• driving git
• opening pull requests
• waiting on CI
• deploying
• health checks
• rollback

The clearest example is what happens when a developer drags a ticket to Git Push. From that moment there is no agent involved. The orchestrator runs eight steps:

If the VM isn't healthy afterwards, the orchestrator rolls it back to the previous images and reverts the merges, on its own, and then tells the thread what happened. If it is healthy, the ticket moves to Completed and the developer gets a notification. The ticket's stack is torn down after that.

Why we took the steps back from the agent

We tried it the other way first, with the agent doing more of the operational steps. It was slower, it was more expensive, and it was worse at exactly the tasks that need to be boring.

That last one is the reason that matters. Merging, deploying and rolling back should happen the same way on every ticket, and that sameness is the whole value. An agent deciding how to merge is not a feature. A script that always merges the same way is.

The scripted part is deterministic and tested. When it breaks, I read a log and fix it.

Where the agent earns its place

The agent's job is the engineering work itself. In the walkthrough video I asked for a camera layout change on a device page. The agent shipped it and posted the preview. I replied in the thread asking it to also handle more than 4 cameras with a horizontal scroll, and to set up 8 test cameras on that device. It did both, including creating the test data in the ticket's own database copy, and it picked up with the same context it had before.

Isolation is what lets it do that safely. Because every ticket has its own stack, an agent that does something silly to the database did it to a copy. A developer can have several tickets in flight at once, and one ticket can sit in QAQC for a week while three others ship.

Two more lines we drew on purpose:

• QA is human. The developer tests the preview and decides whether the ticket moves forward. An agent grading its own work would remove the one review that matters most.
• This pipeline stops at dev. Auto-merge, rollback and VM updates all apply to the dev environment only. Production goes through a separate path where people make the call, on a slower cadence.

The order I'd build it in

If you're setting up something similar, this is the order we ended up with:

1. Isolation first. If every task can't run in its own branch, its own services and its own data, nothing after it is safe.
2. A handoff surface the team already uses. For us that was the ticket.
3. Only then, the agent inside the box.

And keep the box scripted. The agent should be the smallest part of the system.

The longer write-up, with the video walkthrough, is The ticket is the interface.

If your team runs coding agents, which step did you hand to the agent that a script should own?