Green build, broken invariant: one Kafka session where every fix passed and nothing held

Every fix compiled and passed lint. Every fix was followed by a different breakage. All of them broke one sentence about asynchronous TID processing.

You know this loop. A bug report comes in. You find something wrong, fix it, and the system fails again, somewhere else, in a new way. You fix that. It fails a third time. An hour later you have a diff that touches half a dozen files and a growing suspicion that the bugs were never separate.

At Explyt, where we build an AI agent for JetBrains IDEs, we read session logs to see what debugging with an agent looks like outside a demo. This post is about one of those logs. Yurii Kostyukov, our product manager, went through it after the fact; he did not run the session, and neither did anyone else on our team. The developer worked in our agent, in plain chat mode, on a current frontier model. Debug mode stayed off. The codebase was Kotlin and Java on Spring Boot with Kafka, in a payments system.

The longer narrative is on our blog as One symptom, ten problems: a Kotlin + Spring Boot + Kafka debugging case. Treat the "ten" there as a working title; we could not verify a count from the log. This version is shorter, skips most of the story, and ends with something you can paste into your own repository.

TL;DR

• A single symptom started the session. The developer and the agent fixed defect after defect, roughly half of them unrelated to that symptom, and every one of them violated the same rule about how a terminal ID moves through the asynchronous path.
• The agent helped with the small things: SQL to inspect state, risk warnings, local code explanations, no new debug logging. It never took over the investigation. The developer picked every next step.
• Fixes came before localization. The agent checked compilation and lint, then stopped. Whether the symptom went away was nobody's check, and several fixes did not survive the next run.
• Every runtime fact reached the agent through the chat window, pasted by hand. Its own terminal activity was limited to git.
• Debugging a distributed system means finding the invariant the defects share. We list four things an agent needs for that, which of them Explyt covers today, and which it does not.

The rule everything broke

The developer summed up the scope of the session in one sentence:

Any operation on a TID must go through the single asynchronous path and must not let TMS and EMV drift apart.

TID is a terminal ID. TMS and EMV are two downstream systems that have to agree about it. A rule like that lives between components; no single class owns it and the compiler does not check it. Each defect in the session was a place where the code stepped outside that rule.

We are leaving the symptom, the defects and the code out on purpose, since they would identify the developer's project, and the shape of the session is enough to carry the lesson.

Helpful in the small, absent in the large

The left column is real value; a developer alone types all of that themselves. The right column is where the hours went.

Take the third row. A fix arrives, it compiles, lint is clean, the developer applies it, and the next run breaks somewhere new. The log shows the agent verifying the build and stopping there. The symptom itself was never the acceptance criterion, for the agent or, in the moment, for the developer.

The fourth row is the one we keep returning to. Without a model of the entities, every defect looks like a fresh bug with its own fix. With the model, they collapse into one rule violated in several places, and the search changes: the question stops being "why did this fail" and becomes "where else does TID state get written outside the async path".

Who ran the experiments

Nobody but the developer. SQL snippets with their output, log lines from Spring Boot and from Kafka, HTTP responses, Kubernetes and Docker Compose output, configuration files: a human copied every one of them into the chat, while the agent's own terminal use stayed at git.

Consider what that does to an agent's judgment. It sees the running system only through excerpts someone else chose, in an order someone else chose, with a framing already attached. From that position it cannot run an experiment; it can only comment on the experiments a human decided to run. The passivity in the table above follows directly from this, and no prompt wording changes it.

The internal review of the session proposed two remedies. First, instruments: a browser and a terminal the agent operates itself, with instructions to use them. Second, a written debugging method: a journal of hypotheses, experiments and results, plus a running model of the data and its invariants. The references the review pointed to were systematic-debugging from Superpowers and parallel-debugging from wshobson/agents. Both remedies assume the agent can act and observe without a human relay. A method with no instruments is a checklist someone else has to fill in.

Four things a debugging agent needs

1. A written working theory outside the chat scroll, so that "verified" and "assumed" do not blur over a two-hour session.
2. A way to execute the experiment behind the current hypothesis and read the outcome without a human in between.
3. State observed first-hand: a variable value at a breakpoint, a row returned by a query the agent ran. A pasted excerpt is already somebody's interpretation.
4. A standing question after the second defect in the same area: what do these two have in common?

The session had none of the four. The model was current; what it had to work with was a chat window, file access and git, pointed at an asynchronous payments pipeline. The harness is ours, and so are the gaps.

What Explyt covers today

We want to be precise here, because this is our product.

Debug mode addresses item 3 and part of item 2. It applies when the failure reproduces under a test, an application launch or one of the project's IDE run configurations. The agent confirms the cause in the debugger before editing: breakpoints, variable values, call stack. A minimal fix follows, and after it the same scenario runs again together with the related tests, which is the loop this session never entered.

Run configurations address the rest of item 2. Builds and tests go through the IDE and come back as structured results: console output, test outcomes, compilation errors. The agent reads a red test instead of a green lint. The docs also say what a single debugger run does not prove: it confirms the fix for that scenario and does not stand in for the related tests.

Items 1 and 4 stay open. Explyt's Memory Bank persists facts and project agreements between chats; a per-bug hypothesis journal is a different artifact, and the product does not keep one for you. Reading Kafka topics, querying your database or pulling Kubernetes logs unaided is likewise absent from the documentation. That evidence still comes from the developer, or from an MCP server you attach, and each server decides what it exposes.

So, had the failure in this log been reproducible in a test or a run configuration, Debug mode would have altered the middle of the loop: the state at the moment of the write would have been visible, and each fix would have been followed by a rerun instead of a lint pass. The edges would have stayed where they were. Someone still brings the infrastructure evidence, and someone still notices that the newest defect rhymes with the first one.

Try this on Monday

Nothing in this section describes the session. It is what we would set up before the next one, with any agent that reads repository instructions.

Step 1. Give the working theory a file. One per bug, outside the chat:

# Working theory: <bug slug>

## Verified
- <fact> (how: breakpoint in X / query Y / test Z, when: <timestamp>)

## Assumed
- <claim nobody has checked yet>

## Current hypothesis
<one sentence>
Experiment that would refute it: <command / test / breakpoint>

## Invariant candidates
- <a rule that more than one defect has violated>

Step 2. Make the agent keep it. A rule for AGENTS.md or your vendor's equivalent:

## Debugging

Maintain .tasks/<bug>/theory.md and update it after each experiment.
Every entry under "Verified" names the command, test or breakpoint
that verified it. Everything else goes under "Assumed".

Do not propose a fix until the failure is reproduced in a test or a
run configuration and the cause is confirmed at a breakpoint or in
a query result you executed yourself.

Success means the original scenario passes on rerun. A clean build
and a clean linter are preconditions for a fix; they do not confirm it.

When a second defect appears in the same area, stop. Write down what
the two have in common before fixing either.

Step 3. Start under the debugger. Pick one failing test, reproduce it with a breakpoint, and make the agent read the variables before it edits anything. If your agent has no debugger, the rule still helps: every time "Verified" stays empty, the missing instrument is visible in the journal.

Limits

One session, one developer, our own agent, reviewed after the fact by one person who did not run it. We are not publishing the defects or the code. The log also does not say whether a local test could reproduce the failure, which is what Debug mode requires. Treat this as one case and nothing more.

Where to go next

This post keeps the story short on purpose. The full write-up, with the session walked through step by step and a closer look at each of the four failures, is on our blog:

→ Read the full case on explyt.ai: One symptom, ten problems

Our earlier case, Superpowers vs Plain Old Debugger in Explyt, covers the opposite shape: one bug, one cause, two agent runs.

When your agent fixed a bug and something else broke right after, did you stop to ask what the two had in common? Tell us the invariant you found in the comments.

Sources

• Site version of this case: One symptom, ten problems: a Kotlin + Spring Boot + Kafka debugging case
• Earlier case: Superpowers vs Plain Old Debugger in Explyt
• Explyt docs: Debug mode, Run configurations, Tools and integrations, Memory Bank
• Debugging method references: obra/superpowers systematic-debugging, wshobson/agents parallel-debugging