Starting from v0: Why Plan the Tools, and a Few Real Problems

In the second post we worked out why the documentation had to drop horizontal layering and switch to a vertical organization along three levels: project, module, and feature. Most people's first reaction to this is: the structure is clear, so let's get the team writing docs against the new template right away.

Can people write it? Yes. But there's no need to.

If we still leave people to maintain this structure by hand, the whole system loses its main point. Traditional documentation falls behind the code and contradicts itself for one basic reason: it depends on people reconciling it by hand inside a fast development rhythm. In the age of AI coding, AI can generate twenty features' worth of code in an afternoon. If people are still expected to update the docs by hand, they become the slowest physical bottleneck in the whole development chain.

Since we have already nailed down the structure of the new documentation, defined each section, and made the boundaries clear, maintaining the docs is something we fully can — and should — hand to AI.

So for this documentation system to actually run, we should plan a dedicated set of tools for it. The job of those tools is to let AI read and write documents in a strict format and track constraints automatically, instead of having a person open a Markdown editor and type. That line at the end of the second post — "every hard constraint that must be followed... can never fall back on manual review" — was about this toolchain.

But this toolchain wasn't designed perfectly on paper from the start. Today's shape is what it became through use in our own projects. The whole thing has gone through a dozen or so versions from v0 to now; below I've picked the three pitfalls that hit us hardest.

Looking back, those three pitfalls map exactly onto three ways a line of defense in the tooling can fail: never installed, installed wrong, installed too late.

1. Capability layer mismatch: the defense was never installed

On one project I upgraded the execution toolchain to a new version but only got around to updating the matching model capability file a few days later. Neither side reported an error. Everything looked normal.

The problem showed up when creating an "issue". An issue is the core action in our tool-driven doc maintenance: list the technical proposals up for discussion, review them one by one, and only after one is chosen does the tool write it to disk. The issue created for review that day was inexplicably missing two options.

Misdiagnosis: my first reaction was model drift. This had happened before: while restructuring content, the model treated old entries as outdated information and cleaned them up. For that reason I had added a hard constraint to the prompt — the content of an issue under review may only grow, never shrink. This time that rule failed.

Cause: tracing it down, the problem wasn't in the model at all. The execution tool had been upgraded, but the capability file on the model side hadn't kept up: the old capability file contained no instructions for calling the new tools. And that "only grow, never shrink" review constraint happened to be mounted on a base layer that requires the new tool's identifier. Without that identifier, the whole review rule never got loaded into the context.

Worse, the underlying tool did report an error, but the first thing in the chain to catch it was the model, not a person. When a call failed, the model just tried a different phrasing and kept going; the error was quietly digested by the AI.

Defense: after that, I stopped treating a tool upgrade as "replacing a binary". The capability layer, the instruction layer, and the version declaration are now bound tightly into one release unit: they go up together, and if a layer is missing, the thing refuses to start.

Judgment: a line of defense that exists is not a line of defense that works. A constraint written in a file is not a defense; only one successfully loaded into the context is. The first is a static documentation problem, the second a dynamic runtime problem — and the one that fails is usually the second.

2. Interface mismatch: the entrance was installed wrong

Once I used the tool to update the documentation for the system log library. The intent was to add a new section, "audit channel", while keeping the existing "runtime log" section.

Symptom: while the AI was generating, I glanced at the tool-call log and saw it pass empty content to the "runtime log" section. I didn't think anything of it at the time. But when I opened the document, the previously fine "runtime log" section was gone entirely.

Misdiagnosis: revert, redo. The second time I watched closely: same place, same empty content, same vanished section. At first I assumed model drift again and hoped a redo would get lucky. But landing in the same pit twice in a row, the same way both times, can't be put down to chance.

Cause: the fault was in the tool's API design: the write tool didn't check for empty input. Pass empty content in, and the underlying logic treated it as deleting that section. Write and delete shared a single entrance — a null value carried two opposite meanings at once: "do nothing this time" and "delete this whole section".

From the AI's point of view, it only wanted to keep that section exactly as it was, and the most intuitive way to say that was to pass empty (it reasoned that if it wasn't changing anything, there was no need to regenerate it). The underlying tool received the null and faithfully executed a delete. And because our documents demand extremely strict paragraph structure, that tidy structure made "replace the whole section by structure" look like a completely safe move. Which is exactly what hid the risk: an operation being easy doesn't mean it's correct.

Defense: afterwards, "write" and "delete" were split into two separate methods for good. The write operation strictly forbids empty values, and delete became a high-risk action that must be called explicitly.

Judgment: destructive semantics must never hide in a default value. Leave a parameter empty and a person reads "no change", while the tool reads "clear it". Irreversible operations have to be blocked explicitly at the entrance; you can never count on the caller — human or AI — to remember "don't pass empty" out of tacit agreement.

3. Commit granularity: the defense was installed too late

Strictly speaking this one isn't a coding pit. It's the cost of a mental habit.

Symptom: a technical proposal needed adjusting, and I planned to roll back a feat commit from a few days earlier as a whole. It turned out I couldn't roll it back at all — that commit had a documentation update, two new features, and an interface I'd refactored along the way all mixed into one. Rolling back meant throwing out the whole lot, and half of that code was code I wanted to keep.

Misdiagnosis: in traditional hand-written development I would most likely have taken the shortcut too and squashed an afternoon's work into one commit. I had always assumed that one commit represents one development intent, and that rolling back means returning to before that intent happened.

Cause: AI tools shattered that premise. In one afternoon AI can finish a documentation revision plus three features; physically those things happen at the same time, but logically they are four completely independent intents. When I sliced commits by "how much work got done this time", what I cut wasn't logical intent at all — it was a block of physical time.

Defense: the commit flow was later changed to semantic splitting by pathspec: new feature feat, refactor refact, defect fix fix, proposal update docs, test cases test, miscellaneous chore, each one its own commit. Judging the semantics and drawing the boundaries went to the AI — it reads a diff accurately enough to tell what's new and what's refactored, far better than a human eye scanning it; the person only reviews whether the split makes sense and no longer performs it by hand.

Judgment: commit granularity must be cut by "the smallest intent that might need to be rolled back on its own", never by "how much work got done this time". You think a commit records "what I did"; what it actually decides is "what can be undone separately later". Once a commit mixes several intents, the right to roll back one of them separately is gone for good. That right isn't lost on the day you roll back — it's lost the moment you press Commit.

Conclusion

These three pitfalls share one thing: at no point could anyone see an error. The first one's error was swallowed by the model, the second one's delete was a "successful" call, and the third one's one-pot commit looked entirely compliant at the time. All of them were hidden problems that only showed their true shape in the post-mortem.

So the job of tools is never just to make AI faster. It's to make hidden errors less likely to happen, and to force them into view as fast as possible once they do — which is also the essence of a harness.

Honestly, this system still can't catch "a line of defense nobody thought to build". These three pitfalls were all lessons bought with real incidents; the places where nothing has gone wrong yet are probably still blind spots. Tools can backstop the rules that have already been written down, but not the ones nobody has thought of yet.

In the next post I'll take apart the eight SKILLs one by one: what problem each development capability package actually solves, which judgment sits behind it, and how you can get them running in a real project.