I gave AI agents a design tool. Here is what they got wrong.

The cover image of this post was made by an AI agent with one tool call. No Figma, no headless Chrome, no screenshot.

The first attempt came back with another company's logo in the corner.

Nothing crashed. The call returned 200, a URL and a charge of one credit. The template was attached to a sample brand I had forgotten about, the agent never checked, and the result looked perfectly professional and completely wrong.

That is the whole problem with giving agents a design tool, in one render. I have spent the last few months building an MCP server for bannx, a tool that renders templates into PNGs, PDFs and MP4s. Around 40 tools, driven by Claude Code, Cursor and Codex. These are the six things agents got wrong, and what I changed in the server each time.

1. Agents are bad at markup, so I stopped accepting it

My first instinct was the obvious one: let the model write HTML or SVG and render that. It demos well and it falls apart on the second edit. The agent rewrites the whole document to move one headline, loses a layer on the way, and you cannot tell what changed.

So the server takes no markup at all. A design is a document of nodes, and the agent edits it with a batch of operations:

{
  "templateId": "66f1…",
  "operations": [
    { "op": "updateScene", "sceneIndex": 0, "patch": { "fill": "#0f172a" } },
    {
      "op": "addNode",
      "type": "text",
      "props": {
        "text": "{{ use(headline, 'Your headline here') }}",
        "x": 600, "y": 280, "w": 900, "h": 120,
        "fontSize": 72, "fill": "#ffffff", "align": "center"
      }
    }
  ]
}

Two rules made this workable:

• All or nothing. The batch is validated first. If op 3 is wrong, nothing is written, and the error says so: op 3 (addNode) props.w: …. An agent can fix a named op. It cannot fix "something went wrong".
• dryRun: true. The agent can ask what a batch would do before it does it.

Small diffs, named failures. It is the same reason we review pull requests instead of whole repositories.

2. Agents invent IDs

Give a model a node called headline and it will happily address headline-2 next, a node that has never existed.

Node ids are now opaque 8-character strings, and they are only valid if they came from a snapshot of the document. There is nothing to guess, so the agent has to read before it writes. For building several nodes in one call, new nodes can carry a ref that later operations in the same batch point at, so the agent never needs an id it has not been given.

If your tool has identifiers, make them unguessable. A readable id is an invitation.

3. The worst bug is the one that returns 200

A template has variables: title, price, photo. You declare them, bind them to layers, and callers pass values at render time.

When I audited our public gallery, 3 of 21 templates declared variables that no layer actually read. Every render succeeded. Every render ignored the caller's data and returned the sample design. Nobody got an error, so nobody noticed.

From the agent's side this is a perfect crime. It declared the variable, the call succeeded, the image looked good, task complete.

The fix was not a stricter error. It was making the response talk back. Create and edit calls now return the list of declared variables that nothing reads, as warnings, in the same reply. The agent sees it at the exact moment it can still do something about it.

An agent will not go looking for problems. Put them in the result it is already reading.

4. Your output schema is a contract too

This one touched billing, which is why I remember it.

MCP tools can declare an output schema. Strict clients validate the result against it. I added two fields to the render result, creditsCharged and balanceAfter, and production was still advertising the older schema.

So the render ran, the file was created, the credit was most likely charged, and then the client rejected the reply with data must NOT have additional properties. The agent never saw the URL. From its point of view the tool was broken. From the user's point of view they paid for nothing.

I had tests for the handler. I had no test where a real MCP client validated a real reply. Now there is one, and it is the test I would write first if I started again.

5. Nobody reads your tool descriptions the way you hope

I started by putting everything in the tool descriptions: the coordinate system, the node types, how variables bind. Descriptions are loaded into context on every session whether the agent needs them or not, and long ones get skimmed like terms of service.

What works better:

• Short descriptions that say what the tool is for and when to call it.
• A get_guide tool that returns the long-form manual by topic. The agent pays for those tokens only in the sessions that design something.
• Server instructions with an order of work. Read the guide, resolve the brand, create or edit, read the warnings, then render. Agents follow a numbered sequence far more reliably than they follow advice.

Related: the expression language inside templates has no method calls. It is slice(list, 0, 3), never list.slice(0, 3). Models write the JavaScript version every single time unless the instructions say otherwise in one blunt sentence. So they do.

6. Make the dangerous calls boring

Agents are confident. That is fine for adding a rectangle and less fine for deleting a customer's template or posting to their LinkedIn.

• Every tool carries MCP annotations (read, write, destructive), so a client that asks for confirmation only asks when it matters.
• Deleting a template requires confirmName, the template's exact current name. A wrong name deletes nothing and the error states the name it expected. The agent has to look at the thing before it destroys it.
• The tools that post to social accounts are opt-in per API key. A key cannot post publicly unless someone ticked that box on purpose.
• Every render result says what it cost and what balance is left. An agent that can see the meter spends differently from one that cannot.

What I have not solved

The logo on my cover.

The agent cannot see. It can render an image and get a URL back, but nothing in that reply says "this text overflows its box" or "this is white on pale yellow" or "that is not your brand". A human spots it in half a second.

The next thing I am building is a check tool that returns design problems the way a linter returns code problems: contrast below 4.5:1, text clipped at its longest value, a raw hex colour where a brand token exists. Rule, node, suggested fix. Until then, the honest state of agent-made design is: fast, cheap, and needs a look before it ships.

Try it

The server speaks stateless Streamable HTTP, so the config is a URL and a key:

{
  "mcpServers": {
    "bannx": {
      "url": "https://bannx.com/mcp",
      "headers": { "Authorization": "Bearer YOUR_API_KEY" }
    }
  }
}

This is the actual call that produced the cover above, on the second try:

{
  "templateId": "…",
  "format": "png",
  "size": "1000x420",
  "scale": 2,
  "variables": {
    "page": {
      "title": "I gave AI agents a <mark>design tool</mark>",
      "description": "Here is what they got wrong."
    }
  }
}

No agent? The same render is one POST:

curl -X POST https://bannx.com/api/v1/templates/TEMPLATE_ID/render \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "format": "png", "variables": { "title": "Summer sale" } }'

There is a free tier, and the MCP docs list every tool.

If you have built an MCP server: what did agents get wrong with yours? I am especially curious whether anyone has a good answer to the "it cannot see its own output" problem.