Dev.to AI 🤖 Ai 👁 0 📖 3 min read

A small company asked me for an agent. What I ask before I say yes

A small company says "build us an agent" and means a hundred different things. Before I write any code I settle four questions: which job, what counts as passing, who reviews the output, and what it costs per month. I'm

A small company says "build us an agent" and means a hundred different things. Before I write any code I settle four questions: which job, what counts as passing, who reviews the output, and what it costs per month. I'm Denys, a Forward Deployed Engineer. I wrote this with an AI assistant and checked the numbers against my own notes.

Pick the job by what happens when it is wrong

Forget what sounds impressive. A good first job has a clear input, a clear output, a person who can tell quickly that the answer is wrong, and a cheap mistake. Drafting a weekly report from numbers in their own systems fits. A first reply to a customer message from their own documents fits too, if it hands over to a person when it is unsure.

What does not fit: anything with money, contracts or legal weight as the first job, and anything where every case needs a human decision anyway. There an agent adds a step instead of removing one. Also check the documents. If two versions of a price list disagree, the agent picks one and sounds sure. Fix the documents first.

Write the pass mark before you build

Ask for 20 to 30 real examples of the job, with the answer a person gave. Agree the pass mark in writing, for example "at least this many of the 30 right, and every wrong one flagged as unsure". Then build. If you skip this, the pass mark gets decided after the demo, by whoever is least happy.

Count the hours the job takes per month today. That is the number the whole thing is judged against.

Pick the review level from the cost of a mistake

I use three levels.

A draft or an internal summary: a person samples the output once a week. A record that feeds a report: the agent flags what it is unsure about, a person checks those, plus a sample of the rest. Money, contracts or anything legal: a person approves every item, and the agent only prepares it.

The test I keep coming back to is whether the agent can say "I am unsure". If it can't, it is not ready for the middle level. Ask how it decides to hand over, and what happens to the item afterwards.

"Done" has to produce something

My own worst case was a scraping job that went through 359 dates in 12 minutes, saved 0 rows and reported success. It now stops after 25 empty dates and says why. Whatever you build, add a check that the run produced output, and an alert for anything that can go quiet. A failure nobody sees costs more than one that crashes.

The monthly cost math

The model calls are the part you can calculate. The line is steps per task times tasks per month times the price of one step.

Take a small workload, 20 tasks a day with 4 model calls each. That is 2,400 calls a month, at 4,000 tokens in and 500 out per call. With API list prices of 15 October 2026 from my own cost article, the model bill is:

Model Small: 20 tasks/day, 4 calls Medium: 200 tasks/day, 6 calls
DeepSeek V4.1 Flash $1.09 $16
Claude Haiku 5.5 $1.56 $23
Claude Sonnet 5.5 $31 $468
Claude Opus 5.5 $62 $936

On the small workload the cheapest and the dearest differ by about 57 times (my division of $62 by $1.09). A small model handles most sorting and field pulling. Keep the big ones for the steps that need reasoning. Prices move fast, so re-check before you quote.

The model is not the whole bill. You also pay for the server and for the tools the agent drives, and some of those cost more than the model. In one of my benchmark runs the proxy traffic alone was about $3.08. Then add someone reading the alerts. Put a cap per run in code, not only a monthly limit on the account, so a loop cannot run the bill up.

I keep the longer version of this, with the six jobs I would try first and where agents fail, in a guide on AI agents for small business. The numbers above come from it.

What to send back to the client

Not a demo. One page with the job, the pass mark, the review level, and a monthly cost range at their volume. If they cannot agree to those four lines, there is no agent to build yet.

How do you handle the pass mark when the client has no examples to give you? I would like to hear it in the comments.

Written with AI assistance, checked by me.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.