GPTBot is hammering your API. Block it, or charge it?
Check your logs for last month. Filter user-agents for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, bingbot. For most SaaS apps and content APIs in 2026 that's now a real percentage of total requests โ traffic you
Check your logs for last month. Filter user-agents for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, bingbot. For most SaaS apps and content APIs in 2026 that's now a real percentage of total requests โ traffic you pay to serve, from callers who never convert, never see an ad, never subscribe.
The reflex is robots.txt or a blocklist. Three problems:
- Blocking is leaky. Polite bots honor robots.txt; the traffic that costs you most doesn't announce itself.
- Blocking is blunt. Some agent traffic is your future distribution โ an AI assistant citing your data to a user who becomes a customer.
- Blocking earns $0. The traffic has value to the caller. Blocked value is just destroyed, not captured.
There's now a third option, because two standards landed:
- Web Bot Auth (RFC 9421 profile): agents cryptographically sign requests; you verify against the operator's published keys. Identity stops being a user-agent string anyone can type.
- HTTP 402 + x402: a machine-readable "this costs $0.002, here's how to pay" that agents can settle without a human in the loop.
Which turns your bot problem into a pricing decision per lane:
- Verified, signed agents โ charge per call on the routes that cost you money. They're businesses; API fees are a line item to them.
- Declared-but-unsigned bots โ cheap tier or tight rate limits.
- Undisclosed automation โ paywall or block. Sneaking shouldn't be free.
- Humans โ free, always. They never see any of this.
Wiring it by hand means reading RFC 9421, fetching and caching key directories, building the signature base, and handling replays. Or:
npm i wayleave
app.use(new Wayleave({
directories: { 'https://agents.example/keys': { 'k1': pubkey } },
pricedPaths: { '/api/expensive': 0.002 },
}).express());
One middleware line. Agents get a 402 with a price; your users notice nothing. The interesting question stops being "how do I stop bots" and becomes "what's my agent traffic worth" โ and that's a much better problem to have.
What the middleware doesn't do
It prices the request. It doesn't keep your books.
Charging per crossing means knowing what you're owed: a durable record of every billable request, deduplicated, attributed to a cryptographic agent identity rather than an IP guess, and turned into something you can invoice against. The middleware emits a usage event for each priced call and hands it to you. What happens next is your problem โ a queue, a table, a reconciliation job.
I'm considering building the other half: a hosted meter that receives those events, keeps the ledger, handles key directory fetching and rotation, and produces billing records per agent operator. Never touching your funds โ settlement stays on a payment rail you choose. The meter is the meter, not a middleman.
It isn't built. Not a beta, not a private preview โ unbuilt. Whether it gets built depends on whether people running real agent traffic actually want it, so there's a list:
โ wayleave.dev
One email if it becomes real, nothing otherwise. There's one optional question on the form โ roughly how much agent traffic you see โ and that answer is worth more to me than the signup.
The middleware is free and finished today either way:
npm i wayleave
MIT, zero dependencies, source at github.com/gibrancorbin11-hub/wayleave.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes โ full credit and traffic to the original publisher.