Verifying Humans by Replaying Their Input: a Different Take on CAPTCHA

How we built a bot-protection system that doesn't check what you clicked — it checks how you moved.

Every CAPTCHA you've ever solved had the same weakness: the answer is static. Pick the traffic lights. Type the distorted word. Click every image containing a bus. Whatever the answer is, a script (and lately, a cheap vision model) can compute it too.

We're a small bootstrapped team, and when our own forms got hammered by bots we decided to build protection the way we wished it existed. This post is a technical breakdown of the approach we landed on: server-side trajectory replay.

The core idea

The challenge is not a fixed asset — it's generated:

1. The server creates a random seed for each challenge
2. The client deterministically builds the layout from that seed (same RNG, same layout, every time)
3. You solve it — dragging a disc through a maze, tapping a bouncing ball, straightening a tilted photo
4. Your input events (timestamps + coordinates) are sent back
5. The server re-simulates your exact path through the same layout and verifies the solve

The answer is not "which squares contain a bus". The answer is how you moved: your timing, your speed curve, your micro-corrections. That can't be pre-computed, because the layout didn't exist until the server generated it, and the trajectory is measured against physical plausibility.

What we check on replay

• Teleport detection — a human finger can't jump 200px in one 40ms frame. |Δx| > k·Δt + c → rejected
• Speed caps — per-step velocity above a generous human ceiling → rejected
• Scripted cadence — screen-acting AI agents move the pointer in metronomic, perfectly-timed steps. Humans produce irregular intervals (OS scheduling + muscle jitter). We flag near-perfectly regular input sequences
• Per-challenge replay — puzzle solves, rotation angles, final positions — all recomputed server-side from the same seed

What we deliberately DON'T check

We keep the checks one-directional — flag perfection, pass imperfection:

• Shaky hands? Trackpad? Tremors? Your input has high variance — the opposite of scripted. It passes
• Short traces and slow movements are fail-open. We'd rather let a borderline human through than block one
• No behavioral tracking across sites, no advertising profiles, no personal data

Why this matters now

Two things changed in the last year:

1. Vision models got free. A local multimodal model solves image puzzles at zero marginal cost. Any challenge with a fixed, publicly-visible answer format will fall.

2. Computer-use agents arrived. GPT-class models now take screenshots and act — moving the mouse, clicking buttons, filling forms, exactly like a person. The old defenses don't even see them as bots.

The only durable lever left is cost per solve. Puzzles cost a bot nothing. Motion-physical verification costs real engineering: an agent must generate trajectories that pass teleport, speed, and cadence analysis while also solving the challenge correctly. That raises the price of abuse from ~free to real money — which is the only currency spammers understand.

The honest open problems

• Accessibility: any motion-based check risks flagging motor-impaired users. Our checks are deliberately one-directional (flag perfection, pass imperfection), and sites can serve proof-of-work to everyone, escalating to interactive only for suspicious traffic — but a proper accessibility fallback (audio / alternative challenge) is still on our drawing board
• The arms race is real: an agent that adds realistic jitter to its movements is coming. Every signal we add, someone will model it. We think replay verification raises the bar further than puzzles did, but we're not claiming immunity
• Friction: interactive challenges take 3–5 seconds. That's fine for signup and password reset, too much for search bars — which is why risk-based selection (one-click for trusted IPs, interactive for suspicious ones) matters more than any single challenge type

Try it

All 35 challenges run live in the browser, no signup: msrfteam.com/captcha

Every card on that page plays a real challenge preview, and every solve is verified the way this post describes.

We're a tiny team (zero funding, working from home) building an ecosystem of developer tools — a captcha platform, a package manager, an AI gateway. If you've built or evaluated bot protection, we'd genuinely love to compare notes: what signals would you add to replay verification?