A recaptcha test that proves something, and the ones that only go green
Most reCAPTCHA problems I get asked about turn out not to be captcha problems. They are tests that pass for the wrong reason: a green pipeline, a production form that keeps rejecting real people, and nobody able to say w
Most reCAPTCHA problems I get asked about turn out not to be captcha problems. They are tests that pass for the wrong reason: a green pipeline, a production form that keeps rejecting real people, and nobody able to say which of the two changed. Separating the two is mostly a matter of deciding what you actually mean by a recaptcha test.
Three different things people call "testing the captcha"
When someone says their suite covers the captcha, it is usually one of these:
- The widget renders. The
g-recaptchaelement exists, the iframe loads,grecaptcha.renderdid not throw. Runs in milliseconds, works offline, says nothing about your backend. - Verification works. Your server posts whatever token it received to Google's siteverify endpoint and branches on
success. - The whole flow works for a person. A real challenge was shown, a real token was minted, the form submitted, and a row landed in the database.
Levels 1 and 2 belong on every commit. Level 3 belongs in a release check, run once and on purpose. Mixing them is how a suite turns green while the signup form stays broken.
Google's v2 test keys, and why the warning banner matters
For reCAPTCHA v2, Google's FAQ publishes a fixed pair: site key 6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI and secret 6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe. The wording on that page is blunt — you will always get "No CAPTCHA" and all verification requests will pass. No grid, no checkbox click, no waiting.
The part people skip: the widget renders a warning banner while those keys are in use. That banner is not an inconvenience, it is the safety mechanism. It makes a test key on a live page visible to anyone who looks, which is what you want, because a test key in production waves every bot straight through.
Two practical consequences:
- Both sides have to be swapped together. A test site key in the browser and your production secret on the server is not "mostly configured" — verification still fails, and the failure looks like a token problem rather than a configuration problem.
- A staging domain does not need to be in the key's allowlist when the test key is used, which is why these keys are the standard answer for localhost.
localhostis not in a normal key's supported domain list by default; you have to add it explicitly in the admin console, and Google recommends not doing that on the production key at all.
reCAPTCHA v3 has no test keys, so stop asserting the score
There is no always-pass v3 key. Google's guidance is to create a separate key for test environments and to expect the scores not to be accurate, because v3 scoring depends on seeing real traffic. That is not a documentation gap, it is the design: a score is a statement about the traffic, and CI traffic is one machine in one datacenter.
So an assertion like expect(score).toBeGreaterThan(0.5) is a measurement of your runner's IP reputation, not of your product. Assert the parts that are actually deterministic instead:
- a token value appears in the hidden field inside your timeout;
- siteverify returns
success: truefor that value; - your form rejects a request where the field is empty or missing.
A v3 token also lives about two minutes and can only be verified once. Any test that renders the page, waits, and then submits is exercising the expiry path whether it means to or not.
Turnstile is the provider that does this properly
If you have a choice of provider, Cloudflare's dummy keys are worth knowing, because they cover the failure paths and not just the happy one:
-
1x00000000000000000000AA— always passes -
2x00000000000000000000AB— always fails -
3x00000000000000000000FF— forces the interactive challenge - secret
1x0000000000000000000000000000000AA— always passes validation - secret
3x0000000000000000000000000000000AA— returns the token-already-spent error
Two rules come with them. The dummy token is only accepted by the dummy secrets, so client and server keys get swapped as a pair. And the dummy keys work on localhost, which production keys deliberately do not.
The cheap tests you should write first
Before anything that mints a token, write the negative tests. They need no keys, no network to a solver, and they catch the mistakes that actually hurt:
- POST the protected endpoint with no token field at all — expect your own 400, not a 500 and not a success.
- POST with an empty string for the same field — same expectation.
- POST the same token twice. Google's siteverify answers a second attempt with
success: falseandtimeout-or-duplicateinerror-codes, because a token can only be checked once. If your code turns that into a generic 500, users will read it as a broken form. - Wait past the token lifetime and submit. On a v2 checkbox the code expires after roughly two minutes and the widget flips back so the user has to tick again — your page should say something, not spin.
None of these need a captcha provider, and together they cover the majority of real incidents I have seen reported.
Where stubs stop being enough
Nothing above touches the parts that break in production: the domain missing from the key, invisible v3 running on page load so the token is stale by submit time, a widget sitting inside a third-party iframe where querySelector cannot reach it, or a bot-detection layer on your own egress IP blocking the whole flow.
For one deliberate run per release you want a genuine token. That is where a recaptcha test leaves the realm of stubs and needs something that can complete the challenge outside the browser — an API your test runner calls, or your staging server calls, whichever fits the architecture.
What a token request wants
The request is small, and every field is there because leaving it out lowers the pass rate:
-
sitekey— required, thedata-sitekeyfrom the page -
url— required, and it has to be the real page URL, notabout:blank -
useragent— recommended, and worth keeping identical to the browser the flow normally uses -
data— extra parameters for the specific type, for example reCAPTCHA v3'saction -
enterprise— set when the target site runs Enterprise
Submitting returns a job id, the job runs in the background, and you poll it. The type name is part of the path and has to be spelled exactly right — a typo does not come back as an unknown-type error, it comes back as a generic failure, which is a genuinely annoying twenty minutes to debug.
The plumbing rules that decide whether the test is stable
-
error: 14means the job is unfinished. It is not a failure and it costs nothing extra. Poll again — every 2 to 3 seconds for token jobs, not tighter. Resubmitting because you saw a 14 leaves several jobs racing, which is slower and billed. - A rejected submit (bad parameter, no credit, no upstream capacity) costs nothing. Credits are deducted the moment the upstream accepts a job, and stay deducted whatever happens afterwards, timeouts included. There are no refunds, so a test that resubmits in a loop is a test that spends money.
- Jobs expire: recognition jobs after 3 minutes, token jobs after 5. The token they hand back is good for roughly a minute or two — use it immediately, never cache it between runs or between parallel workers.
- Default concurrency is 4 and the per-key rate ceiling is 20 requests per second. Both overflows come back as
error: 11. In a CI matrix that runs specs in parallel, this is the first thing to fail, and it fails intermittently, which is the worst way to fail. - Quota resets daily at midnight Beijing time (UTC+8) and does not roll over.
- Almost every error arrives with HTTP 200. Branch on the
errorfield, never on the status code — only 401, 405 and 404 break that pattern. - Authentication is a header,
Authorization: Basic nsk_..., and the key goes in verbatim. It is deliberately not base64, so encoding it makes it unrecognisable. - A balance check is a good first assertion in a smoke test: the status endpoint returns remaining credit, the reset countdown and the concurrency limit in one call.
- Python's stdlib
urllibgets 403'd by the front-end CDN with an empty response body. Userequests, or giveurlliba normal User-Agent.
Cost shape
The two entry points are priced differently, and it is worth picking the cheap one when the test allows it. Solving the image and clicking the tiles yourself is one credit; asking for a finished reCAPTCHA v2 token is twenty. If a test only needs to prove the token wiring works end to end, a one-credit text captcha proves the same plumbing at a twentieth of the cost. Signup includes 200 free credits, which is more than enough to build the fixture before you pay anything.
The short version
Test keys on every commit. Negative tests for the failure paths. One real, deliberate token run before you ship, with the URL, sitekey and user agent copied from production rather than invented. And one rule about scores: never assert them, because they are not about your code.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.