I've been working on Synapse Shield on my own for a while now. It's a self-hosted bot protection library that watches how a visitor moves the mouse, types, and scrolls instead of serving CAPTCHA images, while keeping all of that data on your own server.

With v0.10.4, I shipped Zero-PII keystroke masking so the server would never see what people actually type. I was pretty proud of that release for about three days, until an outside review came back with 10 bugs. I went through and tested every single one myself before I believed the report, and all 10 turned out to be real.

The painful ones

The worst bug was in the privacy feature itself. My masking logic used a regex to decide which key names were safe to send, and that regex accidentally let whole words through. If autocomplete or an IME filled Mustafa into an input field, the server received Mustafa in plain text, even though preventing that was the whole point of v0.10.4 and the headline of my release notes. I ripped out the regex and switched to a short allowlist of actual control keys, masking everything else.

Another bypass was almost funny once I saw it. Sending a request with no mouse movement, no clicks, and no keystrokes, just a single scroll event, came back with 0% bot risk. I had spent all my time tuning movement math and never once tested the laziest bypass possible.

At the same time, real people on phones were getting blocked. Tapping a button four times in a row returned a 403 on the fourth tap. A single tap has no movement path, so my code recorded empty placeholder values for it, and after a few taps the rule that catches repeated identical behavior kicked in. I'd spent plenty of time testing bots and barely any time using the thing on a phone like a normal person.

DELETE requests also bypassed protection across all six framework adapters because I had typed out POST, PUT, and PATCH in six different files and forgotten DELETE every time.

The other six issues were smaller, though still worth fixing right away: a Vue proof-of-work solver that could never succeed, a missing payload size limit in one adapter, a timestamp check that only read one of four time fields, a Rust CI job that failed to link on Linux, and a few detection gaps like a dead-straight 25px mouse line scoring 0% risk.

Fixing them and testing the tests

For each bug on the list, I reproduced it first, patched it, and added a test that fails if the old behavior ever creeps back in. That brought the suite up to 536 Python tests, 78 JS tests, and 42 Rust tests.

I also tried mutation testing for the first time. It breaks your source code on purpose (turning < into <= or deleting a branch) to see whether your test suite actually notices. My first run reported 92%, which felt great for roughly an hour until I noticed the tool had left broken files behind and inflated the score. After cleaning up and re-running it properly, the real number was around 88%. That run happened before this batch of fixes, so I still haven't measured the new code.

Two things that break on upgrade

If you're upgrading an existing setup, watch out for two changes:

1. A DELETE request without a token on a protected path now returns a 403. Since DELETE requests usually don't have a body, send the token in the X-Synapse-Token header.
2. The /metrics endpoint on the standalone gateway now requires the admin secret. Leaving it open to everyone was my mistake. If you scrape that endpoint, add the secret to your scraper config.

What I still don't know

I still don't have a real false-positive number from live users. All of the human traces in my test suite pass, but synthetic test data only tells you so much. My next step is running a shadow-mode pilot where it logs what it would have blocked on real traffic without actually blocking anyone.

Nobody independent has audited the codebase either. One outside pass found 10 bugs in code I thought was finished, so I'm assuming there are more waiting to be found.

Trying it out

pip install synapse-shield

If you want to give it a spin, start with shadow mode (mode="shadow") and check what it flags on your own traffic before letting it block requests. If you find a hole, please report it through GitHub Security Advisories instead of a public issue. I'd much rather get ten more reports like the last one than find out later.