CrowdSec alert from my own IP: the 'attack' was my phone's photo app

The worry was simple: CrowdSec was "being hammered". The numbers looked alarming, and I wanted to know who was attacking my homelab.

The answer, after a proper look on 21 September, was me. This post covers how that happened, what the big numbers actually meant, and the bugs I found in my own detection while working it out.

The only local alert in a week was my own address

My setup: services sit behind a Cloudflare tunnel, CrowdSec parses the logs, and bans are pushed to Cloudflare's edge. Alongside it I run warden, a small detect-only tool I wrote to read the logs CrowdSec doesn't.

Over seven days CrowdSec had raised exactly one local alert: the crowdsecurity/http-probing scenario. The source was our own home WAN address. The user agent was a photo-backup app on a phone. The trigger was 11 thumbnail requests returning 404 in 4 seconds, all for photos that had been deleted.

The phone was asking for thumbnails of pictures that no longer existed. To a probing detector, a burst of 404s from one address looks exactly like someone scanning for paths.

CrowdSec's own console had a label for it: a security engine reported itself. Warden agreed independently: 27 of the 29 application-layer events it recorded that week came from the same address.

cscli metrics is not a count of attacks on you

The part that made it feel like an attack was cscli metrics. The decision counts were large:

• 16,676 under http:scan
• 2,420 Tor exit nodes
• 1,015 from a FireHOL list

None of these were addresses that had touched my network. They come from the community blocklist (CAPI) and third-party lists: addresses blocked pre-emptively because other people's CrowdSec instances reported them. Local alerts in the same period: 1.

This is the most useful thing I learned: never quote cscli metrics decision counts as evidence of traffic against your estate. They are the size of a shared blocklist. The number that means "something happened here" is the local alert count, and you still need to open each alert and look at the source.

Why I can't just ban locally

A common first instinct is fail2ban-style blocking with iptables. Behind a Cloudflare tunnel that cannot work. At layer 3, every connection comes from the tunnel connector, not from the client. The attacker's real address only exists in the HTTP layer, in a forwarded header that ends up in the logs.

So a local firewall ban keyed on the attacker's address never matches a packet. And banning what does appear at layer 3 would ban Cloudflare and cut off everything. That is why warden has no iptables code at all, and why enforcement happens at Cloudflare's edge.

That also has a wrinkle. Cloudflare retired the legacy Firewall Rules API, and the official CrowdSec Cloudflare bouncer I tried (v0.3.0) created its IP list fine, then crashed creating the rule. I replaced it with a small sync script that reads CrowdSec's decisions, takes only local bans (not the community list, which is far too large), and mirrors them as Cloudflare IP Access Rules. I tested it end to end with a fake ban: the rule appeared at the edge, then disappeared when the ban was cleared.

Your own address is the easiest thing to ban

Once I knew the alert was self-inflicted, the real risk became obvious. If my own address can trip a detector, my own address can end up banned at the edge.

Warden already allowlists my own addresses before scoring. But on 10 September, reviewing the code before publishing it, I found a gap: warden can also propose a /24 range ban, and although my address was removed from the score, the proposed range could still contain it. I added a check (net_overlaps_allow()) that refuses any range ban overlapping an allowlisted address, and published the repo with it.

On 21 September I discovered that fix had never been backported to the copy actually running on the server. The public repo was right; production wasn't. It's backported now. A published repo and a running copy drift silently, and you have to check both.

Three bugs from building warden

Warden has been detect-only since I built it on 30 August: it records what it would ban and changes nothing. That turned out to be the right default, because the first version would have banned the wrong people.

1. Event time came from ingestion, not the log line. The first run reads a whole backlog at once. Storing "now" as the event time put weeks of July history inside the ban window, all 19 events of it. With enforcement on, that would have banned addresses for things they did in July. After switching to per-source timestamp parsing, 0 landed in the window.
2. A fixed lookback longer than the timer double-counted every event. A ten-minute lookback on a five-minute timer means every line is read twice, inflating scores toward a false ban. It now uses a real timestamp watermark plus a hash-based dedup table.
3. Per-address scoring missed activity spread across a subnet. Seven addresses in one /24 each scored below the per-address threshold. I added a /24 roll-up that needs both a higher total and several distinct addresses, because a range ban is a much blunter instrument. Those particular hits turned out to be old OIDC client-authentication failures from July and August, not a live brute force. Check event dates before calling something an attack.

A later one, found on 16 September: warden had read zero lines from the Authelia log for two weeks. The container it read through had exited and not restarted, and the pattern it matched didn't fit Authelia v4's wording anyway ("Unsuccessful 1FA authentication attempt"). A detector reading nothing looks exactly like a quiet week.

The URL-decoding bug

Building a Cloudflare Worker to alert on edge probes, I had it score pathname + search. That keeps percent-encoding, so a classic SQL-injection phrase with a space in it arrives as %20 and never matches a pattern written with a space. A unit test caught it.

The Cloudflare WAF rule I had written for the same pattern had the identical flaw. It now decodes the URL first. Without the test, it would have looked right and blocked nothing.

WAF rules run before Workers

Once both were live, I sent a probe the WAF blocks. It got a 403, and the Worker raised no alert. WAF custom rules run first, so the Worker only sees what the WAF lets through. Adding a WAF rule for a pattern removes the Worker's visibility of it. They don't double up. I test through the edge with curl --resolve, because my LAN resolver has split-horizon rewrites and a normal request would never reach Cloudflare.

A smaller Cloudflare API trap from the same day: GET /accounts returning 200 with an empty list does not mean the token lacks Workers access. Listing accounts is its own permission. I concluded "no Workers access" from it and was wrong. To tell read from edit, probe a write: deleting a script that doesn't exist returns 403 for read-only and 404 if you can write.

LAN scanning needs a learning window

The same work added an ARP sweep to spot new devices. Two sweeps twelve minutes apart found 48 then 46 devices, and six "new" ones were just sleeping smart bulbs, a Nest and the TV. An ARP sweep only sees what's awake, so the scanner learns for days before it alerts. A sweep that returns zero hosts is treated as an error, not as everything vanishing. The dashboard also shows suppressed findings on purpose: an invisible filter looks identical to a dead detector.

What I'd tell someone else

• Before panicking about CrowdSec, check whether the source is you. Phone apps retrying against deleted content look like probing.
• cscli metrics decisions are mostly community blocklists. Count local alerts.
• Behind a tunnel, enforce at the edge, never on the host.
• Take event time from the log line, and never let a range ban contain your own address.
• Run new detectors in detect-only first, and test that they're actually reading something.

🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.