A rule that loads fine and never fires: what we measured on our own Wazuh rig

A custom rule can pass the loader, sit in etc/rules for months and never fire, and nobody notices, because the absence of an alert looks exactly like a quiet day. An agent can show active in the API with a keepalive a few seconds old and send nothing at all, and the dashboard stays green.

We are the team that built Detection Watch, a container that sits next to a Wazuh manager and looks for those two things. This post is what it found on our own lab rig on 7 November 2026, how it works, what it deliberately does not do, and where it stops.

What we measured

The rig is our own lab: Wazuh 4.14.7 in docker, seven days of alerts (1 to 7 November), 128,901 alerts in 8 files (alerts.json plus the rotated ones). The scan took 3 s and produced three kinds of finding.

An agent that looked healthy and sent nothing. One agent, an OT lab agent, was reported active by the API with a keepalive a few seconds old. Its alerts per day over the window were 3 0 0 30 38 7 0: from 30 and 38 a day down to 7, then none on the last day of the window. No rule had any reason to fire, because no event arrived. We did not stage this; it appeared on its own. Detection Watch wrote a source_silent finding, and through the shipped rules file that became rule 190002, level 10, in the manager's alerts.json.

A stock rule that went quiet. Rule 591 (level 3) fired on 5 of 6 days, 12 alerts in total, then none on the last day. That is rule_quiet, level 7 in Wazuh.

Eleven custom rules that never fired. Each one carries a label with the likely reason, using the same labels as Rule Doctor Lite (NO-MATCH, SHADOW-CANDIDATE and so on). One of them came back as NO-MATCH: sibling 100223 fired; this rule matched nothing: a rule in the same file was matching events, this one was not. That is custom_never_fired, level 5.

One note on the window. This was a hand run on the afternoon of 7 November with the window set through that day. The scheduled 02:00 run stops at the last complete day, so a partial day never counts as quiet on its own.

This is the finding line for the first case, as written to the findings file, with the agent name replaced and the bookkeeping fields removed:

{"dw": {"kind": "source_silent", "severity": "high", "rule_id": "", "agent": "ot-lab-01", "last_seen": "2026-11-06", "detail": "agent ot-lab-01: alerts/day 3 0 0 30 38 7 0, none in the last 1 day(s); API status active, last keepalive 2026-11-07T10:39:37+00:00", "measured_window": "2026-11-01..2026-11-07"}}

The measured window is printed with every finding, so when someone reads the alert a week later they know which days were counted.

How it works

Detection Watch is one read-only container next to the manager. It mounts /var/ossec/logs (alerts.json and the rotated files) and the rule directories, all read-only. It calls three API endpoints and no others: authenticate, /agents and /logtest.

From the alert files it builds the set of rule ids and agents seen per day. A rule that fired and then stopped becomes rule_quiet; a rule that is seen on fewer than 2 days is not reported as quiet, because one firing is not a pattern. A custom rule with no alerts in the window becomes custom_never_fired, with a label for the likely reason. An agent whose alerts stop while /agents still says active becomes source_silent.

Then it replays samples.json, a file of scrubbed log lines you provide, through PUT /logtest, at about 1.3 s per event as we measured it. It keeps a baseline of which rule wins for each sample and reports a replay_regression when a sample starts landing on a different rule, or none.

It also reads ossec.log for 7617 and 7619, the lines the manager writes when it ignores a rule at load (dropped_at_load), and for event-loss lines (event_loss).

Findings go out as JSON lines to one file. The manager tails that file with a <localfile> json block, and a shipped rules file, 190000 to 190006 in group detection_drift, turns each finding into an alert. The levels in that file are: source_silent 10, replay_regression 9, event_loss 9, dropped_at_load 8, rule_quiet 7, custom_never_fired 5. So the findings land on the dashboard the SOC already watches, with the levels the SOC already filters on.

<localfile>
  <log_format>json</log_format>
  <location>/var/detection-watch/out/findings.json</location>
</localfile>

What it deliberately does not do

It does not write to the manager. The only change on your side is the one <localfile> block and the rules file you copy in. It does not need the docker socket, and the replay goes through the API rather than a second logtest container.

It does not send anything out of your network. The licence is verified offline; there is no phone-home.

It does not tell you a rule is wrong. It tells you the rule has not fired, over which days, and the most likely reason. Whether that is a dead rule, a decommissioned source or a quiet week is still your call, and the finding gives you the numbers to make it.

It does not replace a rule test suite. The replay goes through the manager's own logtest, so it inherits that behaviour, including any place where logtest and the live pipeline differ.

Limits

Wazuh 4.x only. We have tested on 4.14.7. We have not tested the replay on Wazuh 5.0; the analysis engine changes there, so treat 5.0 as unsupported.

This is early access, v0.1. The three signals above are the ones we have seen fire on real data. The replay regression, event-loss and dropped-at-load paths are implemented but produced no finding on our rig in this window (the replay ran against our samples and each landed on the expected rule), so we cannot show you one of those yet.

The 2-day floor means a rule that fires once a month will never be reported as quiet. That is a deliberate trade against noise, and it is a gap.

How to try it

The tarball is at https://vct.atkvn.com/dl/dw/detection-watch-0.1.0.tar.gz, with the sha256 in the .sha256 file next to it. Setup is docker compose up -d, one API user with read and logtest permissions, your samples.json, a copy of the rules file, the <localfile> block above, and a manager restart. A 14-day trial licence for one manager comes from the form on the Detection Watch page (the canonical link of this post); after that it is USD 29 per manager per month. If you only want a one-off look at which custom rules are likely dead, Rule Doctor Lite on PyPI is free: pipx install rule-doctor-lite.

We would like to know how you catch quiet rules today. A saved search that someone remembers to run? A dashboard panel of rule ids per day? Nothing, and you found out the hard way? Tell us in the comments; the cases we have not seen are the ones we most need.