I re-ran my collector to be safe. It invented 171 changes.

Last Monday I re-ran a collector I had already run that morning. Not for any good reason. The first run finished in about eleven minutes and I didn't trust it.

The dashboard went from quiet to 171 changes.

Nothing upstream had moved. I checked. Eleven hours, same weekday, same registry dump. The 171 was my own second sweep arguing with my first one, and my page was reporting the argument as news.

I kept a screenshot. It's the most useful bug report I have on the project, and I'll explain why I didn't delete the evidence.

What that page actually counts

One of my series is a weekly census of MCP registry endpoints. It walks the registry, pings every server, and sorts each one into PASS, AUTH_GATED, UNKNOWN and so on. Around 35,000 servers on a normal week.

The interesting output is the delta: which endpoints changed status since last time, which vanished, which came back.

Every run record carries six date and comparison fields, and I added them over a couple of months, each one after getting burned. The two that matter here:

scanDate:                2026-11-02
generatedAt:             2026-11-02T10:16:33.783Z
comparedAgainst:         2026-11-02
comparedAgainstComplete: true
baseline:                false
statusChanged:           171

scanDate is the date the upstream record describes. generatedAt is when my sweep actually ran. comparedAgainst is what the diff was taken against. On this row, scanDate and comparedAgainst are the same string. The run diffed the day against itself.

That should be impossible to report as a change. It wasn't, because nothing in the code asked the question.

Three things I wanted to do, and didn't

Overwrite the old snapshot so the page always shows the newest numbers. Easy, and it would have deleted the only record of what the pipeline did. The whole premise of these series is that the past can't be re-collected. Silently rewriting my own history is a strange way to defend that.

Backfill the bad events with the fixed logic. Also tempting. Same problem in reverse: the log stops being a log. It becomes my current opinion about the past.

Write 0 for the endpoints that timed out. A dashboard without holes looks healthier than it is. A timeout isn't a zero, and I'd have traded a fake 171 for a fake 0.

I left all 41 events alone. 40 of them came from that one same-day re-run — 20 flipping up, 20 flipping back down. They're wrong as claims about the world and accurate as a record of what my collector did on the 2nd.

The rule I should have had from the start

A run may emit a change event only if both of these are true:

1. baseline: true. The run compared against a complete previous snapshot, not a partial one.
2. The comparison crosses an ISO calendar week boundary.

Everything else is a re-scan. It refreshes stored state. It does not touch the published number.

That second condition sounds bureaucratic until you look at the run before the bad one:

comparedAgainst:         2026-10-30
comparedAgainstComplete: false
comparedAgainstCoverage: 9991
serversThisWeek:         35327
addedUncomparable:       25382
removed:                 46

9,991 out of 35,327. The previous snapshot covered 28% of the registry, and the diff cheerfully reported the other 25,382 servers as "added, uncomparable". Both numbers are honest — which is the point. Without comparedAgainstCoverage sitting right there, I would have read 25,382 as growth.

Where the rule change went

Changing what "a change" means is not a code fix, it's a definition change. Applying it backwards would quietly falsify two weeks of published events.

So there's a file on the site listing every definition change, in order, with the date it takes effect, the commit, and the reason. This is the row I added on 2026-11-04:

trust-delta
Change detection moved from "difference between two sweeps"
to "ISO calendar week". Same-day or same-week re-runs only
refresh stored state and produce no events. Events may only come
from baselineEligible anchors, and must cross a calendar week.

Reason: the 2026-11-02 same-day re-run (scanDate == comparedAgainst)
contributed 40 of 41 events (20 up, 20 down) and made the page
report 171 changes. That was re-scan noise, not change over time.

Commit 44a3bc6. Older events keep the old meaning; newer ones follow the new rule; the log is the only place that says which is which. The log gets written before the rule ships, and it never rewrites history. I've broken the first half once already, which is why it's now in the README.

The same bug wearing a different hat

While chasing the 171 I found the sibling problem: a source that answers with an anti-bot page is not a source with no data.

Two rows from collect.events, unedited:

2026-10-30 09:06:48  sb253-reporting  HTTP 403, 919 B (challenge page)
                     retry -> 09:10:34 -> 200, 58,613 B
                     data-source-recovered, no gap in the series

2026-10-30 09:09:34  sb253-program    HTTP 403, 919 B
                     retry -> 09:10:43 -> 200, 61,221 B

919 bytes of HTML isn't a measurement. The state machine now has a branch between success and dead: 202/403/417/429 or an empty body gets one retry with backoff, and if it still fails it lands as data-source-unavailable with the status code and the byte count. The last valid value stays, marked as such.

The old design blocked the whole round on any unavailable page. One challenge page could stop a week of collection without saying anything. Now a source can be declared tolerate: true in the registry with a written reason — a CARB docket that returns 417 to anything that isn't a browser, a CME path that went 404 — and the round keeps going, with the event logged. If a series' primary source is missing, the round still fails. It just fails loudly instead of pretending.

Four gates between the data and the page

Publishing runs four checks in order. Any non-zero exit stops the round.

6 of 6 blocked last time I ran it. The gate writes assurance/.gate/result.json with the head commit and an inputs_hash, and each round exports assurance/artifact-<date>.json with the exit codes and a sha256 per data file.

Which is the part that matters more than any of the above. You can clone the repo, run the gate, and compare hashes. That's the only reason to trust a stranger's dashboard.

I still label every series E0: recheckable by anyone, recomputed by nobody. The pages say verified_by_third_party: false instead of leaving you to assume otherwise.

If you're building a change detector on public data

Some of this is obvious in hindsight, all of it cost me a week:

Keep scanDate and generatedAt as separate fields. The gap between them is where the bugs live.

Store what you compared against and how complete it was. A diff against a 28% base is not a diff.

Give runs an explicit baseline flag and decide in code, once, which runs may anchor future comparisons. Don't decide it in your head at read time; you'll decide differently at 1am.

Make same-day and same-week re-runs incapable of producing events. "Re-run it to be sure" should be a safe operation. Right now, in most pipelines I've seen, it isn't.

Log definition changes before they ship. Never rewrite the events those definitions already produced.

Record the HTTP status and byte count of every failed fetch. Status is how you tell "the source blocked me" from "the source is gone".

I'm not going to claim this is now correct. It's correct for the failure I hit, and the log has a row for the next one.

The method page has the full failure table and the definition log: https://xka0085-byte.github.io/longgame/method.html

Code and data: https://github.com/xka0085-byte/longgame

There's a frozen 2026-11-05 snapshot with a DOI if you want a fixed copy: https://doi.org/10.5281/zenodo.23998349

And a read-only mirror on Hugging Face: https://huggingface.co/datasets/Eidonze/longgame-public-data-series