204 of a board's 232 jobs were "updated" in the same second

I aggregate job boards. One of the features is "posted in the last 24 hours".

For a week it worked. Then I pulled a full board and asked a simpler question than I usually ask: what does the updated_at distribution actually look like?

GitLab's board, 232 live postings:

So my "new in the last 24 hours" feed was reporting 204 jobs as fresh. Their real first-published dates span seven months. Every one of them was 222 days old at the most, and one of them was a March posting wearing an October timestamp.

Nothing errored. The API returned 200, the timestamps parsed, the sort worked. The output was just wrong in the way that survives review — plausible, formatted, and committed.

The timestamp is a column about the record, not about the posting

updated_at is a data-quality field. It answers "when did this row last change in our database", and a board that re-syncs its whole catalogue changes every row at once. That's the 204. It is not a bug on Greenhouse's side; it's a field being asked to answer a question it was never about.

The tell is the histogram. Twenty-nine distinct values across 232 rows is not a posting pattern. Postings arrive one at a time — a company does not hire 204 people in the same second. A single value covering 88% of the board is a write operation.

Compare the other two vendors I hit, same question, same day:

Ashby: 814 postings, 814 distinct timestamps. Lever: 82 and 82. Those two answer "when was this posted" with a per-posting event. Greenhouse answers with a per-row write. Same logical field, opposite reliability — and if you wrote one normaliser and trusted it, you would only be wrong on one third of your sources, which is exactly the amount of wrong that doesn't get noticed.

What actually works

Two fields, one local set.

# DON'T: this is a "board sync" feed, not a "new jobs" feed
new_jobs = [j for j in jobs if is_recent(j["updated_at"])]

# DO: structural date for recency, id for novelty
seen = load_seen_ids()                      # persisted between runs
first_run = not seen
new_jobs = [
    j for j in jobs
    if j["id"] not in seen
    and (first_run or parse(j["first_published"]) > last_run_at)
]
save_seen_ids(seen | {j["id"] for j in jobs})

first_published is the only Greenhouse field that behaves like a posting date. But it still can't tell you newly new on its own — a job published in March and still open is legitimately in the feed forever. Recency is a property of the posting; novelty is a property of your observation. Those are two different clocks and they need two different fields.

The first_run branch matters more than it looks. On a cold start, every posting is unseen, so the difference set is the whole board. Without that branch you either publish 232 "new" jobs on day one, or you silently drop the board and never recover. I hit both.

This is the same failure as the missing-key trap in the first post of this series — a field that exists, parses, and returns a value that means something other than what you assumed. null at least stops you.

Two more things that cost more than they look

?content=true is a 22× tax. GitLab's board, same 232 jobs either way:

The job count is identical. You get the same 232 records with a content key added. Fetch descriptions on the detail endpoint for the jobs you actually keep, not on the listing that exists to be filtered.

And note the middle column versus the right one: 22× and 7× are the same request. If you benchmark bandwidth with curl --compressed and then reason about your JSON parser's memory, you are off by a factor of three and you will never find it, because both numbers are real and neither is labelled. I have made this mistake in a report; the fix is to write the encoding into the sentence, not the number.

limit can be decorative. Remotive accepts ?limit= and ignores it:

limit=3   -> HTTP 200, 173,720 B, job-count 13
limit=50  -> HTTP 200, 173,720 B, job-count 13

Byte-identical. Both responses are the same 13 jobs. A limit parameter that parses, validates, and does nothing is worse than no parameter — you'll build a paging loop on it and silently stop at whatever the server felt like sending. The only way to know a parameter works is to send two values and compare the counts, not the status codes. Both were 200.

The rule I write down now

For every timestamp I consume, I check the histogram before I check the value:

• Distinct values ≈ row count → it's an event field. Trust it for recency.
• One value covering most rows → it's a write field. It records your vendor's cron, not the world.

That check costs one Counter() and it is the difference between a feed and a random-number generator. The 204 rows were not lying to me. They were answering the question I should have asked.

Written with AI assistance and verified by hand — every board, count and byte figure above was pulled live on 2026-10-18. The normalisers are in ats_normalizers.py.