Common Crawl's web graph shed 70% of its links this year. Referring domains barely moved.

Common Crawl publishes a domain-level hyperlink graph every month, each one built from three consecutive crawls. Its own published totals show something odd this year: the number of domains has stayed roughly flat, while the number of links between them has fallen by more than two thirds.

I maintain crawlgraph, which serves those graphs as a backlink API, so I wanted to know whether the drop shows up in what people actually look up: the referring domains of a real site.

The published totals

Every graph is listed, newest first, in Common Crawl's graphinfo.json:

curl -s https://index.commoncrawl.org/graphinfo.json \
  | jq -r '.[] | [.id, .stats.nodes, .stats.arcs] | @tsv' | head -10

From Nov-Jan to Aug-Oct, domains went from 122M to 135M and links went from 6.11B to 1.82B. Common Crawl's release notes do not give a reason, and I am not going to guess one. These are their numbers, not my estimates.

What happened to real sites

crawlgraph moves one full quarter at a time, so it compares Apr-Jun with Jul-Sep: two graphs with no source crawl in common. Across that pair the link total fell 45%. I counted referring domains for fourteen established sites in both releases:

Every count stayed between -1% and +15%. So whatever removed 1.75B links between those two graphs, it was not spread evenly across the referring domains of sites like these. Fourteen sites are a sample, not the whole graph: a small or new site can still move more.

The number that did move: membership

The last column is the interesting one. 22% to 37% of each site's Apr-Jun referring domains were not observed linking in Jul-Sep, and a similar number of new ones took their place. The totals look stable because the churn roughly cancels out.

Two practical consequences if you work with snapshot link data, from any provider:

1. A missing referring domain is an observation gap first. It means the domain was not seen linking in those crawls. Before you email anyone about a "lost link", fetch the page and check. I wrote up the check I use.
2. Compare snapshots that do not share crawls. Consecutive monthly graphs share two of their three crawls, so a diff between them understates real change. A quarter apart, the sets are independent.

Reproduce it

With a crawlgraph key, the per-domain diff is one call. It counts against the same quota as a backlink lookup, and the free tier gives 15 a month:

curl "https://crawlgraph.com/api/v1/changes?domain=basecamp.com&from=cc-main-2026-apr-may-jun&to=cc-main-2026-jul-aug-sep" \
  -H "Authorization: Bearer $CRAWLGRAPH_KEY"

For basecamp.com it returns 11,750 and 11,967 referring domains, with 3,978 added and 3,761 removed. Without a key you can do the same from Common Crawl's raw graph files. They are large: the Jul-Sep edge list is 2.15B rows, so budget disk and memory before you start.

The full release notes, with the crawl IDs and how to pin a release, are here. If you know why the link totals dropped, I would like to hear it in the comments.