Dev.to WebDev 🛠 Dev 👁 0 📖 4 min read

Matching the Same Product Across Five Stores: How Deal Sites Deduplicate Listings

The same air fryer, five listings, five prices — and none of them agree on the name A deal site that tracks a product at Amazon, Walmart, Best Buy, Target, and Woot is not tracking one product. It is tracking five listi

The same air fryer, five listings, five prices — and none of them agree on the name

A deal site that tracks a product at Amazon, Walmart, Best Buy, Target, and Woot is not tracking one product. It is tracking five listings that may or may not describe the same thing. One calls it "Ninja Foodi 6-in-1 8-qt 2-Basket Air Fryer", another "Ninja Foodi DZ201 8 Quart", a third just "Ninja 2-Basket Air Fryer". The prices differ, the photos differ, and the model numbers sometimes disagree with each other. If you compare them naively, you will tell a reader the same air fryer costs $79.99 and $129.99 at the same time — and both numbers will be wrong in different ways.

Deduplication — deciding which listings are the same product — is the unglamorous step that makes every price comparison on a site like Savlume possible. Here is how we think about it.

Why titles alone will betray you

Retailer titles are written to rank in search, not to identify a product. They stuff in keywords, bundle details, and marketing words ("Pro", "Max", "Plus") that may or may not mean a different model. Two listings can share 90% of their words and be different products — a 6-quart and an 8-quart version of the same fryer — while two listings of the identical product can share barely half their words, because one retailer drops the model number and the other leads with it.

So matching starts with identifiers, not words. The strongest signals, in order:

  1. UPC / EAN / GTIN. When two listings carry the same barcode, they are almost certainly the same product. The problem is that many retailer feeds omit the UPC, and marketplace sellers sometimes attach the wrong one. A UPC match is a strong yes; a UPC mismatch is not a strong no.

  2. Manufacturer part number (MPN). Model numbers like DZ201 are the next best anchor. They are usually present on the manufacturer's own pages and on spec sheets, and they survive across retailers better than titles do. But retailers abbreviate them, append color codes, or drop suffixes — DZ201 vs DZ201BK — so matching needs normalization rules, not exact string equality.

  3. Brand + key specs. When identifiers are missing, the fallback is a structured fingerprint: brand, product type, and the two or three specs that actually define the model — capacity for an air fryer, resolution and channel count for a dash cam, suction power for a vacuum. Two listings that agree on brand, type, and all defining specs are probably the same product; two that disagree on any one of them are probably not.

The traps that create phantom deals

The expensive mistakes are not missed matches. They are false matches — merging two different products into one. Three traps cause most of them:

Bundles and variants. A listing for "air fryer + cookbook" or a two-pack is not the single item, even if the title is nearly identical. Our matcher treats bundle indicators (counts, "with", "2-pack", "includes") as a hard split: a bundle never merges with the standalone product.

Color and size suffixes. Retailers encode variants in the title — "Black", "8-qt" — and a matcher that strips those words will merge an 8-quart fryer with a 6-quart one and report a price drop that is really a smaller product. Variant attributes are part of the identity, not noise to be cleaned away.

Refurbished and renewed listings. A renewed unit at $59.99 next to a new one at $99.99 is not a deal; it is a different condition. Condition is a first-class field in the product record, and a refurbished listing can never merge with a new one, no matter how identical the title looks.

How matching connects to price verification

Deduplication is not a one-off cleanup job. It feeds directly into the two-level verification pipeline we described in an earlier article: retailer-verified prices are confirmed on the retailer's own page, source-reported prices come from feeds and snapshots. A price can only be verified against the right listing, and "the right listing" is exactly what the matcher decides. Merge two products by mistake and the verification step will happily confirm a $79.99 price that belongs to a different model.

It also feeds the feeds themselves. Our API at /api/v1/items and the RSS feed at /feed/all.xml serve one record per product, not one per listing. Each record carries the offers from every retailer where the product was found, with each offer's price, retailer, conditions, and verification status. An AI assistant reading llms.txt or querying /api/mcp gets the same deduplicated view a human reader sees on the guide page — one product, several offers, no phantom duplicates.

What we still get wrong

No matcher is perfect, and ours is honest about that. New products with no UPC in any feed, retailers that invent their own model codes, and near-identical models that differ by a single letter are all cases where the system flags a possible match for a human to decide instead of guessing. A missed merge costs us a comparison; a wrong merge costs a reader money. We would rather show two listings for the same product than one listing for two products.

The lesson generalizes beyond deal sites. Any system that compares prices — a shopping assistant, a procurement tool, a price tracker — is only as honest as its entity matching. The price is the easy part. Knowing what the price is of is the hard part, and it is where most "deals" quietly fall apart.

More on how Savlume works: https://savlume.com/

Follow Savlume Deals on WhatsApp: https://whatsapp.com/channel/0029VbE9P7J7DAX7rD8TAC2U

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.