How a retail media network can verify product presence in video creative
A retail media network verifies product presence in video creative by matching the advertiser's approved product images against frames from across the whole video, then confirming each match before the ad is approved. In
A retail media network verifies product presence in video creative by matching the advertiser's approved product images against frames from across the whole video, then confirming each match before the ad is approved. In practice that means four steps: cut the video into time-coded segments; search those segments with the SKU's packshots, label text and product name; have a vision model check each candidate segment for the right product at a usable size; and record the evidence (timestamps, frames, scores, the matched SKU). Borderline cases go to a person. This checks the submitted creative. It does not prove the ad ran or was seen.
Why this check exists
Retail media networks sell placements to brands whose products the retailer stocks. A video ad running on a retailer's site, app or in-store screens is expected to feature the product being promoted, often a specific SKU the retailer carries. Ad review teams usually check this by hand: someone watches the video and ticks a box. That works at a few dozen ads a week. It breaks when a network takes creative from hundreds of brands and agencies, with several cuts and lengths per campaign.
The questions a reviewer is actually answering:
- Is the promoted item on screen? A brand logo alone does not count.
- Is it the right SKU? Same brand, wrong flavour or pack size is a common miss.
- Is it visible enough? On screen long enough and large enough to count, by the network's own rule.
- Where is it? Timestamps a reviewer can jump to, so a human check takes seconds rather than a full watch.
What a verification pipeline looks like
Reference set per SKU. Start from the retailer's catalog: front and back packshots, packaging variants, the product name and the text printed on the label. Matching against this SKU's own images is what separates it from the rest of the brand's range.
Segment the video. Split on shot changes so each piece has a start and end time. A five-second product shot inside a 30-second ad then becomes its own searchable item.
Find candidates three ways.
- Visual similarity between each packshot and each segment's embedding finds the product even when the label is unreadable.
- On-screen text (OCR) catches the product name on packaging or in supers.
- The transcript catches the product named in the voiceover. That supports a match but never proves the product is visible.
Confirm with a vision model. Similarity scores only rank candidates. To decide whether a segment shows the 12 oz original, front-facing and clearly visible, send the top segments, with their frames, to a vision model along with the SKU description and the network's pass rule, and keep its yes or no with a reason.
Return evidence. Store the segment times, a thumbnail, the scores, the model's reason and the matched SKU ID. Approve clear passes, reject clear misses, queue the middle for a person, and keep the record for the advertiser and for audit.
Typical failure cases to test before you trust it: lookalike competitor products, new packaging that is not in the catalog yet, very short appearances (under a second), heavy motion blur, and products held partly out of frame.
Building it on your own creative library
Disclosure: I work on Mixpeek.
Mixpeek is a developer API, so this is something your team builds on top of it. It does not connect to an ad server or a retail media platform and it does not approve ads; you wire its results into your own review queue. The videos stay in your own object storage.
1. Index every submitted ad by scene. The multimodal extractor splits video into segments (by time, scene or silence) with a start time on each, and can add a transcript, OCR text and a description per segment, along with a multimodal embedding that text, images and video share. Pass your creative ID and campaign fields through so every segment carries them.
2. Search the segments with the SKU's packshot. A feature search stage takes an image URL as the query ("input_mode": "content"), embeds it the same way the video was indexed, and returns the closest segments with scores. Filter on the creative ID so you search one ad at a time. Run a second search on the OCR text for the product name, and fuse the two.
3. Confirm presence with an LLM filter. The LLM filter stage judges each candidate against criteria you write, such as "Keep only segments where the {{INPUT.product_name}} package is clearly visible." With include_media: true the model sees the segment itself (video for Gemini, the thumbnail for other providers), as well as its text fields.
4. Write the result back. The retriever returns the kept segments with their times and thumbnails. That list is your evidence; an empty list means "not found" and goes to a reviewer.
On cost, at the public rates when I checked on 17 November 2026 (pricing), video search is 0.066 USD per minute of video and on-screen text 0.1098 USD per minute, billed once at indexing. A week of 500 thirty-second ads is 250 minutes.
Whatever you build it with, send the borderline cases to a person, and measure the misses on your own creative before you let it approve anything on its own.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.