I tested 4 AI image detectors on 100 images. Here's what I'd tell a friend.

I've been building a small AI image checker, and at some point I had to pick the detection engine behind it. Every vendor publishes an accuracy figure. None of them says how it was measured, so I couldn't compare them. I ran my own test instead. It's small and it has gaps, which I list below, but it changed how I think about these tools.

The test set

100 images, all run on October 13, 2026.

50 real photos, mostly from Wikimedia Commons under CC0 or Creative Commons licenses. 38 are original files: phone shots, portraits, landscapes, product photos. 37 of the 50 still had a camera model in their EXIF, from iPhone, Pixel and Galaxy phones. The other 12 are copies I made to look like photos that have travelled: 4 phone-style screenshots, 4 heavily recompressed copies of the kind chat apps produce, and 4 lightly retouched versions.

50 AI images from six generator families: OpenAI GPT Image, Google Nano Banana, Black Forest Labs FLUX.2, Midjourney (v6 to v8.1), ByteDance Seedream and Stable Diffusion XL. 42 come from generators released in 2025 or later. Of the 40 original AI files, I generated 27 for this test (about $1.42 in API calls) and took 13 from openly licensed datasets on Hugging Face. The last 10 are phone-style screenshots of AI images, saved as PNG with no metadata at all.

How I scored it

Every engine got the same files. I turned each answer into "AI" or "real" with one fixed rule per engine: a Sightengine score of 0.5 or higher, Hive's own recommended threshold of 0.9, and the vendor's own verdict for AI or Not and WasItAI. An "unknown" answer counted as wrong.

One caveat before the numbers: only two engines saw all 100 images. I stayed within free trial allowances, so AI or Not ran on a stratified sample of 20 and WasItAI on 9.

The results

Please don't read the bottom two rows as a ranking. Twenty and nine images are too few. AI or Not's sample was also harder than the full set, with a bigger share of screenshots and recompressed copies, and its one false alarm was a lightly retouched photo. On the 9 images that all four engines saw, Sightengine and Hive got 9 of 9, AI or Not 8 of 9 and WasItAI 7 of 9. That's a hint, not a verdict.

By generator, FLUX was the hardest: Sightengine caught 8 of 9 FLUX images and Hive 6 of 9. Both engines caught every Midjourney image.

Why I watch false alarms more than accuracy

"Accuracy" mixes two very different mistakes. My test harness has a fake engine that answers "AI" for every image, used only for wiring things up. On this set it catches all 50 AI images and still scores 50% accuracy, because it also flags all 50 real photos.

The two mistakes don't cost the same. A missed AI image is a shame. A real photo called AI lands on a person: a buyer with a genuine refund claim, an artist who drew every line, someone on a dating app who just takes good photos. If most of the pictures you check turn out to be real, even a small false-alarm rate can produce more wrong accusations than right ones.

Two more things I didn't expect:

• The threshold is a choice. At Hive's recommended cutoff of 0.9, it flagged no real photos and caught 90% of the AI images. At 0.5 it would have caught 96%, but flagged 1 real photo in 50. Every product built on these engines picks a point on that curve, usually without telling you.
• Zero isn't zero. Sightengine and Hive both flagged 0 of 50 real photos. With only 50 photos, the 95% confidence interval still runs from 0% to about 7%. That's a good sign from a small test, not a guarantee.

Content Credentials: the free check most people skip

Some AI tools now sign the files they produce, using the C2PA standard, known publicly as Content Credentials. The record names the tool and is cryptographically tied to the file. Reading it costs nothing and can happen in the browser.

In my set, 24 of the 50 AI images carried Content Credentials, and none of the real photos carried an AI claim. The signers I saw were OpenAI, Google, Black Forest Labs and BytePlus (ByteDance). All 24 were original files: 24 of the 40 AI originals carried credentials. One catch I only took seriously later: a valid signature tells you who signed, not whether to trust them. Only 14 of those 24 were signed by companies on the official C2PA trust list (OpenAI and Google). The other 10 were signed with certificates that aren't on it, and a self-made certificate can sign anything, so my checker treats those, and plain metadata labels like Midjourney's, as a claim and still runs the detection model.

The catches:

• Midjourney files never had them (0 of 7 originals), and neither did Stable Diffusion XL (0 of 5).
• Screenshots lose them. None of the 10 screenshots carried credentials. A screenshot is a brand-new file, so the signature doesn't come along.
• Some of my files may have lost them earlier. Several came from public datasets, where a re-saved copy can drop the record, so a low count doesn't prove a generator never signs.
• Cameras sign too. Two real photos, from a Google Pixel 9 and a Pixel 10, carried camera credentials, and credentials never marked a real photo as AI.

So credentials are close to proof when they're there, and their absence tells you nothing. If you meet a viral image as a screenshot, expect to find none.

The table uses one simple cutoff at 0.5 so the engines can be compared. The tool itself is more cautious: it calls an image AI above 0.8, real below 0.2, and says "can't tell" in between. Reading credentials first and sending only the rest to Sightengine, with those two lines, the result was 98 of 100 correct, 1 "can't tell" (a screenshot of a real photo scoring 0.21), 1 wrong (a FLUX screenshot scoring 0.18, called real), and still 0 of 50 real photos flagged. 16 images were settled by trusted credentials without a model call (14 AI images plus 2 Pixel camera photos signed as real captures); the other 84 went to the model.

Screenshots cost the models a little too. On 10 pairs of an AI original and its screenshot, Sightengine went from 10 caught to 9, and Hive from 10 to 8.

A third answer: "edited with AI"

While I was building the set, one Pixel photo's metadata said "Edited with Google AI". It isn't fully real or fully generated, so I swapped it out of the real photos. Both main engines had called it real, which is fair enough, but it shows that "real or AI" is becoming the wrong question for a lot of phone photos. A useful checker needs a third answer.

What this test can't tell you

• It's small: 100 images, one day.
• Two of the four engines only saw samples.
• It's missing dating-app selfies, heavily filtered portraits, hand-made artwork, and composites or partial AI edits. There are only a few illustrations.
• One real photo went through Sightengine and Hive a few hours after the rest, once their daily free quotas had reset.
• Vendors update their models, so this is a snapshot.
• I'm not neutral. The checker I built now uses Sightengine, with Hive as a backup. No vendor paid for the test or saw it first, and all four ran on free trial credits.

How I'd check a single image

If you just want to know about one picture, this is the order I'd use:

1. Look for Content Credentials. The Content Authenticity Initiative's free Verify tool (contentcredentials.org/verify) reads them. If it names an AI generator, you're mostly done. If it shows nothing, move on; that's normal.
2. Find the earliest copy. Run a reverse image search with Google Lens, TinEye or Bing. The first place an image appeared often tells you who made it and why.
3. Ask whether the scene makes sense. The right uniforms, the right road signs, the right weather for the date. This check doesn't get weaker as generators improve.
4. Then look closely. Small text, hands holding things, reflections, shadows, repeated faces in a crowd. Treat these as clues: one odd detail doesn't prove AI, and a flawless picture doesn't prove a camera.
5. Use a detector as one more clue. Prefer one that shows a range and its reasons, and that is allowed to say "can't tell". Use the original file if you can, because screenshots are harder.

Whatever you find, don't use a detector's answer alone to accuse a real person.

I put this credentials-first flow into a small free checker, and the full method, every number by generator and the limits are on its accuracy page: https://aiimagedetection.app/ai-image-detector-accuracy/. If you have an image it gets wrong, I'd like to see it.

I used an AI assistant to help draft and edit this post. The test set, the runs and the numbers are my own.