Most AI-image benchmarks are built from generators that are two years old. Detectors tuned on them look excellent on paper and miss what is being posted today. So we built a new set in October 2026: AI images from 19 current generators, AI clips from 12 video models, and freely licensed real photos and clips.
Every image also exists as a WhatsApp, an Instagram and a re-shared copy, and every clip as a messenger copy, because that is the form in which content reaches a trust & safety or OSINT team. 60% of the set is public on Hugging Face; the other 40% is held back, and every number on this page comes from that hidden part.
TL;DR
- Images: WasItAI V3 caught 99.5% of AI images at a 1% false-positive rate (AUROC 1.000). ZeroGPT (TruthScan engine) caught 97.2%, the best open-weights detector 48.1%.
- Social-media copies: after WhatsApp, Instagram and re-share compression, V3 still caught 96.2%. The best open detector dropped to 37.3%.
- Video: AUROC 0.987 on original clips and 0.964 on messenger copies. In production settings, 60 of 63 AI clips caught and 1 of 60 real clips flagged.
- Caveat before you quote anything: we built the set ourselves and our model has seen similar content in training. Our numbers are a home-field result.
Images
AUROC measures how well a detector separates AI from real across all thresholds (1.0 is perfect, 0.5 is a coin flip). "Detected @ 1% FP" is the share of AI images caught when the threshold is set so that 1 in 100 real photos is flagged, which is the operating point that matters when you screen millions of files.
| Detector | AUROC | Detected @ 1% FP | AUROC, social copies | Detected @ 1% FP, social |
|---|---|---|---|---|
| WasItAI V3 | 1.000 | 99.5% | 0.998 | 96.2% |
| ZeroGPT (TruthScan engine) * | 0.992 | 97.2% | – | – |
| haywoodsloan/ai-image-detector-deploy | 0.911 | 48.1% | 0.875 | 37.3% |
| OmniAID (CLIP) | 0.840 | 34.6% | 0.710 | 9.8% |
| Smogy/SMOGY-Ai-images-detector | 0.802 | 11.1% | 0.768 | 6.2% |
| Ateeqq/ai-vs-human-image-detector | 0.789 | 4.8% | 0.785 | 5.9% |
| WasItAI V2 (our previous model) | 0.787 | 23.1% | 0.742 | 9.1% |
| DDA (NeurIPS 2025) | 0.742 | 14.4% | 0.677 | 8.5% |
| Organika/sdxl-detector | 0.673 | 0.5% | 0.653 | 0.8% |
| dima806/ai_vs_human_generated_image_detection | 0.617 | 0.0% | 0.627 | 0.0% |
| Cyberdon360/ai-vs-real-image-detector | 0.599 | 2.4% | 0.604 | 2.2% |
| umm-maybe/AI-image-detector | 0.466 | 1.0% | 0.451 | 0.5% |
Hidden split: 208 AI and 314 real originals; social copies are 3 per image (624 AI, 933 real). * ZeroGPT, which uses TruthScan for image and video detection, was run on originals only and returned no score for 65 files (moderation block or error); its row covers the 457 it scored. On those 457, WasItAI V3: AUROC 0.9995, 99.5% detected at 1% FP.
Per generator, V3 caught every hidden original from 18 of the 19 generators at its default threshold, including GPT-Image-2, Nano Banana 2.1 and Pro, FLUX.3 and Qwen-Image 3. Seedream 5 Pro was at 90%.
Video
63 AI clips from 12 models against 60 real clips. The open detectors are image models, so we scored the same frames from each clip with them and averaged.
| Detector | AUROC, original | AUROC, messenger copy |
|---|---|---|
| WasItAI V3 (video model) | 0.987 | 0.964 |
| WasItAI V2 (our previous model) | 0.869 | 0.666 |
| DDA (NeurIPS 2025) | 0.839 | 0.623 |
| OmniAID (CLIP) | 0.813 | 0.618 |
| haywoodsloan/ai-image-detector-deploy | 0.770 | 0.658 |
| Smogy/SMOGY-Ai-images-detector | 0.677 | 0.596 |
At the threshold we run in production, our video model flagged 60 of 63 AI clips and 1 of 60 real clips.
How the set was built
- AI images: 469 new images from GPT-Image-2, Nano Banana 2.1, Nano Banana Pro, Meta Muse Image, Microsoft MAI-Image 2.5, Grok Imagine 2, Seedream 5 Pro, Qwen-Image 3, Qwen-Image 2512, Ideogram 4.5, FLUX.3, FLUX.2 Pro, FLUX.2 dev, Kling Image 3, Recraft 4.1, Luma Uni-1, Z-Image Turbo, HiDream O1, Krea 2.
- AI videos: 164 clips from Veo 3 Fast, Kling 3, Seedance 2.0, Hailuo 02, MiniMax H3, Wan 2.6, LTX 2.3, HunyuanVideo 1.5, Luma Ray 3.2, Grok Imagine Video 1.5, Vidu Q3, PixVerse 6.
- Real: 800 photos and 160 clips under free licenses (CC0, public domain, CC BY, CC BY-SA), with author and source for every file.
- Never trained on: no file in the set was used to train our models.
- Metadata removed: generators embed C2PA credentials ("created by …"). We stripped all metadata so a detector cannot read the answer from the file header. Pixels are unchanged.
- Split: 60% public, 40% hidden, assigned by hash.
About this benchmark
We built and ran this benchmark ourselves, and our models have seen similar content in training, so treat our numbers as a home-field result. That is exactly why 60% of the files are public: anyone can rerun it, and vendors can submit their detector for the hidden split.
Download it and submit your detector
The public 60% is on Hugging Face: images, social copies, videos and a metadata.csv with label, generator, license and source per file. Evaluation scripts that download the split, run any detector and report the same metrics are on GitHub for images and video. The hidden 40% stays private, but the SHA-256 of every hidden file is published in those repositories, so anyone can check later that the hidden split was not changed.
Building a detector? Run it on the public split and email patrick@wasitaigenerated.com. We score submitted detectors on the hidden 40% and list them here, including results that beat ours.
FAQ
Which AI image detector is the most accurate in 2026?
On this test set, WasItAI V3 separated AI from real images almost perfectly (AUROC 1.000) and caught 99.5% of AI images while flagging 1% of real photos. The best open-weights detector caught 48% at the same false-positive rate. We built the test set ourselves, so read our numbers as a home-field result.
Does WhatsApp or Instagram compression break AI image detection?
It hurts every detector. WasItAI V3 went from 99.5% to 96.2% detected at a 1% false-positive rate after WhatsApp, Instagram and re-share compression. The open detectors lost more, the best one from 48% to 37%.
Can AI-generated video be detected?
Yes, but compressed video is the hard case. Our video model reached AUROC 0.987 on original clips and 0.964 on messenger-compressed copies. At the threshold we run in production, it flagged 60 of 63 AI clips and 1 of 60 real clips.
Can I test my own detector on this set?
Yes. 60% of the set is public on Hugging Face. Run your detector on it and email patrick@wasitaigenerated.com; we score submitted detectors on the hidden 40% and add them to this page.
Check an image or video yourself
Upload it to WasItAIGenerated: the same detector used in this benchmark.
Test it free