This report gives the full results behind the accuracy figures on our website. It covers the best result from each AI Aware detector in the 12 weeks to 7 November 2026: text, images and video, and audio. Alongside the headline figures, it shows how each detector performs by type of content, by AI model and on content it has never seen before, including the areas where it is weaker.
Summary
| Detector | Test set | Samples | Accuracy | AUROC |
|---|---|---|---|---|
| Text | AI Aware modern evaluation set v4, 14 types of writing | 10,334 | 98.6% | 0.998 |
| Images and video | Pooled held-out test set | 43,106 | 99.4% | 0.9996 |
| Audio | BRSpeech-DF validation set | 2,000 | 97.6% | 0.995 |
AUROC (area under the ROC curve) measures how well a detector separates real from AI content across every possible cut-off. 1.0 is perfect separation and 0.5 is no better than chance. Accuracy is the share of items labelled correctly at the cut-off we use in the product.
Text detector
The text detector was tested on 10,334 English texts: 5,166 written by people and 5,168 generated by 30 AI models. Texts cover 14 types of writing, from academic papers and legal documents to Reddit posts and recipes. None of these texts were used in training. Before testing, markup is stripped and any text that directly names the AI that wrote it is removed, so the detector cannot rely on giveaways. Texts longer than 2,048 tokens are cut to that length.
| Measure | Result | 95% confidence interval |
|---|---|---|
| Accuracy | 98.64% | 98.42% to 98.86% |
| AUROC | 0.9978 | 0.9970 to 0.9985 |
| Average precision | 0.9983 | 0.9977 to 0.9988 |
| Macro F1 | 0.9864 | 0.9842 to 0.9886 |
| Labelled human | Labelled AI | |
|---|---|---|
| Human text (5,166) | 5,106 | 60 |
| AI text (5,168) | 81 | 5,087 |
By type of writing
| Type of writing | Texts | Accuracy |
|---|---|---|
| Academic | 900 | 99.2% |
| Encyclopedic | 600 | 98.5% |
| Fiction | 800 | 99.4% |
| Instructional | 800 | 99.0% |
| Legal | 1,200 | 97.5% |
| Listings and digests | 334 | 99.7% |
| News | 1,000 | 99.5% |
| Poetry | 500 | 95.6% |
| Recipes | 500 | 94.8% |
| Reddit posts | 800 | 98.8% |
| Reviews | 900 | 99.3% |
| Social media | 400 | 98.5% |
| WikiHow | 700 | 99.9% |
| Wikipedia | 900 | 99.4% |
Accuracy is above 97% for every type of writing except poetry (95.6%) and recipes (94.8%). Both are short, formulaic forms where human and AI writing look alike.
By length
| Length (tokens) | Texts | Accuracy |
|---|---|---|
| Under 128 | 2,390 | 96.5% |
| 128 to 255 | 2,305 | 99.0% |
| 256 to 511 | 3,406 | 99.3% |
| 512 to 1,023 | 1,970 | 99.5% |
| 1,024 to 2,047 | 255 | 100.0% |
| 2,048 and over | 8 | 100.0% |
Very short texts are the hardest. Below 128 tokens, about 95 words, accuracy is 96.5%. From 128 tokens upwards it is 99% or higher.
By AI model
| Source | Texts | Accuracy |
|---|---|---|
| Human (real) text | 5,166 | 98.8% |
| OpenAI GPT-5.6 Terra | 453 | 99.6% |
| Mistral Medium 3.5 | 422 | 99.8% |
| OpenAI GPT-5.6 Luna | 422 | 100.0% |
| Anthropic Claude Haiku 4.5 | 401 | 99.8% |
| DeepSeek V4 Flash | 401 | 100.0% |
| Google Gemini 3.8 Flash | 401 | 99.5% |
| Qwen 3.8 Flash | 401 | 99.5% |
| xAI Grok 4.6 | 401 | 90.8% |
| OpenAI GPT-5.6 Sol | 252 | 99.6% |
| Anthropic Claude Sonnet 5 | 251 | 98.0% |
| Z.ai GLM 5.3 Flash | 150 | 99.3% |
| Google Gemini 2.5 Flash | 140 | 95.7% |
| OpenAI GPT-5 nano | 107 | 95.3% |
| DeepSeek V3 | 103 | 97.1% |
| OpenAI GPT-5 | 102 | 97.1% |
| Anthropic Claude Opus 5 | 100 | 100.0% |
| Moonshot Kimi K3 | 100 | 99.0% |
| Anthropic Claude 3.7 Sonnet | 84 | 97.6% |
| Meta Llama 4 | 69 | 100.0% |
| Anthropic Claude Sonnet 4 | 66 | 98.5% |
| xAI Grok 4 | 59 | 98.3% |
| DeepSeek R1 | 55 | 100.0% |
| OpenAI GPT-5 mini | 52 | 94.2% |
| Anthropic Claude Fable 5.1 | 50 | 94.0% |
| Z.ai GLM 5.3 | 50 | 98.0% |
| Mistral Medium 3 | 27 | 100.0% |
| Qwen Turbo | 23 | 100.0% |
| xAI Grok 4.3 | 14 | 100.0% |
| Google Gemini 3.7 Flash | 7 | 100.0% |
| Qwen Max | 5 | 100.0% |
The detector catches 99% or more of text from most current models, including GPT-5.6, Claude Opus 5, Gemini 3.8 Flash and DeepSeek V4. The weakest result is xAI Grok 4.6 at 90.8%.
Image and video detector
The image detector was tested on 43,106 held-out images: 24,047 real photographs and 19,059 AI-generated images from 28 generators. Real images come from COCO, Wikimedia Commons, Flickr, Pexels and the NTIRE challenge dataset. Video is checked by breaking it into frames and testing each frame with this detector, so these results apply to video too.
| Measure | Result |
|---|---|
| Accuracy | 99.39% |
| AUROC | 0.9996 |
| Average precision | 0.9996 |
| Equal error rate | 0.68% |
| Precision | 99.50% |
| Recall (AI images caught) | 99.11% |
| False positive rate (real images flagged) | 0.39% |
| F1 | 0.9931 |
| Matthews correlation | 0.9876 |
| AI images caught with 0.1% false positives | 98.2% |
| AI images caught with 1% false positives | 99.5% |
| Expected calibration error | 0.0025 |
A low calibration error means the confidence score can be read at face value: when the detector says 90%, it is right about 90% of the time. Compared with the previous checkpoint, AUROC rose from 0.9989 to 0.9996 and the equal error rate fell from 0.93% to 0.68% on the same test set.
By image generator
| Generator | AI images | Caught |
|---|---|---|
| NTIRE challenge fakes | 10,033 | 99.5% |
| FLUX.2 Klein 4B | 3,557 | 99.9% |
| Z-Image Turbo | 3,046 | 99.4% |
| Stable Diffusion 3 | 335 | 96.7% |
| SDXL | 331 | 98.8% |
| DALL·E 3 | 327 | 100.0% |
| Stable Diffusion 2.1 | 308 | 97.7% |
| GPT Image 1.5 | 295 | 98.6% |
| Midjourney 6 | 289 | 96.5% |
| Illustrious | 75 | 98.7% |
| GPT Image 2 | 74 | 78.4% |
| GPT Image 2.5 Flare | 54 | 79.6% |
| GPT Image 1 mini | 54 | 88.9% |
| GPT Image 2.5 Sunburst | 54 | 87.0% |
| ERNIE Image Turbo | 52 | 96.2% |
| Seedream 5.0 | 48 | 95.8% |
| FLUX.2 Klein 9B | 22 | 100.0% |
| ERNIE Image | 21 | 100.0% |
| Qwen Image | 14 | 100.0% |
| Grok Image | 12 | 100.0% |
| Nano Banana | 12 | 50.0% |
| Lumina | 12 | 100.0% |
| Recraft V2 | 8 | 75.0% |
| Ideogram 2.0 | 5 | 40.0% |
The detector catches over 96% of images from the most widely used generators, including FLUX.2, DALL·E 3, Stable Diffusion and Midjourney 6. The newest OpenAI image models (GPT Image 2 and 2.5) and Google’s Nano Banana are harder, at 50% to 89% on small samples.
Real photos wrongly flagged, by source
| Source of real photos | Images | Wrongly flagged as AI |
|---|---|---|
| COCO | 1,610 | 0.25% |
| Wikimedia Commons (people) | 1,230 | 0.65% |
| Flickr | 3,488 | 0.00% |
| NTIRE challenge (real) | 10,210 | 0.69% |
| Pexels | 7,509 | 0.16% |
Audio detector
The audio detector was tested on the BRSpeech-DF and ASVspoof 5 datasets, with real speech also drawn from LibriSpeech and synthetic speech from DFADD. Each set below has 2,000 clips. The product uses a cut-off chosen on the BRSpeech-DF validation set.
| Test set | AUROC | Equal error rate | Accuracy at product cut-off |
|---|---|---|---|
| BRSpeech-DF validation | 0.9949 | 2.75% | 97.6% |
| BRSpeech-DF test (held out) | 0.9728 | 7.75% | 90.0% |
| ASVspoof 5 training holdout | 1.0000 | 0.00% | 100% |
| ASVspoof 5 development set | 0.9993 | 1.10% | 90.0% |
The validation figure of 97.6% is measured on the same set used to choose the cut-off, so it is the best case. The held-out BRSpeech-DF test set, which played no part in training or in choosing the cut-off, gives 90.0%. On the ASVspoof 5 development set, the detector separates real from AI speech almost perfectly (AUROC 0.9993), but the cut-off chosen on BRSpeech-DF gives 90.0% balanced accuracy there.
How we test against unseen content
Results on a detector’s own held-out data are always higher than on content from elsewhere. Real-world images, video and audio are compressed, resized, re-recorded and shared across platforms, and new AI models appear every month. To measure this, we also test on public datasets that our detectors were never trained on.
| Detector | Unseen test set | Samples | AUROC | Result |
|---|---|---|---|---|
| Text | Texts from AI models the detector was not built to identify | 1,001 | n/a | 98.7% accuracy |
| Text | Types of writing the detector was not built to identify | 334 | n/a | 99.7% accuracy |
| Images | Deepfake-Eval-2024 (public) | 786 | 0.898 | 80.2% accuracy; 92.9% precision; 73.2% of AI images caught; 8.9% of real images flagged |
| Audio | Deepfake-Eval-2024 (public) | n/a | 0.866 | 64.5% balanced accuracy; 21.8% equal error rate |
The text detector holds up well on unseen AI models and types of writing. The image and audio detectors are weaker on Deepfake-Eval-2024, a public benchmark of real-world deepfakes collected in 2024. This is typical of the field: published research shows detectors lose accuracy on in-the-wild content. We report these results so that customers can see how the detectors perform beyond our own test sets.
Methodology
- Test sets are held out. No test item was used to train the model being tested.
- Best result in 12 weeks. Models are retrained continuously and only go live when they improve, so we report the best result for each detector between 14 August and 7 November 2026.
- Cut-offs. Text and images use the product cut-off of 0.5. Audio uses a cut-off calibrated on the BRSpeech-DF validation set.
- Confidence scores, not just yes or no. Every detector gives a calibrated probability, and the text detector scores each section of a document separately.
- Scale. Over 15 million detection tests have been run on AI Aware’s detectors.
AI Aware’s detection is developed with computer scientists at City St George’s, University of London, led by our co-founder Dr Tillman Weyde. See our benchmark against other detectors and the research page, or try the detector free.