Why voice deepfakes matter
A few seconds of someone's voice are enough to clone it with a commercial TTS service. Cloned voices are used in calls that impersonate a relative or a CEO, in fake robocalls before elections, and in fabricated clips of public figures. Unlike a doctored image, a phone call leaves the person on the other end almost no time to check.
tropa-voice is built to catch exactly these voices, and tropa-voice-mini is its free open-weights sibling. We ran both against the strongest open-source detectors available, on five public benchmarks, with one evaluation script for every model. The code and our own test set are public, so every number below can be reproduced.
TL;DR
- tropa-voice (hosted API) beats every open-source detector on all five benchmarks.
- Commercial voices: 0.23% EER across 14 TTS systems, including ElevenLabs, MiniMax and OpenAI: up to 10× fewer errors than open alternatives (2.0–7.1%).
- Real 2024 deepfakes: on Deepfake-Eval-2024, about 30% fewer errors than the best open-source detector (EER 11.9% vs 17.2%).
- In-the-Wild: tropa-voice catches 99.4% of deepfakes at a 1% false-positive rate.
- tropa-voice-mini (Apache-2.0) is the strongest open-weights voice detector you can use commercially, with 0.0% EER on ElevenLabs v3 and v4, MiniMax, OpenAI, Gemini and xAI.
The five benchmarks
- In-the-Wild: 31,748 clips of celebrities and politicians collected online: real recordings and deepfakes from 2019 to 2022.
- Deepfake-Eval-2024: Deepfakes that circulated on social media and video platforms in 2024, official test split (773 files). Music, video soundtracks, many languages. The hardest set here.
- Commercial-Voices-2026-mini: Built by us and public: 2,084 clips from 14 commercial TTS systems (ElevenLabs, MiniMax, OpenAI, Gemini, xAI, HeyGen and others) and 2,200 openly licensed real recordings.
- SONAR: Speech from 8 recent TTS systems.
- MLAAD-tiny: 69 TTS systems across many languages, including recent open-source LLM-based voices.
Leaderboard
Equal error rate per benchmark: the error rate at the threshold where missed fakes and wrongly flagged real recordings are equally frequent. Lower is better. The best value in each column is green.
| Detector | License | In-the-Wild | Deepfake-Eval-2024 (test) | Commercial-Voices-2026-mini | SONAR | MLAAD-tiny |
|---|---|---|---|---|---|---|
| tropa-voice | Hosted API | 0.66% | 11.9% | 0.23% | 0.05% | 0.34% |
| tropa-voice-mini | Apache-2.0 | 0.98% | 13.5% | 0.19% | 0.18% | 2.33% |
| Spectra-AASIST | Apache-2.0 | 1.43% | 17.2% | 2.01% | 0.48% | 0.71% |
| DF_Arena_1B | Non-commercial | 0.96% | 27.6% | 7.14% | 1.26% | 3.24% |
| NII AntiDeepfake XLS-R-2B | Non-commercial | 1.34% | 25.1% | 3.99% | 0.43% | 4.72% |
EER in percent. tropa-voice runs behind our voice detection API; tropa-voice-mini is free on Hugging Face. DF_Arena_1B and the NII model are licensed for non-commercial use only.
- tropa-voice has a lower EER than every open-source detector on all five benchmarks.
- Commercial voices are where our models pull furthest ahead: 0.23% and 0.19% EER, about ten times lower than the next open model (2.01%).
- Deepfake-Eval-2024 separates the field most. On clips that went through social media with music and background noise, tropa-voice stays at 11.9% while non-commercial research models rise to 25–28%.
- tropa-voice-mini is the strongest open-weights detector licensed for commercial use.
Detection at a 1% false-positive rate
EER is a neutral comparison, but in production you pick a threshold. With the threshold set so that 1 in 100 real recordings is flagged, tropa-voice catches 99.4% of the deepfakes in In-the-Wild.
The API's ai verdict is set for the same false-positive rate, about 1% on real speech.
Per system: Commercial-Voices-2026-mini
EER per commercial TTS system, each scored against the same 2,200 real recordings. These are the voices scammers and content farms can buy today.
| TTS system | tropa-voice | tropa-voice-mini | Spectra-AASIST | DF_Arena_1B | NII XLS-R-2B |
|---|---|---|---|---|---|
| ElevenLabs v4 | 0.0% | 0.0% | 2.5% | 15.6% | 7.4% |
| ElevenLabs v3 | 0.0% | 0.0% | 0.7% | 9.1% | 2.4% |
| MiniMax Speech 2.8 HD | 0.0% | 0.0% | 0.7% | 8.7% | 3.4% |
| MiniMax Speech-02 HD | 0.7% | 0.0% | 2.7% | 5.4% | 6.8% |
| OpenAI gpt-4o-mini-tts | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |
| OpenAI tts-1-hd | 0.0% | 0.0% | 0.4% | 1.4% | 0.0% |
| Gemini 3.8 Flash TTS | 0.0% | 0.0% | 0.6% | 2.8% | 0.0% |
| xAI TTS | 0.0% | 0.0% | 1.6% | 14.4% | 3.1% |
| HeyGen | 0.8% | 0.2% | 6.2% | 12.0% | 7.3% |
| VibeVoice 7B | 1.4% | 1.5% | 7.4% | 11.8% | 27.9% |
| Qwen Audio 3 TTS | 0.8% | 0.0% | 1.5% | 6.0% | 2.3% |
| Inworld | 0.0% | 0.0% | 1.3% | 7.3% | 2.2% |
| Seed Speech v2 | 0.0% | 0.0% | 0.0% | 2.4% | 0.8% |
| Chatterbox | 0.0% | 0.0% | 0.0% | 0.5% | 0.0% |
Our models separate most commercial voices without a single error, including ElevenLabs, MiniMax, OpenAI, Gemini and xAI. On the hardest systems, VibeVoice 7B and HeyGen, they stay at 1.5% or below, where open alternatives reach up to 27.9%.
Protocol
- Same code, same files: every detector was run by us through one evaluation script on identical files.
- Sliding windows over the whole clip: long recordings are cut into overlapping windows, each window is scored, and the file score is the mean of the top 30% of window scores. A short synthetic passage inside a long real recording is therefore not averaged away.
- Documented preprocessing: external models get the input their authors document (sample rate, window length, normalization), so no model is run outside its intended setup.
- Official splits: for Deepfake-Eval-2024 we use the official test split.
- Metric: equal error rate per benchmark, plus detection at a 1% false-positive rate on In-the-Wild.
Reproduce every number with our benchmark code on GitHub.
Good to know
- Very short clips (under 2 seconds), heavy background music and phone-quality audio are harder for every detector.
- New voice generators appear every month. tropa-voice is updated continuously; tropa-voice-mini is a fixed release.
Code, data and models
- Evaluation code: wasitaigenerated/voice-deepfake-benchmark on GitHub.
- Our test set: Commercial-Voices-2026-mini on Hugging Face.
- Open model: tropa-voice-mini (Apache-2.0), free and CPU-friendly. Details and usage on the tropa-voice model page.
- Hosted model: the AI voice detection API, or upload a clip on the deepfake detector.
- Background on how voice cloning works and how to protect yourself: How to detect AI voice cloning.
Building a voice detector? Run it with the same code and email patrick@wasitaigenerated.com. We add submitted results to this page, including results that beat ours.
FAQ
Which AI voice detector is the most accurate in 2026?
tropa-voice, the model behind our hosted API, beat every open-source detector we tested on all five public benchmarks: In-the-Wild 0.66%, Deepfake-Eval-2024 11.9%, Commercial-Voices-2026-mini 0.23%, SONAR 0.05% and MLAAD-tiny 0.34% equal error rate. On In-the-Wild it catches 99.4% of deepfakes at a 1% false-positive rate.
Can ElevenLabs voices be detected?
Yes. On Commercial-Voices-2026-mini both tropa-voice and tropa-voice-mini had an equal error rate of 0.0% on ElevenLabs v3 and v4. The best other open model reached 0.7% and 2.5%.
How well does voice deepfake detection work on real social-media clips?
Real-world clips are the hardest test for any detector: Deepfake-Eval-2024 contains deepfakes that circulated online in 2024, with music, video soundtracks and many languages. tropa-voice reached an EER of 11.9% there, about 30% fewer errors than the best open-source detector (17.2%). Non-commercial research models were at 25–28%.
What is the equal error rate (EER)?
The error rate at the threshold where the share of fakes a detector misses equals the share of real recordings it flags. Lower is better, and it does not depend on where a vendor chooses to put its threshold.
Can I reproduce these results?
Yes. The evaluation code is on GitHub, Commercial-Voices-2026-mini is on Hugging Face, and the other four benchmarks are public datasets. tropa-voice-mini is free to download; tropa-voice can be tested through the API.
Check a voice recording yourself
Upload a clip to WasItAIGenerated: it runs tropa-voice, the model that leads the table above, with a timeline of where in the clip the synthetic voice occurs.