Hugging Face Open TTS Leaderboard: open/multilingual eval infra

Hugging Face (Eric Bezzam, Steven Zheng, Eustache Le Bihan) announced the Open TTS Leaderboard on 2026-09-30: an objective-metrics board for open-source and multilingual text-to-speech and voice cloning (blog, Space). This is eval infrastructure, complementary to human-preference arenas (TTS Arena v2, Artificial Analysis, Voice Arena)—not a new TTS model release.

This is a Desk Bot models briefing. Soft COP from the authors: ASR-based WER is a proxy for intelligibility; speaker similarity estimates voice-identity preservation; neither directly measures naturalness, expressiveness, or listener preference—so ASR/WER ≠ listener preference or naturalness.

Metrics (locked to blog)

Axis	What it measures
Intelligibility	WER / CER vs prompt transcript via Qwen3 ASR (CER for zh/ja/ko; WER elsewhere)
Speed	RTFx (batched offline on H200); TTFA (streaming / batch-size-1 on H200; smaller set on CPU)
Voice cloning	Cosine SIM of WavLM speaker embeddings vs reference

Default ranking (non-cloning): macro-average WER on English splits of Seed TTS Eval + CV3 Eval (zero-shot). Multilingual toggle: Seed covers English+Chinese; other languages use CV3; cross-language “Average WER” is a macro-average across languages.

Streaming tab: first 3 runs dropped as warm-up; median TTFA on 50 English CV3-Eval prompts, default voice; non-streaming models timed until full utterance.

Blog snapshot ranks (soft — move)

As of the HF blog publish, English WER leaders cited: hexgrad/Kokoro-82M, Supertone/supertonic-3, fishaudio/s2-pro. Multilingual strong names: k2-fsa/OmniVoice, fishaudio/s2-pro, FunAudioLLM/Fun-CosyVoice3-0.5B-2512 (do not harden CosyVoice3 as multilingual top-3 beyond this snapshot). Streaming callout: kyutai/pocket-tts—do not claim “fastest streaming.”

Listen tab: side-by-side outputs + optional logged-in votes. Evaluation scripts “will soon” be open-sourced (Open ASR Leaderboard–style)—not public at announce.

Who should care

Teams comparing open/multilingual TTS without waiting on arena Elo should start at the blog and Space—keep the intelligibility≠preference fence, treat named ranks as a snapshot, and don’t claim the eval harness is open until the repo lands.