VibeVoice ASR

VibeVoice-ASR is a unified speech-to-text model that processes up to 60 minutes of long-form audio in a single pass and generates structured transcriptions identifying who spoke what and when, with support for over 50 languages, code-switching, and customized hotwords.

Key Capabilities

• Single-pass processing of 60-minute audio — maintains global context and consistent speaker tracking across the full duration.
• Rich transcription output — jointly performs automatic speech recognition (ASR), speaker diarization, and timestamping in one end-to-end generation step.
• Customized hotwords — users can supply domain-specific terms or names (e.g., proper nouns, technical jargon) to guide recognition and improve accuracy.
• Multilingual and code-switching support — natively handles over 50 languages without requiring explicit language setting; switches seamlessly within and across utterances.

Benchmark Performance

The model (VibeVoice-ASR-7B, 7 billion parameters) uses a continuous speech tokenizer operating at a 7.5 Hz frame rate. Representative results on public datasets:

DER = diarization error rate, WER = word error rate. Full results are in the technical report.

Additional Capabilities

• Prompt-based context injection — provide textual context (e.g., a topic summary) to improve accuracy on domain-specific terminology and polyphonic character disambiguation.
• Batch inference — process multiple audio files concurrently.
• vLLM inference support — enables faster decoding when deployed with vLLM.
• Fine-tuning — LoRA fine-tuning code is publicly available for adapting the model to custom domains.