ElevenLabs Scribe v3 Live uses conversational context to stop voice agents from cutting off callers mid-sentence

Key Points

- Elevenlabs has released Scribe v3 and Scribe v3 Live, two speech recognition models built for messy audio situations like crosstalk and background noise, covering both pre-recorded files and real-time use.

- Benchmarks from Artificial Analysis put Scribe v3 among the most accurate transcription models on the market, but it runs slower and costs more than rival models from Microsoft and xAI.

- The real-time version analyzes conversational context to avoid cutting callers off during brief pauses, and ElevenLabs is expanding enterprise partnerships with Nvidia and McKinsey.

Elevenlabs updates its speech recognition for recorded conversations and real-time use. Independent benchmarks confirm high accuracy but also show drawbacks in speed.

Elevenlabs has released two new speech-to-text models, Scribe v3 and Scribe v3 Live, built for messy conditions such as people talking over each other in meetings or phone calls with background noise.

Scribe v3 is designed for pre-recorded audio files, while Scribe v3 Live targets low-latency real-time applications. Both models are available now through the ElevenAPI developer interface and the ElevenAgents platform.

According to Elevenlabs, Scribe v3 cuts word errors by about 25 percent compared to its predecessor across 102 languages, and the company says German accuracy improves significantly. Speaker diarization also works more reliably in larger group conversations, according to the company, and Scribe v3 Live makes it available in real time for the first time.

Scribe v3 leads on accuracy but is slower and pricier than rivals

Independent benchmarks from Artificial Analysis confirm that Scribe v3 is among the most accurate transcription models on the market. It edges out competitors from Microsoft, Mistral, xAI, Google, and OpenAI, trailing only a preview model from Alibaba.

Anyone looking to process large volumes of audio quickly and cheaply is better off with other providers, though. Elevenlabs lands mid-pack on speed and price. Models from Microsoft and xAI run many times faster at marginally higher error rates and cost roughly half as much.

However, the benchmark measures only word errors on recorded files. It doesn't account for the improvements Elevenlabs touts in speaker diarization, rare languages, or the real-time behavior of Scribe v3 Live.

Scribe v3 Live aims to keep voice agents from cutting off callers

In voice agents, Scribe v3 Live is meant to fix a flaw in current systems, which often rely on rigid pause timers and cut callers off mid-sentence when they stop briefly to think or slowly read out a string of numbers. The new real-time model analyzes conversational context alongside the audio signal so it can tell when a speaker has actually finished.

Elevenlabs also gives developers more control over the final transcript. The models can be preloaded with thousands of specialized terms like brand names or medications to avoid mishearing domain-specific vocabulary. Developers can set formatting rules for numbers or dates in natural language, and a multilingual mode handles language switches mid-sentence. Enterprise customers get isolated servers in the EU and a zero-retention mode designed to delete data immediately after each request.

Elevenlabs is also expanding its enterprise business alongside the new models. Under an expanded partnership with Nvidia, the company is integrating the open Nemotron 3.5 Super VL language model into ElevenAgents, securing additional compute capacity running on Nvidia, developing local voice control for robots, and running a phone agent for Poland's public health system. The company also announced an alliance with consulting firm McKinsey, a broader partner program for system integrators, and its own marketplace where companies like Salesforce, IBM, and Zendesk offer ready-made voice agents.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now