ElevenLabs Details Cue Model for Character-Level Audio Timing

ElevenLabs detailed Cue in an Oct. 28 post, describing a model that places individual characters from supplied text at their corresponding points in recorded speech. The timing can be used for transcription, subtitles, dubbing and lyric timing.

Cue follows Scribe, ElevenLabs’ speech-to-text model, matching text with recorded audio. It is also available through the Forced Alignment API and can align recordings of any length in a single pass.

In a benchmark of real-world recordings across 29 languages, Cue’s average timing error on full recordings was 179 milliseconds. The open-source tools in the comparison had average errors ranging from 283 milliseconds to more than 6,000 milliseconds.

On English recordings with background noise and other issues, Cue’s average error was 57 milliseconds. The other tested systems ranged from 65 milliseconds to 920 milliseconds.

Cue supports more than 90 benchmark languages. It can align an hour of audio in about half a second on a production GPU; the next-fastest comparison tool took nearly two seconds.