AssemblyAI Benchmark Shows Transcript Leaders Shift With Call Length

AssemblyAI's Oct. 28 Voice Standard Bench update found different transcript-correction leaders across four call lengths, giving developers separate quality, cost and speed measures for choosing models to clean speech-to-text output.

GPT-5.6 Sol led the 30-second test with a score of 98, followed by GPT-6.1 Sol at 94 for two minutes, Opus 5 at 91 for 15 minutes and GLM-5.3 at 88 for 60 minutes. The scores use a 100-point quality scale and are not percentages of words transcribed correctly.

The benchmark paired each model's score with its listed cost per audio hour and median response time. GPT-5.6 Sol cost $0.19 per hour and responded in 1.94 seconds; GPT-6.1 Sol cost $0.11 and took 6.76 seconds. Opus 5 cost $0.41 with a 14.86-second median, while GLM-5.3 cost $0.031 and took 14.73 seconds.

The 60-minute results also separated the lowest price from the top score. Nemotron Nano 9B v2 was the least expensive choice at $0.0019 per hour of audio, but earned a score of 6. The benchmark's Fastest and Lowest cost selections have no minimum quality cutoff. Kelsey Foster wrote, “Check the score next to the Fastest and Lowest cost cards before using them, since neither has a quality floor.”

The benchmark tests editing after speech recognition has generated text, rather than measuring how well a system recognizes spoken words. The correction task removes fillers, false starts and stutters, fixes punctuation, capitalization and clear transcription mistakes, and retains the speaker's meaning. Scores blend measures of faithfulness and improvement in readability.

The correction tests start with real call transcripts that receive simulated errors. Each length group includes 10 transcripts within 15% of its target length. The original transcript is not treated as a perfect answer key because it can contain errors itself. The resulting score measures task quality, not transcription accuracy as a percentage.

AssemblyAI launched Voice Standard Bench on Oct. 13 to compare more than 50 models across voice-related tasks and transcript lengths. TokenPost covered the benchmark's initial release in its earlier report. The Oct. 28 correction results are a snapshot that may change as models are added.