← All runs

Evaluation run

self-correction chains audio

6 July 2026, 13:56 · eval-engine · 0169abd · 140 cases, 140 spoken

Did it hear the words right?

Clean human speech20 clips · real recordings3.4%
Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%

Did it clean up correctly and safely?

55%
Light, ships on by default
77/140 cases passed · 288 ms typical
49%
Polish, experimental
59/120 cases passed · 273 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words  ·  19 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

140 cases match

tts-num-selfcorrectSyntheticneeds review
HeardThe total comes to fifty no wait sixty dollars $215.08 $60. 67% misheard
Was saidthe total comes to fifty no wait sixty dollars
InstantThe total comes $215.08 $60.
LightThe total comes $215.08. $60. 264 ms
PolishThe total comes is $215.08, $60. 375 ms
Why light?the result is missing “$60”; the microphone misheard 67% of the words
Why polish?the microphone misheard 67% of the words
tts-num-timeSyntheticneeds review
HeardLet's meet at four thirty this 4.30 the afternoon. 43% misheard
Was saidlet's meet at four thirty this afternoon
InstantLet's meet at 4.30 the afternoon.
LightLet's meet at 4:30 the afternoon. 312 ms
PolishLet's meet at 4:30 in the afternoon. 273 ms
Why?the microphone misheard 43% of the words
tts-num-time-amSyntheticneeds review
HeardThe meeting is at nine 9 a.m. tomorrow. 43% misheard
Was saidthe meeting is at nine am tomorrow
InstantThe meeting is at 9 a.m. tomorrow.
LightThe meeting is at 9 a.m. tomorrow. 293 ms
PolishThe meeting is at 9 9:00 a.m. tomorrow. 350 ms
Why?the microphone misheard 43% of the words
tts-num-versionSyntheticneeds review
HeardPlease update it to version two point one 2.1. 57% misheard
Was saidplease update to version two point one
InstantPlease update it to version 2.1.
LightPlease update it to version 2.1. 321 ms
PolishPlease update it to version 2.1. 231 ms
Why?the microphone misheard 57% of the words
tts-num-version-32Syntheticneeds review
HeardUpgrade to version three point two 3.2 please. 43% misheard
Was saidupgrade to version three point two please
InstantUpgrade to version 3.2 please.
LightUpgrade to version 3.2 please. 258 ms
PolishPlease upgrade to version 3.2. please. 258 ms
Why?the microphone misheard 43% of the words
tts-num-yearSyntheticneeds review
HeardWe launched back in twenty twenty 2020? 33% misheard
Was saidwe launched back in twenty twenty
InstantWe launched back in 2020?
LightWe launched back in 2020? 273 ms
PolishWe launched back in 2020? 227 ms
Why?the microphone misheard 33% of the words
tts-num-year-2024Syntheticneeds review
HeardWe started this back in twenty twenty four 2024. 38% misheard
Was saidwe started this back in twenty twenty four
InstantWe started this back in 2024.
LightWe started this back in 2024. 345 ms
PolishWe started this back in 2024. 293 ms
Why?the microphone misheard 38% of the words
tts-punct-questionSyntheticclean
HeardAre we still on for lunch today? 0% misheard
Was saidare we still on for lunch today
InstantAre we still on for lunch today?
LightAre we still on for lunch today? 267 ms
PolishAre we still on for lunch today? 206 ms
tts-punct-runonSyntheticclean
HeardThe build passed we can ship it tell the team. 0% misheard
Was saidthe build passed we can ship it tell the team
InstantThe build passed we can ship it tell the team.
LightThe build passed, we can ship it tell to the team. 297 ms
PolishThe build passed, we can ship it and tell the team. 298 ms
tts-punct-runon-2Syntheticclean
HeardI went to the store I bought milk I came back home. 0% misheard
Was saidi went to the store i bought milk i came back home
InstantI went to the store I bought milk I came back home.
LightI went to the store I and bought milk. Then I came back home. 428 ms
PolishI went to the store I and bought milk. Then I came back home. 447 ms