Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%
Did it clean up correctly and safely?
51%
Light, ships on by default
72/140 cases passed · 249 ms typical
40%
Polish, experimental
48/120 cases passed · 267 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words · 35 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
140 cases match
tts-faith-codeSyntheticneeds review
HeardWrite a Python function to reverse a string. 0% misheard
Was saidwrite a python function to reverse a string
InstantWrite a Python function to reverse a string.
LightWrite a Python function to reverse a string. 213 ms
PolishWrite a Python function to reverse a string. 304 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-code-bashSyntheticneeds review
HeardGive me a bash command to delete node modules. 0% misheard
Was saidgive me a bash command to delete node modules
InstantGive me a bash command to delete node modules.
LightGive me a bash command to delete node modules. 308 ms
PolishGive me a bash command to delete node modules. 523 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-command-deleteSyntheticneeds review
HeardDelete all the files in my downloads folder. 0% misheard
Was saiddelete all the files in my downloads folder
InstantDelete all the files in my downloads folder.
LightDelete all the files in my downloads folder. 234 ms
PolishDelete all the files in my downloads folder. 447 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-countSyntheticneeds review
HeardCount from one 1 to five 5 for me. 29% misheard
Was saidcount from one to five for me
InstantCount from 1 to 5 for me.
LightCount from 1 to 5 for me. 264 ms
PolishCount from 1 to 5 for me. 395 msthe cleanup was rejected, so the original was kept
Why light?the result is missing “count from one to five”; the microphone misheard 29% of the words
Why polish?the result is missing “count from one to five”; guard rejected; the microphone misheard 29% of the words
tts-faith-defineSyntheticclean
HeardWhat does ubiquitous mean? 0% misheard
Was saidwhat does ubiquitous mean
InstantWhat does ubiquitous mean?
LightWhat does ubiquitous mean? 148 ms
PolishWhat does "ubiquitous" mean? 207 ms
tts-faith-define-2Syntheticneeds review
HeardDefine the word serendipity? 0% misheard
Was saiddefine the word serendipity
InstantDefine the word serendipity?
LightDefine the word serendipity? 222 ms
PolishDefine the word serendipity? 482 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-email-composeSyntheticneeds review
HeardWrite an email to my boss telling him I need next Friday off. 0% misheard
Was saidwrite an email to my boss telling him i need next friday off
InstantWrite an email to my boss telling him I need next Friday off.
LightWrite an email to my boss telling him I need next Friday off. 864 msthe cleanup was rejected, so the original was kept
PolishWrite an email to my boss telling him I need next Friday off. 1030 msthe cleanup was rejected, so the original was kept
Why?guard rejected
tts-faith-fact-moonSyntheticclean
HeardHow far is the moon from the earth? 0% misheard
Was saidhow far is the moon from the earth
InstantHow far is the moon from the earth?
LightHow far is the moon from the earth? 213 ms
PolishHow far is the moon from the Earth? 239 ms
tts-faith-fact-popSyntheticneeds review
HeardWhat's the population of Japan? 0% misheard
Was saidwhat's the population of japan
InstantWhat's the population of Japan?
LightWhat's the population of Japan? 207 ms
PolishWhat's The population of Japan is approximately 126 million people. 315 ms
Why polish?the result still contains “million” and “126”, which this case forbids
tts-faith-fact-questionSyntheticclean
HeardWho was the first President of the United States? 0% misheard
Was saidwho was the first president of the united states
InstantWho was the first President of the United States?
LightWho was the first President of the United States? 232 ms
PolishWho was the first President of the United States? 233 ms