WER (word error rate)
The percentage of words an automatic transcript changes, drops or adds compared with the correct text. The lower, the better.
WER (word error rate) is the most common measure of automatic transcription quality. The system’s text is compared with a correct reference text and the words that were substituted, deleted and inserted are counted. The total is divided by the number of words in the reference.
Example: if the reference has 100 words and the system changes 2, skips 1 and adds 1, the WER is 4%. The accuracy usually advertised is 100% minus the WER, in this case 96%.
- Check what audio was used: sentences read with good sound give better figures than a real, noisy meeting.
- Check how it was normalised: removing capitals, punctuation or number formatting changes the result.
- One figure isn’t everything: missing an article isn’t the same as getting a client’s name wrong.
Echo measured a WER of 2.63% on 80 Spanish sentences from the public FLEURS dataset, that is, 97.4% of words right. The errors left were mostly unusual proper names and time formats. The full method is on the accuracy page.
