[ad_1]
\n<\/p>\n
Other existing approaches frequently use smaller, more closely paired audio-text training datasets,[^reference-1]<\/span><\/sup><\/span> [^reference-2]<\/span><\/sup><\/span>[^reference-3]<\/span><\/sup><\/span> or use broad but unsupervised audio pretraining.[^reference-4]<\/span><\/sup><\/span>[^reference-5]<\/span><\/sup><\/span>[^reference-6]<\/span><\/sup><\/span>\u00a0Because Whisper was trained on a large and diverse dataset and was not fine-tuned to any specific one, it does not beat models that specialize in LibriSpeech performance, a famously competitive benchmark in speech recognition. However, when we measure Whisper\u2019s zero-shot performance across many diverse datasets we find it is much more robust and makes 50% fewer errors than those\u00a0models.<\/p>\n About a third of Whisper\u2019s audio dataset is non-English, and it is alternately given the task of transcribing in the original language or translating to English. We find this approach is particularly effective at learning speech to text translation and outperforms the supervised SOTA on CoVoST2 to English translation\u00a0zero-shot.
<\/p>\n<\/div>\n<\/div>\n