• LibriSpeech: Common English speech benchmark
• FLEURS: Multilingual speech
• AMI Meetings: Meeting recordings

• Jie Jenn

• Whisper Overview
• OpenAI Audio API
• Jie Jenn

• https://github.com/openai/whisper
  • LibriSpeech
  • Multilingual
• Community Features:
  • Transcription and diarization (speaker identification)
    • openai/whisper#264
    • https://colab.research.google.com/drive/1HuvcY4tkTHPDzcwyVH77LCh_m8tP-Qet?usp=sharing
  • Streaming (real-time)
    • openai/whisper#2
    • https://betterprogramming.pub/color-your-captions-streamlining-live-transcriptions-with-diart-and-openais-whisper-6203350234ef
  • Cpp port (lightweight)
    • https://github.com/ggerganov/whisper.cpp
• Limitation:
  • Hallucination: openai/whisper#679

• ASR

• WER benchmarking
• https://github.com/Picovoice/speech-to-text-benchmark

Word error rate (WER) is the ratio of edit distance between words in a reference transcript and the words in the output of the speech-to-text engine to the number of words in the reference transcript.

Real-time factor (RTF) is the ratio of CPU (processing) time to the length of the input speech file. A speech-to-text engine with lower RTF is more computationally efficient. We omit this metric for cloud-based engines.

The aggregate size of models (acoustic and language), in MB. We omit this metric for cloud-based engines.