Skip to content

Commit

Permalink
add section Available models and languages
Browse files Browse the repository at this point in the history
  • Loading branch information
jongwook committed Sep 21, 2022
1 parent cfd6bdd commit 49a3ffc
Show file tree
Hide file tree
Showing 2 changed files with 4,630 additions and 1 deletion.
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,30 @@ choco install ffmpeg
```


## Available models and languages

There are five model sizes, four with English-only versions, offering speed and accuracy tradeoffs. Below are the names of the available models and their approximate memory requirements and relative speed.


| Size | Parameters | English-only model | Multilingual model | Required VRAM | Relative speed |
|:------:|:----------:|:------------------:|:------------------:|:-------------:|:--------------:|
| tiny | 39 M | `tiny.en` | `tiny` | ~1 GB | ~32x |
| base | 74 M | `base.en` | `base` | ~1 GB | ~16x |
| small | 244 M | `small.en` | `small` | ~2 GB | ~6x |
| medium | 769 M | `medium.en` | `medium` | ~5 GB | ~2x |
| large | 1550 M | N/A | `large` | ~10 GB | 1x |

For English-only applications, the `.en` models tend to perform better, especially for the `tiny.en` and `base.en` models. We observed that the difference becomes less significant for the `small.en` and `medium.en` models.

Whisper's performance varies widely depending on the language. The figure below shows a WER breakdown by languages of Fleurs dataset, using the `large` model. More WER and BLEU scores corresponding to the other models and datasets can be found in Appendix D in [the paper](https://cdn.openai.com/papers/whisper.pdf).

![WER breakdown by language](language-breakdown.svg)



## Command-line usage

There are five model sizes, four with English-only versions, offering speed and accuracy tradeoffs. The following command will transcribe speech in audio files, using the `medium` model:
The following command will transcribe speech in audio files, using the `medium` model:

whisper audio.flac audio.mp3 audio.wav --model medium

Expand Down
Loading

0 comments on commit 49a3ffc

Please sign in to comment.