Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add language_bias parameter to detect_language #2004

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jbaudanza
Copy link

We are doing a lot of multi-lingual transcriptions, especially with language learners. We have found that Whisper will often incorrectly identify the language when a language learner is speaking their target language.

For example, a native Korean speaker will speak English, but with a strong Korean accent, and Whisper will identify the language as Korean, not English.

We've found that adding a language bias of +1.0 to the language learner's target language is enough to nudge whisper in the direction of the user's target language, while still allowing it to properly identify the user's native language, and other languages.

@jmgb27
Copy link

jmgb27 commented May 19, 2024

Hi @jbaudanza did you have any workarounds for this one? looks like they haven't fixed it yet.

@jbaudanza
Copy link
Author

Hi @jbaudanza did you have any workarounds for this one? looks like they haven't fixed it yet.

This PR is the workaround.

Alternatively, you could try using a different model to do the language detection part. Maybe try this: https://huggingface.co/speechbrain/lang-id-voxlingua107-ecapa I haven't compared the results to Whisper though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants