Skip to content

Optimized OpenAI's Whisper TFLite Port for Efficient Offline Inference on Edge Devices

License

Notifications You must be signed in to change notification settings

wbjnpu/whisper.tflite

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

40 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Whisper Enhanced Quantized TFLite Model

This project contains an enhanced version of the Whisper quantized TFLite model optimized for both Android and iOS platforms. The model is designed to perform well on edge devices, making it suitable for a wide range of applications.

Integrate Whisper TFLite for Enhanced Mobile App Performance

If you're interested in incorporating the Whisper TFLite model into your iOS and Android applications, please don't hesitate to reach out to us at [email protected]. Our project features an upgraded iteration of the Whisper quantized TFLite model, finely tuned for optimal performance on both Android and iOS platforms. This model is tailored to excel on edge devices, rendering it versatile for various application scenarios. Contact us for further details and collaboration opportunities

Getting Started

To get started with this enhanced Whisper model, follow these steps:

  1. Clone this repository to your local machine:

    git clone https://github.com/nyadla-sys/whisper.tflite.git
  2. You can now use the enhanced Whisper quantized TFLite model in your projects by refering sample code for Android and iOS.

Demo Apps

Android

You can find a sample Android app in the whisper_android folder that demonstrates how to use the Whisper TFLite model for transcription on Android devices.

DTLN quantized tflite model

Our overarching objective is to incorporate real-time noise suppression through the utilization of a quantized DTLN tflite model, delivering noise-reduced audio data to the whisper tflite model.

Courtesy from breizhn/DTLN

DTLN Paper

TODO

  • Considering adding DTLN noise cancellation tflite model to improve whisper ASR accuracy in noisy environments.

Here are links for comparing TFLite with other Whisper models:

Whisper's Comparative Analysis Speech Recognition Experiments Repository OpenVoiceOS' Whisper TFLite Plugin

Stay Updated

Stay connected to this repository for further developments and updates related to the Whisper enhanced TFLite model. We are constantly working to improve its performance and compatibility with various edge devices.

If you have any questions or encounter any issues, please don't hesitate to open an issue in this repository. We'll be happy to assist you!

Citing:

If you are using the DTLN model, please cite:

@inproceedings{Westhausen2020,
  author={Nils L. Westhausen and Bernd T. Meyer},
  title={{Dual-Signal Transformation LSTM Network for Real-Time Noise Suppression}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={2477--2481},
  doi={10.21437/Interspeech.2020-2631},
  url={https://dx.doi.org/10.21437/Interspeech.2020-2631}
}

About

Optimized OpenAI's Whisper TFLite Port for Efficient Offline Inference on Edge Devices

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 45.9%
  • C 38.4%
  • Python 3.2%
  • Java 1.8%
  • Starlark 1.6%
  • Rust 1.4%
  • Other 7.7%