• (05/1/2021) Trained vanilla Contextnet over Librispeech Dataset https://arxiv.org/abs/2005.03191
• (05/6/2021) Implemented the Low rank decomposition on ContextNet on both spectrogram and raw audio input.
• (05/16/2021) Generated Loss Landscapes for the trained models, see demo_loss
• (05/20/2020) Trained a Low Rank Decomposition based Deep Net with wave input Low rank decomposition model
• (06/7/2020) Generated Integrated Gradients for trained models Keras Integrated Gradients documentation

• What's New?
• Table of Contents
• Publications
• Installation
  • Installing from source
  • Running in a container
• Setup training and testing
• Features Extraction
• Augmentations
• Training & Testing Tutorial
• Loss landscape visualisation and gradient attribution
  • Gradient_Visualisation_Notebook
  • Loss Lists Notebook
• English Dataset
• References & Credits
• Contact

• ContextNet (Reference: https://arxiv.org/abs/2005.03191) See examples/contextnet

• **Raw Wwaveform Based CNN Through LOW-RANK Spectro-Temporal Decoupling ** (Reference: https://publications.idiap.ch/downloads/reports/2019/Abrol_Idiap-RR-11-2019.pdf) See tensorflow_asr/models/encoders

  Low rank spectro-temporal decoupling implementation in this project

  

git clone https://github.com/vaibhav016/FILRCN.git
cd FILRCN
python setup.py build
python setup.py install

docker-compose up -d

• For mixed precision training, use flag --mxp when running python scripts from examples

• For enabling XLA, run TF_XLA_FLAGS=--tf_xla_auto_jit=2 python3 $path_to_py_script)

• For hiding warnings, run export TF_CPP_MIN_LOG_LEVEL=2 before running any examples

See features_extraction

See augmentations

1. Define config YAML file, see the config.yml files in the example folder for reference (you can copy and modify values such as parameters, paths, etc.. to match your local machine configuration)
2. Download your corpus (a.k.a datasets) and run download_links.shscripts folder to download files For more detail, see datasets. Note: Make sure your data contain only characters in your language, for example, english has a to z and '. Do not use cache if your dataset size is not fit in the RAM.
3. [Optional] Generate TFRecords to use tf.data.TFRecordDataset for better performance by using the script create_tfrecords.py
4. Create vocabulary file (characters or subwords/wordpieces) by defining language.characters, using the scripts generate_vocab_subwords.py or generate_vocab_sentencepiece.py. There're predefined ones in vocabularies
5. [Optional] Generate metadata file for your dataset by using script generate_metadata.py. This metadata file contains maximum lengths calculated with your config.yml and total number of elements in each dataset, for static shape training and precalculated steps per epoch.
6. run create_transcripts_from_data.sh from scrpts folder to generate .tsv files(the format in which the input is given is .tsv)
7. For training, see train.py files in the example folder to see the options
8. For testing, see test.py files in the example folder to see the options.

For visualisations, we have two kinds of scripts. cd examples/contextnet/contextnet_visualisation

1. for loss landscapes, cd into context_visualisation/loss_landscape_visualisation.
   1. run generate_lists.py(This generates the loss and accuracy lists)
   2. now run plot_loss.py (From those lists, images are drawn both 2d and 3d)
   3. now run video_create.py(It sews all the images into a single video)
2. For gradient visualisation,
   1. run integrated_grad_vis.py, which will generate the integrated gradients for all the trained models
   2. then run plot_gradients.py
   3. Finally run video_create.py

1. TensorFlowASR
2. Loss landscape visualisation
3. Keras Integrated Gradients

Vaibhav Singh ([email protected])

Dr Vinayak Abrol ([email protected])