augmentation.md

Data Augmentation Pipeline

Data augmentation has often been a highly effective technique to boost the deep learning performance. We augment our speech data by synthesizing new audios with small random perturbation (label-invariant transformation) added upon raw audios. You don't have to do the syntheses on your own, as it is already embedded into the data provider and is done on the fly, randomly for each epoch during training.

Six optional augmentation components are provided to be selected, configured and inserted into the processing pipeline.

• Audio

  • Volume Perturbation
  • Speed Perturbation
  • Shifting Perturbation
  • Online Bayesian normalization
  • Noise Perturbation (need background noise audio files)
  • Impulse Response (need impulse audio files)

• Feature

  • SpecAugment
  • Adaptive SpecAugment

In order to inform the trainer of what augmentation components are needed and what their processing orders are, it is required to prepare in advance an augmentation configuration file in JSON format. For example:

[{
    "type": "speed",
    "params": {"min_speed_rate": 0.95,
               "max_speed_rate": 1.05},
    "prob": 0.6
},
{
    "type": "shift",
    "params": {"min_shift_ms": -5,
               "max_shift_ms": 5},
    "prob": 0.8
}]

When the augment_conf_file argument is set to the path of the above example configuration file, every audio clip in every epoch will be processed: with 60% of chance, it will first be speed perturbed with a uniformly random sampled speed-rate between 0.95 and 1.05, and then with 80% of chance it will be shifted in time with a random sampled offset between -5 ms and 5 ms. Finally this newly synthesized audio clip will be feed into the feature extractor for further training.

For other configuration examples, please refer to examples/conf/augmentation.example.json.

Be careful when utilizing the data augmentation technique, as improper augmentation will do harm to the training, due to the enlarged train-test gap.