End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations. Voice conversion, in which a model has to impersonate a speaker in a recording, is one of those situations. In this paper, we propose Blow, a single-scale normalizing flow using hypernetwork conditioning to perform many-to-many voice conversion between raw audio. Blow is trained end-to-end, with non-parallel data, on a frame-by-frame basis using a single speaker identifier. We show that Blow compares favorably to existing flow-based architectures and other competitive baselines, obtaining equal or better performance in both objective and subjective evaluations. We further assess the impact of its main components with an ablation study, and quantify a number of properties such as the necessary amount of training data or the preference for source or target speakers.

J. Serrà, S. Pascual, & C. Segura (2019). Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion. In Advances in Neural Information Processing Systems (NeurIPS). In press.

@article{Serra19ARXIV,
author = {Serr{\`{a}}, J. and Pascual, S. and Segura, C.},
journal = {ArXiv},
title = {{Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion}},
volume = {1906.00794},
year = {2019}
}

Paper: https://arxiv.org/abs/1906.00794 (latest version)

Audio examples: https://blowconversions.github.io

Suggested steps are:

1. Clone repository.
2. Create a conda environment (you can use the environment.yml file).
3. The following folder structure will be produced by the repo. From the git folder:
   • src/: Where all scripts lie.
   • dat/: Place to put all preprocessed files (in subfolders).
   • res/: Place to save results.

All the following instructions assume you run them from the src folder. Also, check the arguments/code for the scripts below. You may want to run with a different configuration.

To preprocess the audio files:

python preprocess.py --path_in=/path/to/wav/root/folder/ --extension=.wav --path_out=../dat/pt/vctk

Our code expects audio filenames to be in the form <speaker/class_id>_<utterance/track_id>_whatever.extension, where elements inside <> do not contain the character _ and IDs need not to be consecutive (example: s001_u045_xxx.wav). Therefore, if your data is not in this format, you should run or adapt the script misc/rename_dataset.py.

To train Blow:

python train.py --path_data=../dat/pt/vctk/ --path_out=../res/vctk/blow/ --model=blow

To transform/synthesize audio with a given learnt model:

python synthesize.py --path_model=../res/vctk/blow/ --path_out=../res/vctk/blow/audio/ --convert

To execute the classification script:

python classify.py --mode=train --path_in=../dat/wav/vctk/train/ --fn_cla=../res/vctk/classif/trained_model.pt --fn_res=../res/vctk/classif/res_train.pt

python classify.py --mode=test --path_in=../res/vctk/blow/audio/ --fn_cla=../res/vctk/classif/trained_model.pt --fn_res=../res/vctk/classif/res_test.pt

To listen to some conversions (using sox's play command):

python misc/listening_test.py --path_refs_train=../dat/wav/vctk/train/ --path_refs_test=../dat/wav/vctk/test/ --paths_convs=../res/blow/audio/,../res/test1/audio/ --player=play

• If using this code, parts of it, or developments from it, please cite the above reference.
• We do not provide any support or assistance for the supplied code nor we offer any other compilation/variant of it.
• We assume no responsibility regarding the provided code.