How to apply multi-gpus training?

[lifuguan]

Thanks for your wonderful work. Is it possible to train the model using multiple gpus?

[liuyuan-pal]

Experiments are all conducted on a single 2080Ti or 1080Ti GPU (sadly, we don't have too many GPUs) so the current script does not support multi-GPU training. Maybe you can apply pytorch-lightning which requires rewriting the training script.