Improved optimizer.skip_synchronize() API for NVIDIA AMP

Intel MLSL support, improvements to gradient clipping & NVIDIA AMP pe…

…rformance, bugfixes

Apache MXNet 1.4.1 compatibility, improvements to coordination perfor…

…mance at ultra-large scale, stall message improvements, bugfixes

Add horovodrun

PySpark, Apache MXNet, autotuning, TensorFlow eager execution, mixed-…

…precision & embedding improvements

Force allreduce of all gradients in step(), bugfixes

FP16 hierarchical allreduce

PyTorch 1.0, TF-Keras, FP16 ops on GPU

Parallelized hierarchical allreduce

Support for the upcoming PyTorch release