horovod/horovod v0.19.1

TensorFlow 2.2 compatibility, MPI args for horovodrun

on GitHub

TensorFlow

• Added _aggregate_gradients in DistributedOptimizer to support Keras in TensorFlow 2.2. (#1784)

PyTorch

• Removed uses of deprecated PyTorch C++ API to support PyTorch 1.6. (#1731)

Changes to horovodrun

• Added process binding arguments to horovodrun. (#1767)
• Added --tcp flag to horovodrun for TCP only communication. (#1744)

Changes to installer

• Added HOROVOD_BUILD_ARCH_FLAGS to specify architecture-specific compiler flags. (#1751)
• Added Python extras to enforce that Horovod is installed after other frameworks. (#1785)

API changes

• Added mpi_args to horovod.run.run. (#1787)
• Added support for data transformation before train and validation in TorchEstimator (#1750)

Bugs

• Fixed bug in cache dump. (#1739)
• Fixed root rank output handling in MXNet out-of-place broadcast. (#1740)
• Fixed data_type_to_str for SparseVector and DenseVector. (#1780)