Updates
- Efficient and robust compressed training through progressive layer dropping
- JIT compilation of C++/CUDA extensions
- Python-only install support, ~10x faster install time
- PyPI hosted installation via
pip install deepspeed - Removed apex dependency
- Bug fixes for ZeRO-offload and CPU-Adam
- Transformer support for dynamic sequence length (#424)
- Linear warmup+decay lr schedule (#414)