NVIDIA/DALI v1.5.0 on GitHub

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

Extended decoders.image to support WebP decoding (#3206)
Added indexing (NumPy-like) API for tensor slicing (#3200 and #3195)
Extended external_source to support source argument in TensorFlow DALI Dataset (#3215, #3193, #3177 and #3176)
Added examples:
- Tensorflow YOLOv4 (#2883)
- WebDataset usage with external_source (#3153)

Fixed issues

This DALI release includes the following fixes:

Fixed include paths that prevented including some parts of DALI in other C/C++ projects (#3210)
Fixed a crash when only anchors and no shapes were provided in multi_paste (#3166)
In the spectrogram operator, extracted windows are now correctly centered before FFT calculation, when the nfft argument is bigger than length of the window. (#3180)
Fixed a minor memory leak in decoders.image (#3148)

Improvements

Add documentation for indexing. (#3200)
Move to CUDA 11.4U1 (#3213)
Add WebP support to image decoder (#3206)
libtar API implementation (#3198)
Tensor indexing (#3195)
Make TF graph-mode tests faster (#3204)
Add support for ES source in TF DALI Dataset (#3177)
Add tensorflow YOLOv4 example (#2883)
Refactor Python External Source code (#3176)
Update third party dependencies to latest release versions (#3184)
Add deferred deallocation to cuda_vm_resource. (#3154)
Adjust test scripts and section header for webadataset notebook (#3162)
Add Webdataset-ExternalSource Jupyter notebook (#3153)
Update PR template (#3150)
Update PR template (#3129)

Bug Fixes

Fix failing TarArchive tests (#3226)
Build custom libtar in conda (#3223)
Improve validation in DALIDataset (#3215)
Update DALI_DEPS_VERSIOn to include NVIDIA/DALI_deps#19 (#3224)
Fix identity check in _is_generator_function which. Add test. (#3216)
Fix unused imports in test_utils.py (#3214)
Remove the usage of ManagedMemory from the OpticalFlow tests (#3211)
Suppress test using unified memory when it is not supported (#3209)
Remove include prefix from include paths (#3210)
Fix CVE-2021-3246 in libsnd (#3208)
Fix pytorch-lighting test (#3196)
Fix coverity issues + skip tests involving managed memory when not supported. (#3190)
Disable NVJPEG HW decoder for driver < 455 due to performance reason (#3189)
Fix compilation with newer GCC (#3188)
Disallow some types of sources for parallel ES explicitly (#3193)
Center windows when extracting windows to a bigger output window (#3180)
Add a compute cap value before running the GDS test (#3185)
MultiPaste to adjust the region shape to cover up to the end of the input shape (#3166)
Fix wording in docs (#3165)
Fix image decode (#3148)
Fix LastBatchPolicy doc and update Parallel ES wording (#3152)
Fix some errors (#3147)

Breaking API changes

There are no breaking changes in this DALI release.

Deprecated features

There are no deprecated features in this DALI release.

Known issues:

The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
- privileged=yes in Extra Settings for AWS data points
- --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10.2:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda102==1.5.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda102==1.5.0

or for CUDA 11:

CUDA 11.0 build uses CUDA toolkit enhanced compatibility. It is built with the latest CUDA 11.x toolkit
while it can run on the latest, stable CUDA 11.0 capable drivers (450.80 or later). 
Using the latest driver may enable additional functionality. 
More details can be found in enhanced CUDA compatibility guide.

pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==1.5.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==1.5.0

Or use direct download links (CUDA 10.2):

Or use direct download links (CUDA 11.0):

FFmpeg source code:

This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

https://developer.download.nvidia.com/compute/redist/nvidia-dali/libsndfile-1.0.31.tar.gz

NVIDIA/DALI v1.5.0 DALI v1.5.0 on GitHub