NVIDIA/DALI v1.33.0

DALI v1.33.0

on GitHub

Key Features and Enhancements

This DALI release includes the following key features and enhancements:

• Enhanced experimental support for checkpointing (saving and resuming DALI pipelines at arbitrary iteration) (#5232, #5195, #5166):
  • Support data readers checkpointing (#5198, #5213, #5184, #5182, #5165, #5181, #5180, #5183, #5162, #5139).
  • Support randomized GPU operators checkpointing (#5216, #5148).
  • Improved checkpointing documentation (#5230).
• Improved Python annotations and signatures (#5217, #5159, #5167, #5154, #5188, #5158, #5150).
  • Added annotations for JAX and Pytorch iterators (#5129, #5197).
  • Improved PythonFunction annotations (#5207, #5149).
  • Improved data type annotations (#5179, #5153)
• Improved JAX support:
  • Added pmap compatibility for JAX data_iterator (#5185).
  • Improved JAX, Flax and Paxml training examples (#5176, #5205).
• Moved to CUDA 12.3U1 and enbaled GDS and nvJPEG2k support for the SBSA platform (#5209, #5170).
• Added Python 3.11 support and experimental support for Python 3.12 (#5174)

Fixed Issues

• Fixed fn.normalize handling of batch of empty samples (#5223).
• Fixed infinite video decoder seek loop (#5218).
• Fixed computation of maximal threads number for kernels in GPU fn.transpose and fn.normalize. (#5208)
• Fixed handling of empty slices and slicing of empty inputs. (#5204)
• Fixed scalar constant dimensionality inference (#5191)
• Fixed sharding in Caffe reader (#5172)

Improvements

• Mark missed operator stateless, assure checkpointing tests coverage (#5232)
• Add improved checkpointing docs (#5230)
• Support checkpointing in Numpy reader (#5198)
• Set the maximum supported Python version to Tensorflow 2.13 in tests (#5234)
• Dependency update 23/12 (#5231)
• Support checkpointing in External Source (#5213)
• Support checkpointing in other random operators (#5216)
• Remove a redundant call to Executor::GetTensorQueueSizes() (#5225)
• Fix links to release notes and docs archive (#5227)
• Install black with jupyter formatting capabilities (#5226)
• Fix the minimal Python version for TF 2.14 and 2.15, adds 2.13 (#5221)
• Unify positional input name handling in docs and signatures (#5217)
• Add launch_bounds to SliceNormalizeKernel_2D kernels (#5206)
• Update test_RN50_external_source_parallel_train_ddp.py to work with the latest PyTorch (#5219)
• Rework Flax and Paxml training tutorials (#5205)
• Mark all remaining stateless operators (#5195)
• Install black formatter in the aarch64 build (#5211)
• Fix typos in docs and comments (#5194)
• Bump up TensorFlow version in tests (#5175)
• Enbale nvJPEG2k support for the SBSA platform (#5209)
• Extend Python Function and plugin type annotations (#5207)
• Update readme about the Black formatting (#5212)
• Add "Format DALI with black" to blame ignore revs (#5210)
• Add type annotations for JAX plugin (#5197)
• Fix/simplify slice usage in examples. (#5203)
• Re-enable Python linter (#5189)
• Format DALI with black (#5169)
• Adjust configuration for autoformatting with black (#5168)
• Introduce TensorLike to signatures utilizing Array Interface (#5179)
• Update nvJPEG2k to 0.7.5 version (#5202)
• Add doc build artifacts to gitignore (#5187)
• Update FFmpeg to 6.1 (#5186)
• Add pmap compatibility for JAX data_iterator (#5185)
• Support checkpointing in Nemo reader (#5184)
• Support checkpointing in Webdataset reader (#5182)
• Support checkpointing in mxnet and tfrecord (#5165)
• Support checkpointing in Caffe reader (#5181)
• Support checkpointing in experimental video reader (#5180)
• Support checkpointing in sequence reader (#5183)
• Enable Python 3.11 test and 3.12 experimental support (#5174)
• Make sure that GDS on SBSA is not tested for drivers below cuda 12.2 (#5177)
• Add checkpointing benchmarks (#5166)
• Improve WarpAffine input documentation (#5178)
• Refactor JAX basic training example (#5176)
• Support checkpointing in Coco Reader (#5162)
• Move to CUDA 12.3U1 and enable GDS support for SBSA (#5170)
• Expose Tensors types stubs in nvidia.dali.tensors module (#5153)
• Make the video reader warn instead of failing on unreadable files (#5163)
• Implement checkpointing for random GPU operators (#5148)
• Generate overloads for Multiple Input Sets in interface files (#5159)
• Improve API submodules discoverability in some language engines (#5167)
• Expose current pipeline in python functions (#5156)
• Set signature for fn API (#5154)
• Add file to ignore Flake8 commits in blame (#5161)
• Suppress sanitizer leaks reported from xla and prevent hang in git clone (#5152)
• Add checkpointing support to Video Reader (#5139)
• Add type annotations for numba and pytorch plugins (#5129)
• Improve Python Function signature annotation (#5149)
• Dependency update 2023.11 (#5146)
• Bump up CUDA version used for tests (#5100)

Bug Fixes

• Patch libtiff for CVE-2023-6277 (#5224)
• Coverity fixes: fix fn.normalize handling of batch of empty samples, fix broken assertion in copy_with_stride (#5223)
• Avoids infinite video decoder seek loop (#5218)
• Fix and move caching from MaxThreadsPerBlock to MaxThreadsPerBlockStatic. (#5208)
• Remove redundant overload signature for operators without inputs (#5188)
• Fix hanlding of empty slices and slicing of empty inputs. (#5204)
• Fix slice/stack/cat/scalar usage in RNNT pipeline tests. (#5201)
• Fix Python3.11 tests (#5196)
• Fix scalar constant dimensionality & hide constant op documentation (#5191)
• Fix sharding in Caffe reader (#5172)
• Fix the check for dynamically generated number of outputs (#5158)
• Improve symbol visibility when importing (#5150)

Breaking API changes

There are no breaking changes in this DALI release.

Deprecated features

No features were deprecated in this release.

Known issues:

• The following operators: experimental.readers.fits, experimental.decoders.video, experimental.inputs.video, random_resized_crop, and experimental.decoders.image_random_crop do not currently support checkpointing.
• The video loader operator requires that the key frames occur, at a minimum, every 10 to 15 frames of the video stream.
  If the key frames occur at a frequency that is less than 10-15 frames, the returned frames might be out of sync.
• Experimental VideoReaderDecoder does not support open GOP.
  It will not report an error and might produce invalid frames. VideoReader uses a heuristic approach to detect open GOP and should work in most common cases.
• The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
  To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, you can use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
• In experimental debug and eager modes, the GPU external source is not properly synchronized with DALI internal streams.
  As a workaround, you can manually synchronize the device before returning the data from the callback.
• Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when running in Docker with escalated privileges, for example:
  • privileged=yes in Extra Settings for AWS data points
  • --privileged or --security-opt seccomp=unconfined for bare Docker.

Binary builds

NOTE: DALI builds for CUDA 12 dynamically link the CUDA toolkit. To use DALI, install the latest CUDA toolkit.

CUDA 11.0 and CUDA 12.0 builds use CUDA toolkit enhanced compatibility. 
They are built with the latest CUDA 11.x/12.x toolkit respectively but they can run on the latest, 
stable CUDA 11.0/CUDA 12.0 capable drivers (450.80 or later and 525.60 or later respectively).
However, using the most recent driver may enable additional functionality. 
More details can be found in enhanced CUDA compatibility guide.

Install via pip for CUDA 12.0:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda120==1.33.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda120==1.33.0

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==1.33.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==1.33.0

Or use direct download links (CUDA 12.0):

• https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda120/nvidia_dali_cuda120-1.33.0-11414174-py3-none-manylinux2014_x86_64.whl
• https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda120/nvidia_dali_cuda120-1.33.0-11414174-py3-none-manylinux2014_aarch64.whl
• https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda120/nvidia-dali-tf-plugin-cuda120-1.33.0.tar.gz

Or use direct download links (CUDA 11.0):

• https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-1.33.0-11414177-py3-none-manylinux2014_x86_64.whl
• https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-1.33.0-11414177-py3-none-manylinux2014_aarch64.whl
• https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda110/nvidia-dali-tf-plugin-cuda110-1.33.0.tar.gz

FFmpeg source code:

• This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

• https://developer.download.nvidia.com/compute/redist/nvidia-dali/libsndfile-1.2.2.tar.gz