ultralytics 8.4.128

v8.4.128 - Use synchronous OpenVINO batch inference (#25921)

on Python PyPI

🌟 Summary

v8.4.128 improves OpenVINO reliability and batch inference, reduces RAM use during training, strengthens export behavior, and clarifies dataset, augmentation, and tracking workflows. 🚀

📊 Key Changes

• Synchronous OpenVINO batch inference — priority update ⚡
  OpenVINO now submits each input batch as a single synchronous request and consistently uses the LATENCY performance hint. Throughput implementations remain available internally, but mode selection is forced to latency-oriented execution to avoid hangs in AsyncInferQueue, particularly for dynamic INT8 batches on CPU systems.
• Improved OpenVINO efficiency and stability 🧠
  Benchmarks showed that one batched synchronous request was about twice as fast and used roughly one-third the RAM compared with splitting the batch into separate asynchronous requests. This should make OpenVINO exports more dependable in CI and production workloads, though throughput-focused applications may see different performance characteristics.
• TensorRT INT8 optimizations 🔧
  TensorRT 7–10 now keeps only the detection-head Sigmoid layers in higher precision instead of affecting every matching activation. TensorRT 11 no longer applies the unnecessary Sigmoid exclusion. This reduces model size and improves inference speed while preserving confidence calibration.
• More compatible ONNX exports 📦
  ONNX export now caps the opset at 18 to avoid CUDA execution falling back to CPU for unsupported operations. This prevents extra host-memory copies and should improve GPU inference consistency, especially for models such as RT-DETR.
• Safer RKNN exports 📱
  RKNN INT8 export now clearly rejects unsupported non-detection tasks and recommends FP16 instead. Documentation also adds updated YOLO26 FP16 and INT8 benchmarks for Rockchip devices.
• Lower RAM usage for cache='ram' 💾
  Cached images are stored in a shared contiguous memory buffer, preventing DataLoader workers from duplicating the cache during forked training. This should keep memory usage flatter when using multiple workers.
• SAM and FastSAM improvements 🎯
  • SAM auto-mask generation now encodes each crop once and reuses its features across point batches, reducing repeated computation.
  • FastSAM box and point prompts are clipped to image boundaries, preventing negative or out-of-range coordinates from silently selecting incorrect masks.
• More robust training and data utilities 🛠️
  • DDP now ignores externally set RANK and LOCAL_RANK values unless a real multi-process environment is detected.
  • Ray tuning correctly aggregates metrics across multiple datasets and reports the completed epoch.
  • Scalar indexing now preserves bounding-box formats and instance dimensions.
  • Ground-truth candidate selection is more consistent for very small boxes.
• Improved dataset workflows 📚
  NDJSON conversion now supports local image paths, while COCO JSON training documentation adds clearer requirements, cache warnings, and validation guidance.
• Documentation and usability updates ✍️
  Documentation now describes tracking as a mode that runs on detection, segmentation, pose, or OBB models; adds local Docker build instructions; expands task-specific K-Fold guidance; clarifies augmentation behavior; and documents Android ExecuTorch setup.
• AMP check weights are cached globally 📥
  The YOLO26n weights used only for AMP compatibility checks are now stored in the user configuration directory and reused across projects instead of being downloaded into the current working directory.

🎯 Purpose & Impact

• More reliable deployment: Synchronous OpenVINO execution reduces the risk of indefinite hangs and makes batch inference more predictable across Intel and AMD CPU environments. ✅
• Better performance on supported hardware: TensorRT and ONNX changes reduce unnecessary precision constraints and CPU fallback operations, potentially improving speed, memory usage, and GPU utilization.
• Lower training memory requirements: Shared RAM caching is especially beneficial for large datasets and multi-worker training.
• Fewer silent prediction errors: Prompt clipping and format-preserving indexing ensure that invalid coordinates or scalar selections do not produce misleading masks or improperly interpreted boxes.
• Clearer user guidance: Updated documentation makes it easier to choose the right task, prepare datasets, configure augmentations, build Docker images, and deploy models on mobile and edge hardware.
• Compatibility note: OpenVINO now prioritizes latency-safe synchronous execution over automatic throughput mode selection. Users seeking maximum throughput should benchmark their specific hardware and workload after upgrading.

What's Changed

• Refactor Docker quickstart docs by @ambitious-octopus in #18658
• Fix RKNN INT8 "no detections" and update benchmarks by @lakshanthad in #24703
• Add task-specific k-fold split guidance by @AffanBinFaisal in #24527
• Apply class-aware NMS in shared C++ postprocessing by @acgist in #23187
• Cache AMP check weights in global config directory by @gabe-zhang in #23664
• Handle environments where RANK and LOCAL_RANK variables are set outside of DDP context by @Y-T-G in #22724
• Fix cache='ram' memory leak via shared image buffer by @raimbekovm in #24673
• Support local image paths in NDJSON dataset conversion by @Y-T-G in #24357
• Add https://youtu.be/eotJNQwboWA to docs by @RizwanMunawar in #25910
• Fix Ray tuning across multiple datasets by @glenn-jocher in #25922
• Preserve bounding box format when indexing by @aswanth-07 in #25906
• Make GT side clamp in select_candidates_in_gts monotonic by @ShuaiLYU in #25912
• Scope the TensorRT INT8 Sigmoid exclusion to the detection head on both quantization paths by @onuralpszr in #25920
• Cap ONNX opset at 18 for ONNX Runtime CUDA kernel coverage by @Y-T-G in #25915
• Preserve instance axes during scalar indexing by @aswanth-07 in #25907
• Remove unreachable YOLODataset task assertion by @roel-klein in #20870
• Clip FastSAM box and point prompts to image bounds by @JESUSROYETH in #25916
• Correct InfiniteDataLoader iteration documentation by @PhuTd03 in #23578
• Remove unreachable resume-training branch by @noorchauhan in #24163
• Document ExecuTorch Android runtime setup by @blm25 in #22849
• perf(sam): encode each generate() crop once and reuse it across point batches by @JESUSROYETH in #25909
• Correct augmentation parameters and COCO JSON training example in data guides by @raimbekovm in #25919
• Describe tracking as a mode rather than a task in docs and READMEs by @raimbekovm in #25908
• Use synchronous OpenVINO batch inference by @glenn-jocher in #25921

New Contributors

• @roel-klein made their first contribution in #20870
• @acgist made their first contribution in #23187
• @blm25 made their first contribution in #22849
• @noorchauhan made their first contribution in #24163
• @PhuTd03 made their first contribution in #23578
• @aswanth-07 made their first contribution in #25906
• @gabe-zhang made their first contribution in #23664

Full Changelog: v8.4.127...v8.4.128