ultralytics/ultralytics v8.4.157

v8.4.157 - Speed up TensorRT FP16 and INT8 engines up to 20% (#26223)

on GitHub

🌟 Summary

Ultralytics v8.4.157 makes TensorRT inference faster—up to 20% for FP16 and INT8 engines—while improving YOLOE prompt-free support, Apple Silicon performance, validation reliability, and training stability. 🚀

📊 Key Changes

• ⚡ Faster TensorRT engines (PR #26223, @Y-T-G)

  • Calibrates FP16 conversion with a real image instead of random noise, allowing more layers to safely run in FP16.
  • Rewrites SiLU activations into a TensorRT-fusable form, reducing memory-bound activation kernels.
  • Adds CUDA Graph replay for supported static TensorRT engines, reducing per-inference launch overhead.
  • Expected impact: up to 20% faster TensorRT FP16 and INT8 inference, especially on static-shape engines. CUDA Graph acceleration is not used for dynamic engines, DLA execution, or engines with embedded NMS.

• 🎯 Expanded YOLOE-26 prompt-free inference (PR #26201)

  • Supports checkpoints containing both one-to-many and one-to-one detection heads.
  • The nms setting can now select between standard NMS inference and NMS-free inference.
  • set_vocab() can regenerate both branches for custom prompt-free YOLOE models.

• 🍎 Improved Apple Silicon CPU and MPS performance

  • Disables a slow NNPACK convolution path on Apple Silicon CPUs, improving larger-batch training and validation.
  • Reworks MPS box-IoU and task-assignment operations to avoid expensive reductions.
  • Replaces semantic segmentation bincount operations with MPS-friendly alternatives, addressing severe slowdowns and buffer-size failures.
  • Optimizes result export, plotting, and summaries by avoiding repeated per-detection conversions.

• 🧪 More reliable training and validation

  • AutoBatch no longer treats an initial batch-size probe failure as an out-of-memory limit.
  • Mosaic augmentation now closes correctly during very short training runs.
  • Resume errors now report the actual checkpoint or configuration problem instead of incorrectly claiming the file is missing.
  • Invalid settings such as negative patience, non-positive image sizes, invalid worker counts, and non-finite values are rejected earlier.

• 📐 Improved export and dataset behavior

  • Fixed-shape exported models consistently retain their exported image size across repeated predict() calls.
  • TAR dataset extraction now returns the dataset’s actual top-level directory, matching ZIP behavior.
  • Semantic mask caches correctly handle changes to nc=1, preserving foreground pixels in binary mask training.
  • Standalone validation skips labels for classes unsupported by the loaded model instead of crashing.
  • COCO size-specific metrics now remain associated with the correct task, including detection, segmentation, and pose.

• 📦 Smoother dependency installation

  • Corrected Python-version markers and package conflicts for several export extras, improving uv lock and uv sync reliability.

🎯 Purpose & Impact

• Faster deployment: TensorRT users can expect lower inference latency without changing model predictions or application code. ⚡
• Better YOLOE flexibility: YOLOE-26 prompt-free models can now switch between NMS and NMS-free inference as intended.
• Stronger Apple Silicon support: CPU and MPS users benefit from faster validation, semantic segmentation, and training workflows.
• Fewer confusing failures: Configuration, resume, export, dataset, and validation issues are detected earlier with clearer behavior.
• More dependable metrics: COCO evaluation results for boxes, masks, and keypoints are no longer accidentally overwritten or rejected.
• Easier experimentation: Improved dependency resolution and archive handling make setup and dataset workflows more predictable. 🚀

What's Changed

• Disable NNPACK conv2d for CPU inference and training on Apple silicon by @raimbekovm in #26221
• Fix stale semantic mask cache when nc changes to 1 by @cainiao33 in #26227
• Fix check_resume masking real errors as FileNotFoundError by @cainiao33 in #26225
• Keep fixed-shape export imgsz on repeated predict() calls by @cainiao33 in #26241
• Replace box_iou prod reductions with slice products for faster MPS validation by @raimbekovm in #26228
• Fix coco_evaluate writing mask size mAP over the box values and raising on pose by @Nicholas022400701 in #26246
• Fix uv sync and uv lock extra resolution by @hylreg in #26219
• Fix standalone detect val crashing on labels with class index >= model nc by @Nicholas022400701 in #26240
• Speed up TaskAlignedAssigner on CPU and MPS by removing size-2 and middle-axis reductions by @raimbekovm in #26234
• Convert boxes and keypoints once in Results.save_txt, plot and summary by @raimbekovm in #26233
• Fix slow and crashing semantic segmentation on MPS by @Y-T-G in #26239
• Ignore AutoBatch probe failures below the first successful size by @fcakyon in #26224
• Support dual one2many/one2one LRPC heads for prompt-free YOLOE inference by @ShuaiLYU in #26201
• Fix close_mosaic never firing when epochs < close_mosaic by @cainiao33 in #26226
• Reject negative patience at cfg by @wizzseen in #26236
• Floor imgsz at the model stride in check_imgsz by @cainiao33 in #26243
• Fix tar dataset extraction returning the datasets root directory by @cainiao33 in #26244
• Speed up TensorRT FP16 and INT8 engines up to 20% by @Y-T-G in #26223

New Contributors

• @wizzseen made their first contribution in #26236
• @hylreg made their first contribution in #26219

Full Changelog: v8.4.156...v8.4.157