ultralytics/ultralytics v8.4.104

v8.4.104 - New YOLO26-Depth monocular depth estimation task (#25065)

on GitHub

🌟 Summary

YOLO26 gains a complete monocular depth estimation task, enabling per-pixel distance prediction, training, validation, visualization, calibration, and deployment alongside existing Ultralytics tasks. 🌍📐

📊 Key Changes

• New YOLO26-Depth model family 🤖

  • Adds yolo26n-depth, yolo26s-depth, yolo26m-depth, yolo26l-depth, and yolo26x-depth.
  • Uses a DPT-style depth head that fuses multi-scale YOLO26 features to produce dense depth maps aligned with the input image.
  • Supports both unbounded log-depth output for short- and long-range scenes and bounded depth output modes.

• Depth is now a first-class Ultralytics task 🆕

  • Available through the CLI:

    yolo depth train
    yolo depth val
    yolo depth predict
    yolo export

  • Fully supported through the Python Model API, including prediction, training, validation, export, and model.calibrate().

• Improved depth training and evaluation 📊

  • Adds depth-specific loss functions combining scale-aware depth accuracy with multi-scale edge and gradient matching.
  • Adds standard depth metrics such as delta1, delta2, delta3, absolute relative error, RMSE, and SILog.
  • Handles sparse or invalid ground-truth pixels safely.

• Depth-aware data pipeline 🗂️

  • Introduces paired RGB image and .npy float32 depth-map loading.
  • Applies geometric transformations and flips consistently to images and depth maps.
  • Adds validation for missing, corrupt, incorrectly shaped, or unreadable depth targets.
  • Adds Depth8, an 8-image dataset for rapid pipeline testing and debugging.

• Large dataset and benchmark support 🌐

  • Adds configurations and documentation for NYU Depth V2, KITTI, SUN RGB-D, ARKitScenes, DIODE, Hypersim, TartanAir, Virtual KITTI 2, and others.
  • Documents zero-shot evaluation on NYU Depth V2, KITTI, ETH3D, Make3D, and iBims-1.
  • Released models are pretrained on a broad mix of approximately 2.19 million indoor, outdoor, synthetic, real-world, and pseudo-labeled images.

• Depth visualization and results support 🎨

  • Adds DepthMap results, depth heatmap plotting, depth-aware result summaries, and access through result.depth.data.
  • Depth predictions can be converted through the existing .cpu() and .numpy() workflows.

• Export and deployment support 🚀

  • Adds depth export support for formats such as ONNX and TensorRT.
  • Dynamic exported output shapes track the input image dimensions.
  • TensorRT documentation now correctly lists the supported opset and workspace arguments.
  • Explicitly prevents unsupported NMS or IMX export configurations for depth models.

• More reliable training logs 🧾

  • Loss components are now returned as name-keyed dictionaries rather than positional tensors.
  • Fixes incorrect logging of l1_loss as dfl_loss when DFL is not used.
  • Makes loss reporting more robust across detection, segmentation, pose, distillation, classification, and depth tasks.

• Performance and reliability improvements ⚡

  • Computes DFL log_softmax once instead of twice, improving the DFL loss path by approximately 1.5–1.75× in benchmarks.
  • Reduces TaskAlignedAssigner kernel launches by batching candidate accumulation.
  • Caches semantic-mask bit depth to avoid reopening mask files every epoch.
  • Makes plotting threads non-daemon so generated plots finish writing before interpreter shutdown.

• Export, dataset, and workflow fixes 🛠️

  • Converts Path export arguments to strings, preventing exported ONNX metadata from failing to reload.
  • Fixes tar archive cleanup when using safe_download(..., delete=True).
  • Adds depth-target support to NDJSON dataset conversion.
  • Improves macOS CI stability and queues PyPI publishing with retry handling.
  • Adds documented pretrained YOLO26 detection and segmentation checkpoints for Objects365.

🎯 Purpose & Impact

• Expands YOLO26 beyond object recognition 🌟
  Users can now infer scene geometry and approximate camera-to-surface distances from a single RGB image, supporting robotics, navigation, AR/VR, 3D reconstruction, and spatial awareness applications.

• Handles a wider range of environments 🏠🚗
  The unbounded log-depth design avoids a fixed short-range ceiling, making the models better suited to both indoor scenes and long-range outdoor driving data such as KITTI.

• Simplifies end-to-end development ✅
  Depth estimation uses the same familiar Ultralytics workflow for dataset preparation, training, validation, prediction, export, and Python integration.

• Improves custom-dataset adaptation 🎯
  Users can fine-tune pretrained YOLO26-Depth models and calibrate absolute depth scale without retraining the network, helping adapt predictions to a particular camera or environment.

• Makes deployments more dependable 🔒
  Dynamic export shapes, corrected metadata serialization, more accurate TensorRT documentation, and safer archive handling reduce friction when moving models into production.

• Benefits existing tasks as well ⚙️
  The loss-dictionary refactor, DFL optimization, assigner optimization, semantic-mask caching, and CI robustness fixes improve maintainability, logging accuracy, training speed, and reliability across the broader Ultralytics framework.

For production workflows, YOLO26 remains the recommended latest stable model family. Users who prefer managed annotation, training, and deployment can also use the Ultralytics Platform.

What's Changed

• Fix Path values in exported model metadata by @glenn-jocher in #25324
• Queue publish runs and retry PyPI upload failures by @glenn-jocher in #25331
• Harden macos-26 CI parallelism and Streamlit Inference model list by @glenn-jocher in #25337
• Compute DFL log_softmax once instead of two cross_entropy calls by @raimbekovm in #25329
• Batch TaskAlignedAssigner select_topk_candidates into a single scatter_add_ by @raimbekovm in #25326
• Add Objects365 pretrained YOLO26 models to dataset docs by @Y-T-G in #25333
• fix: list TensorRT export arguments by @amanharshx in #25338
• Return loss items as name-keyed dicts from criteria by @Laughing-q in #25330
• Cache semantic mask bit-depth to avoid reopening every epoch by @Y-T-G in #24985
• update ultralytics-inference version to 0.0.30 in documentation by @onuralpszr in #25339
• Fix safe_download tar extraction crash with delete=True by @glenn-jocher in #25340
• Make threaded plot threads non-daemon so interpreter shutdown joins them by @glenn-jocher in #25341
• New YOLO26-Depth monocular depth estimation task by @Bovey0809 in #25065

Full Changelog: v8.4.103...v8.4.104