🌟 Summary
Release 8.4.188 improves QNN exports for Android, removes an unnecessary operation from non-LiteRT depth exports, and updates the LiteRT export toolchain.
📊 Key Changes
- Fixes Android QNN model loading: QNN exports now keep input and output quantization inside the QNN context. This avoids an
io_name_overridesattribute that newer QNN export tools can add but Android’s built-in QNN Execution Provider cannot read. - Speeds up depth exports outside LiteRT: The export-only ReLU is now applied only for LiteRT. It remains there as a workaround for an Intel NPU compiler issue, but is avoided in QNN and other formats.
- Updates LiteRT export requirements: Minimum versions are raised to
litert-torch 0.10.0,ai-edge-litert 2.3.0, andai-edge-quantizer 0.10.0. - Refreshes the export Docker image: It now installs the full export dependency set, including LiteRT support.
🎯 Purpose & Impact
- 📱 Android users should be able to load affected QNN exports with ONNX Runtime’s built-in QNN provider, including in apps using the Ultralytics Flutter plugin.
- ⚡ Depth-model exports may run more efficiently where the ReLU workaround is not needed.
- 📦 LiteRT exporters use a newer toolchain aligned with current mobile NPU runtimes, which can improve compatibility when exporting for supported devices.
- 🧩 These are export and deployment improvements; the release does not change model architecture or weights.
What's Changed
- Raise LiteRT export toolchain floors to LiteRT 2.3 by @glenn-jocher in #26869
- ultralytics 8.4.188 Fix Android loading and depth speed of QNN exports by @glenn-jocher in #26872
Full Changelog: v8.4.187...v8.4.188