⚠️ Manual environments

ort replaced the manual Environment struct with ort::init all the way back in Nov 2023 with 2.0.0-alpha.1 (yes, before rc hell, we used to be in alpha hell). This was done because only one environment was allowed per process anyway. However, almost every other release since then, there has been some kind of issue stemming from the environment being stored as a global variable.

The trouble caused by this now far outweighs the inconvenience of lugging around another variable everywhere, so in rc.14, you must always first create an Environment to start building sessions:

// pre-rc.14
{
    // optional, even!
    ort::init().commit()?;

    let mut model = Session::builder()?
        .with_optimization_level(GraphOptimizationLevel::Level3)?
        .with_intra_threads(4)?
        .commit_from_file("yolov8m.onnx")?;
}

// rc.14
{
    let env = ort::init().build()?; // <-- must build an environment first
    let mut model = Session::builder(&env)?
        .with_optimization_level(GraphOptimizationLevel::Level3)?
        .with_intra_threads(4)?
        .commit_from_file("yolov8m.onnx")?;
}

Note that you don't need to hold onto env unless you intend to create more sessions. Please refrain from storing environments or sessions as global variables -- we can't fix the issues that causes within ONNX Runtime! And library authors, please remember to allow the user to pass their own Environment.

✨ Rust 1.100 compatibility

rc.14 adds support for the upcoming v2 build dir layout that will ship in Rust 1.100. If you intend to update to Rust 1.100, you should upgrade to rc.14 as soon as possible, since older ort versions will fail to build when copy-dylibs is enabled.

Note that, due to the new layout, tests will no longer be able to see dylibs copied by copy-dylibs; you can try to use ort::util::preload_dylib to manually set the path to provider dylibs instead.

📦 You get a binary! You get a binary! Everyone gets binaries!

🟩 rc.14's prebuilt CUDA binaries add support for RTX 50xx/Blackwell cards on x64. We now also ship aarch64 builds for Jetson Orin/Thor, Grace Hopper/Blackwell, & DGX Spark! Turing & Ampere GPUs remain fully supported.

🟥 Team Red gets some love as well: we now have MIGraphX binaries on x64 Linux! In the future, we'll also ship Windows builds with the Vitis AI EP for NPU acceleration on Ryzen AI chips.

🟦 We didn't forget about Team Blue either! We're also shipping OpenVINO builds for Windows & Linux x64.

🤖 x64 Android binaries were also added to make simulation easier.

⚙️ Miscellaneous features

• f2a5f73 Update to ONNX Runtime v1.31.0.
  • ort-web is updated to v1.30.0.
• c1f6e09 Add ort::util::probe to load-dynamic to check the version of ONNX Runtime before you try to load it.
• 23b6602 Remove get_/set_gpu_device.
• aee9f9f Bump MSRV to 1.92 (part of #625)
• #629 Allow custom ops to access preallocated outputs
• 1a4f903 Refactor execution provider internals
  • ℹ️ ep.name() now becomes EP::name() and ep.is_available() becomes EP::is_available().
• #644 Return None instead of Some("") when a ModelMetadata::custom() key doesn't exist
• 0825ab1 Only allow data_ptr/size/clone methods on Tensor-ish values

🛠️ Other fixes

• #618 Update ort::ep::cuda::preload_dylibs to work with CUDA 13.
• bc23566 Use the correct provider option names for the WebGPU EP.
• #621 Fix TensorRefMut::from_raw/Allocator::new treating non-CPU memory as CPU memory
• #622, #623, #624 Improve portability
• #626 Fix a crash involving custom loggers & actually pass the log category
• #632 Fix a double-free in run_async
• beb1b2f Reduce allocation overhead in Session::run
• #636 Fix string attributes for custom operators
• #638 Fix a memory leak in Session::overridable_initializers()
• 9c51509 Bound the lifetime of AllocationDevice to the MemoryInfo it belongs to
• #640 Return the correct length in Sequence::into_iter's ExactSizeIterator implementation
• #650 Fix iteration of SessionOutputs after removeing an output
• 7fdadd1 Prevent tensors from outliving the allocator that created them
• #659 Fix memory leaks/UAF in Trainer