🚀 Foundry Local v2.0.1 Release Notes

We're excited to announce Foundry Local v2.0.1. This release introduces a unified, cross-language inference API, a shared native runtime, and packaging and reliability improvements for Linux and ARM64.

🆕 What's New

🧰 Unified Session API

Foundry Local now provides a consistent inference model built around Model → Session → Request<Item...> → Response<Item...>. ChatSession, EmbeddingsSession, and AudioSession are available across C#, Python, JavaScript/TypeScript, Rust, C++, and the stable C ABI.

The SDKs now share the cross-platform foundry_local native runtime, a versioned C ABI, consistent native error propagation, and deterministic resource ownership. The C++ SDK includes a header-only RAII wrapper over the C ABI.

💬 Sessions, Items, and Streaming

• Stateful multi-turn chat with turn counting and history rollback.
• Typed text, message, image, audio, tensor, byte, tool-call, tool-result, and speech items.
• Multimodal messages with URI-based or in-memory image and audio inputs.
• Synchronous and streaming responses, request cancellation, streaming cancellation, finish reasons, and token usage.
• ItemQueue support for incremental input, including live PCM audio.
• Typed tensor outputs for batch embeddings and detailed speech results with segment and word timing.

🛠️ Tool Calling and Responses

• Session-level tool registration and removal, with auto, none, and required tool choice.
• Typed tool-call outputs and tool-result inputs for multi-turn tool-calling flows.
• Prompt-template application is now handled internally by ChatSession.
• The local HTTP service retains OpenAI-compatible chat completions, models, embeddings, audio transcription, Responses API operations, image/audio inputs, and Server-Sent Events streaming.

🔄 Migration Notes

Installation

APIs

Manager, catalog, model discovery, download, load, unload, and web-service workflows remain largely familiar. Execution-provider handling is now automatic by default: the unified package detects available hardware and selects WinML, WebGPU, CPU, or CUDA as appropriate. Applications that explicitly selected or configured an execution provider in 1.2.4 should review that customization when migrating.

C++ applications require a broader migration to the new RAII wrapper and shared native runtime. Direct OpenAI-style in-process clients remain available but are deprecated and scheduled for removal at the end of 2026. New applications should use the Session API.

🐞 Known Issues

• On RTX Spark (Windows Arm), NVIDIA TensorRT RTX models fail on first load when decoding is set to top_k
• Rust SDK package publishing is in progress

⚡ Improved

📦 Platform Packaging

• There is now one package per SDK; separate -winml packages are no longer needed.
• The unified package detects the available hardware and loads the appropriate WinML, WebGPU, CPU, or CUDA execution provider.
• Hardware acceleration is included in the primary packages.
• All language SDKs share the foundry_local native runtime.
• Linux and ARM64 packaging has been improved.
• Linux packages now target manylinux_2_28 compatibility.

🗂️ Models and Downloads

• Enumerate model versions and select or download a specific version.
• Configure catalog URLs, regions, filters, and version limits with region-aware fallback.
• More accurate cached-model enumeration and refreshed model information across language bindings.
• Resumable, chunked, transactional model downloads with cross-process locking, bounded-memory streaming, progress callbacks, and cancellation.

🧱 Reliability

• More reliable model downloads and concurrent cache access.
• Improved HTTP diagnostics and Linux certificate handling, including common CA bundle discovery and SSL_CERT_FILE support.
• Graceful manager shutdown that stops new operations, cancels active work, closes sessions, stops the web service, and unloads models.

🐛 Fixed in 2.0.1

Linux Packaging

• Linux Python wheels now use valid manylinux_2_28_x86_64 and manylinux_2_28_aarch64 tags.
• Restored Linux ARM64 wheel generation and added architecture-specific artifact validation.
• Builds and release validation now run in pinned manylinux containers with auditwheel repair, glibc compatibility checks, and bundled-library validation.
• Prevented duplicate ONNX Runtime and ONNX Runtime GenAI libraries in repaired Python wheels; they remain declared package dependencies.

Platform Reliability

• Fixed Linux HTTPS failures caused by unavailable CA bundle paths in statically linked curl or OpenSSL configurations.
• Restored explicit dependencies required for older glibc environments.
• Improved catalog-index synchronization and GCC/Clang compatibility for concurrent catalog access.

📚 Resources

💙 Feedback

File issues at https://github.com/microsoft/Foundry-Local/issues and include the SDK version, platform, and relevant diagnostics.