🚀 Foundry Local CLI 0.14.0 (Preview)

Public preview update for the Foundry Local CLI, adding embeddings, local document indexing and semantic search, and ranking and typed-decision models.

🆕 What's New

Updated Foundry Local SDK

This build updates the Foundry Local SDK and Runtime packages to 2.4.1, ONNX Runtime GenAI to 0.19.0, and WinML to 2.5.96. ONNX Runtime remains at 1.31.0.

It includes the features and fixes from SDK 2.4.0 and SDK 2.4.1, including Native AOT support for ranking and typed-decision sessions.

Embeddings

• New foundry embed command generates local text embeddings, with automatic model download and loading.
• Accept separately quoted text inputs, a UTF-8 file, or redirected stdin. Use --input-format jsonl for batches containing one JSON string per record.
• Return full, ordered vectors and token usage. Use --output json for clean, progress-free scripting output.
• Optional --normalize l2 normalizes vectors; --execution-provider selects an existing embedding variant.

foundry model list --type embedding
foundry embed <embedding-model> "First document" "Second document" --output json
foundry embed <embedding-model> --file .\document.txt --output json

Local document indexing and semantic search

• New foundry index create, search, update, and rebuild commands turn UTF-8 text and Markdown files into searchable local collections.
• Search returns passages ranked by cosine similarity, with source paths and line ranges, not generated answers.
• Updates reuse unchanged files. Use --prune to remove missing or newly excluded files, or --dry-run to preview changes.
• Configure chunk size, overlap, include/exclude filters, and model-specific document/query prefixes. The exact embedding variant and settings are saved with the index.
• Failed updates and rebuilds preserve the previous committed index.

foundry index create docs --source .\docs --model <embedding-model>
foundry index search docs "How do I configure GPU support?" --top 5
foundry index update docs --dry-run --prune
foundry index update docs --prune

Here, docs is a user-chosen index name. Indexing initially targets small collections of text/Markdown files, not PDFs, OCR, or a full vector database.

Ranking and typed-decision models

• New foundry rank command ranks candidate answers with text-ranking models such as CLM, returning candidate probabilities.
• New foundry decide command runs typed-decision models such as KEV, supporting predicate probabilities (--noul), choices (--choice), and ordered scores (--score).
• Both commands accept flags, a JSON request file, or redirected JSON stdin; support text and JSON output; and automatically download and load the selected model when needed.
• BYOM registration supports the text-ranking and typed-decision tasks for compatible model packages.

foundry rank <clm-model> --question "Which response is best?" `
  --answer "First response" --answer "Second response"

foundry decide <kev-model> --state '{"ticket":42}' `
  --noul "escalate=Escalate this ticket?" `
  --choice 'team={"sales":"Sales","support":"Support"}' `
  --score 'priority=["low","medium","high"]'

Use compatible ranking or typed-decision model packages, not ordinary chat models. For JSON file/stdin input, omit model from the request body; the command's model argument selects it.

AMD GPU execution provider

Foundry Local now recognizes AMDGPUExecutionProvider and can select it for models that support it on AMD GPUs.

⚡ Improvements

• Better automatic model selection: aliases and default catalog listings use the preferred available execution-provider variant. Variants requiring unavailable providers are skipped for automatic selection but remain discoverable with --variants; explicit variant IDs stay exact.
• Verified model downloads: downloaded ranges are checked using Azure Storage MD5 and pinned to the blob ETag. Corrupt chunks are downloaded again on resume; incomplete downloads are not treated as cached.
• Refreshed CUDA execution-provider bundles for Windows and Linux on x64 and ARM64.
• Better tool calling: the SDK infers the tool protocol from the model's rendered chat template.
• More complete model metadata: context length and capabilities such as reasoning and tool calling are restored from catalog tags when structured metadata is missing.
• Better BYOM guidance: interactive registration prompts for reasoning and tool-calling support when not explicitly provided.
• Improved console editing: move the caret with Left/Right and insert, paste, or delete at that position, with Unicode-aware wrapping and more efficient handling of long input.
• Clearer interactive help: slash-command help includes argument syntax and aliases.

🐛 Fixed

• Interactive Ctrl+C recovery: cancel an active chat/run/transcription turn without forcing the interactive session to exit; Ctrl+C while idle exits.
• CUDA fallback: a reported CUDA-only execution-provider registration failure no longer prevents daemon startup; CPU and other successfully registered providers remain available.
• Live audio cleanup: owned audio buffers are disposed correctly when queue insertion fails.
• macOS packages: corrected runtime-library aliases and executable permissions, with final ZIP and PKG payload validation.
• SDK fixes: Qwen2.5 required and named tool choices no longer return HTTP 500, and image_url vision inputs work in Chat Completions.
• SDK 2.4.1 fixes: ranking results missing a probability are rejected rather than silently treated as zero, and an Objective-C class name conflict with ONNX Runtime is resolved on macOS.

⚠️ Behavior Changes

• The telemetry opt-out environment variable is now ORT_DISABLE_TELEMETRY=1, matching ONNX Runtime. It replaces ORT_TELEMETRY_DISABLED. Update scripts and deployments that set the old variable, then restart the daemon.

Install

Pick the asset for your platform from the downloads list below.

# Windows x64
Add-AppxPackage .\foundry-0.14.0-win-x64.msix
foundry --version

# macOS Apple Silicon
sudo installer -pkg foundry-0.14.0-osx-arm64.pkg -target /
foundry --version

# Linux x64
tar xzf foundry-0.14.0-linux-x64.tar.gz
cd foundry-0.14.0-linux-x64
./lib/foundry --version
# Install system-wide:
sudo bash install.sh

Windows packages and macOS binaries/installers are signed by Microsoft Corporation. SHA-256 digests are available in the GitHub release asset details.

Quick Start

foundry status
foundry model list
foundry model load qwen3-0.6b
foundry chat qwen3-0.6b
foundry transcribe -m whisper-tiny -f audio.wav
foundry server stop

Use --output json on supported commands for scripting. Run foundry --help for the full command surface.

Coming from the older Foundry Local CLI?

The earlier service-based CLI has been replaced. Common commands map roughly as follows:

Links

• CLI reference: https://github.com/microsoft/foundry-local-cli/blob/main/docs/CLI-REFERENCE.md
• Docs: https://aka.ms/foundry-local-docs
• SDK: https://github.com/microsoft/Foundry-Local/tree/main/sdk_v2
• SDK 2.4.0 release notes: https://github.com/microsoft/Foundry-Local/releases/tag/v2.4.0
• SDK 2.4.1 release notes: https://github.com/microsoft/Foundry-Local/releases/tag/v2.4.1
• Discord: https://aka.ms/foundry-local-discord

Feedback

File issues at https://github.com/microsoft/Foundry-Local/issues with [cli] in the title and include the output of foundry report.