github ggml-org/Llama-Windows v0.13.0

2 hours ago

Highlights

  • Speculative decoding, wired automatically — models that ship a draft head (mtp / dflash / dspark) now run up to ~2× faster with identical output. The app detects the head from Hub repos and imported files, configures the server, and offers a one-click reload for models that are already loaded.
  • Your own model folders join the app — point Settings at any folder where you keep GGUF models and they appear — loadable, with metadata and fit verdicts — right next to the Hugging Face cache.
  • Honest "will it fit?" verdicts — three-level fit badges (fits / might fit / won't) computed from real memory estimates (GGUF headers) against this machine's devices and free memory, with the reasoning one hover away.
  • Share to Llama — send images and text from any app straight into the chat composer through the Windows share sheet, and open chats/models with llama:// deep links.
  • Installs are ~60% smaller — 235 MB → 86 MB. The app was carrying an entire Windows-ML runtime and 40 language packs it never used; it now ships only what it needs (of which the app itself is ~2.5 MB).
  • A quieter, faster app — the always-on model poller costs nothing when nothing changes, the model list rebuilds in batches, and JSON round-trips are compiled instead of reflected: less CPU, battery and UI stutter while the app sits in your tray.
  • Safer installs and settings — the installer script now runs in memory (it never touches disk), the server API key is stored encrypted like your Hugging Face token, and every URL the app fetches or opens is validated first.
  • Coming to the Microsoft Store — releases are now submitted to the Store automatically; the listing goes live once Partner Center certification completes.

New features

Speculative decoding (#105)

  • Models with a draft head (-mtp, -dflash, -dspark in the repo's files) are detected automatically — from the Hub at download time and for models already in your cache — and the server is configured to use them.
  • The details view shows the detected head; a loaded model that missed its head gets a one-click reload to engage it.

Enrolled model folders (#106)

  • Settings → General → Model folders: add places you already keep GGUF models (another drive, a private stash). They're scanned recursively and listed as first-class models, with the same metadata, fit and download-less loading.
  • The Hugging Face cache stays the primary storage; enrolled folders are additive and never duplicated.

Fit verdicts (#111)

  • Every model now answers "will it run here?" before you download: memory estimates read from the GGUF header, compared against your RAM/VRAM and devices, rendered as three badges — fits, might fit, won't.
  • The reason is spelled out (what the model needs vs. what the machine has), matching the wording of the pre-download check.
image

Share to Llama and llama:// deep links (#98, #99)

  • Share images and text from other apps into the chat composer via the Windows share sheet — images ride straight into the multimodal pipeline.
  • llama:// links open the chat overlay with a prompt or load a model — the model id is validated against what the server actually has.
image

Microsoft Store distribution (#125)

  • Each version release is now submitted to the Microsoft Store automatically (Partner Center submission behind a human approval gate), so the app can be installed and updated from the Store once the listing passes certification.

Performance

  • Much smaller installs and updates (#122, #123): trimming is enabled and validated (a build-time check now fails on any unsafe reflection usage), and the unused Windows-ML/Search/Widgets runtime is gone from the package — 235 MB → 86 MB for self-contained builds.
  • The /models poller is change-gated: while nothing changes it publishes nothing, drops to a slower cadence, and stops waking the UI thread — a large cut in idle CPU/battery for an app that runs all day (#114).
  • The model list rebuilds once per change instead of per row: opening the flyout with a large cache no longer staggers rows in over multiple frames (#116).
  • JSON serialization is source-generated across the board and settings load in a single parse: faster startup paths and settings round-trips (#120).

Fixes

  • A dead llama server is detected again: a failing GET /models is confirmed against /health, so the app reports "the server stopped responding" instead of showing a phantom "running" forever (#115).
  • Fixed a rare cross-thread race on the shared settings collections that could wedge the background model watcher (#119).
  • Server reads and writes stay outside the MSIX virtualization sandbox, fixing settings/models that could split across real and virtual paths (#108, fixes #107).
  • Removing a managed installation deletes only Llama's own files from the shared install directory (#95).
  • Correct architecture detection on ARM64 machines (#101).
  • Providers without a bundled brand mark show the org's Hub avatar instead of an empty tile (#112).
  • The stale Windows Phone identity is gone from the package manifest (#124).

Security

  • The installer script runs in memory — install.ps1 travels from the validated response into PowerShell without ever being written to disk (#113).
  • The server API key is protected like your HF token — DPAPI-encrypted at rest, managed from the same Settings door, never logged (#109).
  • URLs are treated as untrusted — avatar fetches are restricted to HTTPS Hugging Face hosts with a size cap (including after redirects), and the update banner only ever opens an https github.com release page (#119).
  • Supply-chain hardening (#117) — release tag names are no longer interpolated into CI scripts, every third-party action is pinned to a commit, NuGet dependency locks are enforced at restore time (drift fails the build), and preset INI section names are validated against injection.
  • Trim-safety enforced (#122) — the release build now fails if reflection or dynamic code re-enters the app, keeping the trimmed artifacts trustworthy.

Misc

  • Dependency updates (#92), Microsoft Store publisher identity (#102), version bump to 0.13.0.

Don't miss a new Llama-Windows release

NewReleases is sending notifications on new releases.