koboldcpp-1.124

[ a e s t h e t i c . e d i t i o n ]

• NEW! - KoboldCpp has now integrated a small subset of popular models from audio.cpp by @0xShug0! This greatly enhances the available built-in Music Gen and TTS Gen capabilities!
  • Music Gen: The long awaited YuE2 music gen model is supported. Download, save and load one of the following templates:
    • YuE2, no LLM - Lightest option, only YuE2, bring your own lyrics .
    • YuE2, small LLM - YuE2 with a small 4B LLM for generating lyrics. (Recommended!)
    • YuE2, big LLM - YuE2 with a huge LLM for generating lyrics. Only for large GPUs.
  • TTS Gen: The following TTS models have also been added (ported from audio.cpp)
    • BreezeTTS 2 [Voice cloning] [Instructions] (Recommended!)
    • Ming 3.0 TTS [Voice cloning] [Instructions] (Recommended!)
    • Qwen3-TTS Base [Voice cloning] (Recommended!)
    • Qwen3-TTS VoiceDesign [Instructions] (Recommended!)
    • VoxCPM2 [Voice cloning] [Instructions] (Recommended!)
    • Chatterbox [Voice cloning]
    • Chatterbox Turbo
    • CosyVoice 3 [Voice cloning] [Instructions]
    • Fish Audio S2 Pro [Voice cloning] [Instructions]
    • Kitten TTS 2 [Voice cloning]
    • Maya1 [Instructions]
    • NeuTTS 2E
    • OmniVoice [Voice cloning] [Instructions]
    • PocketTTS [Voice cloning]
    • Soprano 1.1 80M
    • Supertonic 3
    • VibeVoice 1.5B [Voice cloning]
    • A full directory of supported ported TTS models can be found here
  • MusicUI has been updated to support YuE2 and the new TTS models. After launching, access it at http://localhost:5001/musicui
  • MusicUI now has a transcribe tab (use with Whisper models), this can be useful for some of the new TTS models which need reference transcripts.
• NEW! - The latest sd.cpp sync brings some exciting new image generation model support (thanks @wbruna)
  • MOSS Image (Recommended!)
  • Fibo 2 Edit
  • Flux 3 Klein
  • Don't forget the Krea 2 Turbo and Qwen Image 2.1 previously added.
• NEW! - AI Horde now supports Chat Completions workers! This is enabled automatically, simply update to the latest KoboldCpp version. This allows the use of jinja, reasoning, tool calls, streaming and vision over AI Horde - you can enable this by toggling chat completions mode in https://lite.koboldai.net . Special thanks to @tazlin for coordinating this enhancement!
• Added a /mcp wizard for KoboldCpp Agent, allows setting up custom MCP endpoints. Also fixed a MCP loader race condition.
• Added /paste mode to KoboldCpp Agent to handle multiline text.
• Enable CUDA graphs for all architectures, expose upstream lazy loading --lazy-mode and MoE cache --moe-cache-mib, thanks @askmyteapot
• Added --paralleltotalcontext, which can be used with --parallelrequests. It allows setting the total KV context pool that concurrent requests share. Defaults to contextsize if unset. Increase this if the incoming batched requests are all very large - big requests will queue if they cannot fit.
• Fixed a CPU flash attention bug triggered by -Ofast, also detect file types by content instead of extension (thanks @wbruna)
• Now allow a limited amount of cache reuse when in batched (parallel requests) mode
• Improved SWA handling for Gemma4 models, with corrected warmup. SWA quality should be a lot better and more consistent when rewinding.
• Fixed a potential buffer overflows on loading for a few legacy models. (Thanks @professor-moody and @UNILESS)
• Added support for Animated WebP as the default preview for video gen (GIF is still supported, toggle in SDUI settings). AVI remains the default download option.
• Allow using tokenizers baked into the text encoder llm for image gen models. An explicit tokenizer.json can still be provided in the clip2 slot.
• Added support for TAEQI-tinier as a baked in VAE for Qwen Image 2.1, thanks @madebyollin
• Reworked built in WebSearch to give better and longer results in the fetched content
• Widen tool random id range
• Allow using RPC host tensor cache directory by @mfrederiksen, trigger with --rpccache
• Fixed Anthropic api tool handling by @jaeminbird
• Mask passwords in terminal load logs by @Yuvraj-Dhepe
• Disable async mode for ancient AMD GCN cards on Vulkan
• Updated Kobold Lite, multiple fixes and improvements
  • SSE parser improvements, added /v1/responses support, tool call and websearch improvements
• Merged fixes, new model support, and improvements from upstream, including Nemotron 3.5 Bolt 74BA9B, Mistral Large 4, LFM3, Muse Spark and more.

Download and run the koboldcpp.exe (Windows) or koboldcpp-linux-x64 (Linux), which is a one-file pyinstaller for NVIDIA GPU users.
If you have an older CPU or older NVIDIA GPU and koboldcpp does not work, try oldpc version instead (Cuda11 + AVX1).
If you don't have an NVIDIA GPU, or do not need CUDA, you can use the nocuda version which is smaller.
If you're using AMD, we recommend trying the Vulkan option in the nocuda build first, for best support. Alternatively, you can download our rolling ROCm binary here if you use Linux.
If you're on a modern MacOS (M-Series) you can use the koboldcpp-mac-arm64 MacOS binary.
Click here for .gguf conversion and quantization tools
Newer rolling experimental builds can be found here, these are auto-updated and may be unstable.

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag. You can also refer to the readme and the wiki or check our koboldcpp.net website.