koboldcpp-1.122

PSA: KoboldCpp needs YOUR help to fight a fake phishing site (click here for info)!

• NEW: Added an integrated KoboldCpp Agent - KoboldCpp now ships with a bundled Agentic harness. This works like an extremely lightweight replacement for tools like Opencode, Codex or Claude Code. Comes with 9 built-in tools, and a tiny system prompt of only 2k tokens including all tools.
  • To use it, simply enable it from the Admin tab in the GUI launcher, or add --agent to your launch flags.
  • KoboldCpp Agent can also connect to third party backends, or any OpenAI Chat Completions compatible endpoint.
  • Add more tools by loading a mcp.json file, MCP tools will be shared to the agent. Note: MCP tools execute on the KoboldCpp server, while Agent tools execute on the agent client.
  • Comes with 3 approval modes for tool calling confirmation: on/auto/off. Exercise caution when approving tool calls.
  • To function effectively, KoboldCpp Agent requires at least 28k ctx and 8k gen amount, though larger values are recommended. Recommend to have at least 12GB VRAM for a good experience.
  • You can download a .kcppt template to get started with Qwen 3.6 35BA3B here, simply load and launch in the latest KoboldCpp.
  • Supports AGENTS.md, context compaction and many more features
  • Run /help in the Agent to get more information
• Added the ability to specify ubatchsize independently from batchsize, similar to llama.cpp. Pipeline parallel flag removed, it will auto enable when ubatch is smaller than batchsize
• Make sliders more compact, added ubatch slider
• Remove some smartcache spam when in quiet mode
• Increased default contextsize.
• Generation failures during streaming return an error object
• Revert P40 increase VRAM usage issue
• Sync upstream sd.cpp by @wbruna
• Limit max image preview size to handle pixel space diffusers
• Added --autoswapthreshold, now you can set the size limit before autoswap triggers to prevent swapping when small models are being utilized.
• Lowered SD VAE tiling threshold to 512
• Fixed some tool calling parser issues
• Refactor of koboldcpp.sh build/run script by @henk717
  • Note for package maintainers: This release includes a breaking change to koboldcpp.sh. To shorten compiles and make it easier to build from source, koboldcpp.sh now attempts to optimize for the target platform it is run on.
  • What this means: If you use koboldcpp.sh script in your build system, you must now include KCPP_PORTABLE=1 as the environment variable if you seek to produce builds that work on systems outside of your own.
• Allow generation of audio codes when using LLM for lyrics (2 step process), improved music UI
• Added keepalive whitespace padding for tool streaming and text sync requests, similar to image gen
• Smarter handling of max output tokens when close to max context
• Updated Kobold Lite, multiple fixes and improvements
• Merged fixes, new model support, and improvements from upstream

Download and run the koboldcpp.exe (Windows) or koboldcpp-linux-x64 (Linux), which is a one-file pyinstaller for NVIDIA GPU users.
If you have an older CPU or older NVIDIA GPU and koboldcpp does not work, try oldpc version instead (Cuda11 + AVX1).
If you don't have an NVIDIA GPU, or do not need CUDA, you can use the nocuda version which is smaller.
If you're using AMD, we recommend trying the Vulkan option in the nocuda build first, for best support. Alternatively, you can download our rolling ROCm binary here if you use Linux.
If you're on a modern MacOS (M-Series) you can use the koboldcpp-mac-arm64 MacOS binary.
Click here for .gguf conversion and quantization tools
Newer rolling experimental builds can be found here, these are auto-updated and may be unstable.

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag. You can also refer to the readme and the wiki.