koboldcpp-1.123

• NEW: Introducing koboldcpp.net, our new community website! - KoboldCpp now has an official community website!
  • This is a legitimate community site put together with the help of @Autumnlight02 and aims to be a simple and helpful resource for newbies to get started, (and also to help combat other fake websites floating around).
  • So, do check it out at https://koboldcpp.net - contributions are welcome at this repo!
  • Feel free to link and share it with others!
  • Note: Official executable downloads are still always found here at the KoboldCpp Github release page.
• NEW: Added support for Qwen Image 2.1 - You can now generate and edit images using the Qwen Image 2.1 model! Grab a ready to use .kcppt template here! Thanks to @wbruna for the upstream sync efforts and maintenance.
• NEW: Major improvements to integrated KoboldCpp Agent:
  • Changes for /confirm Confirmation (Approval Modes):
    • auto: Auto approve any read/write only within workdir, and always confirm for shell calls or for edits or writes outside of workdir. This means that in auto mode can be safely used to keep the agent running automatically within one directory tree, as any risky commands will require approval.
    • on (default): always require approval for any tool calls.
    • off (caution): always approve any tool calls.
  • Added /save and /load command to save and restore koboldcpp agent sessions. You can also reload a saved session directly from koboldcpp launch using --agentload [file]
  • Allow confirmation to be set both globally and also for each tool individually with /confirm NAME on|off|auto
  • Allow agent approval prompts to be interrupted with [Esc] to do or say something else.
  • Allow configuring max agent automation steps with /maxsteps
  • Added /usage command to see token usage
  • Added /autocompact N to trigger compaction upon reaching some token value
  • Added /paste to allow multiline pasting directly to the agent
  • Fixed incorrect system prompt location
  • Prevent agent shell calls from blocking on stdin
  • Limit agent MCP tool calling only when on localhost, since MCP tool calls affect the device the MCP server runs on.
  • Move Kcpp Agent to dedicated Agent tab in the GUI launcher
• NEW: Major Batched Mode Improvements - KoboldCpp now supports all standard samplers, grammar, tool calls and vision in batched (parallel) mode, so if you're running multiple requests at once, try using --parallelrequests N to get instantaneous responses for multiple requests together.
• Fixed a tool call hanging issue with some bad tools, fix mistral tool calling, fixed tool parser.
• Fixed bug: rep_pen_range being unbounded potentially causing OOM, thanks @SounLabs
• Fixed grammar integer parsing minimum, thanks @SashaMIT
• Fixed vision being broken for oldpc releases.
• Prevent idle unload during a batch request
• Added Spark X2.5 template.
• Deprecated explicit --jinjatools flag, now using --jinja automatically always enables Jinja tools and falls back if they fail.
• Deprecated confusing --multiuser flag, set connection queue limit with --queuelimit instead.
• Cleaned up project root, relocated some files
• Termux: Improved android launcher in android_install.sh
• Fixed remotetunnel binary location if run from a different work path
• Use shared timeout for reqtimeout for model reloading
• Fixed various minor bugs
• Updated API documentation and readme
• Updated Kobold Lite, multiple fixes and improvements
  • Allow saving multiple custom OpenAI endpoints as favorites
• Merged fixes, new model support, and improvements from upstream

Download and run the koboldcpp.exe (Windows) or koboldcpp-linux-x64 (Linux), which is a one-file pyinstaller for NVIDIA GPU users.
If you have an older CPU or older NVIDIA GPU and koboldcpp does not work, try oldpc version instead (Cuda11 + AVX1).
If you don't have an NVIDIA GPU, or do not need CUDA, you can use the nocuda version which is smaller.
If you're using AMD, we recommend trying the Vulkan option in the nocuda build first, for best support. Alternatively, you can download our rolling ROCm binary here if you use Linux.
If you're on a modern MacOS (M-Series) you can use the koboldcpp-mac-arm64 MacOS binary.
Click here for .gguf conversion and quantization tools
Newer rolling experimental builds can be found here, these are auto-updated and may be unstable.

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag. You can also refer to the readme and the wiki.