koboldcpp-1.121

Music generated by @Dampfinchen, clips for music video created entirely in KoboldCpp

• NEW: Media reference support added for Minimax H3! - You will now be able to attach audio clips and multiple reference images to generate minimax H3 videos. The music video attached to this video was created entirely in KoboldCpp (in segments of 7s each)
• Fixed a race condition during streaming.
• Fixed LTX Audio conditioning bug
• Added support for video LoRAs (thanks @wbruna)
• Swapped to CUDA 11.4 for OldPC release binaries (tested and verified by @henk717)
• Fixed a bug in grammar memoization (thanks @Reithan)
• Fixed tool calling support for Kimi models and Deepseek V4 Flash models
• Handle invalid numeric API parameters (thanks @yhz5613813)
• Fixed WebSearch UTF-8 handling (thanks @Erquint)
• --analyze feature can now analyze .safetensor file metadata in addition to .gguf files.
• Hide cache slots in GUI launcher if smartcache is off
• Made help menu in GUI launcher more accessible
• Added a flag --ffncpu which behaves like llama.cpp --n-cpu-ffn, an alternative way to partial offload layers
• Optimized Adaptive-P sampler
• Prevent segfault on model load failure
• Reworked GUI launcher's SaveDataFile selection UI
• Add support for xhigh reasoning effort
• Improved huggingface model downloader
• Jinja tools requests with tools in the payload have a lower default temperature
• NEW: Reworked MusicUI (Music Generation UI) for a more intuitive user experience.
• Multiple Stable-UI (SDUI) GUI Improvements:
  • NEW: Runtime LoRA selector in included stable-ui (http://localhost:5001/sdui) - Thanks to @wbruna, you can now easily view, add and remove image generation LoRAs from your selected LoRA directory at runtime in SDUI with a nice GUI selector.
  • Fixed SDUI Image carousel selection
  • Image Generation Recovery - Enable "Poor connection mode" in SDUI to allow recovering generated images even if you disconnect or accidentally close your browser.
  • Improved cloudflare tunnel stability in SDUI - Sends a keepalive to help prevent tunnel collapse during long requests (e.g. video generation).
  • Added box for sending extra custom fields in SDUI, increased max video frames to 240 (default) or 480 (extended toggle), show elapsed time
  • Set SDUI dynamic LoRA default strength to 1.
  • Prevent viewing generation progress status if genkey is incorrect
• Updated Kobold Lite, multiple fixes and improvements
  • Attached vision images now render in correct turns.
  • Improved autoscrolling behavior
  • Fixed <t2i> tag handling in thinking blocks.
  • Added mini toggle buttons for enabling TTS and voice recognition if detected.
• Merged fixes, new model support, and improvements from upstream

In addition to the above our docker image now has KCPP_LORA support for use with cloud providers, use this variable along with direct links to the lora's you wish to download, they can be separated by a comma.

Download and run the koboldcpp.exe (Windows) or koboldcpp-linux-x64 (Linux), which is a one-file pyinstaller for NVIDIA GPU users.
If you have an older CPU or older NVIDIA GPU and koboldcpp does not work, try oldpc version instead (Cuda11 + AVX1).
If you don't have an NVIDIA GPU, or do not need CUDA, you can use the nocuda version which is smaller.
If you're using AMD, we recommend trying the Vulkan option in the nocuda build first, for best support. Alternatively, you can download our rolling ROCm binary here if you use Linux.
If you're on a modern MacOS (M-Series) you can use the koboldcpp-mac-arm64 MacOS binary.
Click here for .gguf conversion and quantization tools
Newer rolling experimental builds can be found here, these are auto-updated and may be unstable.

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag. You can also refer to the readme and the wiki.