koboldcpp-1.62.2

There and back again edition

• NEW: Img2Img is now supported when generating images using KoboldCpp. An A1111 compatible endpoint /sdapi/v1/img2img is now emulated. When using Kobold Lite, you can now click an existing image, and generate a new image based off it with Img2Img.
• NEW: OpenAI Chat Completions adapters can now be specified on load with --chatcompletionsadapter. This allows you to use any instruct tag format you want via the Chat Completions API. Please refer to the wiki for documentation. The instruct tags should now also handle all stop sequences correctly and not overflow past them when using OpenAI Chat Completions API.
• Added automatic cleanup of old orphaned koboldcpp pyinstaller temp directories.
• Added more usage statistics available in /api/extra/perf/
• Do not display localhost url if using remote tunnel
• Added /docs endpoint which is an alias for /api, containing API documentation
• Embedded Horde Worker job polling URL changed to aihorde.net
• Embedded Horde Workers will now give priority to the local user, and pause/unpause themselves briefly whenever generating on a local active client, and then returning to full speed when idle. This should allow you to comfortably run a busy horde worker, even when you want to use KoboldCpp locally at the same time.
• Try to fix SSL cert directory not found by specifying a default path.
• Fixed old quant tools not compiling
• Pulled and merged new model support, improvements and fixes from upstream.
• Updated Kobold Lite with some layout fixes, support for Cohere API, Claude Haiku and Gemini 1.5 API, and Img2Img features for local and horde.

Hotfix 1.62.1 - Merged command R plus from upstream. I cannot confirm if it works correctly as CR+ is too big for me to run locally.
Hotfix 1.62.2 - CommandR lite template and fix for appending stop sequences in chat completions

To use, download and run the koboldcpp.exe, which is a one-file pyinstaller.
If you don't need CUDA, you can use koboldcpp_nocuda.exe which is much smaller.
If you're using AMD, you can try koboldcpp_rocm at YellowRoseCx's fork here

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag.