koboldcpp-1.29
KoboldCpp Changes:
- Added BLAS batch size to the KoboldCpp Easy Launcher GUI.
- Merged the upstream K-quantization implementations for OpenBLAS. Note that the new K-quants are still not supported in CLBlast yet. Please remain on the regular quantization formats to use CLBlast for now.
- Fixed LLAMA 3B OOM errors and a few other OOMs.
- Multiple bugfixes and improvements in Lite, including streaming for aesthetic chat mode.
To use, download and run the koboldcpp.exe, which is a one-file pyinstaller.
Alternatively, drag and drop a compatible ggml model on top of the .exe, or run it and manually select the model in the popup dialog.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001
For more information, be sure to run the program with the --help flag.
This release also includes a zip file containing the libraries and the koboldcpp.py script, for those who prefer not use to the one-file pyinstaller.