koboldcpp-1.1
- Simplifying the version numbering as I don't think I really need that granularity
- Various small tweaks and improvements, and bugfixes
- Updated embedded kobold lite
To use, download and run the koboldcpp.exe
Alternatively, drag and drop a compatible ggml model on top of the .exe, or run it and manually select the model in the popup dialog.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001
If your CPU is very old and doesn't support AVX2 instructions, you can try running the noavx2 version. It will be slower.