koboldcpp-1.0.10
- Updated the embedded kobold lite to version 19
- Merged the various improvements from the parent repo
- Removed psutil dependencies, reverting calculations to be based on 0.5 x cpu_count
- Changed makefile to hopefully work on ARM
To use, download and run the koboldcpp.exe
Alternatively, drag and drop a compatible ggml model on top of the .exe, or run it and manually select the model in the popup dialog.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001