What's Changed
The most exciting feature that this release of llamafile brings is the first version of transcribefile! 🎉 Our transcribefile is nothing more than a cosmocc-build of the CLI tool from transcribe.cpp, @cjpais' ggml-based Speech-to-Text library that supports 16+ model families 💙 For more details, see CJ's post and Mozilla.ai's annoucement.
In addition to this, a new sync with upstream llama.cpp (which will be soon replaced by a more recent one, as I let too much time pass before the release 😬); few improvements to Vulkan and ROCm acceleration; HTTPS support (both for the server and to download models from HF given their IDs); and pledge/SECCOMP sandboxing.
Thanks everyone for improving llamafile with issues and PRs!
- Probe GPU device count out-of-process on Windows (#988 follow-up) by @aittalam in #994
- docs: redirect GitHub Pages docs to GitBook by @angpt in #993
- ROCm shell script improvements by @aittalam in #995
- docs(support): add GPU backend status table & acceleration verification (#996) by @Anai-Guo in #997
- skill: guardrail llama.cpp update workflow with patch commands by @aittalam in #998
- Update llama.cpp to b9747 (d6d8995) by @aittalam in #1000
- chore: upgrade and digest pin all action digests by @RichardoC in #1002
- CPU / GPU fixes + harness for upstream ggml's test-backend-ops by @aittalam in #1004
- vulkan: clearer message when --gpu vulkan is explicitly unavailable by @aittalam in #1006
- Fixes CTRL+C bug in chat mode by @aittalam in #1009
- transcribefile: CPU build of transcribe.cpp + sync to df1a4ad by @aittalam in #1007
- Updated runner to 16-core by @aittalam in #1005
- transcribefile: Metal GPU support via llamafile's runtime loader by @aittalam in #1010
- Add HTTPS support via cpp-httplib's mbedtls backend by @aittalam in #1011
- feat(help): list all accepted llama.cpp options in --help by @RichardoC in #1003
- Bring back pledge/SECCOMP sandboxing (#930) by @aittalam in #1015
New Contributors
- @Anai-Guo made their first contribution in #997
- @RichardoC made their first contribution in #1002
Full Changelog: 0.10.3...0.10.4