Llamafile
Llamafile v0.9.2 is a significant release. It adds support for:
- DeepSeek Distil R1 Models
- Gemma 3
- IBM Granite
LocalScore
In addition 0.9.2 introduces LocalScore, a benchmarking utility and website.
LocalScore is an open-source tool that both benchmarks how fast Large Language Models (LLMs) run on your specific hardware and serves as a repository for these results. We created LocalScore to provide a simple, portable way to evaluate computer performance across various LLMs while making it easy to share and browse hardware performance data.
LocalScore is now part of the release of Llamafile under the new CLI utility localscore
You can run it ./localscore -m <model>. It is also included in every llamafile so you can benchmark models on your hardware easily using ./llamafile --localscore.
LocalScore was created with support from Mozilla Builders.
What's Changed
- [llamafiler] doc/v1_chat_completions.md: remove duplicate entry by @mseri in #607
- Update server readme with code completion (FIM) example by @heaversm in #637
- URL constructor to get a clean url_prefix (fix #640) by @sizvix in #641
- Fix translation bug from cpp to js in TS highlight by @emilbayes in #681
- Add whisperfile server documentation by @alonsosilvaallende in #707
- Unify button look and rearrange buttons to make them more compact by @corebonts in #712
- add stable-diffusion.cpp to install target (fix #580) by @rgroesslinger in #635
- Improve OpenAI compatibility for /v1/* endpoints by @corebonts in #713
- Update WSL troubleshooting in README.md by @halter73 in #585
- Granite three support by @gabe-l-hart in #608
- Initial support for Gemma 3 models by @corebonts in #717
- Add copy and info buttons to the chat window and improve small screen UX by @corebonts in #719
- Avoid streaming incomplete UTF-8 characters by @corebonts in #727
- Introduce LocalScore CLI by @cjpais in #734
New Contributors
- @mseri made their first contribution in #607
- @heaversm made their first contribution in #637
- @sizvix made their first contribution in #641
- @emilbayes made their first contribution in #681
- @alonsosilvaallende made their first contribution in #707
- @corebonts made their first contribution in #712
- @rgroesslinger made their first contribution in #635
- @halter73 made their first contribution in #585
- @gabe-l-hart made their first contribution in #608
Full Changelog: 0.9.1...0.9.2