github mudler/LocalAI v2.10.0
on GitHub

latest releases: v2.23.0, v2.22.1, v2.22.0...
8 months ago

LocalAI v2.10.0 Release Notes

Excited to announce the release of LocalAI v2.10.0! This version introduces significant changes, including breaking changes, numerous bug fixes, exciting new features, dependency updates, and more. Here's a summary of what's new:

Breaking Changes 🛠

  • The trust_remote_code setting in the YAML config file of the model are now consumed for enhanced security measures also for the AutoGPTQ and transformers backend, thanks to @dave-gray101's contribution (#1799). If your model relied on the old behavior and you are sure of what you are doing, set trust_remote_code: true in the YAML config file.

Bug Fixes 🐛

  • Various fixes have been implemented to enhance the stability and performance of LocalAI:
    • SSE no longer omits empty finish_reason fields for better compatibility with the OpenAI API, fixed by @mudler (#1745).
    • Functions now correctly handle scenarios with no results, also addressed by @mudler (#1758).
    • A Command Injection Vulnerability has been fixed by @ouxs-19 (#1778).
    • OpenCL-based builds for llama.cpp have been restored, thanks to @cryptk's efforts (#1828, #1830).
    • An issue with OSX build default.metallib has been resolved, which should now allow running the llama-cpp backend on Apple arm64, fixed by @dave-gray101 (#1837).

Exciting New Features 🎉

  • LocalAI continues to evolve with several new features:
    • Ongoing implementation of the assistants API, making great progress thanks to community contributions, including an initial implementation by @christ66 (#1761).
    • Addition of diffusers/transformers support for Intel GPU - now you can generate images and use the transformer backend also on Intel GPUs, implemented by @mudler (#1746).
    • Introduction of Bitsandbytes quantization for transformer backend enhancement and a fix for transformer backend error on CUDA by @fakezeta (#1823).
    • Compatibility layers for Elevenlabs and OpenAI TTS, enhancing text-to-speech capabilities: Now LocalAI is compatible with Elevenlabs and OpenAI TTS, thanks to @mudler (#1834).
    • vLLM now supports stream: true! This feature was introduced by @golgeek (#1749).

Dependency Updates 👒

  • Our continuous effort to keep dependencies up-to-date includes multiple updates to ggerganov/llama.cpp, donomii/go-rwkv.cpp, mudler/go-stable-diffusion, and others, ensuring that LocalAI is built on the latest and most secure libraries.

Other Changes

  • Several internal changes have been made to improve the development process and documentation, including updates to integration guides, stress reduction on self-hosted runners, and more.

Details of What's Changed

Breaking Changes 🛠

Bug fixes 🐛

  • fix(sse): do not omit empty finish_reason by @mudler in #1745
  • fix(functions): handle correctly when there are no results by @mudler in #1758
  • fix(tests): re-enable tests after code move by @mudler in #1764
  • Fix Command Injection Vulnerability by @ouxs-19 in #1778
  • fix: the correct BUILD_TYPE for OpenCL is clblas (with no t) by @cryptk in #1828
  • fix: missing OpenCL libraries from docker containers during clblas docker build by @cryptk in #1830
  • fix: osx build default.metallib by @dave-gray101 in #1837

Exciting New Features 🎉

  • fix: vllm - use AsyncLLMEngine to allow true streaming mode by @golgeek in #1749
  • refactor: move remaining api packages to core by @dave-gray101 in #1731
  • Bump vLLM version + more options when loading models in vLLM by @golgeek in #1782
  • feat(assistant): Initial implementation of assistants api by @christ66 in #1761
  • feat(intel): add diffusers/transformers support by @mudler in #1746
  • fix(config): set better defaults for inferencing by @mudler in #1822
  • fix(docker-compose): update docker compose file by @mudler in #1824
  • feat(model-help): display help text in markdown by @mudler in #1825
  • feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 by @fakezeta in #1823
  • feat(tts): add Elevenlabs and OpenAI TTS compatibility layer by @mudler in #1834
  • feat(embeddings): do not require to be configured by @mudler in #1842

👒 Dependencies

Other Changes

New Contributors

Thank you to all contributors and users for your continued support and feedback, making LocalAI better with each release!

Full Changelog: v2.9.0...v2.10.0

Check out latest releases or
releases around mudler/LocalAI v2.10.0

Don't miss a new LocalAI release

NewReleases is sending notifications on new releases.

Get notifications