github mudler/LocalAI v2.17.0
on GitHub

latest releases: v2.18.0, v2.17.1
13 days ago

local-ai-release-2 17-shadow
Ahoj! this new release of LocalAI comes with tons of updates, and enhancements behind the scenes!

🌟 Highlights TLDR;

  • Automatic identification of GGUF models
  • New WebUI page to talk with an LLM!
  • https://models.localai.io is live! 🚀
  • Better arm64 and Apple silicon support
  • More models to the gallery!
  • New quickstart installer script
  • Enhancements to mixed grammar support
  • Major improvements to transformers
  • Linux single binary now supports rocm, nvidia, and intel

🤖 Automatic model identification for llama.cpp-based models

Just drop your GGUF files into the model folders, and let LocalAI handle the configurations. YAML files are now reserved for those who love to tinker with advanced setups.

🔊 Talk to your LLM!

Introduced a new page that allows direct interaction with the LLM using audio transcription and TTS capabilities. This feature is so fun - now you can just talk with any LLM with a couple of clicks away.
Screenshot from 2024-06-08 12-44-41

🍏 Apple single-binary

Experience enhanced support for the Apple ecosystem with a comprehensive single-binary that packs all necessary libraries, ensuring LocalAI runs smoothly on MacOS and ARM64 architectures.

ARM64

Expanded our support for ARM64 with new Docker images and single binary options, ensuring better compatibility and performance on ARM-based systems.

Note: currently we support only arm core images, for instance: localai/localai:master-ffmpeg-core, localai/localai:latest-ffmpeg-core, localai/localai:v2.17.0-ffmpeg-core.

🐞 Bug Fixes and small enhancements

We’ve ironed out several issues, including image endpoint response types and other minor problems, boosting the stability and reliability of our applications. It is now also possible to enable CSRF when starting LocalAI, thanks to @dave-gray101.

🌐 Models and Galleries

Enhanced the model gallery with new additions like Mirai Nova, Mahou, and several updates to existing models ensuring better performance and accuracy.

Now you can check new models also in https://models.localai.io, without running LocalAI!

Installation and Setup

A new install.sh script is now available for quick and hassle-free installations, streamlining the setup process for new users. 

curl https://localai.io/install.sh | sh

Installation can be configured with Environment variables, for example: 

curl https://localai.io/install.sh | VAR=value sh

List of the Environment Variables:

  • DOCKER_INSTALL: Set to "true" to enable the installation of Docker images.
  • USE_AIO: Set to "true" to use the all-in-one LocalAI Docker image.
  • API_KEY: Specify an API key for accessing LocalAI, if required.
  • CORE_IMAGES: Set to "true" to download core LocalAI images.
  • PORT: Specifies the port on which LocalAI will run (default is 8080).
  • THREADS: Number of processor threads the application should use. Defaults to the number of logical cores minus one.
  • VERSION: Specifies the version of LocalAI to install. Defaults to the latest available version.
  • MODELS_PATH: Directory path where LocalAI models are stored (default is /usr/share/local-ai/models).

We are looking into improving the installer, and as this is a first iteration any feedback is welcome! Open up an issue if something doesn't work for you!

Enhancements to mixed grammar support

Mixed grammar support continues receiving improvements behind the scenes.

🐍 Transformers backend enhancements

  • Temperature = 0 correctly handled as greedy search
  • Handles custom words as stop words
  • Implement KV cache
  • Phi 3 no more requires trust_remote_code: true flag

Shout-out to @fakezeta for these enhancements!

Install models with the CLI

Now the CLI can install models directly from the gallery. For instance: 

local-ai run <model_name_in gallery>

This command ensures the model is installed in the model folder at startup.

🐧 Linux single binary now supports rocm, nvidia, and intel

Single binaries for Linux now contain Intel, AMD GPU, and NVIDIA support. Note that you need to install the dependencies separately in the system to leverage these features. In upcoming releases, this requirement will be handled by the installer script.

📣 Let's Make Some Noise!

A gigantic THANK YOU to everyone who’s contributed—your feedback, bug squashing, and feature suggestions are what make LocalAI shine. To all our heroes out there supporting other users and sharing their expertise, you’re the real MVPs!

Remember, LocalAI thrives on community support—not big corporate bucks. If you love what we're building, show some love! A shoutout on social (@LocalAI_OSS and @mudler_it on twitter/X), joining our sponsors, or simply starring us on GitHub makes all the difference.

Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy

Thanks a ton, and.. enjoy this release!

What's Changed

Bug fixes 🐛

Exciting New Features 🎉

  • feat(images): do not install python deps in the core image by @mudler in #2425
  • feat(hipblas): extend default hipblas GPU_TARGETS by @mudler in #2426
  • feat(build): add arm64 core containers by @mudler in #2421
  • feat(functions): allow parallel calls with mixed/no grammars by @mudler in #2432
  • feat(image): support response_type in the OpenAI API request by @prajwalnayak7 in #2347
  • feat(swagger): update swagger by @localai-bot in #2436
  • feat(functions): better free string matching, allow to expect strings after JSON by @mudler in #2445
  • build(Makefile): add back single target to build native llama-cpp by @mudler in #2448
  • feat(functions): allow response_regex to be a list by @mudler in #2447
  • TTS API improvements by @blob42 in #2308
  • feat(transformers): various enhancements to the transformers backend by @fakezeta in #2468
  • feat(webui): enhance card visibility by @mudler in #2473
  • feat(default): use number of physical cores as default by @mudler in #2483
  • feat: fiber CSRF by @dave-gray101 in #2482
  • feat(amdgpu): try to build in single binary by @mudler in #2485
  • feat:OpaqueErrors to hide error information by @dave-gray101 in #2486
  • build(intel): bundle intel variants in single-binary by @mudler in #2494
  • feat(install): add install.sh for quick installs by @mudler in #2489
  • feat(llama.cpp): guess model defaults from file by @mudler in #2522
  • feat(ui): add page to talk with voice, transcription, and tts by @mudler in #2520
  • feat(arm64): enable single-binary builds by @mudler in #2490
  • feat(util): add util command to print GGUF informations by @mudler in #2528
  • feat(defaults): add defaults for Command-R models by @mudler in #2529
  • feat(detection): detect by template in gguf file, add qwen2, phi, mistral and chatml by @mudler in #2536
  • feat(gallery): show available models in website, allow local-ai models install to install from galleries by @mudler in #2555
  • feat(gallery): uniform download from CLI by @mudler in #2559
  • feat(guesser): identify gemma models by @mudler in #2561
  • feat(binary): support extracted bundled libs on darwin by @mudler in #2563
  • feat(darwin): embed grpc libs by @mudler in #2567
  • feat(build): bundle libs for arm64 and x86 linux binaries by @mudler in #2572
  • feat(libpath): refactor and expose functions for external library paths by @mudler in #2578

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

New Contributors

Full Changelog: v2.16.0...v2.17.0

Check out latest releases or
releases around mudler/LocalAI v2.17.0

Don't miss a new LocalAI release

NewReleases is sending notifications on new releases.

Get notifications