mudler/LocalAI v2.21.0 on GitHub

💡 Highlights!

LocalAI v2.21 release is out!

Deprecation of the exllama backend
AIO images now have gpt-4o instead of gpt-4-vision-preview for Vision API
vLLM backend now supports embeddings
New endpoint to list system information (/system)
trust_remote_code is now respected by sentencetransformers
Auto warm-up and load models on start
coqui backend switched to the community-maintained fork

What's Changed

Breaking Changes 🛠

chore(exllama): drop exllama backend by @mudler in #3536
chore(aio): rename gpt-4-vision-preview to gpt-4o by @mudler in #3597

Exciting New Features 🎉

feat: elevenlabs sound-generation api by @dave-gray101 in #3355
feat(vllm): add support for embeddings by @mudler in #3440
feat: add endpoint to list system informations by @mudler in #3449
feat: extract output with regexes from LLMs by @mudler in #3491
feat: allow setting trust_remote_code for sentencetransformers backend by @Nyralei in #3552
feat(api): allow to pass videos to backends by @mudler in #3601
feat(api): allow to pass audios to backends by @mudler in #3603
feat: auto load into memory on startup by @sozercan in #3627
feat(coqui): switch to maintained community fork by @mudler in #3625

Bug fixes 🐛

fix(p2p): correctly allow to pass extra args to llama.cpp by @mudler in #3368
fix(model-loading): keep track of open GRPC Clients by @mudler in #3377
fix(tts): check error before inspecting result by @mudler in #3415
fix(shutdown): do not shutdown immediately busy backends by @mudler in #3543
fix(parler-tts): fix install with sycl by @mudler in #3624
fix(ci): fixup checksum scanning pipeline by @mudler in #3631
fix(hipblas): do not push all variants to hipblas builds by @mudler in #3630

🧠 Models

chore(model-gallery): add more quants for popular models by @mudler in #3365
models(gallery): add phi-3.5 by @mudler in #3376
models(gallery): add calme-2.1-phi3.5-4b-i1 by @mudler in #3383
models(gallery): add magnum-v3-34b by @mudler in #3384
models(gallery): add phi-3.5-vision by @mudler in #3421
Revert "models(gallery): add phi-3.5-vision" by @mudler in #3422
chore(model-gallery): ⬆️ update checksum by @localai-bot in #3425
feat: Added Piper voice it-paola-medium by @fakezeta in #3434
chore(model-gallery): ⬆️ update checksum by @localai-bot in #3442
models(gallery): add hubble-4b-v1 by @mudler in #3444
chore(model-gallery): ⬆️ update checksum by @localai-bot in #3446
models(gallery): add yi-coder (and variants) by @mudler in #3482
chore(model-gallery): ⬆️ update checksum by @localai-bot in #3486
models(gallery): add reflection-llama-3.1-70b by @mudler in #3487
models(gallery): add athena-codegemma-2-2b-it by @mudler in #3490
models(gallery): add azure_dusk-v0.2-iq-imatrix by @mudler in #3538
models(gallery): add mn-12b-lyra-v4-iq-imatrix by @mudler in #3539
models(gallery): add datagemma models by @mudler in #3540
models(gallery): add l3.1-8b-niitama-v1.1-iq-imatrix by @mudler in #3550
models(gallery): add llama-3.1-8b-stheno-v3.4-iq-imatrix by @mudler in #3551
fix: gallery/index.yaml comment spacing by @dave-gray101 in #3585
models(gallery): add qwen2.5-14b-instruct by @mudler in #3607
models(gallery): add qwen2.5-math-7b-instruct by @mudler in #3609
models(gallery): add qwen2.5-14b_uncencored by @mudler in #3610
models(gallery): add qwen2.5-coder-7b-instruct by @mudler in #3611
models(gallery): add qwen2.5-math-72b-instruct by @mudler in #3612
models(gallery): add qwen2.5-0.5b-instruct, qwen2.5-1.5b-instruct by @mudler in #3613
models(gallery): add qwen2.5 32B, 72B, 32B Instruct by @mudler in #3614
models(gallery): add llama-3.1-supernova-lite-reflection-v1.0-i1 by @mudler in #3615
models(gallery): add llama-3.1-supernova-lite by @mudler in #3616
models(gallery): add llama3.1-8b-shiningvaliant2 by @mudler in #3617
models(gallery): add buddy2 by @mudler in #3618
models(gallery): add llama-3.1-8b-arliai-rpmax-v1.1 by @mudler in #3619
Fix NeuralDaredevil URL by @nyx4ris in #3621
models(gallery): add nightygurps-14b-v1.1 by @mudler in #3633
models(gallery): add gemma-2-9b-arliai-rpmax-v1.1 by @mudler in #3634
models(gallery): add gemma-2-2b-arliai-rpmax-v1.1 by @mudler in #3635
models(gallery): add acolyte-22b-i1 by @mudler in #3636

📖 Documentation and examples

docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #3366
chore(docs): add Vulkan images links by @mudler in #3620

👒 Dependencies

chore: ⬆️ Update ggerganov/llama.cpp to 3ba780e2a8f0ffe13f571b27f0bbf2ca5a199efc by @localai-bot in #3361
chore(deps): Bump openai from 1.41.1 to 1.42.0 in /examples/functions by @dependabot in #3390
chore(deps): Bump docs/themes/hugo-theme-relearn from 82a5e98 to 3a0ae52 by @dependabot in #3391
chore(deps): Bump idna from 3.7 to 3.8 in /examples/langchain/langchainpy-localai-example by @dependabot in #3399
chore(deps): Bump llama-index from 0.10.65 to 0.11.1 in /examples/chainlit by @dependabot in #3404
chore(deps): Bump llama-index from 0.10.67.post1 to 0.11.1 in /examples/langchain-chroma by @dependabot in #3406
chore(deps): Bump marshmallow from 3.21.3 to 3.22.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3400
chore(deps): Bump openai from 1.40.5 to 1.42.0 in /examples/langchain-chroma by @dependabot in #3405
chore(deps): Bump openai from 1.41.1 to 1.42.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3401
chore(deps): update edgevpn to v0.28 by @mudler in #3412
chore(deps): Bump langchain from 0.2.14 to 0.2.15 in /examples/functions by @dependabot in #3453
chore(deps): Bump certifi from 2024.7.4 to 2024.8.30 in /examples/langchain/langchainpy-localai-example by @dependabot in #3457
chore(deps): Bump yarl from 1.9.4 to 1.9.7 in /examples/langchain/langchainpy-localai-example by @dependabot in #3459
chore(deps): Bump langchain-community from 0.2.12 to 0.2.15 in /examples/langchain/langchainpy-localai-example by @dependabot in #3461
chore(deps): Bump llama-index from 0.11.1 to 0.11.4 in /examples/chainlit by @dependabot in #3462
chore(deps): Bump llama-index from 0.11.1 to 0.11.4 in /examples/langchain-chroma by @dependabot in #3467
chore(deps): Bump docs/themes/hugo-theme-relearn from 3a0ae52 to 550a6ee by @dependabot in #3472
chore(deps): Bump openai from 1.42.0 to 1.43.0 in /examples/functions by @dependabot in #3452
chore(deps): Bump langchain from 0.2.14 to 0.2.15 in /examples/langchain/langchainpy-localai-example by @dependabot in #3460
chore(deps): Bump openai from 1.42.0 to 1.43.0 in /examples/langchain-chroma by @dependabot in #3468
chore(deps): Bump langchain from 0.2.14 to 0.2.15 in /examples/langchain-chroma by @dependabot in #3466
chore(deps): Bump streamlit from 1.37.1 to 1.38.0 in /examples/streamlit-bot by @dependabot in #3465
chore(deps): Bump openai from 1.42.0 to 1.43.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3456
chore(deps): Bump langchain-community from 0.2.15 to 0.2.16 in /examples/langchain/langchainpy-localai-example by @dependabot in #3500
chore(deps): Bump openai from 1.43.0 to 1.44.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3504
chore(deps): Bump docs/themes/hugo-theme-relearn from 550a6ee to f696f60 by @dependabot in #3505
chore(deps): Bump langchain from 0.2.15 to 0.2.16 in /examples/langchain-chroma by @dependabot in #3507
chore(deps): Bump peter-evans/create-pull-request from 6 to 7 by @dependabot in #3518
chore(deps): Bump openai from 1.43.0 to 1.44.0 in /examples/functions by @dependabot in #3522
chore(deps): Bump langchain from 0.2.15 to 0.2.16 in /examples/langchain/langchainpy-localai-example by @dependabot in #3502
chore(deps): Bump numpy from 2.1.0 to 2.1.1 in /examples/langchain/langchainpy-localai-example by @dependabot in #3503
chore(deps): Bump llama-index from 0.11.4 to 0.11.7 in /examples/langchain-chroma by @dependabot in #3508
chore(deps): Bump langchain from 0.2.15 to 0.2.16 in /examples/functions by @dependabot in #3521
chore(deps): Bump openai from 1.43.0 to 1.44.1 in /examples/langchain-chroma by @dependabot in #3532
chore(deps): Bump yarl from 1.9.7 to 1.11.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3501
chore(deps): Bump llama-index from 0.11.4 to 0.11.7 in /examples/chainlit by @dependabot in #3516
chore(deps): update llama.cpp to 6262d13e0b2da91f230129a93a996609a2fa2f2 by @mudler in #3549
chore(deps): Bump docs/themes/hugo-theme-relearn from f696f60 to d5a0ee0 by @dependabot in #3558
chore(deps): Bump setuptools from 72.1.0 to 75.1.0 in /backend/python/coqui by @dependabot in #3554
chore(deps): Bump langchain from 0.2.16 to 0.3.0 in /examples/functions by @dependabot in #3559
chore(deps): Bump openai from 1.44.1 to 1.45.1 in /examples/langchain-chroma by @dependabot in #3556
chore(deps): Bump setuptools from 72.1.0 to 75.1.0 in /backend/python/autogptq by @dependabot in #3553
chore(deps): Bump securego/gosec from 2.21.0 to 2.21.2 by @dependabot in #3561
chore(deps): Bump setuptools from 69.5.1 to 75.1.0 in /backend/python/transformers-musicgen by @dependabot in #3564
chore(deps): Bump setuptools from 72.1.0 to 75.1.0 in /backend/python/parler-tts by @dependabot in #3565
chore(deps): Bump sentence-transformers from 3.0.1 to 3.1.0 in /backend/python/sentencetransformers by @dependabot in #3566
chore(deps): Bump llama-index from 0.11.7 to 0.11.9 in /examples/chainlit by @dependabot in #3567
chore(deps): Bump weaviate-client from 4.6.7 to 4.8.1 in /examples/chainlit by @dependabot in #3568
chore(deps): Bump setuptools from 72.1.0 to 75.1.0 in /backend/python/vall-e-x by @dependabot in #3570
chore(deps): Bump greenlet from 3.0.3 to 3.1.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3571
chore(deps): Bump setuptools from 70.3.0 to 75.1.0 in /backend/python/diffusers by @dependabot in #3575
chore(deps): Bump setuptools from 70.3.0 to 75.1.0 in /backend/python/bark by @dependabot in #3574
chore(deps): Bump setuptools from 72.1.0 to 75.1.0 in /backend/python/rerankers by @dependabot in #3578
chore(deps): Bump setuptools from 69.5.1 to 75.1.0 in /backend/python/transformers by @dependabot in #3579
chore(deps): Bump setuptools from 70.3.0 to 75.1.0 in /backend/python/vllm by @dependabot in #3580
chore(deps): Bump langchain from 0.2.16 to 0.3.0 in /examples/langchain-chroma by @dependabot in #3557
chore(deps): Bump openai from 1.44.0 to 1.45.1 in /examples/functions by @dependabot in #3560
chore(deps): Bump langchain from 0.2.16 to 0.3.0 in /examples/langchain/langchainpy-localai-example by @dependabot in #3577
chore(deps): Bump openai from 1.44.0 to 1.45.1 in /examples/langchain/langchainpy-localai-example by @dependabot in #3573
chore(deps): Bump pypinyin from 0.50.0 to 0.53.0 in /backend/python/openvoice by @dependabot in #3562
chore(deps): Bump yarl from 1.11.0 to 1.11.1 in /examples/langchain/langchainpy-localai-example by @dependabot in #3643
chore(deps): Bump urllib3 from 2.2.2 to 2.2.3 in /examples/langchain/langchainpy-localai-example by @dependabot in #3646
chore(deps): Bump idna from 3.8 to 3.10 in /examples/langchain/langchainpy-localai-example by @dependabot in #3644
chore(deps): Bump sqlalchemy from 2.0.32 to 2.0.35 in /examples/langchain/langchainpy-localai-example by @dependabot in #3649

Other Changes

feat: external backend launching log improvements and relative path support by @dave-gray101 in #3348
Update quickstart.md by @grant-wilson in #3373
feat(swagger): update swagger by @localai-bot in #3370
fix: devcontainer utils.sh ssh copy improvements by @dave-gray101 in #3372
chore(cuda): reduce binary size by @mudler in #3379
chore(deps): update edgevpn by @mudler in #3385
chore: ⬆️ Update ggerganov/llama.cpp to 7d787ed96c32be18603c158ab0276992cf0dc346 by @localai-bot in #3409
chore: ⬆️ Update ggerganov/llama.cpp to 20f1789dfb4e535d64ba2f523c64929e7891f428 by @localai-bot in #3417
chore: ⬆️ Update ggerganov/llama.cpp to 9fe94ccac92693d4ae1bc283ff0574e8b3f4e765 by @localai-bot in #3424
chore(cli): be consistent between workers and expose ExtraLLamaCPPArgs to both by @mudler in #3428
chore(tests): replace runaway models for tests by @mudler in #3432
chore(model-loader): increase test coverage of model loader by @mudler in #3433
chore(deps): update llama.cpp by @mudler in #3438
chore: ⬆️ Update ggerganov/llama.cpp to a47667cff41f5a198eb791974e0afcc1cddd3229 by @localai-bot in #3441
chore: ⬆️ Update ggerganov/llama.cpp to 8f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48c by @localai-bot in #3445
fix: untangle pkg/grpc and core/schema for Transcription by @dave-gray101 in #3419
chore(deps): update whisper.cpp by @mudler in #3443
chore: ⬆️ Update ggerganov/llama.cpp to 48baa61eccdca9205daf8d620ba28055c2347b64 by @localai-bot in #3474
chore: ⬆️ Update ggerganov/whisper.cpp to 5236f0278420ab776d1787c4330678d80219b4b6 by @localai-bot in #3475
chore: ⬆️ Update ggerganov/llama.cpp to 8962422b1c6f9b8b15f5aeaea42600bcc2d44177 by @localai-bot in #3478
fix: purge a few remaining runway model references by @dave-gray101 in #3480
chore: ⬆️ Update ggerganov/llama.cpp to 581c305186a0ff93f360346c57e21fe16e967bb7 by @localai-bot in #3481
chore: ⬆️ Update ggerganov/llama.cpp to 4db04784f96757d74f74c8c110c2a00d55e33514 by @localai-bot in #3485
feat(swagger): update swagger by @localai-bot in #3484
chore: ⬆️ Update ggerganov/llama.cpp to 815b1fb20a53e439882171757825bacb1350de04 by @localai-bot in #3489
chore: ⬆️ Update ggerganov/whisper.cpp to 5caa19240d55bfd6ee316d50fbad32c6e9c39528 by @localai-bot in #3494
fix: speedup and improve cachability of docker build of builder-sd by @dave-gray101 in #3430
chore: ⬆️ Update ggerganov/whisper.cpp to a551933542d956ae84634937acd2942eb40efaaf by @localai-bot in #3534
chore(deps): update llama.cpp by @mudler in #3497
chore(gosec): fix CI by @mudler in #3537
chore: ⬆️ Update ggerganov/llama.cpp to feff4aa8461da7c432d144c11da4802e41fef3cf by @localai-bot in #3542
chore: ⬆️ Update ggerganov/whisper.cpp to 049b3a0e53c8a8e4c4576c06a1a4fccf0063a73f by @localai-bot in #3548
feat: auth v2 - supersedes #2894 by @dave-gray101 in #3476
chore: ⬆️ Update ggerganov/llama.cpp to 23e0d70bacaaca1429d365a44aa9e7434f17823b by @localai-bot in #3581
Revert "chore(deps): Bump setuptools from 69.5.1 to 75.1.0 in /backend/python/transformers" by @mudler in #3586
chore(refactor): drop duplicated shutdown logics by @mudler in #3589
Revert "chore(deps): Bump securego/gosec from 2.21.0 to 2.21.2" by @mudler in #3590
chore: ⬆️ Update ggerganov/llama.cpp to 8b836ae731bbb2c5640bc47df5b0a78ffcb129cb by @localai-bot in #3591
chore: ⬆️ Update ggerganov/whisper.cpp to 5b1ce40fa882e9cb8630b48032067a1ed2f1534f by @localai-bot in #3592
chore: ⬆️ Update ggerganov/llama.cpp to 64c6af3195c3cd4aa3328a1282d29cd2635c34c9 by @localai-bot in #3598
feat(swagger): update swagger by @localai-bot in #3604
chore: ⬆️ Update ggerganov/llama.cpp to 6026da52d6942b253df835070619775d849d0258 by @localai-bot in #3605
chore: ⬆️ Update ggerganov/whisper.cpp to 34972dbe221709323714fc8402f2e24041d48213 by @localai-bot in #3623
chore: ⬆️ Update ggerganov/llama.cpp to 63351143b2ea5efe9f8b9c61f553af8a51f1deff by @localai-bot in #3622
chore: ⬆️ Update ggerganov/llama.cpp to d09770cae71b416c032ec143dda530f7413c4038 by @localai-bot in #3626
chore: ⬆️ Update ggerganov/llama.cpp to c35e586ea57221844442c65a1172498c54971cb0 by @localai-bot in #3629
chore: ⬆️ Update ggerganov/llama.cpp to f0c7b5edf82aa200656fd88c11ae3a805d7130bf by @localai-bot in #3653
test: preliminary tests and merge fix for authv2 by @dave-gray101 in #3584

New Contributors

@grant-wilson made their first contribution in #3373
@Nyralei made their first contribution in #3552
@nyx4ris made their first contribution in #3621

Full Changelog: v2.20.1...v2.21.0