av/harbor v0.3.4

v0.3.4 - llama-swap

on GitHub

llama-swap

llama-swap is a lightweight, transparent proxy server that provides automatic model swapping to llama.cpp's server.

# [Optional] pre-pull the image
harbor pull llamaswap
# Edit the swap config
open $(harbor home)/llamaswap/config.yaml
# Run the service
harbor up llamaswap

Misc

• boost
  • docs revamp
  • fixing plain proxy without modules
• tgi - HF cache normalised
• raglite - adding missing traefik config

Full Changelog: v0.3.3...v0.3.4