github beamivalice/sushi v1.0.2
sushi v1.0.2

latest releases: v1.6.1-dev1, v1.6.0, v1.6.0-dev3...
one month ago

v1.0.2 — Hotfixes

  • Stability: two cached conversations can no longer share a prompt-cache key, a failed long-context cache copy no
    longer frees memory twice, and very low temperatures behave the same with and without MTP.
  • Memory and loading: two resident models no longer over-commit memory at admission, and packs with invalid EXL3
    rate stamps are refused by name.

Don't miss a new sushi release

NewReleases is sending notifications on new releases.