github tashfeenahmed/freellmapi v0.6.2

latest release: v0.6.3
4 hours ago

A second release on top of v0.6.1, focused on proxy support, routing health, and making model metadata editable.

Fixes

  • SOCKS proxies with remote DNS now worksocks5h:// and socks4a:// are accepted, and the standard ALL_PROXY / HTTPS_PROXY / HTTP_PROXY / NO_PROXY environment variables are now detected (explicit PROXY_URL and dashboard settings still take precedence) (#630, #353, #643)
  • Cloudflare Workers AI streams no longer truncate mid-response when the client omits max_tokens — a sensible default output budget is applied per provider (#553, #644)
  • Key health checks no longer fire synchronized bursts at one provider — probes interleave across providers with per-provider spacing and interval jitter (#553, #644)
  • Chronic timeouts now lower a model's speed score instead of only its reliability, and observed speed ranks feed the speed-sort presets (user-set overrides always win) (#619, #645)
  • Models retired by the upstream (HTTP 410 / end-of-life) are auto-quarantined with a "retired" badge instead of burning a fallback slot on every request; a later catalog re-listing reinstates them (#634, #647)
  • Provider "not allowed to access model" errors (400/401 bodies) are now treated as access denials rather than transient failures, ending an infinite cooldown-retry loop (#618, #647)

New

  • Playground file upload — attach images (with client-side downscaling) and text files; non-vision models are flagged while images are staged (#325, #648)
  • Custom endpoints: "Fetch models" discovers the models your relay actually serves and bulk-registers them; newly added custom models are seeded at sensible ranks so auto-routing considers them (#488, #649)
  • Intelligence/speed ranks and RPM/RPD/TPM/TPD limits are now editable per model in the dashboard, with per-field override indicators (#551, #646)

Full diff: v0.6.1...v0.6.2

Thanks to everyone who reported issues.

macOS note: the DMG is not notarized — on first launch use right-click → Open (or xattr -d com.apple.quarantine /Applications/FreeLLMAPI.app).

Don't miss a new freellmapi release

NewReleases is sending notifications on new releases.