github tashfeenahmed/freellmapi v0.6.0
v0.6.0 — Settings dialog, 60 languages, router discipline
on GitHub

5 hours ago

A dashboard you can use in your own language, a settings surface for theme and preferences, and a large router-discipline pass that stops the gateway causing its own rate limits.

Settings dialog & theming

  • New Settings dialog reachable from the account menu, alongside license-gated Upgrade and Sign out (#607)
  • System / Light / Dark theme — system now tracks the OS preference live, and your choice persists across reloads

60 dashboard languages

  • The dashboard ships in 60 languages (up from 6), with a searchable language picker (#607)
  • Full RTL support — Arabic, Hebrew, Persian, Urdu and friends flip <html dir> immediately
  • Non-English dictionaries are lazy-loaded, so the initial bundle doesn't grow
  • README now lists every supported language and how to contribute a translation (#610)

Routing & reliability

  • In-flight request leases + a provider-wide per-minute gate — concurrent requests no longer all pass the same pre-check and collectively overshoot a free tier's limit (#598)
  • Per-key scoring — key selection samples each key's own measured reliability and speed instead of treating every key on a provider alike; platform:model_id is now a hard pin (#605)
  • Cooldown-probe recovery — heuristically benched keys are re-probed in the background and released early instead of idling out the timer (#601)
  • Client disconnects abort upstream — a closed connection stops the fallback ladder, frees the lease, and is no longer recorded as a provider failure (#601)
  • Honest exhaustion statuses — 413 for context too large, 429 with Retry-After, 404 for unknown models, 503 when nothing is configured, 502 for upstream failures (#601)
  • Streaming first-byte grace — long prefills are no longer killed by the mid-stream stall watchdog (#605)
  • Health false negatives fixed — a transport blip no longer benches every key on a provider, and serving a live request self-heals a key stuck at error (#598)
  • Local Ollama endpoints stop getting 24h rate-limit benches from plain timeouts (#603)
  • A keyless entry in declarative config no longer crashes the server at boot (#603)
  • Per-provider timeouts via PROVIDER_TIMEOUT_<PLATFORM> and a stream stall watchdog

New capabilities

  • /v1/audio/transcriptions — OpenAI-compatible speech-to-text, catalog-driven, with Groq and Cloudflare adapters (#602)
  • reasoning_effort on chat completions and responses, mapped natively to Gemini thinking budgets, with inline <think> blocks extracted into reasoning_content (#602)
  • Per-attempt routing traces — every fallback attempt is persisted; GET /api/analytics/requests/:id returns the ordered ladder (#602)
  • Served-model drift detection — when a provider serves something other than what was asked for, the gateway records it (#603)
  • Gemma tool calling passes through instead of being stripped (#605)

Dashboard

  • Failover-ladder drill-down in Analytics: per-attempt dialog, status/provider filters, per-provider breakdown (#603)
  • Audio tab split into text-to-speech and speech-to-text, with per-model pages and curl snippets (#602)
  • Honest scores — merged model rows show min–max ranges and "best of N"; providers with no traffic say "no data" instead of showing an identical prior (#605)
  • Notifications — success/info auto-dismiss after 5s, errors stay until dismissed, timers pause on hover (#605)
  • 70% smaller bundle over the wire — gzip plus immutable caching for hashed assets, 1390 KB → 411 KB (#574)

Security & operability

  • Gemini key moved out of the ?key= query string into the x-goog-api-key header at every call site — it was landing in proxy logs and error strings (#597)
  • Console credential redaction installed before anything logs (#597)
  • Provider daily caps count from midnight UTC instead of a sliding window; Retry-After clamped to 24h (#597)
  • Cooldowns are visible and clearable — listed on GET /api/keys, cleared with DELETE /api/keys/:id/cooldowns (#597)
  • SQLite busy_timeout plus 0600 on the database and its WAL sidecars (#597)

Fixes

  • TTS: OpenAI voice names mapped to provider voices, so default OpenAI-SDK requests work out of the box (#523)
  • Responses report the routed model id instead of upstream placeholders like Reka's "default" (#568)
  • Android / Termux support via a node:sqlite adapter; better-sqlite3 is now optional so a failed native build can't break install (#308, groundwork by @PSYCH0patch)
  • Health-check failure reasons captured, redacted, and shown beneath the affected key (#536)
  • New docs/README.md index and README documentation links (#436)
  • NVIDIA NIM DEGRADED function 400s treated as provider capacity, not an invalid request
  • Docker image ships server/node_modules so the proxy can load undici
  • Catalog sync guards model override keys against unknown columns (#458)
  • macOS DMG falls back to an unsigned build when Apple signing secrets are absent, instead of failing the release (#563)

Full diff: v0.5.0...v0.6.0

Thanks to @NirvanaCh7, @qq97693453, @AskingConical, @emv33, @Arman-Espiar, @MetaMysteries8, @lujun880726, @ousamabenyounes, @cagedbird043, @jasnoorgill, @Joey9024, @ProAlit, @Mohamed3nan, @wangzhenhui1992, @PSYCH0patch and everyone who reported issues.

macOS note: the DMG is not notarized — on first launch use right-click → Open (or xattr -d com.apple.quarantine /Applications/FreeLLMAPI.app).

Check out latest releases or
releases around tashfeenahmed/freellmapi v0.6.0

Don't miss a new freellmapi release

NewReleases is sending notifications on new releases.

Get notifications