tashfeenahmed/freellmapi v0.8.2

on GitHub

A fixes-only release: one batch of verified reliability, accounting, and deployment corrections (#941). No new providers, no schema-breaking changes; one Docker packaging change to note below.

Fixed

• Streamed token counts no longer lost when usage rides the last content frame (#941). Providers that bundle usage onto the final choice-bearing chunk had their real counts discarded, leaving accounting on the chars/4 estimate. Usage is now captured wherever it lands and reaches the client exactly once, after the finish chunk.
• Wrapped transport errors now fail over instead of 502-ing (#941). undici hides ECONNRESET / EPIPE / socket hang up / UND_ERR_* inside err.cause; a bounded, cycle-safe cause walk now classifies them retryable. Client and hedge aborts remain non-retryable, and account terminations stay fatal.
• Routing penalties could be inert (#941). Disabling models leaves holes in the priority sequence larger than the maximum penalty, so a permanently failing model could stay pinned at the head of its chain. Candidates are re-ranked to dense positions before penalties apply.
• Gemini structured-output 400s (#941). JSON-Schema type unions ("type": ["number","null"]) now translate to Gemini's type + nullable: true, and local $ref targets are inlined from $defs instead of collapsing to an empty schema.
• Hugging Face backups actually persist (#941). Uploads went to the read-only /resolve/ route and silently vanished, so HF Spaces restored nothing on cold start. They now go through the commit API as base64 text; legacy blobs still restore, and a missing token fails loudly.
• GitHub Models stops burning a fallback hop on long requests (#941). Output tokens are capped at the platform's real ceiling and oversized histories are trimmed (system prompt + newest turns) for the GitHub attempt only — the next candidate still sees the full conversation.
• Cooldown escalation is bounded for providers with no published daily limits (#941). Transient RPM jitter could escalate a route to the 24-hour quarantine; unknown-limit routes now cap at 10 minutes, while genuine RPD/TPD exhaustion keeps the full ladder. Expired cooldown rows are also swept once at startup.
• Global z.ai keys validate (#941). Zhipu was pinned to the domestic endpoint; keys issued on the global console now autodetect and route via api.z.ai, with no behavior change for domestic keys.
• SOCKS5 proxies receive hostnames, not IPs (#941). Rule-based proxy clients (Clash) can now route by domain.
• maskKey no longer echoes short keys (#941). Keys of 8 characters or fewer leaked up to their full value in masked display fields.

Performance

• The rate-limit ledger sweep is off the hot path (#941). An unindexed DELETE ran on every request; it now runs at most once a minute.
• In-memory rate windows no longer grow unbounded (#941). On a healthy database the arrays were appended every request and never pruned; memory tracking is now the degraded-mode fallback only, and prunes itself as it records.

Deployment

• The Dockerfile no longer declares VOLUME (#941). The anonymous volume broke or degraded PaaS builds (Railway, Coolify, Dokploy, CapRover) and shadowed same-path bind mounts. Persistence is the deployment's job — the shipped compose file already uses a named volume; docker run users should pass -v. If you relied on the implicit anonymous volume with plain docker run and no -v, add an explicit mount before upgrading.