This release adds Brave Search as a web search engine, soft budget alerts for keys with monthly caps, and opt-in keep-alive pings for long-thinking models. It also fixes a Groq bug that could leave a model refusing almost every request.
Search
- Brave Search. New web and news search engine. Add a Brave key on the Keys page and it appears under Models → Search, with fallback to your other engines. Brave gives $5 of free credits a month (about 1,000 searches); a key that runs out of credit waits for the reset instead of being retried. It arrives through the catalog: Premium now, free tier on 22 November (#1650, #1641).
Providers
- Groq output-token limits. Some Groq accounts have a separate limit on output tokens per minute. One rejection on that limit used to be saved as the model's whole token limit, so later requests were refused before being sent. Output limits are no longer saved, and input-token limits are read correctly (#1546).
- Custom endpoints. Models discovered on your own endpoint that it labels as audio, image or embedding models are now added as those types instead of as chat models (#1556, #1506).
Budgets
- Soft budget alerts. When a key reaches 80% and again 100% of its monthly request or token cap, a
key_budget_thresholdevent appears on the Logs page, once per key per month. SetBUDGET_ALERT_WEBHOOK_URLto also send each alert to a webhook, such as a Slack or Discord workflow (#1665, #1158). - Cap-usage bar. The Edit key dialog shows how much of each monthly cap a key has used, amber from 80% and red at the cap (#1669).
Streams
- Keep-alive pings (opt-in). Set
STREAM_KEEPALIVE_MS(for example10000) to send a content-free ping on that interval while a model hasn't started answering, so clients with an idle timeout don't disconnect. Once a ping is sent, that attempt can no longer fail over silently, which is why it's off by default (#1674, #1642).
Docs
- The Fetch Relay note in
.env.examplepoints to the doc that exists (#1550).
Updating
Desktop installers below cover Windows, Apple Silicon and Intel macOS, and Linux. Docker: ghcr.io/tashfeenahmed/freellmapi:v0.16.9 (latest also points here). Existing keys and profiles are retained. No new migrations in this release.
Thanks to everyone who contributed to this release; you're credited in the README (#1676).
Full changelog: v0.16.8...v0.16.9
⭐ Like the free router? Go Premium — the live signed catalog, $19/yr, cancel anytime.