Bifrost HTTP Transport Release v2.6.0
🔒 Security
- Go 1.27.4 - Security release with fixes to
net/http(a possible HTTP server crash, CVE-2026-97054), cgo, the go command, the compiler and the linker. Bifrost now builds with Go 1.27.4, and the Docker images (Dockerfile,Dockerfile.local,Dockerfile.redhat) move togolang:1.27.4-alpine3.24. Custom Go plugins must be rebuilt with Go 1.27.4 to load. - Dependency security updates -
golang.org/x/net0.62.0 (CVE-2026-78661 excessive resource use parsing HTML, CVE-2026-97045 incorrect HTML rendering),github.com/klauspost/compress1.20.3 (CVE-2026-116591 crash processing compressed data) andgithub.com/andybalholm/brotli1.2.8 (CVE-2026-111497 excessive memory use decompressing data).
✨ Features
- Model Routers - Named model routers classify each request into SIMPLE, MEDIUM or COMPLEX with a semantic, decision-model or LLM classifier, and send each tier to its own weighted provider/model/key targets with fallbacks. A router takes the requests its routing matches (global, team, customer, virtual key or user scope, an optional CEL condition and a priority) after the routing rules run. Routers are managed from the new Routers section in the dashboard, under
/api/routing/model-routers, or in config.json, and request logs record which router routed each request. The existing Complexity Router config keeps fillingcomplexity_tierfor routing rules unchanged, and the router page warns about routing rules that still readcomplexity_tier - Complexity Router Phrase Suggestions - A model router can sample its traffic, cluster it on a schedule within a spend budget and suggest phrases for each tier. The suggestions view shows their impact on past traffic, adds accepted phrases to the router, backfills its window from existing logs, and can classify a test statement against the router's tiers. Runs can also be started on demand through the API. The dashboard view is behind a feature flag
- Bifrost MCP Server -
POST /v1/mcpserves Bifrost's own HTTP API as MCP tools, so MCP clients such as Claude Code can manage a gateway and call its inference routes. Every tool call runs through the same auth and permission checks as a direct/apirequest. Configured withclient.bifrost_url,client.bifrost_mcp_allow_writes(reads only by default) andframework.pricing.openapi_url, orBIFROST_MCP_SPECfor air-gapped installs - Warp Management Actions - Warp answers questions about the gateway's configuration and can create resources through the Bifrost MCP. Every write ends the turn with a confirmation card that shows the tool, its arguments and whether it is destructive, and runs only after the user confirms. Writes are off unless enabled in Warp settings. New endpoints shared with Warp:
GET /api/session/me, user limit lookups,GET /api/logs/search(semantic search) andGET /api/logs/{id}/trace - Claude Code Gateway Sign-In - Claude Code can sign in to Bifrost with
/login(Cloud gateway) at<issuer_url>/gateway/claude-code: device-code sign-in through the OAuth consent page (virtual key, user or anonymous), silent token refresh, Messages inference resolved to the approved virtual key, managed settings, and optional forwarding of Claude Code's OTLP telemetry to your collectors, stamped with the signed-in virtual key, team, customer or user. The consent page requires the code shown in the terminal and can deny a sign-in, users with no access are refused, and enabling a gateway or changing its managed settings needs an admin session. Configured underoauth2_server_config.gateways.claude_codeor on the new Settings → Gateways page, independent of the MCP auth mode
The gateway reserves the/gatewayroute prefix. Aserver.base_pathwhose first segment isgatewayis refused at startup, so move such a deployment to another prefix before upgrading.
- Realtime Speech and Live Sessions -
/v1/realtimeserves Gemini Live on the Gemini API and Vertex AI, ElevenLabs Scribe and Soniox real-time transcription (intent=transcription). GenAI SDK clients in Vertex mode can open Live sessions on/genai. A new WebSocket at/v1/audio/speech/streamstreams text in and audio back while the text is still arriving, billed once per session and logged with the full spoken text - ElevenLabs Drop-In Routes - ElevenLabs SDKs and raw clients can point at
{gateway}/elevenlabsand send a Bifrost virtual key inxi-api-key: text-to-speech (with and without timestamps), the model list filtered to the key's models, and the native Scribe and text-to-dialogue WebSockets, with governance, logging and cost tracking./elevenlabs_passthroughforwards any other ElevenLabs route - Vertex Text-to-Speech - Vertex serves
/v1/audio/speechthrough Cloud Text-to-Speech (Chirp 3 HD and other Google voices) and through Gemini TTS models, including streaming for Gemini TTS - Soniox Provider - New
sonioxprovider for speech-to-text (async jobs behind/v1/audio/transcriptions), text-to-speech with streaming, and real-time transcription - Baseten Provider - New
basetenprovider for Model APIs and dedicated deployments: chat, Responses, embeddings and transcription,/v1/decisionsthrough System One, image generation on dedicated deployments (including the dashboard's/predictURL), and rerank on dedicated BEI deployments - Kimchi Provider - New
kimchiprovider for chat, Responses, embeddings and list models (thanks @arnasnn!) - MachGen Provider - New
machgenprovider for MachGen's image and video models: text-to-image, image edit, and async video generation with retrieve and download, settled and priced by the video job sweeper./v1/modelslists the models each key can use, with their task types. Set up from the UI orproviders.machgenin config.json (thanks @nishkarsh-machgen!) - Sarvam Token Factory v2 - Sarvam chat and model listing move to Token Factory v2, and Sarvam Prism serves
/v1/decisions(thanks @vikashmaddi-sarvam!)
sarvam-30bis no longer served. Usesarvam-105b.
- Gemini Decisions - Gemini decision models (
gemini-decision-*) serve/v1/decisionsnatively, and a newinput_filepart carries PDFs, audio and video. Empty or mistyped input is rejected before the call (thanks @Xytronix!) - Regex Key Aliases - A key alias can be a
regex:pattern over the requested model, with amodel_idtemplate that reuses its capture groups (for exampleregex:(claude-.*)→global.anthropic.$1), so one row aliases a whole model family. The model catalog resolves pattern aliases, and the key deployments table shows pattern rows - Datasheet Overrides - Model pricing and capabilities can be edited from a datasheet editor under Models, through new override endpoints or
datasheet_overridesin config.json. Overrides are stored separately and survive every upstream datasheet sync - Usage Estimates for Cancelled Streams - A chat, text or Responses stream that is cancelled or times out has its missing usage estimated with a tokenizer from the output already received, and priced like reported usage. Estimated fields are marked in
usage.estimation, and provider-reported fields are never replaced - Bedrock Native Structured Output - Bedrock Converse sends
json_schemaresponse formats through nativeoutputConfigfor models that support it, and falls back to the synthetic tool for other models, for schemas Claude cannot accept, and once more if Bedrock rejects the native schema. Thesupports_native_structured_outputdatasheet field overrides the default per model - Custom Tools on Every Provider - Custom (freeform) tools such as Codex's
apply_patchare sent as function tools to every model without native custom tool support, and the model's calls come back to the client as custom tool calls on both Chat Completions and Responses, streaming included. Models markedsupports_custom_toolskeep the native form - Codex WebSocket Through ChatGPT Sign-In -
/chatgpt_passthrough/backend-api/codex/responsesaccepts the WebSocket Codex opens withsupports_websockets = trueand relays it to ChatGPT with the caller's ChatGPT token. Each turn is checked against the virtual key and logged and billed like a turn sent over HTTP. A request without a ChatGPT token gets a 426, which moves Codex to HTTP right away. The Codex model picker lists only the models the caller's virtual key allows - MCP Gateway Settings -
slug_code_modeturns Code Mode on or off for one client's/mcp/<slug>endpoint only.mcp_client_metadata_urllets Bifrost present a self-hosted Client ID Metadata Document when upstream authorization servers cannot reach it.mcp_server_max_protocol_versioncaps the MCP protocol version Bifrost's own/mcpserves, applied without a restart - Configurable OpenAPI MCP Spec Size - The OpenAPI MCP spec size limit rises from 5 MiB to 32 MiB by default and is set with
mcp_openapi_max_spec_mb(up to 256) - MCP Logs Dashboard - MCP logs show a metric strip with sparklines and previous-period trends, a latency histogram and cancelled counts, readable error cards with formatted upstream JSON, and user, team and customer filters
- Stop Reason Column - The logs table has an optional stop reason column (thanks @Constantine3!)
- Trace Dimensions on Child Spans - An OTEL profile with
apply_trace_dimensions_to_child_spansset merges the request'sx-bf-dim-*dimensions onto every exported span (LLM call, plugin, retry, fallback, MCP tool), not only the root HTTP span. Span attributes win on conflict, andpath,methodand keys in reserved attribute namespaces (gen_ai.,http.,bifrost., ...) are skipped (thanks @devin-williams!) - Provider Queue Wait Metric - New
bifrost_provider_queue_wait_secondshistogram records how long an attempt waits for a provider worker - opencode Sessions - Requests from opencode carry a stable
x-opencode-sessionheader per virtual key, resolved from the client's headers (thanks @sstreichan!) - Agent Tracing and Log Previews - Agent traces are easier to follow, and log rows for agent-loop requests preview the latest tool call or tool result instead of a blank message
- Anthropic Thinking Block Binding -
thinking.block_bindingis forwarded to the Claude API with its beta header - Fable 5.2 Fast Mode on Vertex -
"speed": "fast"is sent to Vertex forclaude-fable-5-2 - Databricks Scoped OAuth Secrets - Databricks service principals whose OAuth secret is limited to specific scopes work without configuration: Bifrost retries the token request with the scopes its routes need, and a scope 403 is reported as the provider's own message
- Virtual MCPs on Virtual Keys - The virtual key details sheet lists the Virtual MCPs assigned to the key
- FIPS-Approved Key Derivation - In FIPS 140-3 mode, the encryption data key is derived with PBKDF2-HMAC-SHA-256 and passwords are hashed with PBKDF2-HMAC-SHA-256. Standard builds keep writing the earlier formats, and both formats stay readable, so a rolling deploy is safe
🐞 Fixed
- Mid-Conversation System Reminders - Claude Code's trailing system reminder now keeps one form (native or
<system-reminder>) across turns, decided from the messages actually sent. On Bedrock InvokeModel the changing form had invalidated the prompt cache every turn, cutting cached input and raising cost. The<system-reminder>envelope also drops its trailing newline, and the cache breakpoint stays on the marked block. Native placement is now controlled by thesupports_native_mid_conversation_systemdatasheet field - Vertex Prompt Cache Misses - Vertex Claude request bodies always end with
tools,system,messages, so turn-to-turn cache reads no longer miss when a client sendsmessagesfirst - Cached Tokens Counted Twice - Anthropic streaming usage, on chat and Responses streams and on the passthrough route, counts each cached prompt token once, including upstreams such as kimi-code that report the cache split on
message_delta(thanks @is911 and @wangkanai!) - Prompt Cache on Aliases - Prompt cache auto-injection checks an alias's canonical model, so Bedrock application inference profile aliases for Claude get cache markers (thanks @bardrop!)
- prompt_cache_options -
prompt_cache_optionsreaches only the models that accept it, instead of failing requests to other providers - Anthropic Deferred Web Search - A deferred web search is replayed without a made-up
web_search_tool_result, and its late result stays in the next assistant message on replay, Responses and streams, so neither the continuation nor later turns 400 - Anthropic MCP History on Chat Providers - Messages requests whose history holds
mcp_tool_use/mcp_tool_resultor server-tool blocks no longer send a roleless message to OpenAI-compatible providers, and Responses items with no chat equivalent are dropped on the Responses-to-Chat fallback (thanks @1AbdullahButt!) - Bedrock Structured Output Streams -
/anthropic/v1/messageswithoutput_config.formaton Bedrock Claude streams one well-formed text block, and a plain-text answer from a model that skips the synthetic tool is no longer lost - Bedrock base_url - Bedrock honours
network_config.base_urlfor Converse, InvokeModel, the OpenAI-compatible runtime, Nova Sonic realtime and ApplyGuardrail, and its HTTP client applies the same dial-time address policy as other providers - Bedrock Beta Headers - A caller's
anthropic_betaon Bedrock Converse is merged with the betas Bifrost derives instead of replacing them, so fine-grained tool streaming and server-tool betas keep working (thanks @fus3r!) - Bedrock tool_reference Blocks - Bedrock InvokeModel keeps
tool_referenceblocks inside a clienttool_result, so deferred tools are expanded (thanks @kunalgoesbyken!) - Bedrock Computer Tools - Claude Responses requests with a
computerorcomputer_use_previewtool are served through InvokeModel instead of losing the tool, and replayedtool_useblocks keeptoolset_name(thanks @davidelct!) - Vertex Anthropic Betas - Vertex receives Anthropic betas in the
anthropic-betaheader, including on CountTokens, instead of a body field it rejects - Bedrock Mantle OpenAI Models - Unsupported effort-dependent sampling parameters are stripped for OpenAI models on Bedrock Mantle, as they are on OpenAI
- Logprobs Across Formats - Logprobs survive conversion between Chat Completions and Responses, streaming and non-streaming
- Gemini Function Call IDs - Gemini function calls without an ID get unique IDs, so parallel calls to the same tool pair with the right responses
- Nested Custom Tool Names - Chat custom tools keep the
nameanddescriptionnested undercustom - Gemini Image Output Format -
output_formatandoutput_compressionmap to Gemini image output options on Vertex, and the Gemini API no longer receives options it rejects (thanks @carlreid!) - Mistral Reasoning Effort -
reasoning_effortfollows each Mistral model's ladder instead of being rewritten tohigh - ElevenLabs Timestamps - Speech
with_timestampsreturns character times instead of null (thanks @abmorrill4!)
Breaking for Go SDK users:SpeechAlignment.CharStartTimesMs/CharEndTimesMsare renamed toCharacterStartTimesSeconds/CharacterEndTimesSeconds, and the JSON fieldschar_*_times_msbecomecharacter_*_times_seconds, in seconds.
- ElevenLabs WAV and Scribe Access - Non-streaming
wavspeech returns a real WAV file, and Scribe transcription works with wildcard virtual keys - Empty Transcriptions - Transcribing silence with
text,srtorvttoutput returns empty text instead of a 500 - Unsupported Streaming Speech Models - Streaming speech on OpenAI
tts-1andtts-1-hdreturns a 400 instead of a 500 - GenAI Audio by URL - A
/genairequest with an audiofileDataURL is answered as a multimodal prompt instead of being rejected as a transcription - Vertex Model Listing - Model listing on
usandeukeys uses the same regional endpoint host as inference, and token errors show the underlying cause - allowed_models and Key Aliases - A virtual key that allows a model also allows a key alias that resolves to it (thanks @aharshit123456!)
- Unprefixed Models on Keyless Providers - Catalog-based provider resolution honours
SkipKeySelection, so bare Claude Code model names work on the Anthropic passthrough (thanks @cgtarmenta!) - Model Parameters After URL Sync - Stored model parameters stay in effect after the background model-parameters URL sync (thanks @Constantine3!)
- Deterministic Config Hashes - Provider config hashes no longer change with map order, so API-added keys are not dropped on restart (thanks @rojinebrahimi!)
- Seedance Video Tokens - Seedance video settlements record their output tokens, so logs show them and virtual key token limits count them
- Active Requests Gauge Leak - A stream cancelled before the upstream's first byte no longer leaves
bifrost_active_requestsraised or its log row stuck in processing - Upstream Latency -
bifrost_upstream_latency_secondsno longer includes provider queue wait, retry backoff or other plugins' hooks (thanks @sam-klaviyo!)
Dashboards built onbifrost_upstream_latency_secondswill show lower values after upgrading. Queue wait is now inbifrost_provider_queue_wait_seconds.
- Large File Uploads - Multipart uploads are streamed to temp files and on to the provider instead of being held in memory, on the Files API (OpenAI, Azure), passthrough routes and transport plugins. Body size limits and their 413 still apply
- Virtual Key Budgets - Updating the budget of a config.json virtual key no longer fails with
invalid reset duration format(thanks @High-PO!). Moving a quarterly budget to another window, or updating a team's or customer's rate-limit maximum alone, no longer fails with a 500, and a budget window added with Reset Usage starts at zero - Team and Customer Saves - Saving a customer or team no longer rewrites its teams, virtual keys or customer, and a team's virtual keys are charged to its new customer right after the change
- Duplicate Provider Configs - A virtual key naming the same provider twice is refused with a 400
- MCP OAuth Consent - The consent page no longer gets stuck when an expired SSO session is refreshed mid-flow, and abandoned client registrations are kept long enough for clients to retry
- OAuth Grant Expiry - The OAuth Grants page uses the configured access token TTL
- OpenAPI MCP Tool Arguments - OpenAPI-derived MCP tools give every argument a name clients accept, so they are no longer dropped
- A2A Agents - Valid A2A v1 agent cards register, native HTTP+JSON errors are preserved, and Agent Logs show Task artifacts and history
- Log Retention on Postgres - Retention deletes shrink their batch when they hit the statement timeout instead of failing on the same rows every pass, and retention tables get tighter autovacuum settings so they stop growing between vacuums
- ClickHouse Hybrid Storage - Object uploads update
has_objectin batches, which stops memory growth and dropped uploads - Log Filters - Relay Logs filter values are read from the last 30 days, and MCP log team and customer filters match calls recorded under several of them
- Encoded Request Paths - Transport plugins see the raw encoded path, so an encoded traversal cannot disguise the route
- Plugin Downloads - URL-based plugins download only from hosts listed in
BIFROST_PLUGIN_URL_ALLOWLIST, checked on every redirect
The allowlist is empty by default, so no plugin URL is fetched until its host is listed. SetBIFROST_PLUGIN_URL_ALLOWLIST(comma-separated hosts,*.example.comor*) before upgrading if you load plugins from a URL.
- Dashboard - Log headers and the date picker fit on mobile, the upgrading screen appears only when the server confirms a chunk is gone, and the datasheet editor hides entries without a provider
🔧 Maintenance
- Dependency Upgrades - brotli 1.2.8, A2A SDK 2.8.0 and Dependabot updates
🗄️ Database Migrations
This release adds columns to the `logs` table (`logs_add_model_routing_columns`) and the `agent_logs` table (`agent_logs_add_caller_correlation_id_column`). Bifrost handles concurrent access on the application side, but on large log tables the change still takes locks and adds load. Deploy during a low-traffic window.- add_usage_estimation_tokenizers_json_column - Adds the nullable tokenizer settings column to
config_clientfor usage estimation. Reversible: rolls back by dropping the column. - add_datasheet_overrides_table - Creates
governance_datasheet_overrides. Non-reversible: dropping it would delete user datasheet edits. The table is additive and older binaries ignore it. - add_databricks_scopes_column - Adds
databricks_scopestoconfig_keys. Reversible: rolls back by dropping the column. - add_model_routing_suggestion_schedules_table - Creates the phrase suggestion schedules table. Reversible: rolls back by dropping the table.
- add_model_routers_table - Creates
model_routers. Non-reversible: dropping it would delete every model router. - add_model_router_targets_and_routing - Creates
model_router_targetsand adds routing columns tomodel_routers. Non-reversible: dropping it would delete every router's targets. - add_model_router_suggestions_tier_columns - Adds suggestion tier columns to
model_routers. Reversible: rolls back by dropping the columns. - add_mcp_client_server_type_on_slug_column - Adds the nullable
slug_code_modesetting toconfig_mcp_clients. Reversible: rolls back by dropping the column. - add_mcp_client_metadata_url_columns - Adds
config_client.mcp_client_metadata_urlandoauth_configs.client_metadata_document. Non-reversible: dropping them would discard the hosted document URL and which OAuth configs use Bifrost's document. - add_client_config_mcp_openapi_max_spec_mb_column - Adds
mcp_openapi_max_spec_mbtoconfig_client. Reversible: rolls back by dropping the column. - add_client_config_bifrost_url_and_bifrost_mcp_columns - Adds the Bifrost URL and Bifrost MCP settings to
config_client. Reversible: rolls back by dropping the columns. - add_framework_config_openapi_url_column - Adds the OpenAPI URL to the framework config. Reversible: rolls back by dropping the column.
- add_warp_management_writes_column - Adds
management_writesto the Warp config. Reversible: without the column Warp offers reads only. - add_client_config_entity_access_profile_mode_column - Adds
entity_access_profile_mode(defaultunion) toconfig_client. Reversible: rolls back by dropping the column. - add_budget_business_unit_id_column - Adds a nullable
business_unit_idowner column togovernance_budgets. Reversible: rolls back by dropping the column. - add_budget_business_unit_id_index - Builds the
business_unit_idindex concurrently on PostgreSQL. Reversible: rolls back by dropping the index. - add_oauth2_authorize_requests_device_code_columns - Adds the nullable
device_code_hashanduser_code_hashcolumns tooauth2_authorize_requests, with a bounded lock wait on PostgreSQL. Reversible: only short-lived sign-in state is dropped. - add_oauth2_authorize_requests_device_code_indexes - Builds their unique indexes concurrently on PostgreSQL, so writes are never blocked. Reversible: rolls back by dropping the indexes.
- add_client_config_mcp_server_max_protocol_version_column - Adds
mcp_server_max_protocol_versiontoconfig_client. Reversible: without the column/mcpserves every version, as before. - agent_logs_add_caller_correlation_id_column - Adds
caller_correlation_idtoagent_logs. Non-reversible once any row records a value: the rollback refuses while one does. - logs_add_model_routing_suggestion_tables - Creates the log-store tables for phrase suggestions. Reversible: rolls back by dropping the tables.
- logs_add_model_routing_columns - Adds model router columns to
logs. Non-reversible once any log records a router: the router a request was routed by cannot be recaptured, so the rollback refuses while one does. - logs_tune_retention_autovacuum - Sets
autovacuum_vacuum_scale_factor=0.01andautovacuum_vacuum_threshold=10000on the Postgres retention tables. Reversible: rolls back by resetting the options.
All new migrations add tables, nullable columns or indexes and never rewrite existing rows, so they are safe for a rolling deploy.
🐙 Closed GitHub Issues
- #123 - Files API Support
- #3770 - Propagate x-bf-dim-* headers to all spans in a trace, not just the root HTTP span
- #4965 - [Bug]: Bedrock never activates fine-grained-tool-streaming - tool.eager_input_streaming is silently dropped
- #5272 - [Bug] Bedrock provider ignores Helm network_config.base_url (and AWS_ENDPOINT_URL_*), falls back to default public runtime URL
- #5354 - [Bug]: Anthropic streaming usage double-counts cache-read tokens (~2× prompt_tokens inflation)
- #5510 - [Bug]: Anthropic passthrough double-counts cache-read tokens for Kimi-compatible streams
- #5792 - [Bug]: Anthropic passthrough rejects bare (unprefixed) Claude Code model names with "could not auto resolve a provider"
- #6406 - Vertex: prompt cache mostly misses because
messagesis serialized beforesystem/tools - #6815 - [Feature]: forward x-opencode-session on opencode requests
- #7963 - [Feature]: Claude Code gateway sign-in (/login → Cloud gateway) for Bifrost
- #8073 - Custom model-parameters rows stop applying after the background model-parameters URL sync
- #8094 - [Bug]: Provider config hash is non-deterministic when maps have 2+ entries; API-added keys dropped on restart
- #8467 - [Bug]: ElevenLabs Scribe is blocked by wildcard virtual-key permissions despite appearing in Models
- #8976 - [Bug]: /bedrock/model/{id}/invoke drops tool_reference blocks inside tool_result, so deferred tools are never expanded
- #9276 - [Feature]: Add Kimchi as a first-class provider
- #9579 - [Bug]: bifrost_upstream_latency_seconds includes provider queue wait
- #9591 - [Bug]: /anthropic/v1/messages injects web_search_tool_result after a deferred server_tool_use, so the mixed-tool continuation 400s (tool_use without tool_result)
- #9721 - [Bug]: Anthropic→OpenAI conversion drops role on messages containing mcp_tool_use/mcp_tool_result blocks, causing 400 from all OpenAI-compatible providers
- #9731 - Mistral: every reasoning_effort is rewritten to "high" (Mistral now accepts low…max per model)
- #9788 - [Bug]: ElevenLabs speech with_timestamps returns null char_start_times_ms / char_end_times_ms
- #9846 - [Feature]: Add option to show stop reason as a column.
- #10026 - [Bug]: prompt_cache auto_inject checks r.Model, not the canonical model
- #10132 - [Bug]: Updating the budget of a config.json virtual key fails with "invalid reset duration format: "
Installation
Docker
docker run -p 8080:8080 maximhq/bifrost:v2.6.0Binary Download
npx @maximhq/bifrost --transport-version v2.6.0Docker Images
maximhq/bifrost:v2.6.0- This specific versionmaximhq/bifrost:latest- Latest version (updated with this release)
This release was automatically created with dependencies: core v1.16.0, framework v1.13.0. All plugins have been validated and updated.