maximhq/bifrost core/v1.8.0

Core v1.8.0

on GitHub

Core Release v1.8.0

• fix: forward OpenCode Responses requests directly to /v1/responses @mohammadrezwankhan
• feat: add the VideoEdit operation with BifrostVideoEditRequest, VideoEditInput and VideoEditParameters for prompt-driven edits, upscaling and background removal on an existing video supplied as bytes, a URL or a provider video ID; implemented for OpenAI (/v1/videos/edits) and Runware (videoInference, upscale, removeBackground), with the model optional when the source is a video ID and the prompt optional for asset-driven task types (#6270)
• feat: batch accounting: MergeBifrostLLMUsage promoted to schemas, Endpoint on BifrostBatchResultsResponse, BatchResultItem.Failed(), BatchRequestCountsFromResults, BatchRequestCounts.IsZero(), raw-JSON Gemini batch result parsing and custom_id validation in ConvertRequestsToJSONL; the settlement engine (AccountBatchResults with runner-ID ownership fencing, idempotent aggregate log writes and governance reporting) and a sweeper that polls due jobs with capped, jittered backoff; aggregate log entries carry a bifrost/<version> user agent via BifrostContextKeyRuntimeVersion (thanks @SahilChoudhary22!) (#5291, #5294, #6474)
• feat: Claude-on-Vertex batch support: ToVertexBatchCreateRequest resolves Anthropic families to publishers/anthropic/models/..., vertexConvertRequestsToJSONL emits Claude-on-Vertex instances, custom_id round-trips through batchResultsByKey, and GeminiBatchGenerateContentRequest keeps tools, toolConfig, cachedContent, labels and the display name (#5368)
• feat: add HTTPTransportPreAuthHook to the HTTPTransportPlugin interface, a phase that runs before transport authentication; HTTPTransportPreHook now runs after it (#6375)
  
  Breaking for plugin authors: Go plugins implementing HTTPTransportPlugin must add HTTPTransportPreAuthHook (.so plugins that predate it are skipped for that phase), and any plugin that injected a credential such as x-bf-vk or Authorization from HTTPTransportPreHook must move that work to HTTPTransportPreAuthHook, since the pre-hook no longer runs before auth.
  
• feat: add semaphore_size and inject_timeout to PluginConfig so observability Inject calls are context-bounded per plugin (#6341)
• feat: Runware provider expansion: chat completions, streaming and Responses through its OpenAI-compatible /v1/chat/completions endpoint (Responses muxed via ToChatRequest()), ListModels sweeping the curated modelSearch catalog with the AIR as the model ID, image upscale via /v1/images/edits (type=upscale) and image-to-3D via /v1/videos (type=3d), a shared settings extra-param coercion for multipart and JSON callers, prompt-optional asset-driven operations, and input handling for edit, upscale and video task shapes (#6260, #6372, #6208)
• feat: OpenAI ultrafast service tier: BifrostServiceTierUltrafast, capability-gated forwarding via serviceTierForModel on chat, Responses and compaction, and ultrafast preserved through WithDefaults (#6396)
• feat: JSON bodies on /v1/images/edits: ImageInput accepts a bare string or { "url", "image" }, typed extra params reach providers with their real types, and images is a known field (#6418)
• feat: EmbeddingData.EncodingFormat with typed int8, uint8, binary, ubinary and base64 vectors; Bedrock Titan V2 embeddingTypes and Cohere embedding_types on Converse, native invoke and LangChain BedrockEmbeddings compatibility (#6381)
• feat: rerank: RerankDocument.Data for structured documents, RerankResult.ID, RerankParameters.NextToken, ReturnDocuments forwarded to Cohere and Vertex, ToCohereError for Cohere-shaped errors, /genai/v1/rank served cross-provider via x-model-provider, cross-provider responses converted back to the caller's wire shape with ToBedrockRerankResponse, ToCohereRerankResponse and ToVertexRankResponse, and rerank cost accounting for Bedrock and Cohere (#6301, #6328)
• feat: datasheet-backed compatibility flows: Anthropic, Bedrock, Cohere and Gemini request shaping (adaptive-only thinking, adaptive thinking, native effort, disable-reasoning, mid-conversation system turns, computer-use and text-editor tool generations, default max output tokens, tool validation, thinking-budget zeroing) is resolved through schemas.ResolveModelCaps instead of hardcoded model-name checks (#6281, #6492)
• feat: Gemini 3 per-model thinkingLevel support table (geminiThinkingLevelSupport) with clampThinkingLevel snapping requested levels to the nearest rung (ties break upward) and lowestThinkingLevel for reasoning_effort: "none", so setThinkingBudgetZeroIfSupported sets the floor level on Gemini 3+ instead of zeroing thinkingBudget (#6280)
• feat: Bifrost overhead latency accounting: upstream_latency and overhead_latency on BifrostResponseExtraFields (PopulateOverheadLatency, BifrostContextKeyRequestStartTime, populateLatencyExtraFields so logging plugins see both at hook time); per-phase overhead spans across the request pipeline (queue-wait, attribute-population, convertor, request-marshal, response-parse, handle-setup, pipeline-pre, pipeline-post, worker-setup, key-pool, Bedrock request-sign and credentials-fetch, response-finalize) with StampWorkerHandoff on ChannelMessage.sentAt; lock-free stream overhead accumulators for per-chunk parse, conversion and backpressure installed via ResetStreamOverhead, StampStreamTransport for the outbound marshal and client-write time, and defaultSSEDataReader.ReadDataLine attributing socket reads to upstream; and IsOverheadBreakdownSpan, WithoutOverheadBreakdownSpans and the OverheadSpanConsumer interface so breakdown spans stay out of connectors that do not opt in (#5533, #6388, #6389, #6433, #6470, #6495)
• feat: input/output/additional cost split (BifrostCost) on inference usages, extended to speech, transcription and OCR usages
• feat: Notification, NotificationInput, NotificationSeverity, NotificationAudience and the NotificationPublisher function type for the dashboard notification center (#6207)
• feat: BifrostContextKeySkipModelCheck short-circuits the virtual key model allowlist for evaluate-only requests such as /inspect while keeping every other governance rule (#6479)
• feat: HarnessSessionHeaders and MaxSessionIDLength so Claude Code, Codex CLI and OpenCode session headers can fall back into the session ID (#6333)
• feat: RedactSensitiveHeaders, with IsSensitiveHeader extended to Cloudflare Access (cf-access-*), AWS ALB OIDC (x-amzn-oidc-*) and generic jwt/assertion headers (#6371)
• feat: ResponsesResponseError.Type and a shared Responses stream-error normalizer so terminal error/response.failed events inside an HTTP 200 Azure SSE stream surface as errors with their nested type, code and message on both create-stream and retrieve-stream paths (thanks @dani29!) (#6302)
• feat: ServiceTier on StreamAccumulatorResult, with Anthropic's service_tier from message_start latched onto the final chunk of chat and Responses streams (#6236)
• feat: OpenRouter speech and transcription through the OpenAI-compatible audio handlers instead of returning unsupported-operation errors (#5734)
• fix: preserve max_tokens for OpenCode-compatible chat endpoints (thanks @Alex-wangyang!) (#6458)
• fix: HuggingFace chat streaming completed with zero tokens and therefore zero cost while non-streaming calls on the same models priced correctly, for two reasons: HuggingFace was listed as a provider that omits the [DONE] marker (it sends one), which made the shared OpenAI streaming loop break on the first finish_reason and discard the trailing usage-only chunk that several router inference providers emit; and stream_options.include_usage never reached the router because the shared streaming handler returns early when a provider supplies a custom request converter. Both are corrected, and an explicit stream_options from the caller still wins (thanks @elliottrabac!) (#6478)
• fix: preserve the caller's JSON Schema key order for structured outputs - ChatParameters.UnmarshalJSON holds response_format as raw bytes and the new ChatResponseFormat reader splices them verbatim into OpenAI, Anthropic, Bedrock, Gemini (unless a union type array needs normalizing) and Cohere requests, and ResponsesTextConfigFormatJSONSchema re-encodes in the decoded key sequence, because OpenAI structured outputs generate fields in the declared order and a re-sorted schema silently changes model behavior (#6235)
• fix: open reasoning stream items that carry both an encrypted payload and a visible summary as thinking blocks instead of redacted_thinking on the Anthropic egress, with isReasoningItem and reasoningPayloadAndSummary shared by the native-reasoning and misclassified-function-call branches (#6292)
• fix: replayed thinking blocks through the Anthropic ingress with a bedrock/ model prefix: content-less tool_result blocks are kept, interleaved text/tool-use/thinking order is preserved by the grouped converter, incomplete maps to error on Converse toolResult.status, and buffered reasoning is consumed by the item that owns it, so multi-turn tool use no longer wedges (#6346)
• fix: Gemini/Vertex HTTP 400s on Claude Code traffic routed through /anthropic/v1/messages: trailing assistant prefills are trimmed on both the Responses and chat paths, mid-conversation system messages are inlined in place instead of hoisted into systemInstruction, and AnthropicMessageResponse gains ExtraFields (#6363)
• fix: alias Bedrock toolUseId/toolResultId values longer than 64 characters or outside [a-zA-Z0-9_.:-] (such as Gemini thought-signature IDs) with a deterministic hash prefix, applied identically on tool_use and tool_result in both the Responses and chat converters (#6300)
• fix: route Grok (xai.) models through the openai/v1 Mantle path on Bedrock and Bedrock Mantle, since they have no Converse equivalent (#6022)
• fix: register Bedrock Mantle in ProviderSendsDoneMarker so its streams end after finish_reason instead of waiting for a [DONE] marker (#6021)
• fix: include OpenRouter embedding models from /v1/embeddings/models in ListModels, merged case-insensitively and best-effort (#6264)
• fix: force reasoning.effort to "none" for models that reason by default but do not support reasoning with tool calls when they advertise supports_none_reasoning_effort, instead of dropping reasoning outright (#6293)
• fix: backfill upscale output resolution on Replicate from the target/factor params and metrics.resolution_target bands so resolution-tiered pricing bills the real output size (#6083)
• fix: filter forwarded Accept-Encoding to the codecs CheckAndDecodeBody can decode (gzip, x-gzip, deflate, br, zstd, identity), restrict streaming endpoints to gzip/identity via SetPassthroughHeadersForStreaming, and decode chained content encodings in reverse order (#6360)
• fix: tool_sync_interval handling: negative values are rejected (the "disable sync" semantic is gone now that the connection checker drives discovery and liveness together), ResolveToolSyncInterval follows the global setting for sub-second values, a fresh per-call checker starts on EnableClient and on a sticky-to-per-call flip, and MCPManager.UpdateToolSyncInterval, ConnectionCheckerManager.SetGlobalInterval/ApplyGlobalInterval/RetimeClient and ClientConnectionChecker.SetHealthyInterval hot-reload the global cadence and re-time running checkers in place; GetMCPConfig carries the stored global interval (#6502)
• fix: SetClientTools and UpdateClientCredentials replace the MCP tool map instead of maps.Copy-merging into it, so a tool removed upstream is evicted from memory once the database has dropped it (#6484)
• fix: per-call shared-credential MCP clients (oauth, headers, none) refresh tools synchronously on credential update instead of returning ErrMCPReconnectNotApplicable; disabled per-call clients and per-user auth types keep the sentinel (#6483)
• fix: park a failed EnableClient dial at Disabled instead of Unstable, add ErrMCPEnableConnectFailed so callers do not roll back the persisted disabled flag, and guard isEnableable on both state and config so the admin can retry (#6431)
• fix: output_item.done replaces server-side tool item shells (web_search_call, code_interpreter_call, image_generation_call) in the Responses streaming accumulator so their full payload survives (#6475)
• feat: send s3:// image and document references to Bedrock Converse as the s3Location source member instead of downloading the bytes and re-uploading them - Converse resolves the object itself, which skips a round trip and the 25 MiB inline cap entirely. Image format is derived from the object extension since nothing is fetched and there is no Content-Type to read, and an extension-less object is rejected up front rather than producing an opaque 400 (#6239)
• feat: resolve Vertex URL sources per model family rather than inlining everything - a gs:// URI is now forwarded to Gemini/Gemma as fileData.fileUri (the documented form, resolved under the caller's own project IAM, and the only thing that keeps multi-hundred-MB video inputs viable) and read from Cloud Storage with the request key's own Google credentials for Claude-on-Vertex, which accepts base64 sources only. http(s) is still always fetched: forwarding one was measured against the harness and Vertex rejected every endpoint shape with URL_REJECTED-REJECTED_FC_TOO_MANY_PENDING (#6239)
• fix: always emit a Gemini candidate carrying its finish reason on generateContent, even when nothing visible was generated - a thinking model that spends its whole output budget before emitting a token is a successful 200 with an empty answer, but Candidates is omitempty, so dropping that candidate produced a body with no candidates key at all and left a lone usageMetadata object that every Gemini-shaped client dereferences blind (#6239)
• fix: drop payload-free Gemini parts when assembling a candidate - every Part field is omitempty, so such a part marshals to exactly {}; the harness observed one on the wire when a transcription request for an unintelligible tone came back as parts:[{}], where it is noise a client will try to read and it masks the contentless case by making the parts slice look non-empty (#6239)
• fix: accept a bare model identifier on Bedrock rerank by synthesizing the foundation-model ARN from the resolved region - Rerank is the one Bedrock surface that names its model by ARN rather than by bare ID, so all three rerank drop-ins in the provider harness 400'd on amazon.rerank-v1:0. The partition is derived from the region (aws, aws-cn, aws-us-gov) so GovCloud and China build a correct ARN, and an explicit ARN still passes through untouched (#6239)
• fix: stop stripping file_url from OpenAI-shaped chat file blocks on marshal - dropping it produced {"type":"file","file":{}} and an upstream complaint about a missing file_id, which hid the fact that a source had been discarded. Providers that cannot take a URL now say so by name, and any OpenAI-compatible endpoint that does accept one keeps working without a Bifrost change (#6239)
• fix: leave URL content sources Bifrost cannot download in place on the OpenAI and native-Anthropic paths instead of failing the request - only http(s) is fetched, and whether a gs://, s3:// or scheme-less reference is usable is the provider's call, so the source now travels as {"type":"url"} and the platform answers for itself (#6239)
• perf: JSON serialization on the hot path: shared MCP tools cache their serialized bytes on ChatTool (EnsureSerialized, precomputeToolSerialization) so catalog tools are marshalled once, and OrderedMap.MarshalJSON writes compact JSON directly into a buffer with an inline HTML-safe string escaper instead of re-routing every nested map through MarshalSorted, pinned by a byte-identity fuzz harness (#6241, #6242)
• perf: allocation and tracing reductions on the request path: StartSpanID and SpanFromHandle on the tracer with Span.SetAttributes for bulk writes, a resolved-once attribute block in executeRequestWithRetries, reusable worker delivery timers, Span.Reset keeping map capacity, reservedKeys as a set, pre-sized userValues, logging context reads deferred to the final chunk, no redundant fmt.Sprintf in logger calls, cached plugin span names, compact JSON request bodies, math/rand/v2 in GetRandomString, and a HasPluginLogs guard before draining plugin logs (#5657, #5956, #5957, #6211)
• chore: remove the legacy gen_ai.*-namespaced Bifrost-internal attribute constants, AttrPromptTokens/AttrCompletionTokens, AttrLegacyRetryCount and the nanosecond AttrTimeToFirstToken in favor of the canonical bifrost.* keys (#6403)
• chore: build with Go 1.26.6 (#6269)

Installation

go get github.com/maximhq/bifrost/core@v1.8.0

This release was automatically created from version file: core/version