smg-project/smg v1.14.0-rc.1 on GitHub

SMG v1.14.0-rc.1

1061 commits since v1.13.0 on release/1.14 (the merged pull requests, by type):

Features

feat(gateway): Symphony is the default parser; --parser legacy and the X-SMG-Parser header keep the older parsers (#4677)
feat(gateway): TokenSpeed engines serve every role on the direct ZMQ lane (#4649)
feat(symphony): Llama 3.1 and 3.3 Instruct read with the json table under their own ids (#4655)
feat(engine_zmq_adapter): a TokenSpeed engine's prompt logprobs, top-k candidates and abort status are read by the column names it declares (#4630)
feat(gateway): media over the SGLang direct-ZMQ wire is assembled as the gRPC path assembles it, and the ZMQ lane runs the multimodal module (#4580)
feat(tau2): add shared budget reservations (#4217)
feat(servicer): a Rust servicer for the TensorRT-LLM gRPC contract, with its binding and launcher (#4494)
feat(mock-worker): select repeatable output from prompt tokens (#4451)
feat(servicer): the TokenSpeed servicer's control RPCs ride the wire's control call when the scheduler declares its reply (#4453)
feat(servicer): the Rust SGLang servicer serves the EPD language side (#4413)
feat(servicer): the Rust SGLang servicer carries multimodal inputs to the scheduler (#4405)
feat(health): the readiness body names each served model with its readiness and a reason (#4412)
feat(servicer): the Rust SGLang servicer serves prefill/decode disaggregation (#4411)
feat(metrics): first-token and per-output-token latency histograms with their own bucket edges (#4410)
feat(servicer): the Rust TokenSpeed servicer serves multimodal items of the Qwen3.5 MRoPE lineage (#4415)
feat(servicer): the Rust TokenSpeed servicer serves prefill/decode disaggregation (#4408)
feat(observability): the media pipeline's stages, items and fetches are series on the router and the worker side (#4377)
feat(metrics): the router error counter carries the reason behind each error class (#4345)
feat(helm): router.discovery.provider selects the worker discovery provider, and the Role exists only for what reads the Kubernetes API (#4391)
feat(metrics): one per-worker routing-state series names why a worker is out of rotation (#4353)
feat(metrics): the cached prompt tokens are a token type of the tokens counter and every cache-aware decision observes its match ratio (#4360)
feat(metrics): the Symphony failure classes of a served model's table exist at zero from its first registration (#4348)
feat(servicer): StartProfile and StopProfile on the vLLM wire, both servicers (#4369)
feat(metrics): the KV capacity behind the engine's usage ratio and the block size the index routes by are gauges (#4357)
feat(helm): a dashboard over the emitted families and a PrometheusRule with ten serving alerts (#4343)
feat(helm): a gRPC worker can serve the servicer's Prometheus metrics from the engine pod (#4341)
feat(mock-worker): the mock gRPC worker hosts the standard grpc.health.v1 service (#4338)
feat(gateway): configure PD readiness capacity thresholds (#4331)
feat(multimodal): expose media fetch timeouts (#4330)
feat(servicer): a /metrics endpoint with request, token and latency metrics on every wire (#4149)
feat(discovery): a Consul provider that discovers a service's passing instances (#4137)
feat(glm): add native GLM-5.3 rendering and align Flash media processing (#4036)
feat(multimodal): Nemotron-H Omni images shrink to a request's token budget, as the engine's tiler shrinks them (#4120)
feat(multimodal): make the Hugging Face Hub download an optional hf-hub feature (#4119)
feat(cache_aware): opt-in spill pricing for long-session prefill pools (#4089)
feat(e2e): a mock engine row in the harness, with the answers stated by the test (#4070)
feat(mock-worker): scripted answers on every wire, from a keyed file or the admin API (#4066)
feat(discovery): a Slurm provider that discovers the nodes of running jobs (#4013)
feat(symphony): the DeepSeek R1 line reads with the deepseek_v3 table by model id (#3780)
feat(gateway): a request configured for Symphony that the legacy parsers read is counted by reason (#4007)
feat(helm): the Oracle credentials reach the router from a Secret, not in the pod arguments (#3631)
feat(symphony): a table for the Mistral list of calls, the spelling before the v11 tokenizer (#3770)
feat(symphony): Llama 3.2 reads with the json table, which drops the python tag the model writes (#3769)
feat(symphony): the Qwen2.5 Instruct and Coder-Instruct checkpoints read with the qwen2.5 table by id (#3773)
feat(observability): expose in-flight HTTP request snapshots (#3984)
feat(multimodal): an opt-in TurboJPEG policy refuses to start without libjpeg-turbo instead of decoding JPEGs with the fallback decoder (#3707)
feat(symphony): a table for Llama 4's Python list of calls, bare in the turn and every value quoted (#3768)
feat(symphony): tables for DeepSeek V3 and V3.1, the two spellings the older deepseek parsers read (#3735)
feat(symphony): a table for DeepSeek V3.2 reads its DSML dialect, the one the older deepseek32 parser read (#3759)
feat(symphony): every legacy parser name answers to its table or is listed as unanswered, with the reason (#3784)
feat(symphony): a table for the Kimi K2 line reads its block of calls and the thought its prompt opens (#3772)
feat(gateway): a Symphony table's malformed runs, suppressed calls and refused outputs are counted per table and written as one sampled line (#3747)
feat(symphony): the MiMo V2.6 checkpoints read with the qwen3_tagged_direct table, as V2.5 does (#3779)
feat(symphony): Inkling-NVFP4 joins the Inkling group, whose three checkpoints ship one template (#3778)
feat(symphony): Muse Glimmer 30B joins the Muse Spark group and Phi-4 multimodal the plain one (#3737)
feat(helm): the history store URL reaches the router from a Secret, not in the pod arguments (#3608)
feat(rate_limit): the usage settlement carries the engine's cached and reasoning counts (#3611)
feat(symphony): a table may bound the runs of whitespace in a JSON answer (#3728)
feat(symphony): the v1 Nemotron Super and Ultra checkpoints read with the llama_nemotron table (#3777)
feat(gateway): a bound on the HTTP listener's open connections, --max-connections (#3588)
feat(cli): --bucket-adjust-interval-secs and the worker mTLS flags reach the Rust CLI (#3570)
feat(gateway): worker-side media processing forwards every family the gateway's pipeline serves to workers running it (#3652)
feat(gateway): the access log carries the client address, tenant, key fingerprint, user agent, byte counts and routed worker on request (#3598)
feat(gateway): an admin listener for the control-plane routes, --admin-port and --admin-host (#3597)
feat(gateway): the serving keys and the store URLs read from the environment or a file, and no secret is echoed (#3589)
feat(multimodal): LLaVA, Qwen3-Omni, Qwen3-ASR and DeepSeek V4.1 opt their anchors in for worker-side expansion (#3634)
feat(symphony): the Muse Spark table answers to the name muse, in the gateway's name checks and the CLI's choices (#3537)
feat(beam): add day-0 Reflection's Beam support (#3538)
feat(multimodal): Muse Spark Open images, video and audio (#3526)
feat(grpc_servicer): the TokenSpeed servicer advertises audio for a checkpoint whose config declares a waveform tower (#3521)
feat(symphony): the Muse Spark table (#3519)
feat(router): forward x-override-priority to HTTP and PD workers (#3510)
feat(symphony): a table for the DSML spelling before V4.1's, for DeepSeek V4-Flash (#3518)

Fixes

fix(zmq): a prefill- or encode-registered direct-lane worker needs an engine that declares its bootstrap address (#4708)
fix(servicer): the launchers advertise the structural-tag vocabulary of the engine's xgrammar (#4691)
fix(zmq): an encode-registered direct-lane worker needs an engine that lists the encode request (#4698)
fix(rl): a direct-lane worker is controlled over its engine connection only when its handshake declares the control reply (#4699)
fix(gateway): every log line and failure message that names a worker writes its address masked (#4591)
fix(gateway): the routed-worker header and the request-sent events show a worker URL with its userinfo masked (#4579)
fix(gateway): every iteration of a Responses request takes the tenant rate-limit reservation a chat request takes (#4449)
fix(gateway): a request with tool_choice none still renders its tools, as the engine's own server does (#4129)
fix(symphony): the structural tag in the older vocabulary for a worker that advertises it (#4688)
fix(e2e): recognize nixl_rocm in PD transport checks (#4671)
fix(symphony): the blank line before the calls block is the block's opener on the DeepSeek V4 and V3.2 tables, as on V4.1's (#4073)
fix(gateway): chat logprobs cover every generated token on the gRPC path, as the engine's own server returns them (#4634)
fix(gateway): the request deadline on gRPC worker calls is opt-in (--grpc-request-timeout-secs, 0 = no bound by default) (#4581)
fix(liveness): a busy PD prefill worker's long prompts no longer time out as silent (#4620)
fix(gateway): make shutdown propagation window configurable (#4631)
fix(external-router): log lines and the health answer show a worker URL with its credentials masked (#4603)
fix(servicer): the Rust vLLM servicer withholds the KV-transfer rejection notice wherever a Mooncake connector runs, read off the raw config (#4601)
fix(config): --dp-minimum-tokens-scheduler together with --dp-aware logs a start-up warning, and the help names its scope (#4595)
fix(tokenizer): the native encoders render a developer message as the system message the engine hands them (#4598)
fix(health): a demoted worker returns on consecutive passing probes, not consecutive intervals (#4577)
fix(gateway): a client that leaves a stalled Harmony Responses stream aborts the engine request at once (#3720)
fix(servicer): the PD decode side withholds the KV-transfer rejection notice where a Mooncake connector runs (#4281)
fix(worker): the rank workers of one data-parallel engine share one load poll per tick (#4583)
fix(mcp): preserve engine HTTP requirements in CI (#4351)
fix(servicer): the PD decode side's pre-admission notice stops once the engine owns the request (#4260)
fix(gateway): an explicit thinking.disabled wins over reasoning_effort (#4284)
fix(tokenizer): the opt-in v1 pipeline cuts a Metaspace tokenizer's added tokens before the word marker is prepended (#4574)
fix(gateway): a worker URL's userinfo is masked whole in every rendering, the start-up lines and the configuration refusals included (#4572)
fix(engine): the engine image sets UCX's TCP segment size so a prefill/decode pair over NIXL across hosts pulls KV at line rate (#4276)
fix(loads): preserve native DP rank attribution (#2429)
fix(e2e): the harness's dependency list names pyyaml, which its worker module imports (#4569)
fix(gateway): Messages, completions and harmony streams anchor PD TTFT at the prefill dispatch (#4497)
fix(gateway): a client that leaves a stalled Responses stream aborts the engine request at once (#3706)
fix(gateway): --request-timeout-secs bounds a request to a gRPC worker from its dispatch to its last message (#4001)
fix(servicer): a TokenSpeed encode worker refuses FlushCache, StartProfile and StopProfile by name (#4507)
fix(worker): a data-parallel rank worker indexes its own rank's KV events (#4499)
fix(chat): the request's add_generation_prompt is read, so false renders the conversation without a generation header (#4280)
fix(gateway): a client that leaves a stalled stream aborts the engine request at once (#3632)
fix(gateway): a gRPC worker whose model info names the model as model_id only gets a tokenizer job (#4493)
fix(worker): a data-parallel rank worker's HTTP load probe goes to its engine's address (#4486)
fix(servicer): GetServerInfo and the zero-filled loads count the connected ranks (#4485)
fix(python): declare the protobuf floor the generated proto modules check at import (#4484)
fix(tests): scope the shared-memory path import to Linux (#4450)
fix(servicer): the negative video frame count refusal names its grid row (#4474)
fix(servicer): a warm-up prompt render that raises is a WARNING with its traceback (#4472)
fix(servicer): a TokenSpeed engine abort ends the stream with the engine's status and words (#4444)
fix(gateway): the response reader's panic guard holds the reading's future once (#4470)
fix(servicer): a negative video frame count is refused by name, not as an overflow (#4466)
fix(servicer): the Rust SGLang servicer serves a prefill or decode leg of more than one scheduler rank (#4460)
fix(gateway): a parser panic ends its own request with an error frame instead of a cut stream (#4135)
fix(metrics): a KV event replay request counts no missed batches (#4440)
fix(servicer): the planned warm-up counts a request the engine ended with an abort as failed (#4434)
fix(servicer): the Rust SGLang servicer places a request on the data-parallel rank the Router pins (#4414)
fix(servicer): the Rust SGLang servicer reads an unset top_p and top_k as their neutral values (#4459)
fix(router): the HTTP router's worker-address error log fields are written log-safe (#4462)
fix(metrics): the registration step publishes a Symphony table's failure classes once the worker is in the registry (#4382)
fix(servicer): the SGLang language-only worker runs the planned warm-up before it reports SERVING (#4426)
fix(servicer): the TokenSpeed wire pins the seed of a greedy request as the engine does (#4420)
fix(servicer): the planned warm-up stops short of what the engine's scheduler refuses, and a one-process engine plans the ladder alone (#4438)
fix(external_router): the Messages tool loop counts an upstream error status and a broken stream (#4441)
fix(servicer): the SGLang Python servicer reads unset sampling scalars as their neutral values (#4416)
fix(external_router): the Messages route counts a send that fails below HTTP (#4429)
fix(gateway): the PD count-tokens route logs the worker and endpoint of a call that failed below HTTP (#4387)
fix(servicer): SGLang hidden states are served on the Rust servicer; the Python servicer refuses them by name (#4407)
fix(metrics): a collected answer records its token and generation families as a streamed one does (#4399)
fix(kv_index): the string tree's snapshot merge compares a remote edge that continues below a local child against that child's own children (#4394)
fix(worker): the worker API refuses the smg.ai/discovery-* labels, and a PUT or a label PATCH of a discovered worker (#4392)
fix(tokenizer): a request without tools renders the processor's chat template where the checkpoint ships one that differs (#4373)
fix(tracing): the trace layer keeps its own filter, so --enable-trace exports the spans at every log level (#4379)
fix(gateway): the SSE re-chunker keeps a tool call's tail for its closer (#4365)
fix(tokenizer): tojson(indent=...) renders as Python does for a negative, zero, string or bool indent (#4356)
fix(gateway): a worker's address stays out of the client's answer for a call that failed below HTTP (#4368)
fix(servicer): a readiness hold admits only the warm-up's own requests (#4250)
fix(chat): a history tool call's arguments may be a JSON object, and the template gets the object the engines hand it (#4279)
fix(helm): a worker Deployment's rollout deadline leaves 600 s over its startup probe's budget for the node and the pull (#4364)
fix(gateway): a media host that does not deliver is a 502 under the media's code, the request's own URL a 400 (#4355)
fix(e2e): Worker.stop() releases the log handle and the ports of a process that already exited (#4262)
fix(symphony): the tagged-XML assembler refuses a call nested past the JSON reader's bound instead of growing its stack (#4354)
fix(helm): the router Deployment's progress deadline covers the engine's bring-up now that /readiness follows the engine (#4286)
fix(tokenizer): a chat template file reaches the renderer byte for byte: no escape translation of its backslash-n sequences (#4232)
fix(servicer): the load report's aggregate sums saturate (#4346)
fix(symphony): the Qwen-tagged and XML assemblers refuse a call's leftovers, never content (#4303)
fix(symphony): the cut token in the unnamed-invoke test is marked for the spell check (#4339)
fix(tests): the prefill-decode side-failure test reads the shared harness crate, not a module that is gone (#4340)
fix(e2e): a worker that dies during the gateway's readiness wait fails it at once with its death report (#4336)
fix(e2e): a gRPC worker is healthy on SERVING only; the engines' ports lie below the ephemeral range (#4325)
fix(servicer): the warm-up's end line carries why the engine's cache was not flushed (#4321)
fix(gateway): GET /v1/models lists every alias of a served model (#4289)
fix(gateway): a demoted worker returns to routing on a health read, not on its first contact (#4249)
fix(tokenizer): a chat template file is read as written, so the newline a template opens with reaches the prompt (#4212)
fix(gateway): a prefill or decode side that fails is named the same way by the HTTP and the gRPC prefill-decode paths (#4205)
fix(cache_aware): gate the miss path and ignore idle throughput gauges (#2810)
fix(symphony): a call the token limit cuts inside a string value's closing tag keeps the value and none of the tag (#4307)
fix(symphony): a keyed call the token limit cuts keeps the cut member in its arguments (#4223)
fix(e2e): an HTTP worker that dies during start-up fails the wait at once with the death report (#4261)
fix(gateway): a connection reset under a call is answered 502 like other transport failures, not 500 (#4309)
fix(servicer): the TokenSpeed warm-up also visits every prompt length up to the chunk size before SERVING (#4231)
fix(symphony): bytes a tagged call's syntax cannot read are refused, never content (#4291)
fix(kv-events): a clear resets the worker's block of record for its next cache incarnation (#4300)
fix(symphony): a Gemma call the token limit cuts stays open and keeps the cut member (#4186)
fix(tokenizer): the legacy-normalizer test takes only the space-to-marker replacement (#4301)
fix(symphony): a pythonic call the token limit cuts keeps the cut argument in its arguments (#4179)
fix(tokenizer): a Llama-class checkpoint encodes as its reference class, not as its legacy file pipeline (#4248)
fix(tokenizer): the string filters a chat template applies to a list or a dict start with Python's str, as {{ }} does (#4247)
fix(kv-events): a hybrid engine's prefix-tail entries are left out of the index, not read as parent-less stores (#4170)
fix(gateway): embeddings, classify and transcription take the tenant rate-limit reservation every other gRPC endpoint takes (#4202)
fix(multimodal): a GLM 5 family clip is sampled by the family's own loader rule, not the generic loader's (#4203)
fix(tokenizer): a chat_template shipped as a list of named templates is read: default for a plain request, tool_use when tools are declared (#4230)
fix(health): discovery's default removes only the failed workers it discovered (#4297)
fix(symphony): a call the token limit cuts inside a held value keeps the value in its arguments (#4167)
fix(tokenizer): an explicit null special token in tokenizer_config.json is no token, and a guess names only added special tokens (#4240)
fix(servicer): the Rust vLLM servicer withholds a PD decode side's rejection notice on the Mooncake connector (#4287)
fix(servicer): the TokenSpeed servicer maps an engine abort's status with the shared table (#4173)
fix(gateway): a tool-call-only assistant turn reaches the chat template as empty content, as the engine hands it (#4210)
fix(gateway): a Messages response refused for a tool_use input no value can carry is counted under its table (#4252)
fix(deps): upgrade tokio to 1.54.1 (#4296)
fix(tokenizer): a DeepSeek V4 architecture checkpoint that ships a chat template renders through it; the encoder port stands in only where there is none (#4236)
fix(response-template): \w takes the 12,860 word characters the regex module adds past Unicode 17 (#4227)
fix(symphony): a string value left open at the call's closing marker is closed there and said (#4207)
fix(engine_zmq_adapter): a stale ipc socket that vanishes between the stat and the unlink counts as removed (#4215)
fix(servicer): the KV-event relay joins the engine's publisher within a second of its bind, and the planned warm-up waits for the subscription (#4204)
fix(tokenizer): a list or a dict a chat template prints is written as Python's repr, the bytes the engine's prompt holds (#4220)
fix(data-connector): an item created again under a stored id is the stored item on every history backend (#4194)
fix(servicer): a request that reaches a closing connector is UNAVAILABLE, not INTERNAL (#4193)
fix(data-connector): the Redis history store's item and identifier indexes expire with the records they list (#4180)
fix(servicer): the KV-event relay keeps the learned cache groups across a clear, so its snapshot carries the prompt stored with the reset whole (#4178)
fix(gateway): an output whose decoder fails under Symphony is counted as the table's failed output (#4175)
fix(protocols): Kimi K3 and GLM-5.3 models served under an alias keep their series' rules under every name (#4169)
fix(multimodal): the video sampling rate is carried as a double from the config and the request to the frame picks (#4163)
fix(observability): close_logging waits for the writer threads past the guards' bounded moment (#4166)
fix(helm): worker images default to the repository the engine images are published under; README and examples on the current release tags (#4188)
fix(grpc_servicer): a recipe's capture-cap ceiling applies when it leaves --max-num-seqs to the engine; the README says what a ceiling buys per running window (#4158)
fix(protocols): a DeepSeek V4.1 model served under an alias keeps its reading of the reasoning effort under every name (#4164)
fix(symphony): a Llama 4 quoted value that ends in a backslash closes at its quote (#4168)
fix(servicer): a long prefill on a token-batch wire is not a dead engine, and the silence bound is configurable (#4068)
fix(gateway): a transcription answer is read under the same payload cap as every other worker body (#4143)
fix(engine): one prompt admission check on the three ZMQ wires (#4142)
fix(servicer): a MultiConnector is projected onto the child that moves the KV, a store or MoRI-IO child included (#4155)
fix(gateway): a stop the gateway matches itself aborts the engine request instead of draining it (#4040)
fix(grpc-client): a chat request's no_stop_trim reaches vLLM and TRT-LLM as it reaches the other engines (#3915)
fix(symphony): a string value the model leaves open at its own end of output is closed there and said (#4134)
fix(liveness): a PD decode side's first output counts as its prefill worker's answer (#4136)
fix(gateway): the console log is coloured on a terminal, not in a file or a pipe (#4130)
fix(servicer): the servicer reports SERVING only after a planned warm-up over every running-batch size and prefill length class the engine admits (#4132)
fix(engine): an engine's grammar abort is one explicit error the gateway does not retry, where the engine attributes it (#4112)
fix(zmq): a JSON-schema answer with thinking on lands in content over the gateway's own engine link (#4110)
fix(symphony): text where a call's syntax has structure keeps the model's whitespace between its words (#4125)
fix(symphony): a value held whole from a CDATA section keeps its bytes when stray text follows the closer (#4127)
fix(symphony): text released as content keeps the model's whitespace between its words (#4116)
fix(protocols): a DeepSeek V4 forced tool choice with thinking on is served instead of refused (#4109)
fix(tokenizer): a DeepSeek V4.1 request's medium reasoning effort renders the checkpoint's budget instead of a 400 (#4100)
fix(tool_parser): a surrogate pair escape decodes to one character (#4009)
fix(gateway): the worker HTTP client keeps its connect timeout below the request timeout (#4075)
fix(tokenspeed): forward PD rank pins and correct per-rank capacity (#4131)
fix(external-router): a streaming MCP tool call with non-JSON arguments is counted as an errored call, and both loops word the error alike (#4115)
fix(servicer): finish draining when admitted requests complete (#4117)
fix(symphony): prose after a bare call list keeps the model's whitespace between its words (#4123)
fix(servicer): the KV-events default follows the engine's own word and leaves TokenSpeed's publisher off for a model whose State groups span no tokens (#4107)
fix(zmq): log unexpected input monitor events (#4087)
fix(multimodal): a Qwen2-VL image anchors on the template's <|image_pad|> and expands to image_token_id (#4121)
fix(servicer): the start-up lines logged during the launcher's argument parse reach the engine log on a base that configures logging after the parse (#4096)
fix(symphony): a refused call block that follows the answer's text goes out as its bytes before the finish (#4090)
fix(zmq): bound queued payloads with upstream transport fixes (#4082)
fix(tokenizer): the DeepSeek V4.1 renderer writes the effort budgets of the checkpoint's encoder revision (#4067)
fix(vllm): reject oversized prompts before engine admission (#4076)
fix(symphony): a JSON answer's tag derives for the selected worker's start, so a pool of mixed starts gives each worker its own (#4071)
fix(servicer): a constraint the engine's grammar backend cannot enforce is refused before the engine on every servicer (#4069)
fix(tokenizer): a control-token spelling typed in a message stays text under the renderers that encode their own prompts (#4072)
fix(symphony): the blank line between a text preamble and the calls block is the block's opener on the DeepSeek V4.1 table, not content (#3982)
fix(protocols): accept tool_reference in tool results and unsigned thinking (#4028)
fix(gateway): a request timeout on an HTTP worker ends the request once instead of replaying it on another worker (#4064)
fix(gateway): a gRPC worker registers only once its engine reports healthy (#4065)
fix(zmq): a request over the gateway's ZMQ engine link ends at --request-timeout-secs when the engine goes silent (#4060)
fix(servicer): SGLang sampling params ride as the API prefix and the handshake checks the scheduler's field order (#4062)
fix(symphony): a Messages call whose arguments stop being JSON gets the prefix's value in the whole message as in the stream (#4059)
fix(tokenizer): a template that writes a history call's arguments as text gets them as the request sent them (#4058)
fix(symphony): a call to an undeclared function is counted as unknown_tool and its line names the function (#4056)
fix(placement): keep cache-aware PD prefill placement clear of the overload veto (#4024)
fix(liveness): give a PD prefill worker's new run of prompts its backlog's time to answer (#4015)
fix(symphony): a released refused call block goes out from its opener, with the bytes between two blocks (#4049)
fix(symphony): an answer of nothing but a refused call block goes out as its bytes before the finish (#4044)
fix(gateway): a template that takes content parts gets a message's media where the request put it (#4043)
fix(protocols): accept tool_choice auto without tools on responses (#4038)
fix(zmq): a tool schema the backend cannot enforce is refused inside a forced call's structural tag too (#4041)
fix(symphony): an answer of whitespace alone goes out as content and is counted as empty (#4033)
fix(symphony): an answer of whitespace alone goes out as content and is counted as empty (#4033)
fix(zmq): every structured request carries the backend structural tags compile on unless the engine's configuration pins another (#4030)
fix(protocols): accept client_metadata on Responses requests (#4032)
fix(zmq): include deferred requests in scheduler waiting loads (#4023)
fix(zmq): the startup handshake bound is a flag, an hour by default, and a handshake takes the READY of an engine an earlier one greeted (#4021)
fix(symphony): a JSON answer on an open thought is the schema alone where the engine does not say where it starts (#4019)
fix(symphony): the malformed-run counter labels every reason by the crate's own name; other only for a reason the build does not know (#4018)
fix(symphony): the inkling table starts the answer at the text marker written inside a thought, and the tag derivation tells a kind said again from a close (#4011)
fix(routing): ignore output throughput for prefill waits (#4017)
fix(symphony): a forced list with no closer ends its tag with its own bracket, and every table's tag is checked for one (#4016)
fix(external-router): a Responses stream stopped at max_tool_calls ends with response.completed and its record, as the non-streamed response does (#4010)
fix(gateway): a decode leg the engine fails before its first token probes the pair's bootstrap port, and names a dead one to the client, in the log and in the rendezvous family (#3913)
fix(grpc-client): a plain generate request's no_stop_trim reaches vLLM and TRT-LLM as it reaches the other engines (#3909)
fix(responses): a history window cut inside a turn begins at the next turn boundary (#3722)
fix(gateway): a worker's labels follow its engine when it comes into service (#3410)
fix(symphony): a call to a function the request did not declare is announced by an UnknownTool event of no bytes before its start, on every syntax (#4020)
fix(symphony): the inkling table reads a kind marker inside a body as the kind, so a JSON answer written behind an engine's reasoning gate comes back bare (#4006)
fix(symphony): the json table reads every object of a sequence written on blank lines as a call (#4012)
fix(responses): a tool result with no call before it is refused under the contract profiles, as an unknown call id is (#3592)
fix(symphony): the Qwen3, GLM 4.6, MiniCPM5 and dots3 tables carry their templates' newlines after the thought's close, so a forced call there does not begin with the call marker (#4008)
fix(placement): place a fully over-threshold PD pool by its policy, not by load (#4014)
fix(servicer): one table from an engine's HTTP status to the gRPC code, behind every engine (#3726)
fix(grpc_servicer): an engine's workers are tracked from its spawn, so an early exit leaves none behind (#3653)
fix(external-router): a provider's error body loses the operator's account identifiers on the realtime paths too, through one shared answer (#3930)
fix(symphony): a pythonic parameter without a value is reported and skipped, not written as an empty string (#3880)
fix(responses): a conversation longer than the history window replays its newest items (#3600)
fix(multimodal): worker-side media placement with no worker that processes media references is refused in preparation and holds readiness (#3584)
fix(symphony): a pythonic call the region's close cuts inside its parentheses says so and closes nothing (#3792)
fix(gateway): the engine's reported silence on a step-reporting wire is judged against the prefill its servicer reports in hand (#3981)
fix(routing): isolate pushed loads by virtual DP rank (#3983)
fix(symphony): a forced call on the DeepSeek V3.1 table is written by the table's own block and read back as a call (#3992)
fix(symphony): the fuzz crate names its events' lifetimes (#4005)
fix(symphony): the Kimi K2 table's test helper names its events' lifetime (#4004)
fix(gateway): a failed Responses stream ends the same way on both routers (#3997)
fix(symphony): a forced call on the Kimi K2 table is written by the table's own tag and read back as a call (#3990)
fix(servicer): the tokenizer archive is built off the event loop and once (#3989)
fix(golang): Close frees the stream's converter handle after a request that ended on its own (#3568)
fix(symphony): the scanner's buffers come back empty when a scan is reclaimed (#3996)
fix(servicer): the Python servicer turns the engine's KV event publisher on by default, as the Rust path does (#3558)
fix(multimodal): the cap on one video decode's work is measured in pixels, so a low-resolution clip hours long is no longer refused when the budget fits it (#3629)
fix(policies): the bucket's adjustment stays in the size domain (#3842)
fix(servicer): an SGLang health probe that fails at once is answered unhealthy at once, not at the 20 s deadline (#3988)
fix(gateway): the auth middleware's key map is shared, not cloned, on every request (#3904)
fix(symphony): a JSON answer on the inkling table comes back without a second text marker, and a marker the table did not read is counted (#3969)
fix(multimodal): a modality no worker of the model takes is refused before the media is fetched or decoded (#3979)
fix(symphony): a keyed call block the syntax cannot read is never content, and the failure is counted (#3977)
fix(tokenizer): a chat request's tools reach a DeepSeek V3.1 prompt as the block its template never writes (#3946)
fix(symphony): a forced call after the thought's close begins with the template's newline, not the call marker (#3978)
fix(multimodal): an item above the engine's encoder budget is refused before the engine, which would never schedule it (#3948)
fix(symphony): the tagged reader reads every function block of a call region, not the first alone (#3971)
fix(servicer): the load entries before the engine's first snapshot carry the servicer's own figures (#3963)
fix(gateway): a data-parallel rank whose event stream is quiet follows the fallback poll instead of its last pushed record (#3968)
fix(symphony): the json table reads a call behind Llama 3.1's python tag and the calls a semicolon separates (#3964)
fix(symphony): the inkling table's no-tools variant keeps the message framing out of the content (#3962)
fix(servicer): a data-parallel worker's load feed carries each rank's own hit rate and generation throughput (#3959)
fix(servicer): a data-parallel worker's KV events reach the gateway from every rank under every servicer (#3941)
fix(symphony): a pythonic call the stream's end cuts before its close says so before its end (#3791)
fix(rl): slime data-plane parity, TokenSpeed e2e lane and a trainer-side refit example (#2668)
fix(gateway): an origin that does not deliver a request's media in time is the request's 400, and a size refusal has its own code (#3947)
fix(config): the per-policy load_check_interval_secs goes, and a configuration that still sets it is refused by name (#3838)
fix(gateway): the completion API reads a backend's finish reason through one normalizer, and the engines' signals pass through (#3879)
fix(gateway): a gateway API key ends at the auth middleware and is never forwarded to an upstream as the caller's credential (#3918)
fix(symphony): a call whose arguments are not JSON keeps them as the model wrote them and says so once, not a call cut at the bad token (#3945)
fix(external-router): the realtime WebSocket relay strips the operator's account identifiers from a provider's error event (#3954)
fix(multimodal): 16-bit greyscale PNGs clip each sample to 255, as the engine's own loader does (#3344)
fix(ci): the mixed-line-ending hook leaves the recorded SSE streams as recorded (#3936)
fix(external-router): a provider's error body loses the operator's account identifiers on the chat and Messages paths too (#3919)
fix(gateway): the WASM middleware answers an unreadable body in the error envelope every other gateway error uses (#3912)
fix(gateway): the selection queue bound is read on the runtime's clock, and its later-run test on a paused one (#3867)
fix(gateway): a stopping gateway leaves the mesh before it drains (#3854)
fix(mesh): the status broadcasts dial with the node's TLS identity, so Suspected, Down and Leaving reach the members under mTLS (#3853)
fix(release): the version tool's git calls read the root they are given, and its files are read and written as UTF-8 (#3887)
fix(grpc-client): a plain generate request that names two structured constraints is refused on every engine alike (#3895)
fix(mcp): the inventory refresh keeps a server's tools when its listing fails, awaits outside the map's lock and sees a recovered server; the four tool listings share one function (#3905)
fix(auth): the JWKS provider bounds a response body as it reads it, for the key set and for the discovery document alike (#3901)
fix(symphony): a JSON call closes whole only with arguments a strict reader takes (#3790)
fix(symphony): a tool_use input that cannot carry the model's arguments is named, and the gateway refuses the message instead of {} (#3885)
fix(symphony): a JSON call the end or the close cuts right after its header says so before its end (#3876)
fix(policies): a draining prefill worker's size range goes to its neighbour at once (#3840)
fix(servicer): the TokenSpeed servicer reads /dev/shm tensors through the shared reader, which refuses a symlink (#3865)
fix(tests): building an application context applies its configuration's liveness budgets (#3850)
fix(grpc-client): a plain generate request's seed reaches vLLM and TRT-LLM, and its stop token ids TRT-LLM (#3846)
fix(mesh): the join peer is dialed until the membership table has another member, not once in the first round (#3847)
fix(symphony): the one-call-only rule keeps the bytes of the calls it drops, so the content ranges after them hold (#3844)
fix(mesh): the peer-connections gauge counts a peer's open streams, so an older stream's close does not zero a live one (#3851)
fix(mesh): the peer-connections gauge carries the dialer's name, never an empty peer label (#3839)
fix(bindings): the Go binding's chat responses carry the engine's weight version as system_fingerprint, none without one (#3832)
fix(policies): a prefill worker listed after the first split takes its share of the sizes (#3774)
fix(symphony): the Gemma syntax bounds its nesting where the JSON prefix parser does (#3760)
fix(gateway): Symphony's registry reads an unlabeled model it knows before a fleet-wide parser name (#3787)
fix(symphony): a Gemma member without a value is reported and skipped, not written as an empty string (#3788)
fix(deps): zip 9.0 closes the extraction advisories and the audit's ignore list records every remaining warning (#3805)
fix(symphony): a raw control character inside a JSON string reaches the client as its escape (#3755)
fix(symphony): a Gemma member without a key is reported and skipped, as the tagged readers leave one (#3781)
fix(policies): a removed prefill worker leaves the bucket policy's size partition, and removal reaches every policy (#3763)
fix(auth): OIDC discovery refuses a document whose issuer is not the configured issuer (#3806)
fix(symphony): a call the closing marker cuts inside a string value is left open and says so (#3776)
fix(gateway): a window the operator set by hand is the worker's only window, at registration and on a label update (#3446)
fix(symphony): a tool call's id counts from a nonce of the request's own, never from the call's index (#3705)
fix(symphony): a tagged call the stream's end cuts says so before its end (#3767)
fix(symphony): the chat adapter's test builds its malformed reason in the borrowed form (#3794)
fix(symphony): prose after a JSON value alone in the turn keeps its whitespace (#3643)
fix(bindings): the chart render test runs helm by name, and the Python lint job passes on main again (#3798)
fix(kv_index): the string tree's insert replay re-validates each edge before it attaches a tenant (#3785)
fix(router): every answer a router gives on its own is the JSON error object, under a code (#3743)
fix(servicer): a health probe closes the stream it leaves at its first output (#3753)
fix(servicer): the default KV-event publisher takes a free port, and the e2e job ends what it started (#3786)
fix(helm): the Grafana dashboard reads the families the gateway emits (#3756)
fix(kv_index): the insert replay of the fused walk re-validates each edge before it attaches a tenant (#3782)
fix(symphony): a call cut inside its arguments value says so before its end (#3754)
fix(liveness): an engine building a kernel for a prompt shape it has not run is not a wedge (#3730)
fix(grpc): a retry derives the table's auto tag for the start of the worker it takes (#3727)
fix(messages): thinking.display "omitted" returns the thinking blocks without their text (#3731)
fix(symphony): a turn the prompt leaves unopened starts inside the thought on the DeepSeek V4 table (#3711)
fix(worker): the mesh health hint and the drain step decide a worker's status under its mutation lock (#3614)
fix(servicer): the vLLM launcher switch refuses a launcher it cannot bind at the argument parse, before the engine is built (#3569)
fix(launcher): the six CLI flags the binding could not carry reach the launcher (#3565)
fix(messages): a system-role message sent after the first is a user turn for a renderer whose reference writes no turn for it (#3712)
fix(symphony): a table's tag takes the engine's grammar start into account where the prompt leaves the thought open (#3648)
fix(multimodal): the anchor parity checker follows the engine's current processing API (#3713)
fix(golang): a request that fails after its worker is selected gives the worker's load back (#3714)
fix(messages): thinking.display takes the documented set, updates included (#3694)
fix(workflow): a replacement that changes a worker's group refreshes the pool size of the group it leaves (#3704)
fix(symphony): a call whose object has no arguments member is a call with an empty object (#3702)
fix(install): the toolchain version lines in install-smg.sh run after PATH is exported and stop the build when a tool is missing (#3664)
fix(symphony): a response template's strip: false field keeps whitespace alone as written (#3697)
fix(grpc): Symphony keeps a table's calls wherever the legacy path runs a tool parser (#3572)
fix(symphony): the whitespace a content stay begins with is content only when text follows it (#3676)
fix(grpc): filter workers by advertised context capacity (#3670)
fix(grpc_servicer): a vLLM gRPC launch derives the CUDA-graph capture cap from the admission step (#3642)
fix(responses): the kinds the history does not replay are named one by one, and the left-out line costs a name (#3661)
fix(grpc_servicer): load_record lists poll_timeout_for among its exports (#3662)
fix(multimodal): an empty image batch is refused as an empty batch by every vision processor (#3659)
fix(cli): bound clap construction stack usage (#3663)
fix(tokenizer): enforce the L1 cache memory budget (#3657)
fix(grpc): a retry stays on workers that decide the constraint alike (#3638)
fix(glm): the media-reminder gate probes the part kinds a template is handed, and its doc and test name the family it checks (#3644)
fix(responses): the history a request continues keeps a stored turn's tool calls and their outputs (#3582)
fix(multimodal): a seek run of a long clip ends right after its last pick, so a missed pick falls back to the sequential decode at once (#3621)
fix(grpc_servicer): the host:port rendezvous constructor takes its flag in either spelling (#3630)
fix(servicer): the Rust vLLM servicer takes audio media references for the engine's own processors (#3633)
fix(grpc_servicer): the SGLang and TokenSpeed health Watch answer an unknown service without failing the stream (#3635)
fix(config): the listen IP must parse and the worker-thread count has a bound (#3594)
fix(grpc): a request's constraint is decided once, on the worker selected for it (#3624)
fix(data-connector): the Redis conditional writes store the output-item keys and the expiry with the record (#3580)
fix(glm): render media reminders without preprocessing (#3587)
fix(symphony): a word under way in a call's arguments is held until it is spelled or refused (#3593)
fix(grpc_servicer): the engine's --mm-processor-kwargs reach smg's own media pipeline on the Rust servicer (#3585)
fix(symphony): an optional call the stop closes has no tag again where more than one may come (#3619)
fix(symphony): a Gemma member that follows a value without a separator keeps its first byte (#3601)
fix(symphony): a value's held newline leaves before the start of a tag the stream cut (#3603)
fix(symphony): an ignored terminal inside a held call keeps its place among the bytes (#3591)
fix(multimodal): a long clip sampled sparsely is decoded by seeking to its picks, and a decode over its budget ends at once (#3578)
fix(servicer): the vLLM launcher switch binds the launcher run as main, on every supported release (#3577)
fix(gateway): the auto-constraint test asserts the engine's schema again for a registry table's JSON response_format (#3576)
fix(grpc_servicer): the worker-side media path raises a tiny image to the family's short-side floor, as the router's own pipeline does (#3575)
fix(grpc_servicer): the Python servicer refuses --mm-processor smg before the engine is built, not after the model loaded (#3571)
fix(gateway): a failed start-up flushes and shuts the tracer down before the error returns (#3566)
fix(discovery): the file provider's lost-watch retry fires under a short check interval (#3564)
fix(data-connector): the Redis conditional writes treat a record without an object field as open, as the SQL stores do (#3563)
fix(gateway): the gRPC chat gates read the tools a request declares anywhere, so a tool declared on a message keeps its call markers (#3562)
fix(gateway): a worker's model card follows its labels when they are updated (#3561)
fix(responses): a retried terminal persist whose earlier write landed links its items (#3560)
fix(gateway): the Harmony encoding's first load runs on a thread of its own (#3559)
fix(symphony): a call written as Python where a keyed block's name goes names no call (#3555)
fix(golang): the gRPC request path keeps the engine's special tokens for the tool parser (#3552)
fix(launcher): --drain-settle-secs and the CLI's second spellings reach the launcher, a bare --storage-context-headers entry is refused, and a test pins the flag parity (#3548)
fix(data_connector): the Oracle store checks the schema version before it creates any table (#3545)
fix(mock-worker): honor gRPC aborts and close the listener on shutdown (#3511)
fix(gateway): a JSON response_format leaves a tag that is the schema alone to the engine (#3543)
fix(multimodal): the exact audio resampler admits the odd rates real headers carry (#3542)
fix(symphony): the walks behind an invoke's number leaves have an allowance of their own, and size what is sent (#3533)
fix(tool_parser): the walks behind an invoke's number leaves have an allowance of their own (#3534)
fix(multimodal): follow-ups to the Muse Spark media review (#3532)
fix(gateway): a Harmony chat choice of header tokens only still opens with its role chunk at its Complete (#3531)
fix(gateway): a Harmony chat choice whose only engine output is its Complete opens with its role chunk and ends with its finish chunk (#3529)
fix(symphony): the XML call grammar's leaves are bounded in bytes (#3528)
fix(grpc_servicer): the MRoPE fact reads both rope sections of the checkpoint config (#3530)
fix(gateway): a JSON response_format takes the table's answer tag under Symphony for every table (#3527)
fix(engine_servicer): a vision grid is refused on the TokenSpeed ZMQ wire only where it feeds MRoPE positions (#3525)
fix(tool_parser): a MiniMax forced-call grammar's leaves are bounded in bytes, and a fragment carries only the definitions it reaches (#3524)
fix(gateway): a chat choice whose only engine output is its Complete opens with its role chunk (#3523)
fix(grpc_servicer): TokenSpeed's Generate keeps a choice's empty first Chunk and closes the engine iterator before its finished check (#3520)
fix(symphony): a shaped call's refused JSON value ends where it was refused, so its trailer can close the call (#3507)
fix(server): disable Nagle buffering for TLS streams (#3516)
fix(tool_parser): a tool schema's wide fan-out no longer unfolds the MiniMax forced-call grammar without bound (#3512)
fix(symphony): the XML call grammar is bounded by an element budget (#3515)

Performance

perf(multimodal): resize MiniMax M3 images once from source dimensions (#4681)
perf(gateway): a worker's constrained-decoding labels are parsed once at construction, not on every request (#4718)
perf(symphony): a streamed call's argument fragments borrow the delta (#4706)
perf(gateway): a Messages stream keeps one buffer for the content-block machine's events (#4703)
perf(worker): a worker's client lookup no longer carries the connect path's state on every call (#4683)
perf(gateway): the protected routes' middleware is one stack in every configuration (#4673)
perf(tenant): the copies of a request's route meta share its stamps instead of cloning them (#4672)
perf(tokenizer): the byte-level table knows at load which tokens are whole, so a step reads them without validation (#4323)
perf(engine): the decoded output batch of a ZMQ engine message is sized to its outputs, not to the allocator minimum (#4645)
perf(tokenizer): a prepared tool keeps its tojson text, so a repeated tool set is not serialized per request (#4597)
perf(registries): the per-request name lookups allocate nothing (#4623)
perf(servicer): the Rust SGLang servicer sends large multimodal tensors as auxiliary zmq frames the scheduler's plugin reads in place (#4582)
perf(symphony): a format's scanner markers are built once and shared, not copied per request (#4611)
perf(gateway): the HTTP layers are applied as one stack per group, not one boxed level per layer (#4596)
perf(gateway): the request deadline is applied to a stream's wrappers in place (#4570)
perf(http): the streaming relays keep one wait for the client across the chunks (#4552)
perf(gateway): the stalled-stream watch registers once per stream, not once per delta (#4550)
perf(metrics): the per-request records that still built a key per call go through kept handles (#4529)
perf(engine_zmq_adapter): a request's one sub-stream is polled without the merge (#4522)
perf(engine_zmq_client): the scheduler stats decode the load fields and skip the rest (#4492)
perf(engine_zmq_client): an engine output's rare payloads ride boxed (#4487)
perf(gateway): the ZMQ submit future is boxed at the backend dispatch, so a gRPC request's future is not sized by it (#4468)
perf(gateway): the template value of a request's tools is prepared once per distinct tool set (#4427)
perf(tokenizer): the stop decoder checks each token against sorted stop ids instead of two hash sets (#4282)
perf(tokenizer): the matcher's ASCII class tests read a membership table (#4337)
perf(tokenizer): the decoders append a token's text to the caller's buffer instead of returning a string per token (#4292)
perf(metrics): the per-worker gauges set on every request go through kept handles (#4264)
perf(stream): a streamed chat delta is written from the buffer it was read into, not copied into a chunk of its own (#4295)
perf(tokenizer): the pre-tokenizer matcher runs a compiled program with an explicit choice stack (#4245)
perf(tokenizer): the pre-tokenizer matcher skips the alternatives a character cannot begin and walks one-way nodes in place (#4228)
perf(tokenizer): the prefix cache learns the boundary after a hit, so a shared prefix behind a leading token is served (#4209)
perf(multimodal): the Pillow-parity resize runs one 32-bit RGB pass pair, and the DeepSeek V4.1 pad and patchify stop copying (#4219)
perf(gateway): the stream-or-buffer decision runs as a lean route layer, boxing a future only for a request it may stream (#4144)
perf(gateway): the legacy admission runs as a lean route layer, boxing a future only for a request that waits for a token (#4148)
perf(engine): large tensors ride as auxiliary zmq frames on every typed-tensor wire (#4145)
perf(gateway): the request body timeout layer runs lean, and the legacy concurrency layer is installed only with a rate limiter (#4114)
perf(gateway): the chat and generate dispatcher stages box their inner stage once, not twice (#4113)
perf(gateway): a request's context moves into its pipeline future once, not four times (#4111)
perf(gateway): the text log lines are written with the stock format's bytes and a third of its work (#4105)
perf(tokenizer): the chat template's tojson filter writes a value straight into its JSON text, not through a tree (#4103)
perf(gateway): the tenant route-meta layer runs lean, and the serving auth and wasm layers are installed only when they have work (#4101)
perf(symphony): a streamed delta is built without the role the stream drops (#4095)
perf(vllm): send large tensors as auxiliary zmq frames (#4093)
perf(protocols): a typed JSON body is parsed once; only a body that fails is parsed again to name the failing field (#4086)
perf(gateway): the Messages envelope, SSE keep-alive and trace-context layers cost nothing on requests they have nothing to do for (#4083)
perf(symphony): the reader keeps a shared table's no-calls and no-reasoning variants instead of rebuilding them per request (#4084)
perf(gateway): a UUID v7 takes its random bits from the thread's generator, not from a getrandom call each (#4080)
perf(symphony): a step's events fill the caller's buffer and borrow the delta's text (#3764)
perf(symphony): the scanner reads a delta after held bytes without allocating, seeks a marker's first byte a word at a time, and is not rebuilt at the end of a stream (#3987)
perf(protocol): a chat request's tools are shared with the response spec, not copied into it (#3925)
perf(symphony): a reader reads the request's tool schemas in place instead of indexing them first (#3916)
perf(gateway): where a Symphony table reads the response, no legacy reasoning parser is built to answer for it (#3874)
perf(policies): the registry keeps whether a policy wants the request text where policies change, not per request (#3910)
perf(gateway): the one-pass wire write is the default now that it beats the two passes at every image size (#3852)
perf(gateway): the body limit caps the request alone and the load guard rides in the body's own stream (#3627)
perf(symphony): a table is built once and shared by every request and engine that reads with it (#3721)
perf(symphony): a streamed delta is read in place, without the copies and allocations each delta made (#3723)
perf(router): the selection snapshot keeps no copy of the prompt text next to token ids (#3675)
perf(symphony): the decoder's special-token check is decided once per model, tokenizer and markers (#3719)
perf(router): Harmony detection reads a worker's model cards in place (#3674)
perf(metrics): the router's per-request records keep their handles per worker instead of rebuilding the key on every record (#3599)
perf(gateway): the HTTP edge wraps a response body once, not four times (#3590)
perf(gateway): a chat request's tool schemas are shared, not copied, between preparation, the spec and the reader (#3703)
perf(responses): a tool-call delta copies the call's id and name only when it announces the item (#3701)
perf(build): the release profile optimizes for speed (opt-level 3) (#3715)
perf(responses): a previous-response chain is read in place, not copied out of the memory store on every turn (#3708)
perf(symphony): the registry's repository names are worked out once, not on every lookup (#3689)
perf(tokenizer): the direct encoder encodes a long prompt unit by unit on the thread pool (#3447)
perf(tokenizer): the BPE word-cache bound goes on the model as it is read, not on a copy of it (#3583)
perf(gateway): the log writer threads write queued lines in batches instead of one write per line (#3573)
perf(tokenizer): the exact-match cache hashes a request once, with a keyed hash at memory speed, and compares the text on a hit (#3550)
perf(multimodal): the one-pass wire write beats the two passes at every image size, with the same bytes (#3549)

Documentation

docs(gateway): the EPD selection comment says what the encode pool holds since the direct lane took the encode role (#4711)
docs: three private rustdoc links read as code and -D warnings gates the workspace docs (#4690)
docs(router): the streamed request path's help text says the body reaches the worker as the client sent it (#4687)
docs(servicer): the subdirectory shape of the shared socket directory says the mount must be writable by the gateway's uid (#4628)
docs(servicer): the headless-over-ZMQ section says why a shared ipc socket directory is refused and how a pod hands it to the gateway (#4616)
docs(gateway): the rendered-prefill reasoning doc names its crate-private helper as code (#4575)
docs(gateway): the test trap's doc comment names its cfg(test) arming function as text (#4482)
docs(gateway): the two logprobs conversion docs write their return type as code (#4464)
docs(servicer): the warm-up refusal comments point at UnservedRequest for the stop rule (#4473)
docs(servicer): the plain-DP prefill refusal sentence belongs to SGLang (#4469)
docs(gateway): the policy registry's key-order comment says the header comes before the rid (#4465)
docs(engine_zmq_adapter): the encode dispatch sink's doc names the crate-private notice as code (#4455)
docs(gateway): the tenant rate-limit module's docs describe the wired state (#4458)
docs(engine_zmq_adapter): the TokenSpeed module's four private-doc links resolve or read as code (#4446)
docs(engine_zmq_adapter): six rustdoc links name their private targets as code, and one names the method it means (#4268)
docs(gateway): five private-doc links resolve or read as code (#4428)
docs(crates): fourteen private-doc links in seven sibling crates resolve or read as code (#4423)
docs(symphony): eleven private-doc links name their targets by path or read as code (#4425)
docs(crates): two rustdoc links in the tokenizer and the gRPC client name their private targets as code (#4409)
docs(multimodal): the rate-walk sampler's doc names its private helper as code (#4400)
docs(gateway): the least-load policy's doc names its private helpers as code, and the model-id-from flag's placeholders are code spans (#4378)
docs(gateway): the streamed assistant choice's doc names the opening-choice helper that carries the role (#4380)
docs(crates): eight rustdoc warnings in the tokenizer, the protocol crate and Symphony read as code or sit in one code span (#4329)
docs(gateway): the start-up launcher's doc names where it is declared and reached, and the mock MCP server drops its old-path line (#4313)
docs(crates): five rustdoc links in the servicer and the multimodal crate name their private targets as code (#4312)
docs(tokenizer): twelve rustdoc lines name their private targets and a generic as code (#4275)
docs(symphony): sixteen rustdoc links name their private targets as code, and one drops its redundant path (#4274)
docs(tool_parser): seven reference URLs are hyperlinks and two rustdoc links name their private targets as code (#4273)
docs(protocols): eleven reference URLs are hyperlinks and two rustdoc links name their private targets as code (#4272)
docs(engine_zmq_client): ten rustdoc lines name their private targets as code, and the SGLang request type by its path (#4270)
docs(crates): fourteen rustdoc links in six crates name their private targets as code (#4267)
docs(multimodal): eight rustdoc lines name their private targets and tensor indices as code (#4269)
docs(grpc_client): the media reference's proto comment reads its URL schemes as code (#4278)
docs(crates): three rustdoc lines in the auth, MCP and reasoning-parser crates read their examples as code (#4277)
docs(gateway): two rustdoc links to the private liveness module read as plain code (#4246)
docs(servicer): the NIXL in-place restart recipe carries its measured cost and the sweep's real timing (#4213)
docs(tau2): the validation note says why TP=1 held for the 27B model (#4201)
docs(servicer): restarting a prefill/decode leg in place on one host over NIXL (#4200)
docs(symphony): the mistral notes say which template a request with tools takes (#4199)
docs(tau2): the validation note drops an environment-specific detail (#4189)
docs(wasm): the crate README's layout tree and interface links name the paths the crate has today (#4185)
docs: drop two pointers to design documents that are in neither repository (#4184)
docs(contributing): drop the SMG_RUNNER_GPU_8 row, a runner variable no workflow reads (#4183)
docs(readme): the two feature links that answered 404 point at the pages' published routes (#4182)
docs(gateway): eight doc comments in the worker and observability modules name private items as code, and one ambiguous link names its target (#4176)
docs(gateway): six doc comments in the worker and config modules name private items as code, not as links (#4108)
docs(gateway): the middleware's ten doc comments name private items as code, not as links (#4104)
docs(gateway): the routers' ten doc comments name private items as code, not as links (#4106)
docs(gateway): twenty-one doc comments name private items as code, not as links (#4102)
docs(gateway): ten doc comments name their items so rustdoc resolves them (#4099)
docs(gateway): the crate root carries a module map (#4097)
docs(gateway): the routing snapshot's doc line stands on RoutingState again (#4025)
docs(multimodal): the worker-mode README names the encode-pipeline exception to the 503, and the registration warning says media references, not multimodal requests (#4026)
docs(multimodal): the decoder report's comment says the gauges are set on every report and the log lines once (#3980)
docs(symphony): the DeepSeek V3 table intro says the table reads a thought its template never writes (#3972)
docs(mesh): the README describes what an operator can read of the mesh, not HTTP routes the gateway does not serve (#3834)
docs(symphony): the architecture document (#3732)
docs(servicer): the test fixture's notes say that teardown puts the shell's selection back (#3793)
docs(symphony): only the Chat Completions stream shows a kept state's whitespace alone (#3699)
docs(symphony): a kept state's stream can carry whitespace the whole answer drops (#3698)
docs(tokenizer): the configured BOS is added on request, and a cross-check compares with the same pipeline (#3579)

Maintenance

build(deps): wasmtime and wasmtime-wasi 49.0.4 -> 49.0.5 close RUSTSEC-2026-0420, -0422, -0423 and -0424 in the lockfile (#4763)
chore(release): kv-index 2.0.1 and smg-grpc-servicer 0.15.0 step past the versions the registries hold
build(deps): the lock moves pulldown-cmark to 0.13.5, the release that closes the advisory the audit names (#4736)
refactor(zmq): the direct lane compares the declared disaggregation role as a worker type, not as strings (#4714)
ci(release): release branches cut from main, moved by pull requests, published on dispatch (#4694)
test(discovery): two label-selector tests read the module's own PD configuration (#4720)
test(gateway): the gRPC router and factory tests build their contexts through one module (#4719)
test(zmq): the direct-lane roles' untested branches get their tests (#4717)
refactor(gateway): the engine's declared bootstrap rendezvous is read through the backend peek (#4716)
refactor(zmq): the admin control calls' wait rules and profile id live once, in the adapter (#4715)
test(responses): the usage tests of the three serialization paths share one wire assertion (#4713)
ci(fixtures): the reference pin moves to the bellwether tree that records Kimi K3 in the engine's form; two render entries heal and the replay's prompt statement moves with it (#4712)
test(mesh): the worker-sync tests build their namespace, registry and adapter through one helper (#4709)
refactor(gateway): worker/worker.rs becomes worker/basic.rs and the module_inception expectation goes (#4705)
ci(gpu-guard): the pre-test GPU guard names who holds a dirty card and waits for it to drain before failing the job (#4696)
test(servicer): the role-link tests take the handshake reset as the refusal (#4695)
test(responses): the stream tests run their chat frames through one transform helper (#4700)
refactor(gateway): the chat and Messages request builders share one backed-plan sequence (#4701)
ci(fixtures): the reference pin moves to the bellwether tree that records Llama 3.3 70B Instruct, and the gated checkpoint joins the unloaded lists (#4697)
refactor: items the compiler proved dead go; is_builtin compiles for tests (#4692)
refactor(gateway): the RL control plane's registry view lives under worker/, not at the crate root (#4693)
ci(e2e): the classes on the mock engine run on a CPU job of every pull request (#4689)
ci(pr-test): the tier table names the posture log and the tenant module where they live (#4686)
test(mock-worker): the SIGTERM tests keep their client served while they wait for the exit (#4680)
refactor(gateway): mesh router-peer discovery lives under mesh/, not at the crate root (#4684)
test(symphony): the test modules read the testing module's arguments and malformed readers (#4678)
test(symphony): the test modules share their dropped event and weather fixture (#4675)
build(deps): the audit's zstd ignores go now that only the fixed releases are in the lock (#4676)
ci(fixtures): the reference pin moves to the bellwether tree whose manifests name processor_config.json as an oracle input (#4674)
build(tokenizer): the opt-in v1 tokenizers pipeline moves to its 1.0.0-rc.3 release, whose vocabulary store no longer prints the vocabulary on a repeated key (#4600)
test(symphony): the test modules share their tool builder (#4669)
test(tokenizer): the chat template loading tests read the shared tokenizer.json (#4668)
test(responses): the MCP web-search cases skip on the search backend's no-results answer (#4664)
test(gateway): the cache-aware tests route through the existing helpers (#4665)
refactor(symphony): the carried wrapper bytes leave through one drop_carried helper (#4667)
ci(multimodal): the vision golden generator installs a pinned transformers release (#4670)
ci(e2e): a gateway-only change runs the chat and router suites on one engine instead of every engine's chat lanes (#4658)
test(symphony): the tagged tests share their event readers (#4666)
test(gateway): the tests' unreachable workers hold a refused address, and the admission test judges by its releases (#4656)
test(data-connector/external-router/multimodal): the tests' unreachable peers hold a refused address (#4661)
refactor(kv-index): the two trees intern tenants through one pool (#4653)
build(wasm): the runtime dependency names its features, so the unused module cache and its zstd 0.13 leave the lock (#4660)
refactor(gateway): the workflow data types' uncalled validate_initialized methods go (#4654)
test(gateway): the local stop tests connect their ZMQ pair through one helper (#4657)
build(deps): the audit's zstd and zstd-safe findings under wasmtime's cache crate are recorded as ignores with their reason (#4651)
test(gateway): the policy tests build their workers through one builder (#4652)
refactor(kv-index): the two trees intern tenants through one pool (#4653)
test(symphony): the format test modules read the testing module's event folds (#4650)
test(mesh): the chunking tests share their receiver and its td stream (#4648)
test(mcp): the tool inventory tests share their one-tool inventory (#4647)
test(tokenizer): the render tests share their minimal tokenizer.json (#4646)
test(gateway): the consistent hashing tests build their selection info through two constructors (#4644)
test(pd-router): the side-failure release test judges by the dispatch's end and holds its refused port (#4643)
test(multimodal): the vision goldens read one npz loader pair for both Phi models (#4642)
test(mock-worker): the capture and script tests share their generate reader (#4641)
ci(fixtures): the reference pin moves to the bellwether tree whose last three hand-written developer-role cases carry the engine's message rules, and the entries they heal leave the list (#4639)
test(multimodal): the preprocess goldens share their FNV-1a fingerprint helpers (#4638)
test(kv-index): the token tree's concurrent tests fan out through one scoped helper (#4637)
test(multimodal): the video sampling tests read one frames-and-sampling helper (#4640)
test(gateway): the cache-aware tests share their initialised worker pair (#4636)
test(gateway): the assemble tests build their inline image frames through one helper (#4633)
test(gateway): the least-load and power-of-two tests share their fed worker pair (#4632)
test(external-router): the provider tests share their transformed content (#4625)
test(gateway): the prefill admission tests spawn their plain attempt through one helper (#4629)
test(gateway): the history tests share their replayed-items load (#4614)
test(data-connector): the hooked storage tests share their capturing conversation store (#4607)
ci(fixtures): the reference pin moves to the bellwether tree whose hand-written render sets carry the engine's message rules, and the entries it heals leave the list (#4609)
test(gateway): the cache-aware burst tests share their gated pair and their picks (#4610)
refactor(cli): the deprecated kv-index flags are removed from the command line and the launcher (#4615)
test(kv-index): the eviction policy tests insert their three pages once (#4618)
test(workflow): the engine tests share their tracking step (#4626)
test(wasm): the runtime config tests vary one field of the default (#4619)
test(gateway): the local stop tests share their stream past a local stop (#4621)
test(mock-worker): the integration tests share their served client (#4622)
test(gateway): the multimodal router tests share their gRPC worker and their registry (#4624)
test(workflow): the engine tests start their workflow through one helper (#4627)
ci(e2e): the harness follows the servicer package's default instead of believing an unset lane means the Python servicer (#4613)
test(gateway): the activation step tests read the crate's test app context (#4605)
ci(runners): the accelerator scale sets share one values base under per-set overlays (#4612)
test(multimodal): the fingerprint tests read the shared seeded image (#4608)
test(multimodal): the qwen video tests share their RGB frame views (#4606)
test(gateway): the multimodal worker tests share their pipeline settings (#4604)
test(workflow): the engine tests wait for a settled workflow through one helper (#4594)
test(gateway): the prefill admission tests share their select-this-worker attempt (#4599)
test(gateway): the Kubernetes discovery tests build their Ready condition once (#4592)
build(gateway): the baked version string follows the checked-out commit in a reused cargo target (#4265)
test(multimodal): the two-image preprocess tests share their batch assertions (#4589)
test(kv-index): the token tree tests build their branching paths once (#4593)
test(gateway): the remaining free test modules call the no-health-check helper (#4587)
test(multimodal): the llama4 registry tests share their tokenizer and config (#4590)
test(multimodal): the preprocessing goldens share their seeded image (#4588)
test(gateway): the responses tests share their completed response (#4584)
ci: validate stacked product changes on CPU and select ROCm setup lanes (#4478)
test(e2e): the PD sweep skips the sole prefill's restart on vLLM over NIXL on CUDA, where an idle decode keeps the dead prefill's memory (#4257)
test(gateway): the gRPC stream-failure tests call the no-health-check helper (#4586)
test(gateway): the multimodal router tests call the no-health-check helper (#4573)
test(gateway): the admission, ledger and models tests call the no-health-check helper (#4578)
test(fixtures): the render-parity harness fetches a tiktoken vocabulary too, so the checkpoint that ships only tiktoken.model is compared (#4571)
test(gateway): the router tests share their error-shape reader (#4568)
test(gateway): the worker tests share their ready worker (#4567)
test(data-connector): the storage factory's memory tests share their round trip (#4544)
test(gateway): the gRPC router tests share one closed-port helper (#4563)
test(gateway): the background-response tests share their pending work (#4565)
test(multimodal): the fetch tests read the request head through one helper (#4566)
ci(fixtures): one tokenizer cache for both harnesses, restored again when the cache service limits the first try (#4548)
test(gateway): the scheduler tests drop their function-local map imports (#4561)
test(workflow): the timing tests build their steps through one helper (#4564)
test(gateway): the registry tests share their replacement pair (#4562)
test(gateway): the bucket boundary tests share their first phase (#4558)
test(multimodal): the vision processors' tests share one test-image builder (#4557)
test(tokenizer): the encode-cache parity tests share their prompt reader (#4555)
test(gateway): the policy tests share one healthy-workers helper (#4554)
test(kv-index): the token-tree tests that repeat another one go (#4556)
test(gateway): the tree-sync adapter tests build their adapter through one helper (#4537)
test(mesh): the CRDT map tests share their compacted post-tombstone snapshot (#4542)
test(mcp): the orchestrator's built-in-server tests use the test constructor (#4543)
test(gateway): the create-worker step's two dp-aware tests share their registration (#4545)
test(external-router): the MCP test support's binding helper is private (#4547)
test(grpc-client): the stream-behaviour test binaries share their mock engine (#4551)
ci(fixtures): the reference pin moves to the re-recorded bellwether tree, and the entries it heals leave the lists (#4532)
test(gateway): the admission tests read their gap counters through one recorder type (#4541)
test(gateway): the cache-aware policy tests build their workers through the module's own fixtures (#4539)
test(gateway): the no-health-check worker configuration lives once in the worker test support (#4538)
test(gateway): the EOF streaming tests build their processors through two constructors (#4536)
test(external-router): the MCP tests share their orchestrator and session set-up (#4531)
test(gateway): the two over-limit body tests of the edge layers share their chunked post (#4533)
test(e2e): the function-calling format cases decode greedily on a real engine (#4510)
test(gateway): the two mixed-fleet tests of the gateway router share their fleet (#4528)
test(gateway): the middleware tests' response-lines recorder lives in the logging test support (#4527)
test(tokenizer): the test binaries share their counting allocator and their byte-level BPE builder (#4525)
test(kv-index): the tree test modules lose a duplicate test and share a lyric fixture (#4524)
test(radix-tree): the two differential tests share their two-core subject (#4523)
test(gateway): the remaining default-tail cache-aware configurations call the constructor (#4521)
test(adapter): the vLLM and TokenSpeed streaming tests share their three-item stream read (#4519)
test(gateway): the bucket tests select through the module helper and share one boundary prologue (#4518)
refactor(gateway): the start-up's listener binds and servings are two named phases (#4514)
test(servicer): the three wires' relay-at-boot test shares its publisher and two-batch round (#4513)
refactor(gateway): the tls file's two concerns move under clients and server (#4509)
test(gateway): the two load-scoring policies' tests share their one-rank load builders (#4508)
test(gateway): the EOF streaming tests' tool-call cases share their fixtures (#4501)
refactor(config): the cache-aware policy's default knobs are written once (#4496)
test(gateway): the pipeline's vision-refusal tests use the module's pool and request helpers (#4498)
test(gateway): the gRPC router's count and Messages tests share their router prologue (#4495)
test(tokenizer): the byte-level alphabet helper is one function in the tests' common module (#4491)
test(gateway): the hand-built test AppContext is one fixture (#4490)
test(servicer): the three wires' KV-events replay test shares its socket setup and replay round (#4489)
test(gateway): four bucket policy tests take their workers from the module's helper (#4488)
test(gateway): a collected Symphony message records its token and generation families (#4476)
test(servicer): the SGLang servicer's two-rank tests use harness_with (#4475)
refactor(external-router): the streamed MCP tool loop dispatches each call through named steps (#4463)
chore(deps): the workspace table note on llm-multimodal names the binding as a direct dependent (#4456)
chore(bindings): drop the profile tables cargo ignores in the two binding manifests (#4448)
test(common): the tokenizer test download sends a GET that broke below HTTP again, three times with a short back-off (#4436)
chore(gateway): the build-script dependency on chrono reads its version from the workspace table (#4447)
chore(mesh): four dependency lines read their version from the workspace table (#4445)
test(servicer): the slim-output pin comment names the batch it prints (#4439)
chore(deps): drop the four default-features overrides cargo ignores on workspace-table dependencies (#4442)
chore(deps): seven dependency lines read their version from the workspace table (#4433)
chore(gateway): drop seven dependencies no gateway source names; one moves to the tests (#4431)
test(servicer): the SGLang slim-output pins count the 28th field (#4435)
chore(deps): drop ten dependencies seven crates list and never name (#4430)
chore(mesh): drop twelve dependencies no mesh source names (#4432)
refactor(data_connector): the migration modules drop the whole-file dead-code allowance that outlived what it covered (#4417)
ci(docker): the TokenSpeed engine image build fails when flashinfer's prebuilt kernel cache does not match flashinfer-python (#4418)
refactor(external_router): the Messages route records a failed request's metrics through one helper (#4419)
refactor(gateway): the Messages API utilities drop the whole-file dead-code allowance their first commit carried (#4406)
refactor(gateway): the gRPC router's harmony and selection modules drop the items a dead-code expectation kept (#4404)
test(discovery): the Consul e2e keeps the periodic pass, which applies a change that lands while its instance still registers (#4370)
refactor(gateway): the config builder and the request wrappers drop the twenty-one functions nothing in the tree calls (#4403)
test(routing): the no-hint spread test sends twelve distinct prompts, so a fair tie draw landing every miss on one worker is a one-in-177,147 run, not one in 243 (#4402)
refactor(kv_index): the token tree's insert splits a child under an intermediate node in one place (#4396)
refactor(kv_index): the string tree's merge spells the remote-tenant merge and the trimmed remote clone once (#4390)
refactor(engine_zmq_adapter): the prompt logprobs attach to a stream's responses in one place (#4398)
refactor(kv_index): the string tree's prefix match ends in the tail its fused walk already shares (#4389)
test(servicer): the two-switches test pins the readiness hold, and the two docs that still called admission open are corrected (#4383)
test(discovery): the kind lane registers an EPD mock fleet, each role into its own pool (#4393)
refactor(tokenizer): the Python JSON formatter borrows the indent bytes of the tojson call that owns them (#4386)
build(deps): the audit's RUSTSEC-2026-0387 finding on crypto-bigint 0.5 is recorded as an ignore with its reason (#4397)
refactor(kv_index): the token tree's fixed-tenant match-and-insert is the one fused walk with the tenant known up front (#4384)
test(fixtures): the two LightOnOCR whole-model entries healed by the template-file fix leave the render list (#4367)
ci(pr-test): a tier table in detect-changes decides the e2e lanes a pull request's changed paths can reach (#3864)
test(e2e): the harness waits for the mock worker's standard health service like any engine servicer (#4361)
refactor(gateway): the SSE chunk frames of the gRPC router's streams sit in their own module beside the streaming processor (#4362)
refactor(gateway): the lazy per-worker HTTP client sits beside the worker client cache (#4366)
refactor(servicer): the server start takes the servicer config and hands the wire's connect its inputs (#4374)
refactor(servicer): the three wires connect their engine through one link routine (#4363)
refactor(gateway): the request wrappers of the proto wrapper sit in their own module (#4352)
ci(fixtures): the fixture job unpacks the hand-written sets and every 5,000th benchmark set, and the harnesses list a difference by case name or by set (#4208)
refactor(gateway): a worker's metadata sits in worker/metadata.rs (#4349)
test(gateway): the demoted-worker test is named for the probe that promotes it (#4344)
refactor(servicer): the read-only RPCs share their handshake readers and the load report's arithmetic (#4335)
refactor(gateway): the multimodal payloads of the proto wrapper sit in their own module (#4332)
refactor(gateway): the guarded stream of a response body sits in worker/stream.rs (#4333)
test(protocols): the single-file integration tests fold into one binary (#4327)
test(e2e): the dying-worker wait test's stub sleeps three seconds (#4322)
test(symphony): an invoke the cut found unnamed is refused whole, at the assembler and at the client (#4308)
refactor(gateway): the shared-memory handles of the proto wrapper sit in their own module (#4326)
ci(e2e): the full-warm-up case's worker runs its engine at the level the servicer's lines carry (#4319)
refactor(gateway): the command line's secret settings are a cli module of the binary (#4320)
refactor(gateway): the worker monitor's tests sit in a sibling file, not in the production module (#4317)
refactor(engine_servicer): the three servicers share their request stream, tokenizer prologue, flush validation and capacity fields (#3655)
refactor(multimodal): the media pipeline's video sampling tests sit in a sibling file, not in the production module (#4316)
test(e2e): a worker that dies during start-up reports the cause from its log (#4259)
test(common): the tokenizer test download waits out a rate-limited hub and takes back a poisoned download lock (#4299)
test(bellwether): the render harness lists the Llama-class checkpoint whose pinned reference was recorded from its raw tokenizer file (#4315)
refactor(servicer): the KV event relay's tests sit in a sibling file, not in the production module (#4314)
refactor(gateway): the embedding wrappers and the logprob shapes sit in their own modules beside the proto wrapper (#4305)
test(e2e): the worker-API case's gateway registration wait covers the engine start plus the servicer warm-up (#4256)
refactor(gateway): the command line's value parsers are a cli module of the binary (#4302)
refactor(servicer): the KV relay's subscribed check is the shared state's, and the three servicers delegate to it (#4304)
build(deps): ctutils 0.4.2 -> 0.4.5 in the lockfile (RUSTSEC-2026-0385) (#4306)
refactor(engine): the ZMQ adapter's vLLM translation tests sit in a sibling file, not in the production module (#4311)
build(gateway): the integration tests' shared harness is a crate compiled once, not a module compiled into every test binary (#4290)
test(e2e): a decode engine dies under live streams in the PD topology sweep (#4157)
refactor(servicer): the three servicers' shared state and server lifecycle live in one engine-neutral module (#4221)
refactor(gateway): the completions stream state machine sits in its own module beside the streaming processor (#4293)
ci(e2e): the legs' workers warm up at the fast level; one case per engine runs the full pass to SERVING (#4239)
test(engine): the ZMQ adapter's engine tests connect their client and mock engine through one shared helper (#4165)
refactor(kv-index): the string tree's tests sit in a sibling file, not in the production module (#4242)
ci(fixtures): the fixture job leaves a tokenizer library's vocabulary dumps out of the printed log (#4235)
ci(e2e): the worker start-up wait and the AMD chat row's caps cover the servicer's planned warm-up (#4233)
ci(fixtures): the fixture harnesses wait out the hub's rate limit and remember a file a checkpoint does not serve (#4255)
refactor(gateway): the metrics module's tests sit in a sibling file, not in the production module (#4263)
refactor(engine): the ZMQ connector's tests sit in a sibling file, not in the production module (#4285)
test(e2e): a worker relaunched on its port keeps the previous engine's log (#4253)
refactor(gateway): the gRPC router's PD tests sit in a sibling file, not in the production module (#4258)
refactor(gateway): the multimodal assembly's tests sit in a sibling file, not in the production module (#4254)
build(gateway): the amg alias binary builds no unit-test harness of its own (#4216)
refactor(servicer): the relay's load source over the servicer state is one generic type; each engine supplies its rank's figures (#4211)
test(tool-parser): the single-file integration tests fold into binaries by subsystem (#4195)
test(tokenizer): the single-file integration tests fold into binaries by subsystem (#4181)
test(gateway): the single-file integration tests fold into binaries by subsystem (#4172)
refactor(gateway): the hand-parsed --prefill/--encode flags of the command line are the first cli module of the binary (#4156)
refactor(gateway): the transcription route's metric finalizers are one type (#4198)
test(gateway): the seven metrics tests that drove the global recorder share one binary and one installed recorder (#4206)
refactor(engine): the ZMQ adapter's three per-item tensor splits are functions beside the feature builder (#4162)
test(symphony): the corpus replay counts a Llama 4 reference with a nested apostrophe apart, as the recorder refuses it (#4160)
refactor(external-router): the Responses history loader reads as its three lookups (#4159)
test(gateway): the frozen-worker backlog test reads the run's silence on the worker's clock (#4190)
test(multimodal): the single-file integration tests fold into binaries by subsystem (#4187)
test(symphony): the single-file integration tests fold into one binary where they need no process of their own (#4196)
refactor(zmq-client): the dispatcher's arms are methods on its state, the select loop a page long (#4197)
refactor(multimodal): model specs build vision processors that resolve their preprocessor config once (#4174)
refactor(gateway): the transcription route forwards a worker's answer through two named functions (#4139)
test(gateway): the trace-context test waits for its request log lines after close_logging (#4161)
test(e2e): the TokenSpeed RL lane asserts the refits the engine advertises, so a nightly that accepts disk refits passes (#4034)
build(profile): the dev profile keeps line tables only as its debuginfo (#4151)
test(servicer): the SGLang and vLLM launcher tests' fake servers carry the readiness setters the lifecycle holds (#4150)
ci(scripts): the Rust setup's index refresh survives a third-party apt source that does not answer (#4141)
test(responses): the MCP binding test forces the chained turn's tool call instead of relying on the model's choice (#4147)
test(multimodal): the GLM 5.3 Flash clip rule is proven against the reference on a clip the loader keeps whole (#4146)
test(external-router): the Responses and tool-loop tests build their MCP session fixture once (#4138)
test(multimodal): the GLM 5.3 Flash video rule test expects the loader-then-rate sampling (#4140)
refactor(workflow): a step's run_if gate, attempt and completion are named phases of the engine (#4128)
refactor(tokenizer): the native tokenizer's tests sit in a sibling file, not in the production module (#4126)
test(mcp): the crate's tests build their SSE server configuration once (#4124)
refactor(mesh): the tree-sync adapter subscribes to its three namespaces through one loop and decodes their payloads through one function (#4118)
test(mock-worker): the load client drives the Messages API, embeddings and json_schema answers (#4077)
build(deps): the TLS trust-store crates move past their advisories (rustls-webpki, webpki-roots, webpki-root-certs) (#4098)
ci(evals): fix the AMD BFCL legs' Qwen3.8-27B timeout, artifacts and GPU cleanup (#4057)
refactor(external-router): the MCP tool loop is one upstream round and one call dispatch per phase, over the argument preparation the streaming variant shares (#4094)
refactor(mesh): the tree-sync adapter's tests sit in a sibling file, not in the production module (#4092)
refactor(grpc): the chat utilities' tests sit in a sibling file, not in the production module (#4091)
refactor(gateway): tenancy has a module directory of its own instead of a file at the crate root (#4078)
refactor(gateway): the start-up posture log lives under server/, not at the crate root (#4079)
refactor(worker): the registry's tests sit in a sibling file, not in the production module (#4081)
refactor(kv-index): the token tree's tests sit in a sibling file, not in the production module (#4085)
refactor(gateway): the HTTP client factory lives in a clients module, not at the crate root (#4074)
build(deps): bump tokio from 1.53.1 to 1.53.3 (#4051)
build(deps): bump str0m from 0.23.1 to 0.24.1 (#4052)
build(deps): bump ubuntu from 534baea to 68ee220 in /docker (#4054)
build(deps): bump crossbeam-utils from 0.8.22 to 0.8.23 (#4055)
test(gateway): the unattended ZMQ worker tests take an OS-assigned handshake port (#4048)
test(grpc-servicer): the TokenSpeed pin-forwarding tests build a servicer with an async_llm, so they pass where TokenSpeed is installed (#4035)
test(gateway): a refusal-line test sees every line its thread writes while its neighbours run (#4039)
build(deps): wasmtime and wasmtime-wasi 49.0.3 -> 49.0.4 close RUSTSEC-2026-0378 and RUSTSEC-2026-0379 in the lockfile (#4042)
test(gateway): the legacy-reason test names a tool-call parser no table answers to (#4022)
ci(pr-test): the Rust lint and tests run as parallel shards, each behind its own path filter (#3620)
refactor(gateway): the chat and Messages preparation stages build the request's constraint and the special-token decision through one function (#3999)
refactor(gateway): the chat and Messages preparation stages resolve media and tokenize the rendered prompt through one function (#3877)
test(symphony): every table that reads calls writes a forced tag in every configuration, the known gaps listed (#3995)
refactor(symphony): the list reader takes its framing as one value the factory maps the syntax to (#3994)
refactor(gateway): worker.rs: the worker runtime moves to worker/runtime.rs (#3993)
test(symphony): the fuzz harness checks that a pythonic call that ended without its close said so and stays open (#3862)
refactor(gateway): the chat and Messages request-building stages decide the worker's constraint and Symphony's assignment through one function (#3991)
refactor(gateway): worker.rs: the routing-key load and the load guard move to worker/load.rs (#4000)
refactor(gateway): the two Responses tool loops partition and cap an iteration's calls through one function (#3939)
refactor(gateway): the completion stream's phases are methods of one stream state, as the chat and Messages streams' are (#3921)
refactor(gateway): the chat and Messages readers split the reasoning and parse the tool calls through two shared steps (#3892)
refactor(gateway): the Symphony seam is three modules, name resolution, constraint derivation and the reader (#3907)
ci(e2e): add an AMD row to the 1-GPU vLLM chat lane (#2870)
test(e2e): the Llama tool-choice class serves the conflicting-defs pair under the Symphony parser, as the Qwen class does (#3973)
refactor: the test harness, the Python servicer and the contributing guide say side, role and pool where they said leg (#3970)
refactor(gateway): the integration tests and the engine crates say side and worker where they said leg (#3967)
refactor(gateway): the bootstrap probe, the liveness bounds and the registration steps say worker and side where they said leg (#3965)
refactor(gateway): the gRPC prefill-decode executors and streams say side where they said leg (#3961)
refactor(gateway): the HTTP prefill-decode router and the pair placement say side, role and pool where they said leg (#3957)
refactor(gateway): start-up's graceful shutdown is a named phase with its budget (#3960)
refactor(tokenizer): the DeepSeek V3.2 and Kimi K3 renderers render a message by role, one function per arm (#3958)
refactor(gateway): a worker's prefill-decode function is its role, not its leg (#3956)
refactor(gateway): the Messages stream's phases are methods of one stream state, as the chat stream's are (#3955)
refactor(grpc-client): the engine clients take generate and the abort-on-drop hook from the shared macros, the TokenSpeed client included (#3900)
refactor(response-template): the lowering's three long builders are split by the part of the call they lower (#3951)
refactor(response-template): the pattern parser's quantifier arm is a function of its own (#3953)
refactor(response-template): the template reader's two long functions are split by what they read (#3952)
refactor(gateway): start-up's workflow engines and mesh preparation are two named phases (#3935)
refactor(gateway): the prefill-decode executors are named phases over one dispatch context (#3922)
test(policies): the cache-aware policy's tests move to a tests.rs sibling (#3931)
refactor(gateway): the Responses-to-chat conversion folds one item at a time, and the Messages builder decides in named steps (#3933)
refactor(response-template): the pattern compiler's three long functions are split by node kind (#3944)
refactor(protocols): the deprecated builder constructors leave the crate (#3886)
refactor(gateway): the Responses background module runs a storage future under the request context through one function (#3943)
refactor(grpc-client): the fields no request builder sets are named once per engine client (#3899)
test(gateway): a model family's marker tokens are pinned once across the crates that spell them (#3938)
refactor(gateway): start-up's observability and discovery are two named phases (#3934)
test(tokenizer): the cache tests' byte-splitting mock encoders are one test encoder (#3932)
refactor(policies): the cache-aware policy derives a request's namespaced key in one place (#3929)
refactor(gateway): the multimodal plan processor is named phases over one context, and a batch preprocesses through the cache or whole (#3928)
refactor(tokenizer): the DeepSeek v4 renderer renders a message by role, one function per arm (#3927)
refactor(gateway): the worker-selection stage gathers its inputs, then picks, and the EPD selection reads its legs through one helper (#3924)
refactor(gateway): the PD HTTP router's three dispatch bodies share their leg setup, their leg-failure answers and the decode head (#3923)
test(policies): the bucket policy's adjustment tests assert on the workers' bucket (#3920)
refactor(grpc-client): the one-of-three structured-constraint rule is read once, each engine client keeps only its variant (#3898)
refactor(gateway): the non-streaming readers decode a choice once, and read its finish and matched stop from one place (#3878)
test(gateway): the log-dir start-up test takes a port as served only on the gateway's own word (#3868)
test(symphony): the fuzz run stops a fork child that stopped working (#3861)
test(symphony): the fuzz harness's explain prints the events of both replays on request (#3859)
chore: upgrade DeepSeek-V4.1-Flash to the 1017 checkpoint (#3823)
test(symphony): the fuzz harness names the three breaches of its whole-JSON promise apart (#3858)
refactor(external-router): the Responses route counts its errors through one helper (#3843)
test(symphony): the fuzz harness judges whole JSON by its syntax, not by a reader's number range (#3857)
refactor(mesh): the gossip controller's round is written as named phases, the dial as one function with its outcome (#3917)
refactor(gateway): the KV-event subscription loop reads as rounds, with the backoff and the exit in one place each (#3889)
refactor(protocols): the builder constructors nothing in the tree uses are deprecated (#3869)
refactor(gateway): the older parser names a table answers to, and the names none does, live with the gateway's parser resolution (#3870)
refactor(gateway): the gRPC client dispatches a request to its engines once per request shape, and eight accessors without a caller go (#3881)
refactor(gateway): the HTTP router and the PD HTTP router relay a streaming body through one task (#3890)
refactor(gateway): the gRPC executors take their client-shape mismatch answer from the selection (#3897)
refactor(protocols): a Responses content part says its text once; the routing-text walk and the text-only read use it (#3902)
refactor(multimodal): the model-spec trait drops processor_kwargs, which nothing reads (#3903)
test(symphony): the made-up-empty-string check reads a whole member through its closing byte, and the contract test's made-up-quote rule matches the harness (#3906)
chore(mesh): the eight public functions nothing calls go, the certificate rotation monitor that never started among them (#3908)
refactor(mesh): the two CRDT engines keep their operation log through one keeper (#3911)
chore(gateway): five more functions no code in the tree calls are removed (#3914)
refactor(gateway): the WASM middleware runs its request hook and its response hook as two functions (#3882)
test(grpc-client): the six tests that construct a proto and drop it go (#3872)
refactor(harmony): the decode streams read through a per-stream state, one method per phase (#3888)
test(e2e): the MoRI-IO idle check fails when a worker's metrics are missing (#3894)
refactor(data-connector): the Oracle and Postgres stores write their shared row, filter and insert pieces once (#3883)
chore(tokenizer): three public functions without a caller, test or mention go (#3884)
refactor(tokenizer): the DeepSeek renderers take the family special tokens from one definition (#3891)
refactor(auth): the JWKS provider refreshes its key set in one step, for an expired cache and an unknown kid alike (#3896)
test(gateway): the start-up tests wait for the start-up line they assert on, with the child's log level pinned (#3875)
refactor(worker): the builder writes its defaults once and the profile admin calls share one implementation (#3873)
refactor(gateway): the five reserved gRPC routes run their pipeline call through one reserved execution (#3871)
test(symphony): the fuzz harness checks that a reader makes no byte of the arguments up (#3856)
refactor(symphony): the JSON family's grammar builders live in their own module beside the other syntaxes' (#3863)
refactor(symphony): the crate's surface is its root, pinned by a test, and Declared has its own module (#3860)
test(symphony): the fuzz harness's whole-JSON promise covers the stream's end (#3855)
ci(release): a dispatch can ask for the engine's source build explicitly (#3841)
chore(release): the version check is a tested Python tool with one component walk (#3830)
refactor(gateway): the Symphony seam decides for a chat and a Messages request through one set of functions over what the request asks, and names its failures' classes (#3849)
test(servicer): a test that drives a launch leaves no handshake address or spawn method in the environment (#3866)
refactor(harmony): the Responses stream's completed terminal is one emitter method (#3848)
test(symphony): the modules' tests share one test-support module (#3828)
refactor(pd-router): the six route handlers take one path, the per-route facts are a table (#3845)
test(symphony): the Kimi K2 tests see the id fallback under every cut and hold the rendered turns' shapes, and the docs say what they hold (#3833)
refactor(symphony): the gateway's reader takes its parsing policy from the crate (#3837)
ci(pr-test): the e2e lanes' gate is decided once in detect-changes (#3810)
refactor(grpc-client): the engine clients read the response format, the stops and the plain sampling through one module (#3836)
test(symphony): the corpus harness knows the Kimi K2 table: its probe group and its six slugs (#3831)
ci: one release workflow builds the engine images, with the engine as an input (#3829)
test(symphony): the recorded slices and the shape patterns' oracle are read from the fixture pin, not the repository (#3826)
test(e2e): run the PD topologies over MoRI-IO, and add AMD rows to the PD nightly (#2845)
ci: one composite action downloads the build artifacts for every job (#3818)
refactor(symphony): the named call and the named arguments read through one header-framed reader (#3824)
refactor(protocols): every validator builds its refusal through the crate's constructor (#3820)
test(gateway): the integration binaries share one in-process gRPC harness (#3813)
ci: the gh CLI install and the pod API-key export are one composite action each (#3827)
test(symphony): the corpus harness reads each model's table from the registry (#3733)
refactor(golang): the proto parser reads each field through one reader per type (#3815)
refactor(config): the validators build an invalid-value error through one constructor (#3821)
chore(mcp): drop the backoff dependency nothing in the crate uses (#3819)
refactor(wasm): the execution counters live on the module manager alone (#3822)
test(e2e): the worker's servicer-implementation switch is one table and one block (#3816)
ci(evals): add a MiniMax-M3 AMD leg to the BFCL A/B (#3540)
test(e2e): the suites read a streamed tool call through one collector (#3809)
refactor(protocols): the interactions validator refuses through the crate's own constructor (#3817)
test(gateway): one helper builds the test context over the mock tokenizer for thirteen integration binaries (#3812)
refactor(symphony): a started call the block's end cuts short names what was cut (#3811)
test(servicer): the launcher smoke test loads only this package's vLLM plugin (#3814)
refactor(bindings): the Go binding's proto parser reads its counts and ids through two helpers, with a test of the mapping (#3807)
refactor(mcp): the client handler drops the tool inventory it never read (#3804)
refactor(symphony): a malformed region's reason is one of a closed set (#3734)
refactor(zmq-client): the startup handshake drives HELLO, INIT and READY in one loop (#3803)
test(symphony): the throughput bench reads a recorded slice per table family, splits the setup and times the adapters, with a baseline recipe (#3750)
refactor(auth): the crate drops the five items nothing calls (#3802)
refactor(worker): the circuit breaker's state-entry bookkeeping lives in one place for the swap path and the CAS path (#3801)
test(gateway): a registry table's JSON answer under always follows the thought the prompt leaves open (#3799)
refactor(symphony): a table is spelled once in the registry, and the rest of the registry reads its row (#3740)
test(symphony): coverage-guided fuzz targets for the engine over every table, seeded from recorded output slices (#3749)
refactor(grpc): the chat SSE stream's phases are methods of one stream state (#3752)
refactor(auth): the control-plane middleware admits a caller and refuses a request through one path each (#3800)
refactor(symphony): the two JSON object readers account for their argument bytes through one judge (#3741)
test(symphony): a throughput and setup benchmark for the crate (#3729)
refactor(grpc): the chat and Messages request builders share their media assembly and their request finishing (#3748)
refactor(policies): the load-aware policies read the worker monitor's shared load snapshot (#3761)
test(gateway): the gateway's lib tests build again after two merges crossed (#3796)
refactor(symphony): the registry's checkpoint groups are a data file the build compiles into the same tables (#3739)
test(tokenizer): the render-fixture parity test's known-differences list is empty again, as the pinned fixtures have none (#3746)
refactor(symphony): a malformed region's fixed reason is borrowed, not allocated (#3700)
refactor(metrics): nine recorders nothing calls go, six of them with the family they alone described (#3745)
test: a test for a rule that was removed goes, and the sixteen other bare ignores say why (#3744)
test(servicer): every servicer test starts with no implementation selected by the environment, and leaves none (#3751)
test(gateway): the Responses and chat spec tests build their shared set-up through one helper per variant (#3758)
refactor(mesh): both ends of a gossip stream share one sender round and one inbound frame handler (#3742)
refactor(symphony): the keyed and attributed assemblers write their arguments object through one writer (#3696)
refactor(symphony): the named-call and named-arguments readers read their header in one place (#3672)
test(symphony): the recorded outputs, corrupted, never panic a table and keep the contract (#3605)
refactor(responses): the tool loops share a call's argument preparation and its record, and the stream's per-call body has a name (#3717)
refactor(external): the chat route's retry attempt is three named pieces (#3789)
refactor(servicer): the --servicer-impl registration has one home, the three engines keep their choices (#3738)
refactor(grpc): a stage reports a missing earlier output through one helper, under one code per output (#3736)
refactor(symphony): the text between a format's structure is split into runs in one place (#3671)
refactor(workflow): the pool-size gauge's key has one home (#3718)
ci(benchmarks): add a single- and multi-worker nightly benchmark on AMD (#2931)
refactor(servicer): the three servicer configurations embed one block for the listener, the engine link and the start-up bounds (#3716)
test(gateway): three tests that could never fail pin something or go (#3710)
ci(evals): add a DeepSeek-V4.1-Flash AMD leg to the BFCL A/B (#3539)
refactor(python): the router's settings bind by keyword after a short positional prefix, in alphabetical order (#3709)
refactor(golang): the per-chunk converter is split into named steps under a recorded table of the chunk shapes (#3692)
refactor(golang): the FFI entry points read their arguments and write their results through two helpers (#3688)
refactor(gateway): a DeltaObserver seam on the SSE re-chunk stage, shared by the gRPC pipeline and the HTTP relay (#3641)
refactor(multimodal): painting an image onto a normalized planar canvas is one transform (#3677)
refactor(multimodal): the encoder input's wire write is one dispatch per form, its half-width conversions fixed at compile time (#3678)
test(bindings): the CLI parity reader accounts for every argument attribute of the CLI's source (#3695)
refactor(workflow): the tokenizer registration step runs as named steps (#3687)
refactor(workflow): the scheduler loop runs over named phases (#3691)
test(python): the launcher's arguments and the Rust router's constructor are pinned against each other (#3693)
refactor(response_template): the regex machine's run resumes a frame and executes an instruction in named steps (#3669)
refactor(workflow): one pool-size gauge refresh for the registration and removal steps (#3681)
refactor(symphony): the keyed, tagged and attributed call grammars share their block wrapping and arguments builder (#3690)
refactor(golang): the single and the policy client share one chat request path (#3686)
test(cli): accept extended flatten attribute spellings (#3684)
refactor(workflow): the external worker is built once, with or without model cards (#3665)
test(cli): follow flattened argument groups in parity checks (#3680)
test(grpc): a CPU-only stress bed for the gRPC router, with its load client and a nightly run (#3645)
refactor(responses): the emitter's per-chunk step names its three deltas (#3654)
refactor(data_connector): the Oracle and Postgres migration runners share their pending-selection prelude (#3656)
test(symphony): the stated exception puts a call's bytes back at the last place that fits, in an entry missing them (#3660)
refactor(multimodal): the built-in vision processor registrations are one table (#3667)
refactor(grpc_servicer): the vLLM model section reads its HF attributes through the shared readers (#3668)
test(symphony): every registry table keeps the contract on the outputs recorded for it (#3628)
test(config): the worker-thread tests keep their host-derived counts within the ceiling (#3636)
test(symphony): every scramble is checked; a breach on an output holding a broken escape is forgiven, a panic never (#3646)
refactor(grpc_servicer): the engine config views share their lenient readers (#3647)
refactor(grpc_servicer): the KV event stream and the KV relay build their optional load recorder in one place (#3649)
refactor(grpc_servicer): one chunk loop streams the tokenizer bundle, the three engine servicers yielding from it (#3640)
test(symphony): every table reads seeded scrambles of its own markers without a panic and keeps the contract (#3595)
ci(bench): the benchmark jobs fail a pull request on a regression against its base, measured in the same job (#3612)
refactor(multimodal): the strict usize extra-key rule lives on the preprocessor config (#3625)
refactor(grpc_servicer): the four engine health servicers share one frame (#3639)
test(tokenizer): the render-fixture parity test names the four vocabularies whose reference split combining marks from their letters, with a test per script (#3615)
refactor(multimodal): one-per-item prompt replacements have a name, and five specs use it (#3626)
refactor(grpc_servicer): one reader for a host:port rendezvous flag, the SGLang and TokenSpeed probes built from it (#3623)
refactor(symphony): the JSON outline reads a value's bytes on the shape, not in the per-byte step (#3618)
refactor(worker): the registry transitions a worker's status in one primitive and states its transition table (#3586)
refactor(worker): the liveness code-set predicate is is_connection_code, named for what it judges (#3581)
test(tokenizer): bench the direct encoder's cost against the volume encoded and the piece cache's capacity cliff (#3574)
test(responses): a store whose reads fail while its writes land never takes a late result over a closed or deleted record (#3557)
test(symphony): the DeepSeek V4 table reads every turn the reference encoder renders as its own parser does (#3556)
test(grpc-client): the connect-failure tests dial a refused loopback port and a malformed endpoint (#3554)
test(tokenizer): the recovery-counter tests read the counters of their own thread (#3553)
test(servicer): the lifecycle tests say when a core exits and wait for a spawn with patience (#3551)
ci(tokenspeed): track the newest nightly again (#3547)
ci(aarch64): lint and test the crates with AArch64-only code on an arm64 runner on every pull request (#3546)
refactor(kv_index): the chain index is the only KV index, the positional indexer goes (#3544)
test: main's tests after #3525 and #3526, the renamed translation call and the namespace-assignment detection (#3535)
ci(evals): add AMD legs to the BFCL A/B (#2832)
test(grpc_servicer): one fixture loads the TokenSpeed servicer module for its tests (#3522)
test(e2e): a TokenSpeed worker on constrained_decoding always is constrained under auto (#3506)
refactor(grpc): the response specs are built once and read in place (#3508)

Docker images

(no engine images for v1.14.0-rc.1 yet: run scripts/release_docker_notes.sh v1.14.0-rc.1 once they are published)

smg-project/smg v1.14.0-rc.1 SMG v1.14.0-rc.1 on GitHub

SMG v1.14.0-rc.1

Features

Fixes

Performance

Documentation

Maintenance

Docker images

smg-project/smg v1.14.0-rc.1
SMG v1.14.0-rc.1

on GitHub