llm-d/llm-d-router v0.10.0-rc.1

on GitHub

• docs: add release-note fragment for PR #1657 by @llm-d-router-release-notes[bot] in #1660
• release: configure and pass LATENCY_PREDICTOR_TAG via Makefile by @liu-cong in #1675
• fix(sidecar): inject W3C trace context on outbound proxy requests by @gyliu513 in #1667
• Add operations.md guide for right sizing EPP container resources by @jtechapps in #1616
• deps(go): bump llm-d-kv-cache to v0.9.0 by @vMaroon in #1691
• fix(sidecar): allowlist selector parsing for GA InferencePool API by @pierDipi in #1696
• fix(standalone): require modelServers.matchLabels when InferencePool is disabled by @Jwrede in #1695
• test: consolidate and rename e2e runner scripts by @roytman in #1698
• feat(standalone): align agentgateway configuration with upstream GAIE PR 2905 by @abhay1999 in #1600
• fix(scripts): support single quotes in image check by @weizhoublue in #1676
• fix: resolve mismatched yq path for nested router subchart values by @weizhoublue in #1683
• docs: add release-note fragment for PR #1616 by @llm-d-router-release-notes[bot] in #1687
• docs: add release-note fragment for PR #1667 by @llm-d-router-release-notes[bot] in #1685
• ci: propagate HF_TOKEN to e2e tests to avoid HF Hub rate limits by @Jwrede in #1572
• ci: publish CRD manifests to release assets by @liu-cong in #1713
• deps(actions): bump the github-actions group across 1 directory with 3 updates by @dependabot[bot] in #1715
• deps(go): bump github.com/jellydator/ttlcache/v3 from 3.4.0 to 3.4.1 in the go-dependencies group by @dependabot[bot] in #1709
• fix: bump golang.org/x/crypto to v0.52.0 to resolve Trivy CVEs by @liu-cong in #1720
• refactor(flowcontrol): single-ended SafeQueue backed by container/heap by @LukeAVanDrie in #1690
• docs: add release-note fragment for PR #1690 by @llm-d-router-release-notes[bot] in #1725
• chore: switch default vllm port from 8001 to 8200 for sidecar by @zdtsw in #1723
• Fix inotify pre-flight check for PROM_ENABLED on macOS by @roytman in #1654
• feat(inflightloadproducer): allow configuring output ratio by @asharkhan3101 in #1731
• docs: add release-note fragment for PR #1731 by @llm-d-router-release-notes[bot] in #1743
• feat: Enable the disabling of feature gates that are by default on/active by @shmuelk in #1728
• fix(metrics): reset flow control metrics and correct deprecated references by @ishwar170695 in #1733
• fix(flowcontrol): resolve ManagedQueue eagerly and fix FlowKey.String by @RishabhSaini in #1747
• Update release instruction for fetching the existing release branch by @liu-cong in #1716
• docs: assemble release notes for v0.9.0 by @llm-d-router-release-notes[bot] in #1688
• docs: add release-note fragment for PR #1728 by @llm-d-router-release-notes[bot] in #1744
• fix(inflightload): prevent in-flight counter underflow on endpoint flap by @LukeAVanDrie in #1739
• fix(flowcontrol): return 503 (not 429) when no endpoints exist by @LukeAVanDrie in #1751
• feat(inflightload): bound output-token estimate by client and operator caps by @LukeAVanDrie in #1750
• docs: add release-note fragment for PR #1739 by @llm-d-router-release-notes[bot] in #1829
• docs: add release-note fragment for PR #1751 by @llm-d-router-release-notes[bot] in #1830
• docs: add release-note fragment for PR #1750 by @llm-d-router-release-notes[bot] in #1832
• test: Another step in merging the IGW and router end to end tests by @shmuelk in #1699
• fix: base release-notes assembly branch on main to exclude unrelated diffs by @liu-cong in #1827
• fix: wrong bundle version in artifacts CRDs by @zdtsw in #1742
• [docs] PR template use emphasis in AGENTS.md by @elevran in #1682
• docs(metrics): document all flow control metrics by @RishabhSaini in #1679
• feat(standalone): use typed labels.Selector for endpoint selector by @shichaooutlook in #1701
• fix: clear KV block index entries on endpoint removal by @RishabhSaini in #1738
• Align end to end test with current metrics by @shmuelk in #1841
• feat: implement DumpState for file-discovery by @thc1006 in #1836
• feat: implement DumpState for no-hit-lru-scorer by @thc1006 in #1837
• docs: add release-note fragment for PR #1836 by @llm-d-router-release-notes[bot] in #1847
• deps(actions): bump actions/cache from 5 to 6 in the github-actions group by @dependabot[bot] in #1845
• Ignore deleted IGW files for now by @shmuelk in #1864
• Fix syntax error in update to lychee config file by @shmuelk in #1869
• Do not build and push the dev image on a fork by @shmuelk in #1859
• Only check links in Markdown files, if Markdown files changed by @shmuelk in #1866
• Add filter tracing decorator to scheduling pipeline by @chethanuk in #1707
• feat(sidecar): add P2P connector for OffloadingConnector disaggregation by @nilig in #1888
• feat: add EPP podAnnotations passthrough to router Helm chart by @olgsm in #1883
• docs: add release-note fragment for PR #1883 by @llm-d-router-release-notes[bot] in #1891
• Ensure that plugins that depend on other plugins are instantiated after their dependencies by @shmuelk in #1876
• docs: add release-note fragment for PR #1876 by @llm-d-router-release-notes[bot] in #1898
• test: Final merge and cleanup of End to End tests by @shmuelk in #1852
• burst prefix-cache affinity for prompt-sharing requests by @ezrasilvera in #1727
• docs: remove dead Tokenization Architecture TOC entry by @anxkhn in #1901
• deps(actions): bump crate-ci/typos from 1.47.2 to 1.48.0 in the github-actions group by @dependabot[bot] in #1908
• ci: fix link checker never running on markdown changes by @anxkhn in #1914
• docs: update SIG-router Google Meet link in README by @nilig in #1925
• fix: prevent gRPC payload length overflow panic by @anxkhn in #1916
• epp: add --ha-populate-non-leader-datastore to serve from non-leaders by @mathias-kende in #1884
• Bump Go to 1.25.12 to fix crypto/tls vulnerability by @roytman in #1938
• docs: fix EPP config flag names in architecture guide by @anxkhn in #1941
• feat(epp): add p2p-source-producer for P2P prefix pulls by @nilig in #1924
• docs: add release-note fragment for PR #1884 by @llm-d-router-release-notes[bot] in #1932
• docs(epp): fix filter tutorial factory to use *json.Decoder by @anxkhn in #1902
• docs: add release-note fragment for PR #1888 by @llm-d-router-release-notes[bot] in #1890
• migrate: github.com/llm-d/coordinator @ 876a357 by @roytman in #1872
• fix(tokenizer): support render tokenizer for Anthropic /v1/messages by @zdtsw in #1896
• epp: add --drain-timeout for graceful shutdown by @mathias-kende in #1885
• test(e2e): add cache-affinity test for multimodal routing by @hexfusion in #1562
• Revert "test(e2e): add cache-affinity test for multimodal routing" by @hexfusion in #1955
• Fix lint failure in e2e manifest helpers by @liu-cong with @Copilot in #1951
• ci: add /rebase bot for maintainer-triggered PR rebases by @liu-cong in #1957
• feat: implement DumpState for approx-prefix-cache-producer by @thc1006 in #1835
• add ipv6 setting by @eating-chen in #1920
• Add support for length-based scoring in prefix cache plugin by @azamikram in #1459
• Add generic metric scorer for configured endpoint attributes by @sudoalok in #1620
• feat: add priority based holdback strategies in flow control by @loicmarchal in #1592
• test(e2e): retry post-recovery requests in decode-pod disruption spec by @nilig in #1961
• Add coordinator code owners by @roytman in #1945
• docs: add release-note fragment for PR #1592 by @llm-d-router-release-notes[bot] in #1969
• docs: add release-note fragment for PR #1459 by @llm-d-router-release-notes[bot] in #1967
• test(profilehandler): assert primaryPort in factory success cases by @anxkhn in #1915
• docs: add release-note fragment for PR #1885 by @llm-d-router-release-notes[bot] in #1947
• fix(epp): use real flag names in config-file/config-text error by @anxkhn in #1942
• Move coordinator pipeline-building helpers to pipeline/builder subpackage by @roytman in #1974
• server: add plugin name to default plugin state debug entry by @MOHITPARMAR007 in #1853
• test: cover the not-ready branch in TestPodReconciler by @anxkhn in #1905
• feat: add balanceBy tokens option to burst prefix cache producer by @ezrasilvera in #1978
• Add license scan report and status by @fossabot in #1933
• cleanup(epp): remove deprecated model-server-metrics CLI flags by @ishwar170695 in #1940
• docs: add release-note fragment for PR #1978 by @llm-d-router-release-notes[bot] in #1985
• test: Parallelize the End to End tests - Part one infrastructure by @shmuelk in #1976
• deps(go): bump sigs.k8s.io/structured-merge-diff/v6 from 6.3.2 to 6.3.3 in the kubernetes group by @dependabot[bot] in #1993
• deps(go): bump github.com/go-chi/chi/v5 from 5.3.0 to 5.3.1 in the go-dependencies group by @dependabot[bot] in #1992
• deps(actions): bump the github-actions group with 2 updates by @dependabot[bot] in #1994
• docs: fix broken markdown anchors by @Iceber in #1996
• feat: implement DumpState for predicted-latency-producer by @chethanuk in #1882
• Adding standard deviation metrics for monitoring epp pool load variance by @SachinVarghese in #1871
• add promptTokens-based PD decision to prefix-based-pd-decider by @learner0810 in #933
• Internalize kvcache and kvevents libraries from llm-d-kv-cache by @vMaroon in #1886
• feat: video token estimation formula for prefix caching by @namgyu-youn in #1408
• Add performance load tests to record EPP resource usage by @jtechapps in #1740
• [tests] Adds e2e tests for the /inference/v1/generate endpoint. by @revit13 in #1433
• Revert "feat: video token estimation formula for prefix caching (#1408)" by @capri-xiyue in #2012
• build: bump Go toolchain to 1.26.5 by @zdtsw in #1998
• [Perf Tests] Fix commit conflict by rebasing by @jtechapps in #2017
• Add picker tracing decorator to scheduling pipeline by @chethanuk in #1708
• feat(standalone): make helm --set proxyType=envoy|agentgateway work without overlay files by @abhay1999 in #1926
• Add /v1/images/generations support to openai-parser by @zetxqx in #1936
• feat(sidecar): inject kv_transfer_params.p2p from the KV cache source header by @nilig in #1937
• [Performance Test] Collect pprof profiles and push results to GCS bucket by @jtechapps in #2022
• test(e2e): wait for EPP pod discovery instead of sleeping by @elevran in #1988
• Fix Trivy SARIF scans failing on unfiltered severities by @elevran in #1989
• docs: add release-note fragment for PR #1871 by @llm-d-router-release-notes[bot] in #2001
• docs: add release-note fragment for PR #1886 by @llm-d-router-release-notes[bot] in #2010
• docs: add release-note fragment for PR #933 by @llm-d-router-release-notes[bot] in #2005
• docs: add release-note fragment for PR #1936 by @llm-d-router-release-notes[bot] in #2018
• feat(flowcontrol): add request outcome counter metric by @RishabhSaini in #1636
• Revert "[tests] Adds e2e tests for the /inference/v1/generate endpoint." by @ahg-g in #2015
• Trace scheduler profile execution by name by @AutuSnow in #1983
• Upgraded llm-d-inference-sim to v0.10.2 by @shmuelk in #2030
• fix(coordinator): force non-streaming prefill for chat completions by @roytman in #2002
• refactor(flowcontrol): unify on a single priority queue by @LukeAVanDrie in #1831
• feat(epp): expose per-tier cached-block counts in prefix match info by @nilig in #1962
• feat: consume InFlightLoadProducer in predicted-latency producer by @kaushikmitr in #1856
• refactor(flowcontrol): scrub shard vocabulary and rewrite docs to single-processor model by @LukeAVanDrie in #1846
• feat: deploy vllm-render as standalone service; use simulated render in e2e by @mayabar in #1601
• docs(sidecar): state that --enable-p2p-pull is rejected outside nixlv2 by @nilig in #2044
• datalayer: TLS options (CA verify + mTLS client cert) for metrics scrape by @hexfusion in #1858
• feat: implement DumpState for program-aware-fairness by @thc1006 in #1839
• feat: implement DumpState for mm-embeddings-cache-producer by @thc1006 in #1838
• fix(datalayer): set maxIdleTime to Transport.IdleConnTimeout by @Iceber in #2043
• ci: aggregate matrix job results into single status checks by @elevran in #2028
• docs: add release-note fragment for PR #1858 by @llm-d-router-release-notes[bot] in #2049
• docs: add release-note fragment for PR #1831 by @llm-d-router-release-notes[bot] in #2035
• chore(codeowners): add owners for kvcache and kvevents by @vMaroon in #2013
• docs: add release-note fragment for PR #1937 by @llm-d-router-release-notes[bot] in #2020
• docs(requestcontrol): correct FlowControlAdmissionController.Admit docstring by @chethanuk in #2009
• docs: document flow_control_requests_total metric by @Madhumasa84 in #2034
• epp: let parsers own model-name extraction and rewrite by @zetxqx in #1959
• epp: serve the metrics endpoint over mTLS by @hexfusion in #1919
• docs: add release-note fragment for PR #2034 by @llm-d-router-release-notes[bot] in #2050
• Extract agent-identity as a request metadata by @praveingk in #1754
• docs: add release-note fragment for PR #1919 by @llm-d-router-release-notes[bot] in #2056
• Add generic endpoint attribute filter by @sudoalok in #1999
• feat(epp): match whole prompt when both prefix caps are zero by @kaushikmitr in #2058
• Consolidating images and artifacts by @ahg-g in #2048
• feat: implement DumpState for precise-prefix-cache-producer by @thc1006 in #1840
• chore: remove deprecated sidecar flags by @zdtsw in #2054
• docs: add release-note fragment for PR #2054 by @llm-d-router-release-notes[bot] in #2071
• docs: add release-note fragment for PR #1840 by @llm-d-router-release-notes[bot] in #2070
• docs: add release-note fragment for PR #2048 by @llm-d-router-release-notes[bot] in #2063
• [tests] Adds e2e tests for the /inference/v1/generate endpoint. by @revit13 in #2029
• fix(epp): replace V(1) with logging.DEBUG in token-load-scorer by @Iceber in #2064
• [coordinator] e2e tests: containerize and share deploy components by @revit13 in #1975
• fix(sidecar): add startup validation for CLI port and replica options by @weizhoublue in #1619
• refactor(flowcontrol): remove the queue plugin registry by @LukeAVanDrie in #2036
• fix(epp/metrics): bound model-name label cardinality from user input by @gkneighb in #1909
• test: deduplicate GetFreePort helpers by @chethanuk in #2083
• docs: rewrite metrics.md for current EPP subsystems by @chethanuk in #2082
• clean unused code by @nirrozenbaum in #2078
• Cap max_tokens/max_completion_tokens on synthetic single-token requests by @roytman in #2086
• docs: add release-note fragment for PR #1909 by @llm-d-router-release-notes[bot] in #2087
• fix(sidecar): cap min_tokens on the prefill leg for chat completions by @vMaroon in #2041
• deps(go): bump the go-dependencies group with 2 updates by @dependabot[bot] in #2089
• test: Parallelize the End to End tests - Part two enablement by @shmuelk in #2033
• Add DCGM data source and extractor for GPU utilization by @noalimoy in #1895
• CI: Exploit parallel Router End to End testing in CI by @shmuelk in #2115
• deps(actions): bump the github-actions group across 1 directory with 6 updates by @dependabot[bot] in #2090
• feat(epp): add topology-extractor datalayer plugin by @elevran in #1678
• fix: Remove grafna dashboard from router by @gyliu513 in #2062
• docs: add release-note fragment for PR #1678 by @llm-d-router-release-notes[bot] in #2118
• test(flowcontrol): harden and reorganize the flow-control benchmark suite by @LukeAVanDrie in #1638
• fix(metrics): bound fairness_id label cardinality and prune GC'd flow series by @LukeAVanDrie in #2119
• fix(datastore): retry dropped endpoint registrations instead of silently untracking pods by @LukeAVanDrie in #2092
• deps(go): bump golang.org/x/text to v0.39.0 to fix GO-2026-5970 by @LukeAVanDrie in #2128
• deps(go): bump google.golang.org/grpc from 1.81.1 to 1.82.1 by @dependabot[bot] in #2136
• docs: add release-note fragment for PR #2119 by @llm-d-router-release-notes[bot] in #2126
• docs: align architecture.md terminology with the data layer by @chethanuk in #2084
• feat: added video support for approximate prefix match by @capri-xiyue in #1911
• test: harden port-bind race in EPP integration harness by @chethanuk in #2081
• [Helm] Support agentgateway in standalone service mode by @satyamg1620 in #1613
• test(flowcontrol): deflake integration tests that raced on fixed sleeps by @LukeAVanDrie in #2131
• feat(epp): add soft-reflective-ceiling usage limit policy by @kalantar in #1956
• test(flowcontrol): cover feature-gate wiring and the admission-to-controller seam by @LukeAVanDrie in #2129
• docs: add release-note fragment for PR #1956 by @llm-d-router-release-notes[bot] in #2150
• docs: add release-note fragment for PR #1911 by @llm-d-router-release-notes[bot] in #2144
• ci: Add /area and /milestone commands to PR label sync by @hickeyma in #2113
• test(coordinator): Fix request timeout for multimodal e2e specs by @revit13 in #2111
• docs: add release-note fragment for PR #1613 by @llm-d-router-release-notes[bot] in #2148
• fix(flowcontrol): apply effective defaults for DefaultRequestTTL and per-band MaxRequests by @LukeAVanDrie in #2123
• chore: add RishabhSaini as flow control code owner by @LukeAVanDrie in #2145
• docs: add release-note fragment for PR #2123 by @llm-d-router-release-notes[bot] in #2159
• refactor(flowcontrol): move the priority queue out of the plugins tree by @LukeAVanDrie in #2146
• docs: require monotone ceilings in UsageLimitPolicy contract by @LukeAVanDrie in #2149
• feat(concurrency-detector): Add hybrid mode to concurrency detector by @asharkhan3101 in #1991
• fix: flowcontrol - finalize requests during shutdown by @nicole-lihui in #2156
• Add more performance test configurations and analysis by @jtechapps in #2069
• Add EPP request/response processing latency metrics by @satyamg1620 in #1658
• fix(flowcontrol): add bounded request drop summaries by @ishwar170695 in #2132
• fix(sidecar): rename P2P kv_transfer_params sub-dict keys to match vLLM by @nilig in #2140
• feat(sidecar): support data parallelism with the offloading P2P tier by @nilig in #2075
• feat: add new --model-server-port to replace --vllm-port by @zdtsw in #2142
• docs: add release-note fragment for PR #2142 by @llm-d-router-release-notes[bot] in #2186
• feat(epp): load-aware p2p source selection by @nilig in #2032
• Add GPU utilization filter and scorer plugins by @noalimoy in #1904
• ci(coordinator): Align coordinator CI image build with the router's m… by @revit13 in #2155
• add header-phase-profile-handler scheduling plugin by @roytman in #2176
• Fix image tags presubmit by pinning zmq image version by @jtechapps in #2185
• ci: add /kind, /area, /milestone label sync for issues by @hickeyma in #2157
• feat(coordinator): serve encode/prefill/decode from a single EPP by @roytman in #2134
• deps(go): bump the kubernetes group with 6 updates by @dependabot[bot] in #2207
• deps(go): bump github.com/go-logr/logr from 1.4.3 to 1.4.4 in the go-dependencies group by @dependabot[bot] in #2206
• docs: add release-note fragment for PR #2176 by @llm-d-router-release-notes[bot] in #2198
• deps(go): bump github.com/google/cel-go from 0.26.1 to 0.29.0 by @dependabot[bot] in #2194
• docs: add release-note fragment for PR #1904 by @llm-d-router-release-notes[bot] in #2193
• docs: add release-note fragment for PR #2032 by @llm-d-router-release-notes[bot] in #2187
• docs: add release-note fragment for PR #2075 by @llm-d-router-release-notes[bot] in #2179
• docs: add release-note fragment for PR #2140 by @llm-d-router-release-notes[bot] in #2177
• docs: add release-note fragment for PR #1658 by @llm-d-router-release-notes[bot] in #2163
• docs: add release-note fragment for PR #2156 by @llm-d-router-release-notes[bot] in #2161
• docs: add release-note fragment for PR #1991 by @llm-d-router-release-notes[bot] in #2160
• docs: add SECURITY.md with vulnerability reporting policy by @arijitroy003 in #2189
• [CI]: Remove obsolete Prow GitHub Actions integration by @hickeyma in #2114
• feat(observability): add mm.modality, mm.hash_count, mm.hit to EPP scheduler spans by @hexfusion in #1546
• docs: add release-note fragment for PR #1546 by @llm-d-router-release-notes[bot] in #2211
• feat(epp): add active request filter by @kaushikmitr in #2209
• feat(epp): generalize active request filter into utilization filter by @kaushikmitr in #2218
• feat(epp): add encoder cache features to the latency predictor by @kaushikmitr in #2125
• docs: add release-note fragment for PR #2218 by @llm-d-router-release-notes[bot] in #2219
• fix(EPP): stream base64 image decoding to reduce memory by @Lucas-Fernandes-Martins in #2197
• fix(flowcontrol): return 429 on TTL eviction when pool has endpoints by @mtparet in #2047
• docs: add release-note fragment for PR #2047 by @llm-d-router-release-notes[bot] in #2224
• cicd/bug: query live labels in hold gate instead of (stale) event payload by @elevran in #2200
• docs: update probabilistic-admitter-epp-config.yaml header for flow c… by @Madhumasa84 in #2208
• fix: skip TPOT recording for non-streaming requests by @HipHoff in #2167
• fix: sync deprecated flow control env var into feature gate map by @elinacse in #2076
• feat(Coordinator) Support /inference/v1/generate user messages by @revit13 in #2077
• add subscriber and pool health metrics by @satyamg1620 in #2152
• docs: add release-note fragment for PR #2152 by @llm-d-router-release-notes[bot] in #2231
• docs: add release-note fragment for PR #2077 by @llm-d-router-release-notes[bot] in #2226
• docs: add release-note fragment for PR #2167 by @llm-d-router-release-notes[bot] in #2225
• Cap coordinator prefill/encode legs to a single token per schema by @revit13 in #2202
• add /test/coordinator to the coordinator code owners by @roytman in #2236
• fix(kvevents): attribute KV events to the subscriber serving endpoint by @nilig in #2233
• [CI]: Serialize kind/area label sync per PR and issue by @hickeyma in #2223
• test: wire test/framework packages into unit tests and coverage by @chethanuk in #2121
• fix(kvevents): honor configured engine adapter in precise prefix cache by @wukehu in #2153
• feat(epp): add --tls-min-version and --tls-cipher-suites flags by @andresllh in #2230
• docs: add release-note fragment for PR #2230 by @llm-d-router-release-notes[bot] in #2240
• feat(datalayer): configurable per-source scrape interval by @noalimoy in #2214
• docs: add release-note fragment for PR #2214 by @llm-d-router-release-notes[bot] in #2241
• fix(sidecar): rank-endpoint P2P pulls and wide-EP port-base compensation by @nilig in #2234
• fix(epp): release prefill request count at StartOfStream in inflight load producer by @kaushikmitr in #2243
• docs: add release-note fragment for PR #2243 by @llm-d-router-release-notes[bot] in #2247
• fix(flowcontrol): clamp overflowing SLO TTFT header in ordering by @anxkhn in #1917
• fix(flowcontrol): map pre-admission TTL/cancel to eviction statuses i… by @LukeAVanDrie in #2162
• fix(flowcontrol): make stale-metrics saturation observable and correct pool_saturation semantics by @LukeAVanDrie in #2165
• tokenizer: propagate trace context on vLLM render requests by @mschulist in #2245
• feat: emit per-endpoint scores in EPP dynamic metadata by @adelsam in #2004
• docs: add release-note fragment for PR #2004 by @llm-d-router-release-notes[bot] in #2254
• epp: generalize endpoint identity by @hexfusion in #2147
• fix: use renamed EndpointMetadata.ID in tests after #2147 by @hexfusion in #2256
• [CI][Security]: Run gosec via golangci-lint in the lint job by @hickeyma in #2203
• docs: add release-note fragment for PR #2165 by @llm-d-router-release-notes[bot] in #2250
• CrossReplicaSyncer for EPP by @Lucas-Fernandes-Martins in #1921
• test(coordinator): rewire e2e onto shared inference-gateway and add 3-EPP topology by @revit13 in #2253
• feat: implement EPP plugin stability lifecycle and --allow-experimental-plugins flag (#1912) by @liu-cong in #2073
• docs/configs: replace deprecated maxPrefixBlocksToMatch and blockSize with maxPrefixTokensToMatch and blockSizeTokens by @liu-cong in #2072
• deps(actions): bump the github-actions group with 2 updates by @dependabot[bot] in #2263
• deps(go): bump github.com/google/cel-go from 0.29.0 to 0.29.2 in the go-dependencies group by @dependabot[bot] in #2262
• docs: reflect flow controls GA posture (formerly experimental) by @LukeAVanDrie in #2180
• perf: Reduce allocations on the response-body streaming path by @albertoperdomo2 in #2252
• docs: add release-note fragment for PR #1917 by @llm-d-router-release-notes[bot] in #2249
• docs: add release-note fragment for PR #2073 by @llm-d-router-release-notes[bot] in #2260
• refactor(kvcache): relocate telemetry and utils to their common homes by @vMaroon in #2175
• epp: add multicluster scheduling plugin variants by @hexfusion in #2232
• [docs]: Define area/* label taxonomy by @hickeyma in #2213
• feat(epp): add OpenTelemetry spans for the scheduler scoring path by @mvanhorn in #1834
• Plugin: add request-control Screener extension point by @hasB4K in #2259
• New session_id_header for session affinity scorer plugin by @D-Sai-Venkatesh in #2238
• docs: add release-note fragment for PR #2238 by @llm-d-router-release-notes[bot] in #2278
• docs: add release-note fragment for PR #1834 by @llm-d-router-release-notes[bot] in #2270
• Add native DisaggregatedSet revision routing support by @hasB4K in #2141
• feat(epp): add port parameter to metrics-data-source by @kaushikmitr in #2290
• docs: add release-note fragment for PR #2290 by @llm-d-router-release-notes[bot] in #2291
• docs: add release-note fragment for PR #2141 by @llm-d-router-release-notes[bot] in #2288
• epp: multicluster config-load validation + review cleanups by @hexfusion in #2287
• ci: publish coordinator image alongside epp and sidecar by @roytman in #2304
• update helm chart to use preferred backends for gke by @rlakhtakia in #1681
• feat: replay buffer consumer for KV event index recovery by @RishabhSaini in #1887
• Allow plugins to contribute to EPP readiness by @hasB4K in #2307
• Add topology-aware filter and scorer for P/D routing by @elevran in #2299
• Llmd leader worker clean 2p2d enable by @shikamd123 in #2094
• fix(chart): restore missing --config-file value for EPP by @yuanwu2017 in #2322
• Skip janitor reaping for requests bound to a live context by @LukeAVanDrie in #2313
• fix(epp): select p2p pull sources by CPU-tier cached blocks by @nilig in #2321
• docs(multimodal): fix cache size default and add sizing guidance by @sudoalok in #2246
• docs: add release-note fragment for PR #2321 by @llm-d-router-release-notes[bot] in #2324
• tracing: honor OTLP TLS and auth environment variables by @satyamg1620 in #2312
• CICD: note SHA pin rationale for gosec SARIF upload action by @elevran in #2266
• feat(epp): add error return to PreRequest plugin interface by @dmitripikus in #2215
• feat(chart): allow extra args, init containers, and volumes for token… by @dkhachyan in #2139
• test: move GetFreePort into test/framework/net by @chethanuk in #2122
• fix: remove mid-stream TPOT predictions from predicted-latency producer by @MicheleCampi in #2037
• fix: lint in testcase by @zdtsw in #2331
• docs: add release-note fragment for PR #2037 by @llm-d-router-release-notes[bot] in #2327
• docs: document promptTokens parameter for PrefixBasedPDDecider by @roytman in #2328
• refactor(flowcontrol): UsageLimitPolicy fills a framework-owned ceilings buffer by @LukeAVanDrie in #2171
• refactor(flowcontrol): make queue statistics single-owner and remove the invariant panic by @LukeAVanDrie in #2170
• docs: add release-note fragment for PR #2171 by @llm-d-router-release-notes[bot] in #2336
• docs: replace links to the deleted upstream metrics guide by @marzmesas in #2339
• deps(actions): bump the github-actions group with 2 updates by @dependabot[bot] in #2344
• Added an option to disagg handler to allow running prefill first. by @ahg-g in #2342
• feat(flowcontrol): demand-driven in-flight eviction by @LukeAVanDrie in #2093
• docs: add release-note fragment for PR #2342 by @llm-d-router-release-notes[bot] in #2353
• Added inflight load metrics by @ahg-g in #2354
• docs: add release-note fragment for PR #2093 by @llm-d-router-release-notes[bot] in #2356

New Contributors

• @abhay1999 made their first contribution in #1600
• @ishwar170695 made their first contribution in #1733
• @shichaooutlook made their first contribution in #1701
• @thc1006 made their first contribution in #1836
• @chethanuk made their first contribution in #1707
• @olgsm made their first contribution in #1883
• @anxkhn made their first contribution in #1901
• @mathias-kende made their first contribution in #1884
• @eating-chen made their first contribution in #1920
• @azamikram made their first contribution in #1459
• @loicmarchal made their first contribution in #1592
• @MOHITPARMAR007 made their first contribution in #1853
• @fossabot made their first contribution in #1933
• @SachinVarghese made their first contribution in #1871
• @AutuSnow made their first contribution in #1983
• @Madhumasa84 made their first contribution in #2034
• @praveingk made their first contribution in #1754
• @kalantar made their first contribution in #1956
• @hickeyma made their first contribution in #2113
• @nicole-lihui made their first contribution in #2156
• @arijitroy003 made their first contribution in #2189
• @Lucas-Fernandes-Martins made their first contribution in #2197
• @mtparet made their first contribution in #2047
• @HipHoff made their first contribution in #2167
• @elinacse made their first contribution in #2076
• @wukehu made their first contribution in #2153
• @mschulist made their first contribution in #2245
• @mvanhorn made their first contribution in #1834
• @hasB4K made their first contribution in #2259
• @yuanwu2017 made their first contribution in #2322
• @dkhachyan made their first contribution in #2139
• @MicheleCampi made their first contribution in #2037
• @marzmesas made their first contribution in #2339