Docker image is available at:
docker pull ghcr.io/llm-d/llm-d-inference-scheduler:v0.5.0Notable
- Prefill/Decode disaggregation awareness in filters and scorers
- Support for data parallel serving validated with vLLM and inference-sim
- Various CICD enhancements
- IGW: Flow control scale from/to zero support
- IGW: Standalone EPP
What's Changed
- Fix for flaky sites during lychee md link checker by @pierDipi in #485
- deps(actions): bump actions/checkout from 5 to 6 by @dependabot[bot] in #487
- deps(go): bump google.golang.org/grpc from 1.76.0 to 1.77.0 in the go-dependencies group by @dependabot[bot] in #486
- Use kv-cache-manager based on Go mod version instead of hardcoded by @pierDipi in #484
- update llm-d-kv-cache version to v0.4.0 by @vMaroon in #492
- fix: github action missing Trivy scan on sidecar image by @zdtsw in #481
- [Fix] Enhance macOS Makefile to Support Non-Homebrew Python Installations by @hyeongyun0916 in #489
- feat(allowlist): support both v1 and v1alpha2 InferencePool APIs with flag by @googs1025 in #474
- fix: make 'install-dependencies' and 'build' target by @zdtsw in #493
- skip lint and test when only docs change by @setsunakute in #494
- fix: Fixes for Data Parallel support when also running with Prefix Disaggregation by @shmuelk in #498
- sync gie to v1.2.0 by @nirrozenbaum in #499
- Error if PYTHON_CONFIG is empty by @elevran in #497
- Add GH action to check for signed and verified commits in PR by @elevran in #500
- deps(actions): bump crate-ci/typos from 1.39.2 to 1.40.0 by @dependabot[bot] in #501
- build: make build should use CGO_CFLAGS, CGO_LDFLAGS by @evacchi in #503
- chore: bump gie to v1.2.1 by @nirrozenbaum in #504
- deps(go): bump sigs.k8s.io/gateway-api from 1.4.0 to 1.4.1 in the kubernetes group by @dependabot[bot] in #508
- deps(go): bump the go-dependencies group with 3 updates by @dependabot[bot] in #507
- Miscellaneous dependency updates by @shmuelk in #510
- deps(go): bump the kubernetes group with 5 updates by @dependabot[bot] in #513
- Fix running
make env-dev-kindby @acardace in #512 - test: add precise_prefix_cache_test by @evacchi in #505
- test: reuse upstream data store and enable logr in unit tests by @MregXN in #518
- feat: allow pd_profile_handler to handle diverse plugin types by @hyeongyun0916 in #516
- deps(actions): bump crate-ci/typos from 1.40.0 to 1.40.1 by @dependabot[bot] in #526
- deps(go): bump google.golang.org/grpc from 1.77.0 to 1.78.0 in the go-dependencies group by @dependabot[bot] in #527
- feat(metrics): add model_name label to PD decision metric by @googs1025 in #528
- deps(actions): bump crate-ci/typos from 1.40.1 to 1.41.0 by @dependabot[bot] in #532
- Configure dependabot ignores Go version updates by @elevran in #533
- Updates the architecture description by @davidbreitgand in #525
- Dependabot: exert finer control over package updates by @elevran in #542
- port auto-assign action from llm-d-kv-cache by @vMaroon in #551
- refactor: set python version and pin docker image with tag by @zdtsw in #543
- chore(test): update API version for nixl test by @zdtsw in #555
- deps(go): bump the go-dependencies group with 2 updates by @dependabot[bot] in #558
- deps(actions): bump crate-ci/typos from 1.41.0 to 1.42.0 by @dependabot[bot] in #557
- deps(actions): bump actions/checkout from 4 to 6 by @dependabot[bot] in #556
- Update auto-assign logic by @elevran in #560
- Remove newline in unsigned commit message by @elevran in #561
- bump gie to v1.3.0 rc2 by @nirrozenbaum in #562
- Update OWNERS by @elevran in #559
- refactor: Makefile, update docs by @zdtsw in #463
- feat: add metrics validation in e2e test by @googs1025 in #529
- feat: make no-hit-lru P/D-aware by @evacchi in #522
- Update disaggregated Prefill/Decode inference serving documentation by @mayabar in #571
- deps(actions): bump crate-ci/typos from 1.42.0 to 1.42.1 by @dependabot[bot] in #572
- deps(go): bump github.com/onsi/ginkgo/v2 from 2.27.4 to 2.27.5 in the go-dependencies group by @dependabot[bot] in #573
- fix reviewers auto assign minor bug by @nirrozenbaum in #575
- fix(scorer): make active request pd aware by @kyanokashi in #569
- test(e2e): cleanup kind cluster by @zdtsw in #563
- refactor: add early validation in DP profile handler by @zdtsw in #554
- deps(go): bump the kubernetes group with 2 updates by @dependabot[bot] in #574
- refactor: kv cache manager repo by @sagearc in #570
- bumping IGW version to the full released version by @kfswain in #583
- Enable prefix-cache awareness in active-active multi-replica scheduler deployments by @vMaroon in #578
- Switch to pre-built vLLM wheels for CPU builds by @sagearc in #582
- update llm-d-kv-cache import to v0.5.0-RC1 by @vMaroon in #584
- Use 1.3.0 CRDs by @shmuelk in #586
Updates in Inference Gateway Extension v1.3.0
llm-d-inference-scheduler v0.5.0 has been updated to use the latest version of the Inference Gateway Extension which is 1.3.0.
You can see those changes here
New Contributors
- @setsunakute made their first contribution in #494
- @evacchi made their first contribution in #503
- @acardace made their first contribution in #512
- @MregXN made their first contribution in #518
- @davidbreitgand made their first contribution in #525
- @kyanokashi made their first contribution in #569
- @sagearc made their first contribution in #570
Full Changelog: v0.4.0...v0.5.0