Special mentions
🚀 Rebase to Kubernetes 1.35.1 — #3842 (@xmudrii)
Foundation upgrade to Kubernetes 1.35.1 + Go 1.25.7. XL change touching the entire codebase — API adaptations, test adjustments, separate etcd lifecycle context to prevent shutdown blocking.
🔒 Cross-shard service account lookup — #3973 (@ntnn)
Enables service account validation across shards via a TTL cache. Removes the previous same-shard requirement for SAs. Drops the GlobalServiceAccount feature gate (now always-on).
⚙️ APIResourceSchema Virtual Workspace — #3881 (@mjudeikis)
New virtual workspace enabling providers to access consumer workspace schemas — key for kube-bind.io integration. Requires GlobalServiceAccounts and cross-workspace RBAC.
🔑 defaultSelector for PermissionClaim on APIExport — #3884 (@mjudeikis)
API change: providers can specify default permission claim selectors on APIExport that automatically apply when APIBindings are auto-created via WorkspaceType. Replaces a cache-replication approach (#3859) that had O(workspaces × bindings) scalability concerns.
🛟 Extract Virtual Workspace framework to staging repo — #3959 (@xmudrii)
Moves pkg/virtual/framework and pkg/virtual/options into github.com/kcp-dev/virtual-workspace-framework. External VW developers no longer need to vendor the entire kcp core repo. Also moves OpenAPI defs to SDK and crdpuller to the new repo.
🏋️ Load testing framework & infrastructure — #3796, #3866, #3895 (@SimonTheLeg)
Three-part effort: concept doc, k8s infra setup, and the framework itself. Inspired by clusterloader2, uses Go iterators for tuning sets, supports scenarios like "10,000 empty workspaces" with P99 stats.
🐛 Etcd key poisoning fix — #4011 (@mjudeikis)
Critical data integrity fix: unresolved workspace paths were poisoning etcd keys with malformed cluster names. Adds 404 handling and defense-in-depth filtering.
🔨 CLI permission claims management — #3956 (@rxinui) + #3946 (@ghdrope)
New kcp claims accept / kcp claims reject subcommands plus --accept-all-permission-claims / --reject-all-permission-claims flags on kubectl kcp bind. Significant UX improvement for API consumers.
🎁 SSA (Server-Side Apply) committer — #4002 (@swastik959)
Introduces Server-Side Apply support for controllers, fixing race conditions where JSON Merge Patch would lose concurrent condition updates.
🐞 VW proxy impersonation isolation — #4009 (@officialasishkumar)
Security fix: the shared ReverseProxy in VW was being mutated concurrently, causing impersonation header leakage between requests. Each request now gets an isolated proxy instance.
Honourable mentions
- External etcd for cache-server (#3831) (@gman0)
- Inter-shard auth enhancement (#3965) (@ntnn)
- Parallel resource installation cutting cold-start by ~5s (#3847) (@ntnn)
- Proxy readyz and livez endpoints (#3935) (@nuromirg)
- CVE fixes for opentelemetry SDK (#4013) and go-jose (#4014) (@xrstf)
Changes by Kind
Chore
- Bump google.golang.org/grpc [CVE-2026-33186]
- go.opentelemetry.io/otel/sdk [CVE-2026-24051] (#3972, @xrstf)
- Update go-jose to v3.0.5 [CVE-2026-34986] (#4014, @xrstf)
- Update opentelemetry SDK to 1.43.0 [CVE-2026-39883] (#4013, @xrstf)
- Update to Go 1.25.9 (#4003, @xrstf)
API Change
- Action: Add
defaultSelectorfield toPermissionClaimonAPIExport. When APIBindings are auto-created viaWorkspaceType.defaultAPIBindings, the selector is now taken from the APIExport'sdefaultSelectorinstead of defaulting tomatchAll: true. Existing APIExports withoutdefaultSelectorretain the previousmatchAll: truebehavior. (#3884, @mjudeikis) - Update kcp to Kubernetes 1.35.1
Feature
- Add
/readyznow usesNewInformerSyncHealthz
Add/liveznow usesPingHealthz(#3935, @nuromirg) - Add apiresourceschema virtual workspace for schema access from provider side. (#3881, @mjudeikis)
- Added --accept-all-permission-claims and --reject-all-permission-claims flags for BindOptions. (#3946, @ghdrope)
- Cache-server: added etcd flags
sharded-test-server: added --cache-kubeconfig flag to use an external cache-server (#3831, @gman0) - Enable
kcp claims acceptandkcp claims reject(#3956, @rxinui) - Enable cross-shard service account validation (#3973, @ntnn)
- Extract
pkg/virtual/frameworkandpkg/virtual/optionspackages into a dedicated staging repository (github.com/kcp-dev/virtual-workspace-framework - The compat CLI now supports -old-version and -new-version flags to select which CRD versions to compare. When omitted, it defaults to the first version as before. (#3943, @nuromirg)
Bug or Regression
- BREAKING CHANGE:
Fix {cluster} extract logic for VirtualWorkspaces. Previously, if VirtualWorkspace, used in FrontProxy mapping, had a pathcluster/{cluster}- it was not resolved, and so WorkspaceAuthorizationConfiguration was not run if used inside FrontProxy, but forwarded to VirtualWorkspace without checking. As a result, if one has misconfigured VirtualWorkspace, it might receive traffic intended for another recipient. (#3857, @mjudeikis) - Fix
--shard-virtual-workspace-url,--shard-virtual-workspace-ca-file,--shard-client-key-file,--shard-client-cert-filenot being taken into account when disabling the in-process kcp virtual workspaces server on a shard. (#3955, @xrstf) - Fix an etcd key-corruption bug where an unresolvable multi-segment workspace path in
/clusters/<path>/...on a shard could cause resources to be written to etcd under a key segment containing the raw workspace path instead of the logical cluster name, producing orphaned rows invisible to the normal API read path. The shard now returns 404 for unresolvable workspace paths, and a new defense-in-depth filter rejects any request whose context carries a path-shaped cluster name before it can reach storage. (#4011, @mjudeikis) - Fix concurrent map writes panic in apiexport virtual workspace when
multiple requests share the same user.Info reference. (#3856, @dweidenfeld) - Fix external cache bootstrapping issues that sometimes prevented shards from bootstrapping successfully. (#3974, @xrstf)
- Fix external virtual workspace proxying so concurrent
/services/...requests keep impersonation headers request-scoped. (#4009, @officialasishkumar) - Fix very rare openapi-related panic during startup. (#3833, @xrstf)
- Fixed Inherited APIBindings now inherit permission claim selectors from parent workspaces instead of defaulting to matchAll: true. (#3786, @olamilekan000)
- Fixed TestAPIExportAPIBindingsAccess error when trying to create a ws in a shard setup (#3817, @olamilekan000)
- Fixed events.k8s.io permission denial through APIExport virtual workspace. (#3894, @cnvergence)
- Send initial-events-end bookmark for CachedResource virtual storage (#3875, @maxpain)
- Strip scopes from ServiceAccount tokens in maximal permission policy check (#3867, @mjudeikis)
- Update build version to v1.24.13 for CVE-2025-68121 (#3864, @ntnn)
- Workspace plugin: fix a bug where calling the 'tree' command on a parent workspace that has deleting children can result in a 403 error. (#3843, @neolit123)
Other (Cleanup or Flake)
- All kcp binaries in the container images now have their debugging symbols stripped, saving roughly 25% in total image size. (#3898, @xrstf)
- Change "ts" in JSON logging format to be ISO 8601 instead of UNIX timestamps.
- Deprecate
--external-hostname, determined based on--shard-base-urlor--bind-addressinstead (#3832, @ntnn) - Deprecated the unused flag
--shard-external-urlforvirtual-workspace(#3849, @ntnn) - Internal: consolidate identity secret generation and hashing logic into pkg/identity (#3937, @ghdrope)
Dependencies
Added
- cyphar.com/go-pathrs: v0.2.1
- github.com/Masterminds/semver/v3: v3.4.0
- github.com/jellydator/ttlcache/v3: v3.4.0
- go.uber.org/automaxprocs: v1.6.0
- gonum.org/v1/gonum: v0.17.0
Changed
- cel.dev/expr: v0.24.0 → v0.25.1
- cloud.google.com/go/compute/metadata: v0.6.0 → v0.9.0
- github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp: v1.26.0 → v1.31.0
- github.com/Microsoft/hnslib: v0.1.1 → v0.1.2
- github.com/cncf/xds/go: 2f00578 → ee656c7
- github.com/containerd/containerd/api: v1.8.0 → v1.9.0
- github.com/containerd/ttrpc: v1.2.6 → v1.2.7
- github.com/containerd/typeurl/v2: v2.2.2 → v2.2.3
- github.com/coredns/corefile-migration: v1.0.26 → v1.0.29
- github.com/cyphar/filepath-securejoin: v0.5.1 → v0.6.0
- github.com/envoyproxy/go-control-plane/envoy: v1.32.4 → v1.36.0
- github.com/envoyproxy/go-control-plane: v0.13.4 → v0.14.0
- github.com/envoyproxy/protoc-gen-validate: v1.2.1 → v1.3.0
- github.com/go-jose/go-jose/v3: v3.0.4 → v3.0.5
- github.com/go-jose/go-jose/v4: v4.0.4 → v4.1.3
- github.com/go-logr/logr: v1.4.2 → v1.4.3
- github.com/golang/glog: v1.2.4 → v1.2.5
- github.com/google/cadvisor: v0.52.1 → v0.53.0
- github.com/google/pprof: d1b30fe → 27863c8
- github.com/kcp-dev/embeddedetcd: v1.1.0 → 2cc5c5c
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/api: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiextensions-apiserver: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/apimachinery: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/cli-runtime: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/client-go: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/cloud-provider: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/cluster-bootstrap: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/code-generator: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/component-base: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/component-helpers: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/controller-manager: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/cri-api: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/cri-client: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/csi-translation-lib: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/dynamic-resource-allocation: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/endpointslice: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/externaljwt: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kms: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kube-aggregator: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kube-controller-manager: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kube-proxy: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kube-scheduler: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kubectl: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/kubelet: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/metrics: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/mount-utils: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/pod-security-admission: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes/staging/src/k8s.io/sample-apiserver: 4b3495f → ba4f7d4
- github.com/kcp-dev/kubernetes: 4b3495f → ba4f7d4
- github.com/onsi/ginkgo/v2: v2.21.0 → v2.27.2
- github.com/onsi/gomega: v1.35.1 → v1.38.2
- github.com/opencontainers/cgroups: v0.0.1 → v0.0.3
- github.com/opencontainers/runtime-spec: v1.2.0 → v1.2.1
- github.com/prometheus/client_golang: v1.22.0 → v1.23.2
- github.com/prometheus/client_model: v0.6.1 → v0.6.2
- github.com/prometheus/common: v0.62.0 → v0.66.1
- github.com/prometheus/procfs: v0.15.1 → v0.16.1
- github.com/rogpeppe/go-internal: v1.13.1 → v1.14.1
- github.com/spf13/cobra: v1.9.1 → v1.10.0
- github.com/spf13/pflag: v1.0.6 → v1.0.9
- github.com/spiffe/go-spiffe/v2: v2.5.0 → v2.6.0
- github.com/stretchr/testify: v1.10.0 → v1.11.1
- go.etcd.io/bbolt: v1.4.2 → v1.4.3
- go.etcd.io/etcd/api/v3: v3.6.4 → v3.6.5
- go.etcd.io/etcd/client/pkg/v3: v3.6.4 → v3.6.5
- go.etcd.io/etcd/client/v3: v3.6.4 → v3.6.5
- go.etcd.io/etcd/pkg/v3: v3.6.4 → v3.6.5
- go.etcd.io/etcd/server/v3: v3.6.4 → v3.6.5
- go.opentelemetry.io/auto/sdk: v1.1.0 → v1.2.1
- go.opentelemetry.io/contrib/detectors/gcp: v1.34.0 → v1.39.0
- go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.58.0 → v0.61.0
- go.opentelemetry.io/otel/metric: v1.35.0 → v1.43.0
- go.opentelemetry.io/otel/sdk/metric: v1.34.0 → v1.43.0
- go.opentelemetry.io/otel/sdk: v1.34.0 → v1.43.0
- go.opentelemetry.io/otel/trace: v1.35.0 → v1.43.0
- go.opentelemetry.io/otel: v1.35.0 → v1.43.0
- go.yaml.in/yaml/v2: v2.4.2 → v2.4.3
- golang.org/x/crypto: v0.46.0 → v0.47.0
- golang.org/x/net: v0.48.0 → v0.49.0
- golang.org/x/sys: v0.39.0 → v0.42.0
- golang.org/x/term: v0.38.0 → v0.39.0
- golang.org/x/text: v0.32.0 → v0.33.0
- google.golang.org/genproto/googleapis/api: a0af3ef → b8f7ae3
- google.golang.org/genproto/googleapis/rpc: a0af3ef → b8f7ae3
- google.golang.org/grpc: v1.72.1 → v1.80.0
- google.golang.org/protobuf: v1.36.5 → v1.36.11
- gopkg.in/evanphx/json-patch.v4: v4.12.0 → v4.13.0
- k8s.io/gengo/v2: 85fd79d → ec3ebc5
- k8s.io/kube-openapi: f3f2b99 → 589584f
- k8s.io/system-validators: v1.10.2 → v1.12.1
- k8s.io/utils: 4c0f3b2 → bc988d5
- sigs.k8s.io/json: cfa47c3 → 2d32026
Removed
- github.com/zeebo/errs: v1.4.0
- gopkg.in/yaml.v2: v2.4.0