⚠ Breaking Changes ⚠
Unified Helm chart for simplified deployment
We've streamlined the deployment process by consolidating all different vCluster Helm charts (vcluster, vcluster-k8s, vcluster-k0s, and vcluster-eks) into a single, unified chart. This change is designed to simplify management and upgrading of virtual clusters:
- Single source: No more juggling multiple charts.
- Value conversion: A new vCluster CLI command to convert vCluster v0.19 to v0.20 values is provided (view conversion guide)
- Enhanced validation: We've introduced a values schema JSON to the Helm chart, ensuring that upgrades will only proceed if your configuration matches the expected format to reduce deployment errors.
- Customizable distributions: The new unified chart structure enables easier customization of Kubernetes distributions directly via the Helm chart values:
controlPlane:
distro:
k8s:
enabled: true
New intuitive vcluster.yaml
configuration & docs
We're excited to introduce the new vcluster.yaml
file, replacing the previous Helm values.yaml
. This new configuration features a completely revamped format designed to enhance the user experience:
- Validation: The vCluster CLI and Platform UI now validate configurations when creating virtual clusters. In addition, most IDEs will now automatically provide validation and autocomplete for vCluster configurations.
- Consolidated configuration: All configurations are centralized in the vcluster.yaml file, eliminating confusion previously caused by the mix of CLI flags and Helm values. Please note, this release has a set of unsupported CLI flags (view release notes) however, the vCluster CLI
vcluster convert config
command makes it easy to transition to the new vcluster.yaml format. - Renamed fields: We've updated field names to be more intuitive, making them easier to understand and remember.
- Reorganized structure: Fields are now logically grouped under topical categories, simplifying navigation and enhancing discoverability of related features.
- Docs alignment: Our documentation now mirrors the structure of
vcluster.yaml
, making it easier to cross-reference settings within the file and corresponding sections in the docs.
New vCluster CLI command to convert old values to vcluster.yaml
In order to make it easy to convert your old values (pre-v0.20) to the new vcluster.yaml
format, you can leverage the new CLI command: vcluster convert config
command. For example, let's take these pre-v0.20 configuration values:
service:
type: NodePort
sync:
nodes:
enabled: true
Passing the above old values using the vCluster CLI command vcluster convert config --distro k8s < /path/to/this/file.yaml
will generate the following values:
controlPlane:
backingStore:
etcd:
deploy:
enabled: true
distro:
k8s:
enabled: true
service:
spec:
type: NodePort
statefulSet:
scheduling:
podManagementPolicy: OrderedReady
sync:
fromHost:
nodes:
enabled: true
View configuration conversion guide
Vanilla K8s is now the default distribution
We changed the default distribution for the vCluster control plane from K3s to K8s. This is the least opinionated option, offering greater flexibility and compatibility:
- Flexibility: More customization and scalability options, catering to a broader range of deployment needs.
- Compatibility: In addition to embedded and external etcd, you can now use various storage backends including SQLite, Postgres, and MySQL. This addition addresses previous challenges with using K8s for smaller virtual clusters.
Embedded SQLite is now the default backing store
Embedded SQLite has been set as the default backing store for the K8s distribution. This is to simplify operations and enhance performance for smaller virtual clusters:
- Efficiency: SQLite offers a more lightweight solution for data storage without the overhead associated with more complex choices like etcd.
- Simplicity: Setup is more straightforward, reducing the complexity and time required to get virtual clusters up and running.
Continued Support for etcd: For users with larger deployments or those needing more advanced features, external etcd deployed by vCluster remains a fully supported option:
controlPlane:
distro:
k8s:
enabled: true
backingStore:
etcd:
deploy:
enabled: true
Pro is now the default image
We've updated the default image for vCluster to ghcr.io/loft-sh/vcluster-pro
. This change allows users to seamlessly test and adopt vCluster Pro features without disrupting the existing open-source functionality. The Pro features are integrated into the Pro image but remain inactive by default to ensure that your experience remains consistent with the open-source version unless you specifically activate Pro features.
For users who prefer using the open-source image, simply adjust your vcluster.yaml
configuration to use ghcr.io/loft-sh/vcluster-oss
:
controlPlane:
statefulSet:
image:
repository: ghcr.io/loft-sh/vcluster-oss
Ingress syncing behavior has changed
Pre-v0.20.0-beta.1, when you enabled syncing Ingresses from the virtual to the host cluster, it would also automatically sync all IngressClasses from the host cluster. However, this required a cluster role which some vCluster users don’t have. We’ve now decoupled these syncing behaviors so you can individually enable syncing Ingresses as well as IngressClasses separately.
sync:
toHost:
ingresses:
enabled: true
fromHost:
ingressClasses:
enabled: true
Unsupported CLI flags
vcluster create
Flags:
--disable-ingress-sync
--extra-values
--labels
--annotations
--project
--cluster
--template
--template-version
--link
--params
--set-param
--disable-pro
vcluster delete
Flags:
--project
vcluster connect
Flags:
--project
vcluster pause
Flags:
--project
--prevent-wakeup
vcluster resume
Flags:
--project
vcluster start
Flags:
--kube-config-context-name
--sync
--request-header-ca-cert
--client-ca-cert
--server-ca-cert
--server-ca-key
--kube-config
--tls-san
--out-kube-config-secret
--out-kube-config-secret-namespace
--out-kube-config-server
--target-namespace
--service-name
--set-owner
--name
--bind-address
--port
--sync-all-nodes
--enable-scheduler
--disable-fake-kubelets
--fake-kubelet-ips
--node-clear-image-status
--translate-image
--enforce-node-selector
--enforce-toleration
--node-selector
--service-account
--override-hosts
--override-hosts-container-image
--cluster-domain
--leader-elect
--lease-duration
--renew-deadline
--retry-period
--disable-plugins
--plugin-listen-address
--default-image-registry
--enforce-pod-security-standard
--sync-labels
--plugins
--map-virtual-service
--map-host-service
--host-metrics-bind-address
--virtual-metrics-bind-address
--mount-physical-host-paths
--multi-namespace-mode
--namespace-labels
--sync-all-configmaps
--sync-all-secrets
--proxy-metrics-server
--service-account-token-secrets
--rewrite-host-paths
--sync-node-changes
--fake-kubelets
--fake-nodes
--fake-persistent-volumes
--enable-storage-classes
--enable-priority-classes
--suffix
--owning-statefulset
--disable-sync-resources
--pro-license-secret
--remote-kube-config
--remote-namespace
--remote-service-name
--integrated-coredns
--use-coredns-plugin
--noop-syncer
--sync-k8s-service
--etcd-embedded
--migrate-from
--etcd-replicas
--enforce-validating-hook
--enforce-mutating-hook
All Changes
Features
- merge vCluster charts & new values.yaml by @FabianKramm in #1583
- print error on cli connection problems by @eumel8 in #1594
- feat: allow external etcd for k3s & k0s by @FabianKramm in #1620
- feat: allow embedded sqlite for k8s & default k8s by @FabianKramm in #1626
- added draft for interceptor plugin by @facchettos in #1612
- feat: add migration function by @FabianKramm in #1646
- feat: add "migrate values" command to migrate to new values format by @johannesfrey in #1671
- feat: show a message if user attempts to use pro / platform features by @lizardruss in #1677
- Prevent distro and backingstore switch by @johannesfrey in #1668
Bugfixes
- fix: IPFamilyPolicy not synced for default vcluster service by @rohantmp in #1592
- fix: only apply deploy config once by @FabianKramm in #1600
- fix: delete unused nodes correctly by @FabianKramm in #1607
- fix: override endpoints if service selector is empty by @FabianKramm in #1608
- Fix storage config docs that referenced old yaml structure by @heiko-braun in #1624
- docs: fix helm install command by @pdbrito in #1614
- Files missing from CLI docs regen by @heiko-braun in #1631
- fix: affinity label selector overriden by namespace selector by @rohantmp in #1609
- Fix another chart reference by @johannesfrey in #1649
- Fix comment about default distro by @johannesfrey in #1650
- fix: multi-namespace mode configmap sync by @FabianKramm in #1657
- Fix vcluster create & support orbstack for direct connection by @FabianKramm in #1667
- now takes into account the parameters from the kubeconfig config part by @facchettos in #1672
- now sets up the controller with the exportkubeconfig namespace if it is set by @facchettos in #1676
- fix: add coredns version map with key equales to 1.26 item by @LinPr in #1673
- fix: unhandled error in endpoints sync by @FabianKramm in #1681
- fix: migrate sync ingresses correctly by @FabianKramm in #1685
- fix: migrate command name has changed by @johannesfrey in #1690
- fix: rename embed chart by @FabianKramm in #1691
- fix: let pro commands fail instead of succeed by @FabianKramm in #1692
Refactoring
- refactor: disabled -> enabled: auto & telemetry refactor by @FabianKramm in #1625
- refactor: make function replaceable by @FabianKramm in #1628
- refactor: improvements for plugins by @FabianKramm in #1629
- refactor: rename embeddedSqlite to embedded by @FabianKramm in #1640
- refactor: add concrete webhook types by @FabianKramm in #1643
- refactor: changes for vcluster-sdk by @FabianKramm in #1645
- refactor: make distro optional for default values by @FabianKramm in #1653
- refactor: remove special pro functionality from cli by @FabianKramm in #1665
- refactored pluginconfig by @facchettos in #1664
Chores
- chore: bump values schema by @FabianKramm in #1597
- Schema comments: change vCluster to virtual cluster when not referring to the product itself by @aimeeu in #1598
- license(vCluster): Updated OSS licenses by @loft-bot in #1596
- build: add sync config workflow by @FabianKramm in #1603
- build: update sync-config.yaml by @FabianKramm in #1604
- build: update sync-config.yaml by @FabianKramm in #1605
- build: update sync-config.yaml by @FabianKramm in #1606
- Regenerate CLI docs by @heiko-braun in #1630
- chore: move ipblock comment to field instead of type definition by @pascalbreuninger in #1639
- chore(deps): bump azure/setup-helm from 3 to 4 by @dependabot in #1637
- Added build target to generate schema by @heiko-braun in #1635
- docs: suggest required helm values for scraping kubelet targets in Isolated mode by @neogopher in #1642
- Change chart links by @johannesfrey in #1648
- chore: generate values.yaml comments by @FabianKramm in #1655
- Explicitly mention that Helm must be >= v3.10.0 by @johannesfrey in #1656
- Define global in subchart json schema by @joebowbeer in #1652
- moved withinterceptor so it can catch the info from the context by @facchettos in #1658
- chore: prevent distro switch from k0s by @FabianKramm in #1659
- chore: fix function name in comment by @clonefetch in #1661
- replaced handlername with interceptor wide name by @facchettos in #1662
- Do not encode distro name into Chart URL by @johannesfrey in #1660
- build: release ghcr.io/loft-sh/vcluster-oss by @FabianKramm in #1663
*(config): add pro admonition to jsonschema based on product:pro go tag: by @pascalbreuninger in #1670 - chore: validate passed in values files before passing it to helm by @johannesfrey in #1675
- added check for k0s to make sure we don't use incompatible options by @facchettos in #1674
- chore: adjust helm version check to be more explicit by @johannesfrey in #1680
- Add product:pro to all pro features in config.go by @deniseschannon in #1679
- chore: prevent certain pro commands from executing with v0.20 by @FabianKramm in #1683
- Update pro flags in config.go by @deniseschannon in #1684
- chore: perform early exit when no values file is given for vcluster < v0.20 by @johannesfrey in #1686
- chore: rename vcluster migrate values to vcluster convert config by @johannesfrey in #1688
New Contributors
- @eumel8 made their first contribution in #1594
- @pdbrito made their first contribution in #1614
- @joebowbeer made their first contribution in #1652
- @clonefetch made their first contribution in #1661