Upgrade Guide
To upgrade from previous versions of Rook, see the Rook upgrade guide.
Breaking Changes
- Kubernetes v1.28 is now the minimum version supported by Rook through the soon-to-be K8s release v1.33.
- Several ObjectBucketClaim options were added previously in Rook v1.16 that allowed more control over buckets. These controls allow users to self-serve their own S3 policies. Administrators may consider this flexibility a risk, depending on their environment. Rook now disables these options by default to ensure the safest off-the-shelf configurations. To enable the full range of OBC configurations, the new setting
ROOK_OBC_ALLOW_ADDITIONAL_CONFIG_FIELDSmust be set to enable users to set all of these options. For more details, see the OBC additionalConfig documentation. - First-class credential management added to CephObjectStoreUser resources, allowing multiple credentials and declarative credential rotation. For more details, see Managing User S3 Credentials. As a result, existing S3 users provisioned via CephObjectStoreUser resources no longer allow multiple credentials to exist on underlying S3 users, unless explicitly managed by Rook. Rook will purge all but one of the undeclared credentials. This could be a user observable regression for administrators who manually edited/rotated S3 user credentials for CephObjectStoreUsers, and affected users can make use of the new credential management feature as an alternative.
- Kafka notifications configured via CephBucketTopic resources will now default to setting the Kafka authentication mechanism to
PLAIN. Previously, no auth mechanism was specified by default. It was possible to set the auth mechanism viaCephBucketTopic.spec.endpoint.kafka.opaqueData. However, setting&mechanism=<auth type>viaopaqueDatais no longer possible. If any auth mechanism other thanPLAINis in use, modification toCephBucketTopicresources is required.
Features
- The name of a pre-existing Ceph RGW user account can be set as the bucket owner on an ObjectBucketClaim (OBC), rather than a unique RGW user being created for every bucket. A CephObjectStoreUser resource may be used to create the Ceph RGW user account which will be specified on the OBC. If the bucket owner is set on a bucket that already exists and is owned by a different user, the bucket will be re-linked to the specified user.
- The Ceph CSI 3.14 release has a number of features and improvements for RBD and CephFS volumes, volume snapshots, and many more areas. See the Ceph CSI 3.14 release notes for more details.
- External mons: In some two-datacenter clusters, there is no option to start an arbiter mon in an independent K8s node to configure a proper stretch cluster. The external mons now allow a mon to be configured outside the Kubernetes cluster, while Rook manages everything else inside the cluster. For more details, see the External Mon documentation. This feature is in currently in experimental mode.
- DNS resolution for mons: Allows clients outside the K8s cluster to resolve mon endpoints via DNS without requiring manual updates to the list of mon endpoints. This helps in scenarios such as virtual machine live migration. The Ceph client can connect to rook-ceph-active-mons..svc.cluster.local to dynamically resolve mon endpoints and receive automatic updates when mon IPs change. To configure this DNS resolution, see Tracking Mon Endpoints.
- Node-specific ceph.conf overrides: The ceph.conf overrides can now be customized per-node. This may be helpful for some ceph.conf settings that need to be unique per node depending on the hardware. This can be configured by creating a node-specific configmap that will be loaded for all OSDs and OSD prepare jobs on that node, instead of the default settings that are loaded from the rook-config-override configmap.