strimzi/strimzi-kafka-operator 0.48.0-rc1

on GitHub

⚠️ Important: Strimzi 0.48.0 supports only Kubernetes 1.27 and newer. Kubernetes 1.25 and 1.26 are not supported anymore

Main changes since 0.47.0

This release contains the following new features and improvements:

• Add support for Kafka 4.1.0.
  Remove support for Kafka 3.9.0 and 3.9.1.
• KRaft mode and Kafka Node Pools are now enabled by default.
  The strimzi.io/node-pools and strimzi.io/kraft annotations are not required anymore and will be ignored if set.
• Make properties broker.session.timeout.ms, broker.heartbeat.interval.ms and controller.socket.timeout.ms configurable
• Add monitoring of custom resources using kubernetes-state-metrics (KSM) (see Strimzi proposal 087)
• Ignore users (their ACLs, Quotas and SCRAM-SHA-512 credentials) managed by some other tools based on a configurable pattern in User Operator
• Support for type: custom client authentication (to make it easier to use custom authentication mechanisms such as AWS IAM)
• Added support for Strimzi Metrics Reporter to Kafka Connect, Mirror Maker 2 and Kafka Bridge.
• Add new feature gate ServerSideApplyPhase1 (disabled by default) that adds support for Server Side Apply for ConfigMap, Ingress, PVC, Service, and ServiceAccount according to Strimzi Proposal #105.
• Added distinction between changes of "cluster-wide" broker properties applied dynamically at cluster level, and "per-broker" broker properties applied dynamically at broker level.
• Extend the EntityOperator, Cruise Control and KafkaExporter deployment to support PDB via the template section in the CR spec.
• Added support for KIP-1073
  to get the list of the registered brokers by using the Kafka Admin API. It replaces the usage of the .status.registeredNodeIds field in Kafka.
• Added support for KIP-745 in Kafka Connector and Mirror Maker 2, allowing the usage of
  includeTasks and onlyFailed arguments in Kafka connectors restart.
• Update OAuth library to 0.17.0.
• Additional OAuth configuration options have been added for 'oauth' authentication on the listener and the client.
  On the listener clientGrantType has been added.
  On the client grantType has been added.

Major changes, deprecations and removals

• Fix RBAC naming for KafkaMirrorMaker2 to avoid RoleBinding collisions when a KafkaConnect with the same name exists in the same namespace. KafkaMirrorMaker2 now uses dedicated RoleBinding names.

  Upgrade note for KafkaMirrorMaker2 users (0.47.0 → 0.48.0+): Cleanup recommended

  After upgrading the operator, a new (dedicated) RoleBinding for KafkaMirrorMaker2 will be created and used automatically.
  The old RoleBinding, if it exists, may remain in the cluster but is no longer referenced.

  # List RoleBindings for your KafkaMirrorMaker2 instance (replace <namespace> and <mm2-name>)
  kubectl get rolebindings -n <namespace> \
    -l strimzi.io/kind=KafkaMirrorMaker2,strimzi.io/cluster=<mm2-name>

  If multiple RoleBindings are shown, the legacy one is <mm2-name>-connect-connect-role
  and the new one is <mm2-name>-mirrormaker2-role.

  You can safely delete just the legacy RoleBinding (unused after upgrade):

  kubectl delete rolebinding -n <namespace> <mm2-name>-connect-connect-role

• From Strimzi 0.48.0 on, we support only Kubernetes 1.27 and newer.
  Kubernetes 1.25 and 1.26 are not supported anymore.

• Disable Cruise Control network resource goals when resource capacities are not set.

• The strimzi_resource_state metric in the Cluster Operator is deprecated and is planned to be removed in Strimzi 0.51.
  Use kube-state-metrics based metrics from the examples as a replacement.

• The field .spec.enableMetrics in KafkaBridge is now deprecated and replaced by .spec.metricsConfig.

• The User Operator does not ignore the ACL rules for the * and ANONYMOUS users by default anymore.
  If you need to re-enable this feature, you can do it by configuring the STRIMZI_IGNORED_USERS_PATTERN environment variable to the following regular expression: ^\*|ANONYMOUS$.
  In case you deployed the User Operator through a Kafka custom resource, you can set it using the following YAML directly in the Kafka CR:

  apiVersion: kafka.strimzi.io/v1beta2
  kind: Kafka
  spec:
    #...
    entityOperator:
      template:
        userOperatorContainer:
          env:
            - name: STRIMZI_IGNORED_USERS_PATTERN
              value: "^\\*|ANONYMOUS$" # Double \ needed for escaping
      #...

  In case you use the standalone User Operator, you can set the environment variable in its Deployment.
  Please keep in mind that the ignored users will apply not only to ACLs, but also to Quotas and SCRAM-SHA credentials.

• The field .status.registeredNodeIds in Kafka is now deprecated, it is not used anymore, and it will be removed in the future.

All changes can be found under the 0.48.0 milestone.

Maven artifacts

To test the Maven artifacts that are part of this release, use the 0.48.0-RC1 version in your pom.xml.

Upgrading from Strimzi 0.47.0

See the documentation for upgrade instructions.

Container images

The following container images are part of this release:

New Contributors

• @shunki-fujita made their first contribution in #11787
• @kiku99 made their first contribution in #11862
• @bissquit made their first contribution in #11849

What's Changed

• Fix Strimzi Metric Exporter in Kafka brokers record in CHANGELOG by @scholzj in #11637
• Bump main branch to 0.48.0-SNAPSHOT by @scholzj in #11638
• Added testing Apache Kafka RCs with staging repo on the dev doc by @ppatierno in #11645
• Drop support for Kube 1.25 and 1.26 by @scholzj in #11640
• Fixed ambiguos call with varargs parameter by @ppatierno in #11649
• Add Strimzi 0.47.0 to the main branch by @scholzj in #11655
• Fixed deletion of auto-rebalancing related ConfigMap by @ppatierno in #11653
• Refactor CC capacity configuration classes by @kyguy in #11615
• Remove unused DeploymentConfig operators and other references by @scholzj in #11659
• Use persistent link for Maven used in TF jobs by @Frawless in #11660
• Add Strimzi 0.47.0 to the Upgrade/Downgrade STs by @scholzj in #11658
• feat(kafka): adding new properties as expections to broker configs by @rlanhellas in #11668
• Fix non-default primitive types within KafkaRebalance by @ppatierno in #11672
• docs(kraft): note on migrating zookeeper clusters by @PaulRMellor in #11666
• docs(pdb): clarifies PodDisruptionBudget behavior by @PaulRMellor in #11663
• feat(examples,metrics,kube-state-metrics): extend for 'KafkaAccess' by @sebastiangaiser in #11677
• feat(helm-charts,helm3,strimzi-kafka-operator): add optional deploymentStrategy to CO by @sebastiangaiser in #11681
• Disable CC network resource goals when resource capacities are not set. by @kyguy in #11465
• fix(operator): operator stuck when failing in parse kubernetes version by @rlanhellas in #11641
• Fix gramma and clarity across codebase by @see-quick in #11692
• Remove the KRaft and NodePool annotations by @scholzj in #11686
• Cleanup the CHANGELOG for Strimzi 0.48.0 by @scholzj in #11694
• Call kubeCmdClient clean with timeout by @kornys in #11683
• [ST] Change fields of the YAML files to make more sense and add README for upgrade/downgrade tests by @im-konge in #11698
• Remove check for Kafka version from the validateKRaftJbodStorage method (but keep check for metadata version) by @im-konge in #11697
• [system test] optimize execution time Azure jobs by @see-quick in #11700
• Strimzi Metrics Reporter integration with KafkaConnect and MirrorMaker2 by @OwenCorrigan76 in #11570
• Docs integreate strimzi metrics reporter kc mm2 by @OwenCorrigan76 in #11575
• Update Cruise Control to 2.5.143 by @scholzj in #11710
• Add support for Strimzi Metrics Reporter to the Strimzi Kafka Bridge by @OwenCorrigan76 in #11708
• Use captors in CaReconcilerTest with verify instead of when by @katheris in #11707
• Bumped Vert.x 5.0.2 and Netty 4.2.3.Final by @ppatierno in #11719
• Split example prometheus-operator bits into individual files by @jcpunk in #11715
• Removed unused members in the RestartContext class by @ppatierno in #11737
• Make ignored users in User Operator configurable by @scholzj in #11711
• [DOC] Fix OAUth versus OAuth typos in docs by @scholzj in #11739
• Bumped Vert.x 5.0.3 and Netty 4.2.4.Final by @ppatierno in #11745
• Update Cluster CA secret data when removing old certificates by @katheris in #11723
• Server Side Apply phase 1 - ConfigMap, Ingress, Service, ServiceAccount, PVC by @im-konge in #11693
• Enable storing raw yaml files from test runs by @kornys in #11741
• docs (Metrics Rporter) Add Metrics docs updates for the Bridge related to the Strimzi Metrics Reporter Integration by @OwenCorrigan76 in #11727
• Minor refactoring to remove unused method and rename others by @ppatierno in #11746
• Added dynamic changes on cluster-wide configuration parameters by @ppatierno in #11742
• Clean-up the Util class in operator-common by @scholzj in #11749
• [system test] [doc] topic package by @see-quick in #11691
• Removed unused code within KafkaRollerTest by @ppatierno in #11748
• extend the EntityOperator, CruiseControl and KafkaExporter to support PDB by @KyriosGN0 in #11699
• Remove paralell stuff from unit tests within cluster operator and ope… by @see-quick in #11753
• Simplify ca reconciler test class by @katheris in #11747
• [DOC] Improve the examples of MM2 connector and connector task restart annotations by @scholzj in #11763
• minor: javadoc fix within KafkaRoller by @see-quick in #11766
• PoC of system tests pipeline on GitHub Actions by @Frawless in #11331
• Make sure that pipeline won't be triggered by every comment by @Frawless in #11774
• Add support for type: custom client authentication by @scholzj in #11760
• Add workflows for testing our custom actions by @Frawless in #11769
• Create only commit statuses for our system tests action and deliver couple of fixes by @Frawless in #11777
• Removed unused pom property by @ppatierno in #11778
• [DOC] Fix KafkaTopic typo by @fvaleri in #11776
• Fix CHANGELOG records for the PEM certificates use in Kafka Connect by @scholzj in #11781
• Move methods not related to Vert.x out of VertxUtil class by @scholzj in #11768
• Fix OAuth version in docs (to 0.16.2) by @scholzj in #11795
• docs(deploy): operator handling of resources in namespaces by @PaulRMellor in #11740
• docs(connect): clarifies image/version config for kafka connect by @PaulRMellor in #11779
• Add RequiredInVersions annotation to our CRD generator by @scholzj in #11792
• Update the Prometheus JMX Exporter and the Maven Builder image by @scholzj in #11794
• Checkout merge commit instead of branch HEAD for sts workflow by @Frawless in #11793
• Minor fixes to GHA missing names by @ppatierno in #11796
• Fix NPE when logging User Operator configuration by @scholzj in #11802
• Use proper runner name for building images by @Frawless in #11807
• Cleanup of API CRD classes by @scholzj in #11806
• Add support for deprecated versions to our CRD Generator by @scholzj in #11805
• [ST] Add possibility to specify already built image with tiered storage plugin by @im-konge in #11804
• [system test] [doc] UserST by @see-quick in #11771
• Remove invalid warning about JVM Options being not supported in KafkaBridge by @scholzj in #11812
• docs(fix): fixes typo in loading config values from files procedure by @PaulRMellor in #11814
• Remove unused methods from the Storage API by @scholzj in #11813
• Mention the 3rd party folder deletion when remove Kafka version by @ppatierno in #11815
• [ST] Configure KAFKA_TIERED_STORAGE_BASE_IMAGE env in TestingFarm pipeline by @im-konge in #11810
• Update Vert.x to 5.0.4 and Netty to 4.2.5 by @scholzj in #11819
• Add support Apache Kafka 4.1.0 by @ppatierno in #11642
• Use proper ref for load images and fix results notification by @Frawless in #11811
• [AZP] Add cleanup phase to UTs/ITs job for build by @im-konge in #11821
• Remove support for Apache Kafka 3.9.x by @ppatierno in #11822
• Improve UO error when no operation is specified in ACL rule by @scholzj in #11820
• Fix two leftover examples to Kafka 4.1.0 by @scholzj in #11831
• Added usage of Kafka Admin API to list registered brokers (and unregister them) as KIP-1073 by @ppatierno in #11729
• KafkaConnect / KafkaMirrorMaker2 RoleBinding issues by @shunki-fujita in #11787
• Remove unused parts of KafkaConfigModeGenerator by @scholzj in #11830
• Allow typed APIs to be present only in some API versions by @scholzj in #11835
• Removed use of AccessController by @ppatierno in #11832
• Add workflow for building Strimzi by @Frawless in #11789
• Attemp to fix flakiness CoreFeaturesIT by @see-quick in #11758
• Update Fabric8 Kubernetes Client to 7.4.0 by @scholzj in #11843
• Fix path for check-permissions and move determine-ref to subfolder by @Frawless in #11847
• Add Strimzi test container 0.112.0 by @see-quick in #11824
• Add PR trigger to verify changes in sts workflow for every change by @Frawless in #11833
• Allow having CEL validation rules only for specific CRD API versions by @scholzj in #11851
• [ST] Change testChangingInternalToExternalLoggingTriggerRollingUpdate after removal of Kafka 3.9.x by @im-konge in #11850
• Use parse-comment from main branch by @Frawless in #11861
• Add configuration for codecov service by @Frawless in #11863
• Removed unused method in KafkaRoller by @ppatierno in #11865
• Integrate OAuth 0.17.0 grantType configuration by @mstruk in #11757
• Convert long numeric values to int64 in Helm templates, preventing 1.8e+06 formatting and operator crashes by @kiku99 in #11862
• docs(oauth): adds grantType config to examples by @PaulRMellor in #11848
• Update Bridge to 0.33.0 by @ppatierno in #11852
• docs(affinity): updates the pod scheduling section to use node pools by @PaulRMellor in #11803
• [ST] Add possibility to specify the classpath for Tiered Storage tests by @im-konge in #11873
• chore(monitoring): fix alerts grouping by @bissquit in #11849
• docs(mm2): updates to mm2 config content and structure to improve usability and clarity by @PaulRMellor in #11695
• Update log4j version to 2.25.0 and slf4j to 2.0.17 by @MichaelMorrisEst in #11629
• Handling min.insync.replicas deletion operation when possible based on ELR version by @ppatierno in #11857
• chore(monitoring): fix AbnormalControllerState alert rule by @bissquit in #11878
• docs(fix): format fix for server-side apply feature gate section by @PaulRMellor in #11877
• [ST] Fixes to LogCollector and UserST by @im-konge in #11876
• Disable test-container logging as build is stable by @see-quick in #11882
• Use the new snapshot repository for the Access Operator in STs by @scholzj in #11884
• Defaulting min.insync.replicas to 1 when it's not specified by the user by @ppatierno in #11883
• docs(format): adds missing abstract tags to doc files by @PaulRMellor in #11886
• docs(downgrade): clarifies metadata version support for downgrades by @PaulRMellor in #11885
• Allow configuring additional options when restarting connectors by @rlanhellas in #11797
• Remove unused Apicurio dependency from main pom.xml file by @scholzj in #11897
• Updated bridge to 0.33.1 by @ppatierno in #11891
• Moved access-operator-api dep to systemtest by @ppatierno in #11904

Full Changelog: 0.47.0...0.48.0-rc1