strimzi/strimzi-kafka-operator 0.41.0-rc2

on GitHub

⚠️ Important: Strimzi 0.41 supports only Kubernetes 1.23 and newer. Kubernetes versions 1.21 and 1.22 are no longer supported.

Main changes since 0.40.0

This release contains the following new features and improvements:

• Support for Apache Kafka 3.6.2
• Metrics to monitor CA expiration
• Support for JBOD storage in KRaft mode.
  (Note: JBOD support in KRaft mode is considered early access in Apache Kafka 3.7.x)
• Support for changing topic replication factor by changing the KafkaTopic resources.
• The KafkaNodePools feature gate moves to GA stage and is permanently enabled without the possibility to disable it.
  To use the Kafka Node Pool resources, you still need to use the strimzi.io/node-pools: enabled annotation on the Kafka custom resources.
• Support for configuring the externalIPs field in node port type services.
• The UnidirectionalTopicOperator feature gate moves to GA stage and is permanently enabled without the possibility to disable it.
  If the topics whose names start with strimzi-store-topic and strimzi-topic-operator still exist, you can delete them.
• Improve validation of KafkaMirrorMaker2 resource to make sure the Connect cluster is set to the target cluster
• Support for custom SASL config in standalone Topic Operator deployment to support alternate access controllers (i.e. AWS_MSK_IAM)

All changes can be found under the 0.41.0 milestone.

Watch out also for some notable changes, deprecations, and removals:

• The tlsSidecar configuration for the Entity Operator is now deprecated and will be ignored.
• The zookeeperSessionTimeoutSeconds and topicMetadataMaxAttempts configurations for the Entity Topic Operator have been removed and will be ignored.

Maven artifacts

To test the Maven artifacts that are part of this release, use the staging repository by including the following in your pom.xml:

  <repositories>
    <repository>
      <id>staging</id>
      <url>https://oss.sonatype.org/content/repositories/iostrimzi-1217/</url>
    </repository>
  </repositories>

Upgrading from Strimzi 0.40.0

See the documentation for upgrade instructions.

Upgrading from Strimzi 0.22 or earlier

Direct upgrade from Strimzi 0.22 or earlier is not supported anymore! You have to upgrade first to one of the previous versions of Strimzi. You will also need to convert the CRD resources. For more details, see the documentation.

Container images

The following container images are part of this release:

New Contributors

• @kockas made their first contribution in #9810
• @maciej-tatarski made their first contribution in #9817
• @steffen-karlsson made their first contribution in #9831
• @rao2100 made their first contribution in #9815
• @Gartium made their first contribution in #9857
• @MichaelMorrisEst made their first contribution in #9906
• @dermasmid made their first contribution in #9951
• @rolandjohann made their first contribution in #9961
• @KyriosGN0 made their first contribution in #9963
• @maakuth made their first contribution in #9923
• @john-mcpeek made their first contribution in #9606
• @FlxPeters made their first contribution in #9995
• @blaghed made their first contribution in #10026

All changes

• docs(kafka): expands the content for handling large message sizes by @PaulRMellor in #9790
• Bump main branch to 0.41.0-SNAPSHOT by @scholzj in #9803
• [ST] Fixups for few failing tests in nightlies by @im-konge in #9801
• [DOC] Fix and improve migration docs by @scholzj in #9804
• Add topic replication factor change by @fvaleri in #9483
• [ST] persistent kafka with 1 rep. instead ephemeral by @henryZrncik in #9782
• [ST] Fix the issue with creating Kafka in CC tests in KRaft mode with lower Kafka version by @im-konge in #9808
• #9734 - added toString to whole model, including Password, PasswordSe… by @kockas in #9810
• Add Strimzi 0.40.0 release to the main branch by @scholzj in #9816
• Promote KafkaNodePools feature gate to GA by @scholzj in #9811
• [DOC] Suggest removal of unused fields from Kafka CR after migration to node pools by @scholzj in #9809
• [DOC] Fix some leftovers from 0.40.0 release by @scholzj in #9822
• Fix value for all topics and groups selected in Strimzi Kafka Exporter dashboard by @maciej-tatarski in #9817
• [ST] Fix and improve QuotasST by @im-konge in #9813
• [ST] Update upgrade/downgrade YAMLs and Environment class after 0.40.0 release by @im-konge in #9818
• docs(cruise control): employing cruise control to change topic replication factor by @PaulRMellor in #9788
• [DOC] Add note about aggreated metrics produced by the quotas plugin by @fvaleri in #9827
• [ST] Enhance DrainCleanerST by @im-konge in #9829
• Change chart to table for log size by @maciej-tatarski in #9830
• Update Kafka version in kafka-with-node-pools.yaml example by @im-konge in #9835
• #2779 CrdGenerator validate @JsonPropertyOrder by @steffen-karlsson in #9831
• Add refactoring around replicas change handling by @fvaleri in #9821
• Add a single node KRaft example for use in quickstarts by @scholzj in #9837
• Bump Vert.x 4.5.5 by @ppatierno in #9838
• [ST] Set STRIMZI_USE_NODE_POOLS_IN_TESTS to true by default by @im-konge in #9839
• Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 by @dependabot in #9844
• Some minor refactoring on migration code by @ppatierno in #9833
• [ST] KafkaConnect dynamic log level change test checks addition by @henryZrncik in #9834
• [ST] - Topic replication change feature sts by @see-quick in #9789
• External ip support by @rao2100 in #9815
• [ST] (Kraft)KafkaUpgradeDowngrade decrease tests durations by @henryZrncik in #9852
• [ST] fix flaky test in MultipleClusterOperatorsST by @henryZrncik in #9850
• Added documentation about recommended Kubernetes labels by @ppatierno in #9854
• Promote UnidirectionalTopicOperator feature gate to GA by @fvaleri in #9840
• Bump Vert.x 4.5.6 by @ppatierno in #9865
• [ST] default builders for plain/tls internal clients by @henryZrncik in #9658
• Handle x-kubernetes-preserve-unknown-fields in CRD for Kubernetes Resources object. by @Gartium in #9857
• Abstract client certificate handling by @katheris in #9545
• Remove unnecessary Secret mocks from KafkaReconcilerStatusTest by @katheris in #9874
• Update Fabric8 to 6.11.0 by @scholzj in #9873
• docs(upgrade): updates node draining instructions following drain cleaner changes by @PaulRMellor in #9878
• Move ShutdownHook class from operator-common to cluster-operator by @scholzj in #9877
• Remove Vert.x dependency from MicrometerMetricsProvider by @scholzj in #9879
• Remove the NoStackTraceTimeoutException by @scholzj in #9880
• Move unrelated clases from the package with resource operators by @scholzj in #9882
• Remove Vert.x dependency from the operator-common module and from UO and TO by @scholzj in #9887
• docs(storage): adds a description of tiered storage by @PaulRMellor in #9866
• docs(pvc): updates the procedure to recover from a deleted cluster by @PaulRMellor in #9884
• Bumped Vert.x 4.5.7 by @ppatierno in #9890
• docs(node pools): impact on racks when moving nodes by @PaulRMellor in #9894
• Pom refactoring on OpenTelemetry related dependencies by @ppatierno in #9892
• Enhancing ReconcileResult with type parameter, instead of checking toString by @steffen-karlsson in #9891
• Added roles column definition for the KafkaNodePool CRD by @ppatierno in #9896
• Update Kaniko to 1.22 by @scholzj in #9898
• Reuse Util.decodeFromBase64 by @MichaelMorrisEst in #9906
• #3761 Provide metrics to monitor certificates expiration by @steffen-karlsson in #9861
• Refactor decoding Base64 to byte[] by @MichaelMorrisEst in #9912
• docs(uto): removes some references to using the bidirectional topic operator by @PaulRMellor in #9885
• docs(fips): note on NIST validation by @PaulRMellor in #9913
• docs(fix): fixes link to image environment variables info by @PaulRMellor in #9917
• [ST] Fix LoggingChangeST#testDynamicallySetConnectLoggingLevels for Kafka versions below 3.7.x by @im-konge in #9919
• Fix incorrect path in custom authorizer docs and improve them by @scholzj in #9920
• Add support for Kafka 3.6.2 by @scholzj in #9918
• Fix paused connector status when Connect is scaled to zero by @fvaleri in #9926
• Refactor InitWriterConfig by @Staniel in #9922
• Fixed validation minimum Kafka version for migration by @ppatierno in #9935
• Fixed KRaft detection script when broker in POST_MIGRATION by @ppatierno in #9943
• Added Stopped column to the KafkaRebalance custom resource by @ppatierno in #9944
• [ST] Move AZP profiles from STs' pom.xml and remove profiles related to Jenkins by @im-konge in #9942
• Avoid updating connector status again and again when Connect is scaled to 0 by @scholzj in #9945
• docs(node pools): reversing the migration to node pools by @PaulRMellor in #9950
• docs(overview): updates to the overview for new features by @PaulRMellor in #9953
• Removing the inner polling to check proposal request and rebalancing by @ppatierno in #9940
• fix: use ipv4 for jmx, fixes: #9947 by @dermasmid in #9951
• Updare Fabric8 to 6.12 by @scholzj in #9958
• Improve the separation between the Connect and MM2 operators by @scholzj in #9955
• Add whitespace to fix markdown formatting of inline code by @rolandjohann in #9961
• Fix too long timeout in Kubernetes-based IT tests by @scholzj in #9968
• feat: add revisionHistoryLimit by @KyriosGN0 in #9963
• Fix missing curly bracket in KRaft volume size query by @Frawless in #9971
• MirrorMaker2: Add warning for dangerously invalid configuration by @maakuth in #9923
• Fix auto-restarting of MM2 connectors by @scholzj in #9966
• Refactor key store trust store by @katheris in #9941
• [ST] Fix MigrationST by @im-konge in #9954
• Add JBOD support to KRaft mode by @scholzj in #9936
• Added a custom SASL config option to the Topic Operator #5473 by @john-mcpeek in #9606
• Add trivial refactoring to reflect the change from client to handler by @fvaleri in #9992
• [performance] Introduction to performance testing by @see-quick in #9907
• Add more unit tests for Label.class related to label filtering by @see-quick in #9959
• docs(exporter): adds the showAllOffsets configuration to example Exporter config by @PaulRMellor in #9991
• Fix invalid error message when topic RF change is disabled by @fvaleri in #9989
• Update Grafana dashboards to remove deprecated angular plugins by @Frawless in #9982
• Add validation for unique volume IDs in JBOD storage by @scholzj in #9990
• Separate logging configuration and collecting plugin information from the conenctor reconciliation by @scholzj in #9987
• Fixed deletion of cluster metadata on rollback after JBOD in KRaft support by @ppatierno in #9993
• Remove last traces of ZKClient library from Strimzi by @scholzj in #9996
• fix(helm): add conditional namespace on watchAnyNamespace true by @FlxPeters in #9995
• Add non empty condition to pending but completed log by @fvaleri in #10001
• Update various dependencies (Fabric8, Lombok and Jetty) by @scholzj in #9999
• docs(otel): updates dependencies for Initializing tracing by @PaulRMellor in #10002
• Remove serializable from API by @see-quick in #9997
• docs(diagrams): updates digrams to be more kraft friendly by @PaulRMellor in #9962
• [system test] - adapt JBOD system tests for KRaft and also make it co… by @see-quick in #10007
• [ST] Bump version of test-clients to 0.8.0 and Jaeger's all-in-one image to 1.56 by @im-konge in #10003
• Update TO design documentation with SKIP_CLUSTER_CONFIG_REVIEW by @fvaleri in #10014
• Remove events rule from entity-operator by @fvaleri in #10010
• [performance] - UserOperator bulk batch use case by @see-quick in #10012
• Improve JBOD storage version validation in KRaft by @scholzj in #10017
• Add early-access warning about JBOD in KRaft to CHANGELOG and docs by @scholzj in #10019
• Address some trivial warnings and deprecations in two model test classes by @scholzj in #10024
• Fix offset.show-all switch in Kafka Exporter by @scholzj in #10022
• Remove unneeded state (old certificate secrets) from the Ca classes by @scholzj in #10023
• [system test] - Remove scalabillity test cases by @see-quick in #10029
• Rebuild the Kafka Connect image only when the rebuild annotation is set to true by @scholzj in #10030
• [ST] enable all system test disable due to UTO and add admin client utils by @henryZrncik in #9876
• [performance] - UserOperator capacity tests by @see-quick in #10018
• [performance] Re-name perf test suites to not include in UTs by @see-quick in #10043
• Improve broken ZooKeeperScaler tests by @scholzj in #10046
• docs(kafka): cleans up naming of kafka clusters by @PaulRMellor in #10049
• [performance] - propagate test context into metrics collecting by @see-quick in #10047
• Improve ZooKeeper client handling in migration code by @scholzj in #10045
• Remove PKCS12 files from the server certificate secrets by @scholzj in #10044
• [system test] - KRaft JBOD test cases by @see-quick in #10035
• docs(connectors): annotation to rebuild container image by @PaulRMellor in #10048
• Don't fail reconciliation when Manual Rolling Update fails by @Staniel in #10005
• Relax the KafkaConnect admin/loggers validation by @blaghed in #10026
• [ST] Fix RackAwarenessST, JmxST, and file-sink image build by @im-konge in #10060
• Move dependency analysis from the root pom.xml to the modules by @scholzj in #10062
• [ST] MigrationST - add checks for JBOD storage, change the main tests to be parametrized by @im-konge in #10057
• Fix /controller znode failure handling during KRaft migration rollback by @ppatierno in #10058
• Raise warning when kraftMetadata is used in storage configuration for ZooKeeper-based clusters by @scholzj in #10067
• Cruise Control mock refactoring by @fvaleri in #10056