strimzi/strimzi-kafka-operator 0.40.0-rc1

on GitHub

⚠️ Important: From Strimzi 0.40 on, we support only Kubernetes 1.23 and newer. Kubernetes versions 1.21 and 1.22 are no longer supported.

Main changes since 0.39.0

This release contains the following new features and improvements:

• Add support for Apache Kafka 3.7.0. Remove support for Apache Kafka 3.5.0, 3.5.1, and 3.5.2.
• The UseKRaft feature gate moves to beta stage and is enabled by default. If needed, UseKRaft can be disabled in the feature gates configuration in the Cluster Operator.
• Add support for ZooKeeper to KRaft migration by enabling the users to move from using ZooKeeper to store metadata to KRaft.
• Add support for moving from dedicated controller-only KRaft nodes to mixed KRaft nodes
• Added support for Kafka Exporter offset.show-all parameter
• Prevent removal of the broker process role from KRaft mixed-nodes that have assigned partition-replicas
• Improve broker scale-down prevention to continue in reconciliation when scale-down cannot be executed
• Added support for Tiered Storage by enabling the configuration of custom storage plugins through the Kafka custom resource.
• Update HTTP bridge to the latest 0.28.0 release

All changes can be found under the 0.40.0 milestone.

Watch out also for some notable changes, deprecations, and removals:

• From Strimzi 0.40.0 on, we support only Kubernetes 1.23 and newer. Kubernetes 1.21 and 1.22 are not supported anymore.
• #9508 fixes the Strimzi CRDs and their definitions of labels and annotations. This change brings our CRDs in-sync with the Kubernetes APIs. After this fix, the label and annotation definitions in our CRDs (for example in the template sections) cannot contain integer values anymore and have to always use string values. If your custom resources use an integer value, for example:

  template:
    apiService:
      metadata:
        annotations:
          discovery.myapigateway.io/port: 8080
  

  You might get an error similar to this when applying the resource:

  spec.template.apiService.metadata.annotations.discovery.myapigateway.io/port: Invalid value: "integer": spec.template.apiService.metadata.annotations.discovery.myapigateway.io/port in body must be of type string: "integer"
  

  To fix the issue, just use a string value instead of an integer:

  template:
    apiService:
      metadata:
        annotations:
          discovery.myapigateway.io/port: "8080"
  

• Support for the JmxTrans component is now completely removed. If you are upgrading from Strimzi 0.34 or earlier and have JmxTrans enabled in .spec.jmxTrans of the Kafka custom resource, you should disable it before the upgrade or delete it manually after the upgrade is complete.
• The api module was refactored and classes were moved to new packages.
• Strimzi Drain Cleaner 1.1.0 (included in the Strimzi 0.40.0 installation files) changes the way it handles Kubernetes eviction requests. It denies them instead of allowing them. This new behavior does not require the PodDisruptionBudget to be set to maxUnavailable: 0. We expect this to improve the compatibility with various tools used for scaling Kubernetes clusters such as Karpenter. If you observe any problems with your toolchain or just want to stick with the previous behavior, you can use the STRIMZI_DENY_EVICTION environment variable and set it to false to switch back to the old (legacy) mode.

Maven artifacts

To test the Maven artifacts that are part of this release, use the staging repository by including the following in your pom.xml:

  <repositories>
    <repository>
      <id>staging</id>
      <url>https://oss.sonatype.org/content/repositories/iostrimzi-1213/</url>
    </repository>
  </repositories>

Upgrading from Strimzi 0.39.0

See the documentation for upgrade instructions.

Upgrading from Strimzi 0.22 or earlier

Direct upgrade from Strimzi 0.22 or earlier is not supported anymore! You have to upgrade first to one of the previous versions of Strimzi. You will also need to convert the CRD resources. For more details, see the documentation.

Container images

The following container images are part of this release:

All changes

• Bump main branch to 0.40.0-SNAPSHOT by @scholzj in #9459
• docs(node pools): updates to node pools content to reflect move to beta by @PaulRMellor in #9460
• docs(upgrade): note on topic operator cleanup when upgrading with unidirectional mode by @PaulRMellor in #9464
• Use separate logger for Jetty in Topic Operator by @scholzj in #9468
• Update Cruise Control to 2.5.133 and Kaniko to 1.19.1 by @scholzj in #9472
• Fix the UTO upgrade issue reported in #9470. by @fvaleri in #9474
• Add UnlockExperimentalVMOptions to priority JVM options by @mikekamornikov in #9476
• Add Strimzi 0.39.0 to the main branch by @scholzj in #9479
• Renaming context handler variable on creation by @ppatierno in #9478
• Update supported Kubernetes version to 1.23+ by @scholzj in #9473
• docs(tls): updates the tls communication overview to accommodate kraft by @PaulRMellor in #9481
• Refactor cert handling by @katheris in #9463
• docs(drain cleaner): considerations when using anti-affinity by @PaulRMellor in #9482
• Fix NullPointerException From Missing listenerConfig by @MarkintoshZ in #9489
• Enable TO in KRaft examples by @scholzj in #9490
• Remove the rest of the JMX Trans logic by @scholzj in #9492
• ISSUE-9231 Makes Kafka Exporter offset.show-all configurable by @ilkerkocatepe in #9494
• Remove unused methods by @katheris in #9495
• [ST] Create NamespaceManager for handling namespaces in the tests by @im-konge in #9496
• Fix typo in import control by @fvaleri in #9500
• Use additionalProperties instead of x-kubernetes-preserve-unknown-fields for Map<String, String> fields by @scholzj in #9508
• Set UTO as default standalone deployment by @fvaleri in #9514
• Add new CRD annotation to indicate Strimzi version when a property was added by @scholzj in #9515
• [ST] Enable connector autoRestart test by @jankalinic in #9507
• [ST] Fix detecting kind in systemtests by @jankalinic in #9504
• Remove Cruise Control's cluster.configs.file and related code by @fvaleri in #9499
• Add kind, apiVersion and metadata to our CRDs by @scholzj in #9516
• Split the classes from the io.strimzi.api.kafka.model package to multiple sub-packages by @ShubhamRwt in #9207
• [ST] Update upgrade/downgrade files after 0.39.0 release and enable KRaftStrimziDowngradeST by @im-konge in #9506
• refactor: Rename *Resources.*StatefulSetName and deploymentName methods by @Anefu in #9520
• Correct CRD operator delete predicate, increase test timeout by @MikeEdgar in #8715
• Remove support for Kafka 3.5.0 and 3.5.1 by @scholzj in #9526
• [ST] Upgrade/downgrade - specify file path for Kafka in the YAML files by @im-konge in #9529
• Update Fabric8 to 6.10.0 by @scholzj in #9539
• [ST] Innitial cleanup of AbstractST by @jankalinic in #9525
• Promote UseKRaft feature gate to beta by @scholzj in #9518
• bump kroxylicious-testing to 0.8.1 by @SamBarker in #9550
• Add warnings when unused fields are set when Node Pools or Kraft are used by @scholzj in #9553
• In KafkaAgentClient use generic keystore password instead of p12 password by @katheris in #9538
• [system test] Update Azure KRaft pipelines to run KRaft by @see-quick in #9557
• Move cruise control server, capacity and server logging metrics config to ConfigMap by @Thealisyed in #8977
• docs(feature gates): updates content due to maturity by @PaulRMellor in #9521
• [DOC] Make it clear what happens when brokerCertChainAndKey certificate is updated by @scholzj in #9568
• Properly display disk size in KRaft dashboards by @Frawless in #9567
• [ST] KafkaNodePools Roll only broker role due to broker only config change by @henryZrncik in #9540
• Minor refactoring and fixed warnings in AbstractConnectOperator by @scholzj in #9563
• Improve test coverage of KafkaAssemblyOperator in KRaft mode by @scholzj in #9558
• docs(refactor): brings the custom resource content together by @PaulRMellor in #9572
• Update various dependencies - Vert.x, Cruise Control, Kaniko, Json-Path, ... by @scholzj in #9578
• docs(readme): adds a readme to the documentation folder by @PaulRMellor in #9590
• Refactor connector preparation our of KafkaMirrorMaker2AssemblyOperator by @scholzj in #9593
• Update Cruise Control system tests by @kyguy in #9574
• Allow reconciliation to continue when scale-down is blocked due to brokers still in use by @scholzj in #9585
• Precreate the plugins directory to avoid errors in Mirror Maker 2 pods by @scholzj in #9607
• Update Vert.x to 4.5.2 and other related dependencies by @scholzj in #9618
• Remove the Entity Operator TLS sidecar alert from our sample alerts by @scholzj in #9622
• Check assigned partition-replicas when changing node roles by @scholzj in #9608
• Add Decathlon to adopters file by @ThomasDangleterre in #9626
• Update Micrometer Metrics to 1.12.2 by @scholzj in #9623
• docs(api): puts the api ref into three columns by @PaulRMellor in #9531
• Read and store ST configuration in yaml by @kornys in #9630
• Do not use SLF4J loggers in Topic Operator classes by @scholzj in #9619
• Use KRaft only on Testing Farm to avoid unexpected ZK failures by @Frawless in #9627
• Fix #9270: Incorrect TO ResourceConflict message by @tombentley in #9617
• Fix CC not restarted when API secret changes by @fvaleri in #9616
• [ST] Kafka Roller: enable pod pending test for kraft controller by @henryZrncik in #9584
• [ST] Big refactor of Test Storage usage in ST by @jankalinic in #9569
• docs(node pools): checks for nodes in node pools changing role by @PaulRMellor in #9635
• [ST] Refactor logs collecting of systemtests and collect more data by @kornys in #9637
• [ST] Use real test execution instead of milis from epoch in collecting logs from CO by @Frawless in #9628
• Improve verification of the running test cluster by @kornys in #9639
• [ST] Fixups to few failing tests by @im-konge in #9636
• docs(api): adds formatting for lists to the descriptions of the API properties by @PaulRMellor in #9646
• [ST] Remove unused methods from the KafkaNodePoolResource class by @im-konge in #9650
• Update Vert.x to 4.5.3 by @scholzj in #9653
• docs(refactor): moves drain cleaner content by @PaulRMellor in #9647
• Store secrets and configmaps in folders during log collecting by @kornys in #9659
• Add warning about CRD fix and integer value issue to CHANGELOG.md by @scholzj in #9660
• Updated README with project status to incubating by @ppatierno in #9664
• [ST] Re-enable applying of NetworkPolicies during namespace creation by @im-konge in #9656
• Set default replication factor for CC sample topics by @kyguy in #9471
• [ST] Remove un-necessary check inside testKafkaConnectAndConnectorStatus by @see-quick in #9673
• Use Integer instead of int in KafkaClusterSpec replicas by @im-konge in #9663
• ST: Add missing escape characters for log errors whitelist by @Frawless in #9649
• Updated the Strimzi documentation with little more clarification by @vinod827 in #9672
• docs(annotations): makes format of the annotation commands consistent by @PaulRMellor in #9665
• Set STRIMZI_USE_KRAFT_IN_TESTS=true also for x86 on TF by @Frawless in #9679
• Fix flaky Cruise Control test and fix error when RF is not configured by @scholzj in #9674
• [ST] Env configuration of execution by @kornys in #9677
• [ST] fixing unnecessary error logs when cleaning resources by @henryZrncik in #9667
• Configure OAuth principal.builder.class also for controllers by @im-konge in #9682
• Refactor CC files into CC package by @kyguy in #9683
• Update CC goal documentation by @kyguy in #9678
• docs(node pools): adds a procedure to describe role transitions by @PaulRMellor in #9681
• Inform about reverted scale-down or role-change in .status section by @scholzj in #9676
• Reflect the current roles when rolling Kafka nodes in KafkaRoller by @scholzj in #9686
• [ST] Use ThreadLocal in ResourceManager by @kornys in #9687
• Fixed OAuth configuration on mixed node test by @ppatierno in #9695
• Update KafkaRoller TODOs with the correct issue for tracking by @tinaselenge in #9694
• Changed access modifier as package private for testing by @ppatierno in #9698
• Update Kaniko to 1.20.1 by @scholzj in #9703
• [ST] Add NodePools to all of the test-cases, create NodePoolsConverter, add possibility to specify NP roles for all STs by @im-konge in #9668
• Fix UserOperator config log by @fvaleri in #9711
• Add topicId to KafkaTopic status by @fvaleri in #9708
• docs(review): minor updates and corrections by @PaulRMellor in #9716
• docs(refactor): moves annotations content by @PaulRMellor in #9717
• Fix report tool when getting CO replicasets by @fvaleri in #9721
• docs(build): updates for warnings highlighted in build by @PaulRMellor in #9725
• Cleanup test certificate temp files by @fvaleri in #9707
• Added KafkaAgentClient provider through the supplier by @ppatierno in #9723
• docs(clients): updates the tuning content for producers and consumers by @PaulRMellor in #9260
• [ST] Node Pools role changes. Role change and scale down prevention if replicas present. by @henryZrncik in #9693
• Update Vert.x and related dependencies by @scholzj in #9737
• Replace Fabric8 Kubernetes Mock Server with Kindcontainer by @scholzj in #9722
• [ST] Refactor methods in AbstractST by @jankalinic in #9661
• Added more resources to the mock deletion controller by @ppatierno in #9742
• [ST] add skip mixed role annotation to ST. by @henryZrncik in #9743
• Add support for Kafka 3.7.0 and remove Kafka 3.5.2 by @scholzj in #9747
• initial implementation for tiered storage support by @Staniel in #9727
• Improve UTO logs by @fvaleri in #9718
• Improve validation of certificates in DefaultAdminClientProvider by @scholzj in #9749
• Use native epoll on Aarch64 and not only on x86_64 by @scholzj in #9755
• [ST] fix CC failing test due to default topic rep. factor by @henryZrncik in #9751
• [DOC] Make logging examples less confusing by @scholzj in #9757
• ZooKeeper to KRaft migration by @ppatierno in #9480
• docs(kraft): adds a procedure for zookeeper to kraft migration by @PaulRMellor in #9633
• [ST] Use correct Kafka resource for each mode (ZK without NP / ZK with NP / KRaft) by @im-konge in #9709
• [ST] Add assumption about Kafka version and change Kafka template inside the migration tests by @im-konge in #9767
• Fix Connect's dynamic log level changes by @fvaleri in #9752
• [ST] Fix bridge client network policies creation by @jankalinic in #9719
• Avoid unnecessary patching when resources are being deleted by @scholzj in #9763
• Fail test when Connect doesn't start in time and increse the wait time by @fvaleri in #9773
• Add Drain Cleaner 1.1.0 to the installation files by @scholzj in #9768
• docs(format): format updates for tiered storage by @PaulRMellor in #9779
• More feedback on migration by @ppatierno in #9770
• [ST] Fix NetworkPolicies for Oauth tests by @im-konge in #9776
• Bump up test container 0.106.0 by @see-quick in #9780
• Update Kaniko to 1.21.0 and Maven builder to 1.19 by @scholzj in #9781
• [ST] Fix for incorrect CO log checking after the test by @jankalinic in #9688
• Remove Kotlin from third party libs. by @SamBarker in #9758
• Update OAuth library version to 0.15.0 by @mstruk in #9774
• [ST] AZP - Make it possible to run tests on main branch without building images by @im-konge in #9794
• [ST] Add NetworkPolicies for OpenTelemetryST when running on OCP by @im-konge in #9793
• [ST] Update test-clients to 0.7.0 after the release by @im-konge in #9785
• [ST] Disable flaky assertNoCoErrorsLogged by @jankalinic in #9797
• Fix wrong escaping of OAuth configuration in Mirror Maker 2 by @scholzj in #9799
• Update Kaniko to 1.21.1 by @scholzj in #9800
• Updated bridge to 0.28.0 by @ppatierno in #9787

New Contributors

• @mikekamornikov made their first contribution in #9476
• @MarkintoshZ made their first contribution in #9489
• @ilkerkocatepe made their first contribution in #9494
• @Anefu made their first contribution in #9520
• @vinod827 made their first contribution in #9672
• @Staniel made their first contribution in #9727