strimzi/strimzi-kafka-operator 0.44.0

on GitHub

⚠️ Important: Strimzi 0.44 supports only Kubernetes 1.25 and newer. Kubernetes 1.23 and 1.24 are not supported anymore.

Main changes since 0.43.0

This release contains the following new features and improvements:

• Add the "Unmanaged" KafkaTopic status update.
• The ContinueReconciliationOnManualRollingUpdateFailure feature gate moves to beta stage and is enabled by default.
  If needed, ContinueReconciliationOnManualRollingUpdateFailure can be disabled in the feature gates configuration in the Cluster Operator.
• Add support for managing connector offsets via KafkaConnector and KafkaMirrorMaker2 custom resources.
• Add support for templating host and advertisedHost fields in listener configuration.
• Allow configuration of environment variables based on Config Map or Secret for every container in the container template sections.
• Add support for disabling the generation of PodDisruptionBudget resources by the Cluster Operator.
• Add support for running an automatic rebalancing, via Cruise Control, when the cluster is scaled down or up:
  • after a scaling up, the operator triggers an auto-rebalancing for moving some of the existing partitions to the newly added brokers.
  • before scaling down, and if the brokers to remove are hosting partitions, the operator triggers an auto-rebalancing to these partitions off the brokers to make them free to be removed.
• Strimzi Access Operator 0.1.0 added to the installation files and examples

All changes can be found under the 0.44.0 milestone.

Watch out also for some notable changes, deprecations, and removals:

• When finalizers are enabled (default), the Topic Operator will no longer restore finalizers on unmanaged KafkaTopic resources if they are removed, aligning the behavior with paused topics, where finalizers are also not restored.
  This change matches user expectations.
• The External Configuration (.spec.externalConfiguration) in KafkaConnect and KafkaMirrorMaker2 resources is deprecated and will be removed in the future.
  Please use the environment variables, additional volumes and volume mounts in Pod and container templates instead.
• The Strimzi Canary installation files were removed based on the Strimzi proposal 086 as the project was discontinued and archived.

Upgrading from Strimzi 0.43.0

See the documentation for upgrade instructions.

Upgrading from Strimzi 0.22 or earlier

Direct upgrade from Strimzi 0.22 or earlier is not supported anymore! You have to upgrade first to one of the previous versions of Strimzi. You will also need to convert the CRD resources. For more details, see the documentation.

Container images

The following container images are part of this release:

New Contributors

• @wangshu3000 made their first contribution in #10480
• @Xander-run made their first contribution in #10509
• @padilo made their first contribution in #10451
• @dandotimujahid made their first contribution in #10700
• @varada-sunanda-ibm made their first contribution in #10632

What's Changed

• Bump the main branch to 0.44.0-SNAPSHOT by @scholzj in #10478
• [ST] Add CC API Users test by @im-konge in #10473
• docs(refactor): refactor of the overview guide by @PaulRMellor in #10460
• Update of unit tests to use KRaft - part 10 by @scholzj in #10477
• [ST] Remove few methods/classes that are not used by @im-konge in #10474
• Fix duplicate clustersrolebindings issue by @Eric84626 in #10479
• improve tiered storage system test by @showuon in #10471
• [system test] - re-write some tests to use pattern with trusted certi… by @see-quick in #10475
• docs(cruise control): format fix for cc api users section by @PaulRMellor in #10485
• Disable unmanaged topics reconciliation by @fvaleri in #10481
• Add Strimzi 0.43.0 to the main branch by @scholzj in #10488
• Remove keystore/truststore pwd from kafka agent command by @wangshu3000 in #10480
• Update supported Kubernetes version to 1.25+ by @scholzj in #10490
• Do not generate empty required arrays in OneOf definition by @scholzj in #10492
• [ST] Change ST after release 0.43.0 Upgrade/Downgrade and OLM defaults by @henryZrncik in #10493
• Add release trigger to Packit to trigger system tests on TF (amd64+arm64) by @Frawless in #10413
• Part 11 of unit tests update for KRaft by @scholzj in #10499
• [ST] removal of resources from co-namespace across ST (except for upgrade downgrade STs) by @henryZrncik in #10501
• Part 12 of unit tests update for KRaft by @scholzj in #10505
• Use properly kube version for Kind setup and resolve TF errors by @Frawless in #10507
• Added the scorecard github action and its badge by @harshitasao in #10421
• Improve KRaft based affinity and toleration tests by @scholzj in #10510
• [ST] unused tags removal: internalclients, arm arch unsupported by @henryZrncik in #10497
• Pin the upload-sarif GitHub action in the Scorecard workflow by @scholzj in #10520
• Use different image for RegistryProxy to workaround minikube startup failures by @Frawless in #10519
• Fix port handling in the Kafka Agent by @scholzj in #10516
• fix: Correct the order of generating default and overridden broker capacities in Capacity.processCapacityEntries by @Xander-run in #10509
• Increase minio secret key length for FIPS by @showuon in #10518
• [ST] enable most of formerly kraftNotEnabled tests by @henryZrncik in #10521
• Fix topic operator loop for unmanaged topics by @padilo in #10451
• Fix various JavaDoc style issues by @scholzj in #10525
• Minor clean-up of the test module by @scholzj in #10526
• Fix bugs for automatic refresh of KafkaRebalance by @tinaselenge in #10486
• Improve handling of custom Cruise Control topic configurations by @scholzj in #10511
• Block options and not prefixes in Kafka Connector configs by @scholzj in #10503
• [ST] Parallelize kubeClient interactions in AbstractUpgradeST by @lhunyady in #10514
• docs(oauth): sasl extensions for custom oauthbearer implementations by @PaulRMellor in #10523
• Improve handling of invalid topic configurations by @fvaleri in #10517
• Add Reddit to Adopters file by @tomncooper in #10529
• Fix topicId status update by @fvaleri in #10491
• Promote ContinueReconciliationOnManualRollingUpdateFailure feature gate to beta by @scholzj in #10524
• Move JavaDoc validation to Checkstyle by @scholzj in #10527
• [ST] Bump Test-Frame to 0.6.1 and add handling of the new metric types by @im-konge in #10506
• [ST] Upgrade tests removal of resources from co-namespace by @henryZrncik in #10512
• Add current Raft state metric to the exported metrics by @scholzj in #10533
• Fix checkstyle check by @im-konge in #10537
• Use LockSupport.parkNanos instead of Thread.sleep in systemtests by @kornys in #10522
• Add two more topicId tests by @fvaleri in #10536
• [system test] [doc] bridge package by @see-quick in #10225
• Update Vert.x to 4.5.10 by @scholzj in #10539
• Add Topic Operator refactoring: handlers by @fvaleri in #10412
• [ST] acceptance profile test count reduction by @henryZrncik in #10528
• Run AZP in KRaft mode and create ZK regression pipeline by @im-konge in #10532
• Simple refactoring on editing Kafka custom resource during KRaft migration tests by @ppatierno in #10546
• Using enum rebalance state for KafkaRebalance operator tests by @ppatierno in #10547
• Use Fabric8 Kubernetes client in Kubernetes based integration tests by @scholzj in #10545
• [system test] DOCs generator update to use absolute path by @see-quick in #10549
• [ST] Use the KafkaConnect with Connector upgrade/downgrade scenario as the main one by @im-konge in #10544
• Move the Kube/OpenShift clients and cluster helper classes to systemtest module by @scholzj in #10551
• Suppress deprecatios in KafkaMirrorMakerCrdIT by @scholzj in #10552
• Try to stabilize KafkaConnectorIT tests by @scholzj in #10550
• Continue refactoring of the test module by @scholzj in #10553
• Improve the TestSeparator to indicate if the test failed or succeeded by @scholzj in #10554
• [system test] Run DOCs STs generator only in install goal not verify (during tests) by @see-quick in #10558
• Allow configuring advertised.listeners for controllers with Kafka version 3.9.0 by @tinaselenge in #10530
• Add current raft state to grafana dashboard by @MichaelMorrisEst in #10548
• Add OneOf constraint to Connect external configuration by @scholzj in #10560
• docs(ports): adds a description of port 8443 used by KafkaAgent by @PaulRMellor in #10570
• Deprecate the External Configuration Volumes by @scholzj in #10566
• Update the Kindcontainer used in MockKube tests by @scholzj in #10574
• docs(review): minor doc updates and improvements from review by @PaulRMellor in #10579
• Reorder the Kafka KRaft example based on priority by @scholzj in #10575
• [system test] [doc] connect package by @see-quick in #10567
• [ST] upgrade downgrade STs namespaces creation/deletion minor fix by @henryZrncik in #10568
• Reorder remaining examples based on importance by @scholzj in #10588
• Refactor KafkaCluster tests to not use internal methods by @MichaelMorrisEst in #10590
• docs(keycloak): updates from review by keycloak dev by @PaulRMellor in #10581
• [ST] use namespace as first parameter in methods by @henryZrncik in #10542
• Possibility to override kaniko executor image in systemtest by @akatona84 in #10580
• Roll controller nodes on num.partitions change by @scholzj in #10594
• Add mechanism to manage offsets through KafkaConnector and KafkaMirrorMaker2 resources by @katheris in #10563
• Use init container for Kafka nodes only when needed by @scholzj in #10596
• [ST] Fix NetworkPoliciesST test by @im-konge in #10598
• docs(overview): updates component descriptions by @PaulRMellor in #10595
• Fix typos in Kafka Connect custom volume examples by @scholzj in #10611
• [system test] Move TestUtils methods used only in STs to systemtest m… by @see-quick in #10610
• Remove duplicate methods in Connect operators by @katheris in #10604
• Update setup-kind.sh to correctly set docker login and remove unused variables from the script by @im-konge in #10617
• Remove superflous method from ReadWriteUtils by @MichaelMorrisEst in #10608
• Fixed wrong strimzi domain for rebalance annotation in Javadoc by @ppatierno in #10622
• Update Cruise Control to 2.5.139 by @scholzj in #10621
• Allow templating of host and advertisedHost fields by @scholzj in #10605
• [performance] - Remove bulk and streams use cases from UO and TO. Scallability test added. by @see-quick in #10138
• Add support for Secret or ConfigMap based environment variables in ContainerTemplate by @scholzj in #10623
• [system test] [doc] cruise control package by @see-quick in #10615
• [system test] [doc] Add TestDocsLabels class to remove redundancy by @see-quick in #10633
• [system test] Create a new Tags class by @see-quick in #10624
• Move ContainerEnvVar class to the template subpackage by @scholzj in #10628
• Fix reliability issue with test case by @MichaelMorrisEst in #10635
• [ST] add new util method to check crd presence and use it in UpgradeSTs by @henryZrncik in #10634
• docs(deployment): updates the deployment prep content and download references by @PaulRMellor in #10638
• [ST] Fix StrimziUpgradeST.testUpgradeAcrossVersionsWithUnsupportedKafkaVersion test to run only when there is really an unsupported Kafka version across versions by @egyedt in #10496
• Add env var to disable creation of PodDisruptionBudget by @katheris in #10614
• [ST] ConnectST: switch from (deprecated) External Config volumes to volume mounts. by @henryZrncik in #10609
• Update Fabric8 Kubernetes client to 6.13.4 by @scholzj in #10642
• Update Kafka Configuration Diff and its test by @scholzj in #10639
• Clean up methods in Connect and MM2 Assembly Operators by @katheris in #10640
• Decide if option is custom or not in a single place by @scholzj in #10643
• Configure custom listener certificates and OAuth only on broker nodes by @scholzj in #10646
• Auto-rebalancing on scaling the cluster down/up by @ppatierno in #10583
• Fix rebalance RF change race issue by @fvaleri in #10641
• Add Access Operator 0.1.0 to the packaging directory by @scholzj in #10660
• Fixed autoRebalance status not reset when auto-rebalancing or Cruise Control is not enabled by @ppatierno in #10652
• Add generic labels to docker images by @urbandan in #10671
• Use configuration providers instead of envsubst in Kafka configuration files by @scholzj in #10656
• Refactor the implementation for getting status of rebalance proposal request by @tinaselenge in #10607
• Improve the CA not found error message by @scholzj in #10669
• Update the Helm Chart README with some new features and warnings by @scholzj in #10657
• Bump test frame 0.7.0 by @see-quick in #10675
• [ST] Remove assumption for skipping run on OLM and Helm from few tests by @im-konge in #10673
• Update Kubernetes Config Provider to 1.2.0 by @scholzj in #10667
• Improved KafkaRebalance columns for rebalancing status and template by @ppatierno in #10686
• [ST] Install CO with OLM without need of specifying OLM version for latest released operator by @im-konge in #10681
• Minor refactoring for IDE warnings by @ppatierno in #10692
• Update Cruise Control to 2.5.141 by @scholzj in #10696
• [ST] Use Test-Frame's LogCollector in STs by @im-konge in #10695
• docs(auto-rebalance): format fixes on API descriptions for auto-rebalance properties by @PaulRMellor in #10699
• [system test] [doc] kafka package by @see-quick in #10650
• Updated helm-unittest version for s390x by @dandotimujahid in #10700
• Use valid label values in KafkaST tests by @scholzj in #10708
• docs(metrics): adds missing callout to metrics file list by @PaulRMellor in #10702
• docs(tracing): adds configuration for TLS with OTLP tracing by @PaulRMellor in #10680
• [ST] Remove line for setting null for LMFV in StrimziDowngradeST by @im-konge in #10709
• docs(connectors): adds procedures to manage connector offsets by @PaulRMellor in #10661
• Bump org.eclipse.jetty:jetty-server from 9.4.54.v20240208 to 9.4.55.v20240627 by @dependabot in #10716
• Fixed stuck reconciliation on failed request rebalance by @ppatierno in #10717
• Reverted Python to v3.x working for pre-commit by @ppatierno in #10721
• Follow up on fixing stuck reconciliation on failed request rebalance by @ppatierno in #10720
• Annotate StrimziPodSets before rolling during CA renewal by @katheris in #10711
• Update Jetty to 9.4.56 by @scholzj in #10730
• Remove Canary install files from the packaging folder by @im-konge in #10734
• [DOC] Fix the image name in the TO customization Dockerfile by @scholzj in #10737
• Integrate Strimzi Access Operator into the release bundle by @scholzj in #10738
• Changed variable from class member to local variable by @ppatierno in #10739
• Removed some commented script lines by @ppatierno in #10742
• MirrorMaker2: Check if bootstrap servers are the same for different cluster aliases by @varada-sunanda-ibm in #10632
• Update the Kafka recovery procedure with KRaft notes by @fvaleri in #10728
• docs(auto-rebalance): adds new and updated content for auto-rebalancing by @PaulRMellor in #10698
• [system test] Auto-rebalance feature test case by @see-quick in #10666
• Use Strimzi Test Container in TO tests by @scholzj in #10713
• Bump Test-Frame to 0.8.0 by @im-konge in #10747
• Refactored autorebalance by adding a dedicated Enum for modes by @ppatierno in #10744
• Do not log Kafka Agent configuration at Kafka node startup by @scholzj in #10750
• [system test] Fix testAutoKafkaRebalanceScaleUpScaleDown and make it … by @see-quick in #10754
• allow TLSv1.3 in ZK for FIPS client connection in testCertificates by @showuon in #10726
• docs(storage): updates the storage content for KRaft and node pools by @PaulRMellor in #10731
• Revert "Annotate StrimziPodSets before rolling during CA renewal" by @scholzj in #10762
• docs(access operator): early access instructions for using the access operator by @PaulRMellor in #10752