Introducing Rebalance Safe Commits

This release introduces a new experimental mode that prevent duplicate message processing during a rebalance. This mode has been in the making for 8 months and has been implemented over 9 different PRs.

We have marked Rebalance Safe Commits as an experimental feature because zio-kafka is used in many different and sometimes surprising ways. Some of those are not compatible with the new mode.

RebalanceSafeCommits can be enabled via ConsumerSettings.withRebalanceSafeCommits. Please read the scaladocs to see if this mode is applicable to how you use zio-kafka.

Much more information on the new mode (motivation, why it works, the challenges, etc.), are available in the descriptions of PR Await commits during a rebalance. A manual is forthcoming.

Changes

• Make maxRebalanceDuration configurable @erikvanoosten (#1118)
• Update zio-logging-slf4j, ... to 2.1.16 @zio-scala-steward (#1114)
• Automatically add new flame graphs to gh_pages, prune old @erikvanoosten (#1105)
• Await commit during a rebalance @erikvanoosten (#1098)
• Update README.md @zio-assistant (#1113)
• Update zio-kafka, zio-kafka-testkit to 2.7.0 @zio-scala-steward (#1112)