confluentinc/confluent-kafka-python v2.15.0

on GitHub

KIP-932 Queues for Kafka – Now in Preview

confluent-kafka-python 2.15.0 adds a Preview implementation of the KIP-932 share consumer (Queues for Kafka). Members of a share group cooperatively consume from the same partitions with per-record acquire/acknowledge semantics and redelivery, providing queue-like consumption on top of Kafka.

• New ShareConsumer and DeserializingShareConsumer clients: subscribe, batch poll (poll() returns a Messages batch), and close (with context-manager support).
• Implicit (default) and explicit acknowledgement modes (share.acknowledgement.mode), with AcknowledgeType ACCEPT / RELEASE / REJECT and Message.delivery_count().
• Synchronous and asynchronous commit (commit_sync / commit_async) and an acknowledgement-commit callback.
• Runtime SASL credential updates (set_sasl_credentials) and new IllegalStateException / ConcurrentModificationException exceptions.
• (#2217, #2239, #2241, #2252, #2253, #2265, #2273, #2277, #2279)

See the Share Consumer guide and the Share consumers section of librdkafka's INTRODUCTION.md.

Note: The share consumer is currently in Preview and should not be used in production environments. Its public interfaces may change before General Availability, and known limitations apply (see the guide). The share consumer is single-threaded and not thread-safe. It requires a broker with share groups enabled (generally available in Apache Kafka 4.2.0).

Features

• New optional install confluent-kafka[oauthbearer-aws] provides AWS IAM-based
  OAUTHBEARER authentication via AWS STS GetWebIdentityToken. Activate by setting sasl.oauthbearer.method=oidc,
  sasl.oauthbearer.metadata.authentication.type=aws_iam, and
  sasl.oauthbearer.config="region=...,audience=...". See the
  AWS IAM OAUTHBEARER guide for full configuration, and
  examples/oauth_oidc_ccloud_aws_iam.py
  for a worked example.

Enhancements

• Add support for union-of-pools and auto pool mapping for Schema Registry (#2182)

Fixes

• Fix Encryption fails when expanded union types have two references to the same record (@ChristophMajcenAtXxxlutz, #2262)
• Make orjson optional in JSON serdes with stdlib json fallback (#2281)
• Handle non-http errors during retries (#2292)
• Use TLS certification verification with Hashicorp Vault (#2294)

confluent-kafka-python v2.15.0 is based on librdkafka v2.15.0, see the
librdkafka release notes
for a complete list of changes, enhancements, fixes and upgrade considerations.