confluentinc/confluent-kafka-python v2.12.0

on GitHub

confluent-kafka-python v2.12.0

v2.12.0 is a feature release with the following enhancements:

KIP-848 – General Availability

Starting with confluent-kafka-python 2.12.0, the next generation consumer group rebalance protocol defined in KIP-848 is production-ready. Please refer to the following migration guide for moving from classic to consumer protocol.

Note: The new consumer group protocol defined in KIP-848 is not enabled by default. There are few contract change associated with the new protocol and might cause breaking changes. group.protocol configuration property dictates whether to use the new consumer protocol or older classic protocol. It defaults to classic if not provided.

AsyncIO Producer (experimental)

Introduces beta class AIOProducer for asynchronous message production in asyncio applications.

Added

• AsyncIO Producer (experimental): Introduces beta class AIOProducer for
  asynchronous message production in asyncio applications. This API offloads
  blocking librdkafka calls to a thread pool and schedules common callbacks
  (error_cb, throttle_cb, stats_cb, oauth_cb, logger) onto the event
  loop for safe usage inside async frameworks.

Features

• Batched async produce: await AIOProducer(...).produce(topic, value=...)
  buffers messages and flushes when the buffer threshold or timeout is reached.
• Async lifecycle: await producer.flush(), await producer.purge(), and
  transactional operations (init_transactions, begin_transaction,
  commit_transaction, abort_transaction).

Limitations

• Per-message headers are not supported in the current batched async produce
  path. If headers are required, use the synchronous Producer.produce(...) or
  offload a sync produce call to a thread executor within your async app.

Guidance

• Use the AsyncIO Producer inside async apps/servers (FastAPI/Starlette, aiohttp,
  asyncio tasks) to avoid blocking the event loop.
• For batch jobs, scripts, or highest-throughput pipelines without an event
  loop, the synchronous Producer remains recommended.

Enhancement and Fixes

• Kafka OAuth/OIDC metadata based authentication examples with Azure IMDS (#2083).

confluent-kafka-python v2.12.0 is based on librdkafka v2.12.0, see the
librdkafka release notes
for a complete list of changes, enhancements, fixes and upgrade considerations.