librdkafka v1.8.0
librdkafka v1.8.0 is a security release:
- Upgrade bundled zlib version from 1.2.8 to 1.2.11 in the
librdkafka.redist
NuGet package. The updated zlib version fixes CVEs:
CVE-2016-9840, CVE-2016-9841, CVE-2016-9842, CVE-2016-9843
See #2934 for more information. - librdkafka now uses vcpkg for up-to-date Windows
dependencies in thelibrdkafka.redist
NuGet package:
OpenSSL 1.1.1l, zlib 1.2.11, zstd 1.5.0. - The upstream dependency (OpenSSL, zstd, zlib) source archive checksums are
now verified when building with./configure --install-deps
.
These builds are used by the librdkafka builds bundled with
confluent-kafka-go, confluent-kafka-python and confluent-kafka-dotnet.
Enhancements
- Producer
flush()
now overrides thelinger.ms
setting for the duration
of theflush()
call, effectively triggering immediate transmission of
queued messages. (#3489)
Fixes
General fixes
- Correctly detect presence of zlib via compilation check. (Chris Novakovic)
ERR__ALL_BROKERS_DOWN
is no longer emitted when the coordinator
connection goes down, only when all standard named brokers have been tried.
This fixes the issue withERR__ALL_BROKERS_DOWN
being triggered on
consumer_close()
. It is also now only emitted if the connection was fully
up (past handshake), and not just connected.rd_kafka_query_watermark_offsets()
,rd_kafka_offsets_for_times()
,
consumer_lag
metric, andauto.offset.reset
now honour
isolation.level
and will return the Last Stable Offset (LSO)
whenisolation.level
is set toread_committed
(default), rather than
the uncommitted high-watermark when it is set toread_uncommitted
. (#3423)- SASL GSSAPI is now usable when
sasl.kerberos.min.time.before.relogin
is set to 0 - which disables ticket refreshes (by @mpekalski, #3431). - Rename internal crc32c() symbol to rd_crc32c() to avoid conflict with
other static libraries (#3421). txidle
andrxidle
in the statistics object was emitted as 18446744073709551615 when no idle was known. -1 is now emitted instead. (#3519)
Consumer fixes
- Automatically retry offset commits on
ERR_REQUEST_TIMED_OUT
,
ERR_COORDINATOR_NOT_AVAILABLE
, andERR_NOT_COORDINATOR
(#3398).
Offset commits will be retried twice. - Timed auto commits did not work when only using assign() and not subscribe().
This regression was introduced in v1.7.0. - If the topics matching the current subscription changed (or the application
updated the subscription) while there was an outstanding JoinGroup or
SyncGroup request, an additional request would sometimes be sent before
handling the response of the first. This in turn lead to internal state
issues that could cause a crash or malbehaviour.
The consumer will now wait for any outstanding JoinGroup or SyncGroup
responses before re-joining the group. auto.offset.reset
could previously be triggered by temporary errors,
such as disconnects and timeouts (after the two retries are exhausted).
This is now fixed so that the auto offset reset policy is only triggered
for permanent errors.- The error that triggers
auto.offset.reset
is now logged to help the
application owner identify the reason of the reset. - If a rebalance takes longer than a consumer's
session.timeout.ms
, the
consumer will remain in the group as long as it receives heartbeat responses
from the broker.
Admin fixes
DeleteRecords()
could crash if one of the underlying requests
(for a given partition leader) failed at the transport level (e.g., timeout).
(#3476).
Checksums
Release asset checksums:
- v1.8.0.zip SHA256
4b173f759ea5fdbc849fdad00d3a836b973f76cbd3aa8333290f0398fd07a1c4
- v1.8.0.tar.gz SHA256
93b12f554fa1c8393ce49ab52812a5f63e264d9af6a50fd6e6c318c481838b7f