Confluent's Python client for Apache Kafka
confluent-kafka-python is based on librdkafka v1.1.0, see the librdkafka v1.1.0 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.
- In-memory SSL certificates (PEM, DER, PKCS#12) support (by @noahdav at Microsoft)
- Use Windows Root/CA SSL Certificate Store (by @noahdav at Microsoft)
ssl.endpoint.identification.algorithm=https
(off by default) to validate the broker hostname matches the certificate. Requires OpenSSL >= 1.0.2(included with Wheel installations))- Improved GSSAPI/Kerberos ticket refresh
- Confluent monitoring interceptor package bumped to v0.11.1 (#634)
Upgrade considerations
- Windows SSL users will no longer need to specify a CA certificate file/directory (
ssl.ca.location
), librdkafka will load the CA certs by default from the Windows Root Certificate Store. - SSL peer (broker) certificate verification is now enabled by default (disable with
enable.ssl.certificate.verification=false
) %{broker.name}
is no longer supported insasl.kerberos.kinit.cmd
since kinit refresh is no longer executed per broker, but per client instance.
SSL
New configuration properties:
ssl.key.pem
- client's private key as a string in PEM formatssl.certificate.pem
- client's public key as a string in PEM formatenable.ssl.certificate.verification
- enable(default)/disable OpenSSL's builtin broker certificate verification.enable.ssl.endpoint.identification.algorithm
- to verify the broker's hostname with its certificate (disabled by default).- Add new
rd_kafka_conf_set_ssl_cert()
to pass PKCS#12, DER or PEM certs in (binary) memory form to the configuration object. - The private key data is now securely cleared from memory after last use.
Enhancements
- Bump
message.timeout.ms
max value from 15 minutes to 24 days (@sarkanyi, workaround for #2015)
Fixes
- SASL GSSAPI/Kerberos: Don't run kinit refresh for each broker, just per client instance.
- SASL GSSAPI/Kerberos: Changed
sasl.kerberos.kinit.cmd
to first attempt ticket refresh, then acquire. - SASL: Proper locking on broker name acquisition.
- Consumer:
max.poll.interval.ms
now correctly handles blocking poll calls, allowing a longer poll timeout than the max poll interval. - configure: Fix libzstd static lib detection
- PyTest pinned to latest version supporting python 2 (#634)