Features
- Shadow Indexing (Tech Preview): If cloud storage is enabled, start offset for the partition will take cloud data into account and fetches for data that was already deleted locally will go to cloud storage. A cluster with this feature enabled is not yet supported for usage.
- Schema Registry: Support protobuf schema.
- Cluster health monitor: Ability to query for cluster runtime parameters such as node versions, node availability, node resources and raft group status
- API: The v1/cluster_config/schema admin API endpoint is added for discovering available cluster configuration properties.
- API: Added support for manual partition assignment in CreateTopics API. Currently custom partition assignment is only used during initial partition allocation. Nodes decommissioning and automatic partition re-balancing may change the partition assignment
- rpk: The new rpk group command can be used to list groups, delete groups, describe groups, and modify group offsets. rpk cluster offsets has been deprecated.
- rpk: new
topic add-partitionscommand to add partitions to existing topics
Improvements
- Admin API: For requests that must be submitted to the controller leader node, or to the leader for a particular partition: other nodes will now respond with an HTTP 307 redirect to the leader rather than an HTTP 400 error.
- configuration:
describe_confignow supports key specification inconfiguration_keysfield. - install: Add Archlinux support for installDependencies.sh
- k8s: External ports can now be specified in the Kafka API external listener configuration. When this port is specified, and if it's not available on all the nodes, redpanda will fail to schedule.
- k8s: Operator now by default creates PodDisruptionBudget that defaults to MaxUnavailable=1 and can be configured via CRD
- k8s: rpk-status sidecar is now disabled by default
- k8s: rpk-status sidecar resource limits were increased from 10M to 30M
- k8s: Operator depenencies bumped to 1.21.
- k8s: Operator can now create ingresses for Kubernetes clusters by default.
- raft: Initial leadership information for newly created topics is set more uniformly across the cluster, avoiding some client delays when accessing a newly created topic that has not gone through a leadership election yet.
- rpk: The fstrim tuner, which enables or installs the fstrim.service and fstrim.timer, is no longer enabled by default when running
rpk redpanda mode production
Bug fixes
- ACL: Fix wrong ACL user create arguments
- Idempotent producers: Fix bug affecting idempotent producers, reducing the rate of fatal errors
- kafka/client: Fix retry handling for several operations
- kafka/client: Fix concurrency bug with retry mitigation function
- kafka/server: fix double counting bytes sent metric
- schema_registry: Prevent deleting referenced schema
- schema_registry/store: Use correct shard in get_subject_schema
Note that downgrades from 21.11.1 to earlier releases are not supported.