redpanda-data/redpanda v24.1.1 on GitHub

New Features

Adds new cluster and topic level configurations for write caching feature. by @bharathv in #16924
PR #17009 write caching - raft implementation by @bharathv
Enables write caching by default in dev container mode. by @bharathv in #17677
Add rpk security roles, a new command space to manage your Redpanda roles. by @r-vasquez in [#17538](https://github.com/redpanda-data/ redpanda/pull/17538)
Introduce --allow-role and --deny-role flags for rpk acl commands by @oleiman in [#17416](https://github.com/redpanda-data/redpanda/ pull/17416)
Introduces GET /v1/security/users/roles (Admin API) by @oleiman in #17155
Introduces /v1/security/roles/{role}/members Admin API endpoint for reading and updating RBAC role members. by @oleiman in #17153
#17679 rpk security acl list now supports --format=json by @rockwotj in #17684
Data Transforms now support writing to multiple output topics. The REDPANDA_OUTPUT_TOPIC environment variable exposed in transforms is now removed for REDPANDA_OUTPUT_TOPIC_%d for each output topic specified. by @rockwotj in #16946
rpk transform deploy now supports multiple output topics by @rockwotj in #16950
The golang transform-sdk gains the ability to write to multiple output topics.
This feature can only be used in Redpanda v24.1.x or newer. by @rockwotj in #16978
The rust transform-sdk gains the ability to write to multiple output topics.
This feature can only be used in Redpanda v24.1.x or newer. by @rockwotj in #17007
Publish log (i.e. stderr/stdout) output from data transforms exclusively to an internally managed Redpanda topic (_redpanda.transform_logs). Data transform logs will no longer appear in broker logs. by @oleiman in [#16485](https://github.com/redpanda-data/redpanda/ pull/16485)
Introduce rpk transform logs NAME to view logs for a transform by @rockwotj in [#16923](https://github.com/redpanda-data/redpanda/pull/ 16923)
#16075 Data Transform's Rust SDK now supports a Schema Registry Client. by @rockwotj in #16464
Topic-aware partition balancing, which attempts to spread partition replicas topic-wise across a cluster. This behavior is controlled by the partition_autobalancing_topic_aware config property (enabled by default). by @ztlpn in #17263
Tiered Storage now supports using Azure VM user-assigned managed identities for securely accessing
Azure Blob Storage @andijcr in #17157
Topic recovery and ‘whole-cluster restore’ from Tiered Storage now perform integrity checks on metadata to ensure that each partition can be recovered successfully by @andijcr in #16915
You can now create namespaces in Redpanda Cloud using rpk cloud namespace. by @r-vasquez in [#16685](https://github.com/redpanda-data/redpanda/ pull/16685)
#13175 rpk debug bundle now includes a CPU profile of the requested nodes. by @r- vasquez in #16414
#16107 You can print a schema now using rpk registry schema get --print-schema. by @r-vasquez in #16109
#16623 rpk redpanda config bootstrap now supports bootstrapping your advertised addresses configuration. by @r-vasquez in #16652
new metric vectorized_storage_log_compacted_away_bytes for compaction observability in local storage added by @andijcr in [#17579](https://github. com//pull/17579)
new public metric redpanda_cluster_latest_cluster_metadata_manifest_age to track the age of the cluster_metadata_manifest in cloud storage added by @andijcr in #17404

Bug Fixes

Aggregates partitions in some cloud storage metrics when the aggregate_metrics cluster config is set to true. by @ballard26 in #16336
Fix a bug that could lead to raft log inconsistencies when 2 out of 3 nodes in a configuration are changed. by @ztlpn in [#17675](https:// github.com//pull/17675)
Fix a bug that resulted in Redpanda ignoring until the next restart config values that were reset to their defaults. by @ztlpn in #16504
Fix a bug where logging in a transform could cause the transform to not make progress. by @rockwotj in [#17186](https://github.com/redpanda- data/redpanda/pull/17186)
Fix a crash that happened when a cluster that was partially in recovery mode tried to upload consumer offsets to cloud storage. by @ztlpn in #17013
Fix a memory leak when using transactions with many different producer IDs. by @rockwotj in [#15797](https://github.com/redpanda-data/ redpanda/pull/15797)
Fix a potential cloud storage cache access time tracker file corruption during shutdown. by @nvartolomei in [#16648](https://github.com/ /pull/16648)
Fix a race condition between suffix truncation / delete records and adjacent segment compaction that can lead to crashes and data-loss. by @ nvartolomei in #17019
Fix a rare bug where http client connections would vanish from the connection pool leading to various operations hanging while waiting for an http client. by @nvartolomei in #15681
Fix an issue where rpk transform logs waits for records without the --follow flag specified. by @rockwotj in [#17832](https://github.com/ /pull/17832)
Fix an issue with Cargo.toml when initializing a Rust Data Transform project via rpk transform init by @rockwotj in [#15934](https:// github.com//pull/15934)
Fix initial_leader_epoch/KIP-320 handling in fetch requests. It was ignored until now which prevented consumers to correctly detect suffix truncation. For Redpanda (and Raft), this is a minor problem since suffix truncation is a very improbable event. by @nvartolomei in #17674
Fix internal RPC client connection stall after more than 2^32 requests are sent. by @ztlpn in [#16156](https://github.com/redpanda-data/ redpanda/pull/16156)
Fix large allocation in partition manifest. by @dotnwat in #16160
Fix oversized allocation in storage. by @Lazin in #16642
Fix the starter code for Rust projects in rpk transform init by @rockwotj in #16180
Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled. by @Lazin in #16163
Fixed a few oversized allocations for some admin server endpoints. by @rockwotj in [#16551](https://github.com/redpanda-data/redpanda/pull/ 16551)
Fixed the values for the rpc client in/out bytes metric by @ballard26 in #17933
Fixes rpk transform init --install-deps so that an explicit true value is not needed. by @rockwotj in [#17831](https://github.com/redpanda- data/redpanda/pull/17831)
Fixes a bug in windowed compaction that could cause Redpanda to crash when an error occurs while reading batches. by @andrwng in #16928
Fixes a bug of config_frontend methods getting called on shards other than the controller shard. by @pgellert in [#17088](https://github.com/ /pull/17088)
Fixes a bug that may prevent redpanda from shutting down cleanly when auditing is enabled by @graphcareful in [#16315](https://github.com/ /pull/16315)
Fixes a concurrency issue in transform offset commits pertaining to taking/applying snapshots. by @bharathv in [#17383](https://github.com/ /pull/17383)
Fixes a crash if a WebAssembly function is deployed that immediately crashes. by @rockwotj in [#15939](https://github.com/redpanda-data/ redpanda/pull/15939)
Fixes a crash that could happen when reading from local storage with a large number of segments that all do not contain user data. by @ andrwng in #18075
Fixes a plausible correctness issue with idempotent requests during replication failures. by @bharathv in [#16706](https://github.com/ /pull/16706)
Fixes a race between compaction and Raft recovery for compacted topics that could result in aborted transactional data batches being visible. by @andrwng in #16295
Fixes an an improper initialization of metrics related to controller snapshot uploads. by @andrwng in [#16070](https://github.com/redpanda- data/redpanda/pull/16070)
Fixes an issue where using the CPU profiler with running Data Transforms could cause the process to deadlock. by @rockwotj in #17877
Fixes issue that causes the connection to hang when an unsupported compression type is passed via an incremental_alter_configs request by @ graphcareful in #16399
Fixes lock starvation during transform offset commits. by @bharathv in #17402
Have fetch handler ensure rack awareness is enabled before performing follower fetching by @michael-redpanda in [#15883](https://github.com/ /pull/15883)
Prevent an assertion from being triggered when Wasm VMs fail immediately. by @rockwotj in [#15933](https://github.com/redpanda-data/redpanda/ pull/15933)
Prevent detecting leader epoch advancement when state is not up to date by @mmaslankaprv in [#16560](https://github.com/redpanda-data/ redpanda/pull/16560)
Prevent reactor stalls querying leadership information for large clusters by @rockwotj in [#17473](https://github.com/redpanda-data/redpanda/ pull/17473)
Protect against a very rare scenario where after node restart, some of the partition replicas hosted on that node could not take part in leader elections. by @ztlpn in #16068
Redpanda used to accept an empty string in redpanda.rack in node config. This would cause issues in Kafka operations. Redpanda will now error on startup if redpanda.rack is set to an empty string. by @michael-redpanda in [#15835](https://github.com/redpanda-data/redpanda/pull/ 15835)
Redpanda will now correctly handle an empty rack ID provided in a fetch request by @michael-redpanda in [#15846](https://github.com/redpanda- data/redpanda/pull/15846)
Reduces maximum log line size from 1MiB to 128KiB to reduce occurrences of memory allocation failures by @michael-redpanda in #17922
Report runtime public metrics by task queue for all cores, not just core 0 by @rockwotj in #16154
Return a HTTP 400 error code when deploying a transform to a topic that doesn't exist instead of a 500 by @rockwotj in [#17011](https:// github.com//pull/17011)
Schema Registry: Deleted schemas no longer reappear after certain compaction patterns on the _schemas topic. by @BenPope in #17091
#15042 Fixes a bug in the tiered storage time-based query implementation that could result in a consumer hang when consuming very old data. by @andrwng in #16645
#15201 Fix assertion triggered by interleaving of log flush and log truncation followed by append by @Lazin in #16105
#15603 cluster config aliases are accepted while reading from yaml by @andijcr in #15605
#15674 Fix an issue where new configs would continually revert to legacy defaults after an upgrade. by @oleiman in #15761
#15722 #7946 Fix an issue where create topics responses would show incorrect partition count and replication factor by @oleiman in [#16410](https://github.com/redpanda- data/redpanda/pull/16410)
#15811 Several additional metrics will have their "partition" label aggregated away (i.e., into a single series per remaining label set with no partition label, whose value is the sum of all input series with the same label set and different partition labels). This is already the default behavior for most metrics, but this change extends it to almost all remaining metrics. by @travisdowns in #15966
#15839 safer handle unknown properties in local state by @andijcr in #15840
#15909 Prevent oversized allocs when group fetching from many partitions. by @ rockwotj in #15918
#16129 Fixes a bug in SASL user deletion and update where usernames with a + symbol in the username were prevented from being deleted by @pgellert in #16694
#16251 rpk: fixed a bug where the --password flag could not be used along with the new configuration flag -X pass in clusters where basic authentication was enabled. by @r-vasquez in [#16278](https://github.com/redpanda- data/redpanda/pull/16278)
#16259 Fixes a bug that would previously cause read replicas to report the wrong value for the redpand_kafka_max_offset metric. by @andrwng in #16263
#16320 Prevent oversized allocation with large amounts of controller metadata by @ rockwotj in #16381
#16322 Fix graceful shutdown of the TS archive area retention procedure. by @Lazin in #16382
#16402 Fix invalidate iterator access when yielding control to the scheduler in data transforms debug endpoint. by @rockwotj in #16421
#16479 Fix timequery error that triggered full partition scan by @Lazin in #16503
#16521 Avoid a large contiguous allocation when creating thousands of topics in a single CreateTopics request. by @travisdowns in #16529
#16552 #16064 rpk tune --output-script: Add a missing new line in the ballast file tuner when using the --output-script flag by @r-vasquez in #16576
#16578 Fixes a bug in CreateTopicsResponse to now return all the configs of the topic, not just the topic-specific override configs. by @pgellert in #16922
#16612 fixes small inconsistency between Kafka and Redpanda when trying to query end_offset of an empty log by @mmaslankaprv in #17789
#16643 Fixed deleting Data Transforms with names that had URL unsafe characters by @rockwotj in #16858
#16840 Fixes a bug with TLS metrics where expiration timestamps would not advance on certificate reload by @oleiman in #17233
#17086 fixed enabling cloud storage in existing clusters by @mmaslankaprv in #17112
#17731 Fix incorrect log truncations caused by delayed replication requests. by @ ztlpn in #17895
#17788 Fix problem in Tiered-Storage that could potentially cause consumers to get stuck by @Lazin in #17805
#17827 fix a race between eviction and producer registration that results in an invalid transaction state. by @bharathv in #17880
#2225 Fix reported config source for cleanup.policy by reporting DEFAULT_CONFIG instead of DYNAMIC_TOPIC_CONFIG for the default value. by @pgellert in #17456
ext4 is no longer incorrectly detected as ext2 (all of ext2, 3 and 4 are assumed to be ext4). by @travisdowns in [#13496](https://github.com/ /pull/13496)
fixed a problem leading to UAF error while calculating cloud stage usage by @mmaslankaprv in [#17975](https://github.com/redpanda-data/ redpanda/pull/17975)
fixed incorrect fetch offset validation by @mmaslankaprv in #16146
fixes a bug where compacting away last aborted data batch in a segment may cause readers to be stuck because of a gap in the segment ranges thus blocking raft recovery by @andrwng in #16295
prevents partial consumer group recovery by @mmaslankaprv in #17673
rpk: prevent a segfault when creating a profile from a cloud that is not in ready state. by @r-vasquez in [#17553](https://github.com/ /pull/17553)

Improvements

Add new upload management mechanism by @Lazin in #16684
Add an internal debugging endpoint to see the committed progress of data transforms. by @rockwotj in [#16185](https://github.com/redpanda- data/redpanda/pull/16185)
Add Prometheus metrics for data transforms logging by @oleiman in
Introduce "trust_file_crc32c" metric to export a checksum for each trust file in the system. by @oleiman in [#17539](https://github.com/ /pull/17539)
#16566
Marks the fetch_read_strategy cluster configuration property as deprecated by @ballard26 in [#17191](https://github.com/redpanda-data/ redpanda/pull/17191)
Record result of running microbenchmarks in a file using JSON format by @mfleming in [#15912](https://github.com/redpanda-data/redpanda/pull/ 15912)
#14069 Adds wait_ms parameter to CPU profiler admin API. The API will wait for wait_ms milliseconds then return the profile samples collected during that period of time. by @ballard26 in [#14468](https://github.com/ /pull/14468)
spillover manifests are enabled by default for clusters that did not explicit set a value or null by @andijcr in #16172
new metric vectorized_ntp_archiver_compacted_replaced_bytes by @andijcr in #17627
Removes plan_and_execute_latency_us histograms from internal metrics by @ballard26 in [#17191](https://github.com/redpanda-data/redpanda/ pull/17191)
A clearer error message when the data directory does not exist or is the wrong type by @travisdowns in [#15821](https://github.com/redpanda- data/redpanda/pull/15821)
Add a dedicated CPU scheduling policy for Data Transforms by @rockwotj in #16114
Add error messages when an unsupported Schema Registry client is used in Data Transforms by @rockwotj in #17591
Adds UpdateRoleMembership function to rpk admin API client. by @bojand in #17589
Adds a new cluster configuration property fetch_read_strategy. This property determines which fetch execution strategy Redpanda will use to fulfill a fetch request. The newly introduced non_polling execution strategy is the default for this property with the polling strategy being included to make backporting possible. by @ballard26 in #15328
Adds a new public metric redpanda_raft_recovery_partition_movement_consumed_bandwidth that tracks how much bandwidth is currently in use for raft recovery. This helps tune raft_learner_recovery_rate. by @bharathv in #16842
Adds metric for number of quorum ack requests with write caching. by @bharathv in [#17179](https://github.com/redpanda-data/redpanda/pull/ 17179)
Adds new cluster configuration parameter minimum_topic_replications that permits Redpanda administrators to set a minimum permitted replication factor for newly created topics. by @michael-redpanda in #15777
Adds observability into producer evictions in each shard. by @bharathv in #16724
Caches the connections local address preventing the need to make a system calls to grab this value when auditing events. by @graphcareful in #15931
Changes what the kafka_latency_fetch_latency metric measures to be the time the first fetch_ntps_in_parallel takes. by @ballard26 in #17720
Data Transforms written in Golang now use a non-buffered write mechanism by @rockwotj in
#15922
Data Transforms written in Rust now use a non-buffered write mechanism by @rockwotj in #15923
Fix large wasm module deployments by @rockwotj in #16722
Handle missing data transform logs topic in rpk transform logs by @rockwotj in [#17830](https://github.com/redpanda-data/redpanda/pull/ 17830)
Improve concurrency control and metadata consistency in Tiered-Storage. by @Lazin in [#16774](https://github.com/redpanda-data/redpanda/pull/ 16774)
Improve log segment upload mechanism which avoid reading files from disk directly. by @Lazin in [#16999](https://github.com/redpanda-data/ redpanda/pull/16999)
Improved handling of follower fetching offset validation when used with relaxed consistency by @mmaslankaprv in [#16475](https://github.com/ /pull/16475)
Improves observability by allowing Redpanda to detect that some internal processes are stuck. by @Lazin in [#16466](https://github.com/ /pull/16466)
Increase data_transforms_logging_buffer_capacity_bytes from 100KiB to 500KiB by @oleiman in [#16972](https://github.com/redpanda-data/ redpanda/pull/16972)
Introduces a new non-polling fetch execution strategy that decreases CPU utilization of fetch requests and fetch request latency. by @ ballard26 in #15328
Large allocations are now logged by default (similar to reactor stalls) by @StephanDollberg in [#16828](https://github.com/redpanda-data/ redpanda/pull/16828)
Making the Redpanda README.md more beautiful and functional! by @WillemKauf in [#17822](https://github.com/redpanda-data/redpanda/pull/ 17822)
Prevent metadata oversized allocations when requesting many topics. by @rockwotj in [#15751](https://github.com/redpanda-data/redpanda/pull/ 15751)
Publish total reclaimable space to avoid stuck decommission scenario. by @dotnwat in [#16354](https://github.com/redpanda-data/redpanda/pull/ 16354)
Reduces the number of allocations performed by the auditing subsystem by @graphcareful in [#16056](https://github.com/redpanda-data/redpanda/ pull/16056)
SIMD instructions are generated by default for WebAssembly binaries when building with rpk. by @rockwotj in [#16296](https://github.com/ /pull/16296)
Schema Registry: Improve retry logic for delete_config and delete_subject_permanent by @BenPope in [#17905](https://github.com/redpanda- data/redpanda/pull/17905)
Schema Registry: Improve tombstoning when deleting a subject by @BenPope in #17905
Support changing the timeout for WebAssembly functions by @rockwotj in #15967
Support dynamically changing the limit for WebAssembly binary size by @rockwotj in [#15967](https://github.com/redpanda-data/redpanda/pull/ 15967)
The full_raft_configuration_recovery_pattern cluster config property is now deprecated and will be ignored by Redpanda. by @andrwng in #16890
This PR partially reverts the change such that strict retention remains enabled after upgrade unless it had been explicitly disabled before the upgrade. by @dotnwat in #16077
Transform SDKs for Schema Registry now publish their ABI version for compatibility checks with the broker. by @rockwotj in [#17593](https:// github.com//pull/17593)
Transforms now start at a record relative to the deploy time instead of the time at which the VM starts.
This allows for easier testing of transforms, as one does not have to wait for the VM to boot before producing. by @rockwotj in #17590
Validate transform code at deploy time to ensure the correct SDK is used. by @rockwotj in [#16290](https://github.com/redpanda-data/redpanda/ pull/16290)
#13023 rpk container now prints the container logs in case of failures during startup. by @r-vasquez in #17780
#14116 SR/PP will now reply with a 500 error if an internal service semaphore has been completely exhausted by @michael-redpanda in #15977
#14814 Added new metric to provide Follower Fetching feature observability by @ mmaslankaprv in #15799
#15818 rpk cluster health: now --exit-when-healthy enables --watch when provided. by @r-vasquez in #16106
#15900 Internal kafka client now uses asynchronous compression (when possible) to reduce possibility of oversized allocations and reactor stalls by @michael-redpanda in [#15920](https://github.com/redpanda-data/redpanda/pull/ 15920)
#16162 get_cluster_uuid returns a correctly formatted string by @andijcr in #16183
#16552 #16064 rpk tune --output-script: rpk now creates a file for you if the provided file does not exist. by @r-vasquez in [#16576](https://github.com/redpanda-data/ redpanda/pull/16576)
#16758 cluster: Avoid oversize allocs for topic creation and configuration by @ BenPope in #16982
#16771 Data Transform builds in rpk now uses tinygo v0.31.1 by @rockwotj in #16856
#16795 Added ability to change transactional manage topic properties by @ mmaslankaprv in #16797
#16901 Added EHOSTUNREACH to retry-able error code list by @michael-redpanda in #16902
#17197 more accurate node status reporting by @mmaslankaprv in [#17625](https:// github.com//pull/17625)
#17368 Improves error feedback when Redpanda is given an invalid number of partitions during either topic creation or when the partition count for a topic is increased. by @michael-redpanda in [#17369](https://github. com//pull/17369)
#17925 Allocate and rebalance partition replicas in random order to prevent an undesirable pattern when many partitions have the same replica set. by @ztlpn in #17962
#8809 Node-wide throughput throttling is now fair an responsive. by @BenPope in #16441
[rpk] more informative error message display on create topic failure by @michael-redpanda in [#15837](https://github.com/redpanda-data/ redpanda/pull/15837)
rpk acl user has been moved to rpk security security. Old command is soft deprecated. by @r-vasquez in [#17664](https://github.com/ /pull/17664)
rpk acl has been moved to rpk security acl. Old command is soft deprecated. by @r-vasquez in [#17664](https://github.com/redpanda-data/ redpanda/pull/17664)
rpk profile has been reworked in an attempt to be simpler; see PR #17038 for more detail by @twmb in [#17038](https://github.com/redpanda- data/redpanda/pull/17038)
rpk redpanda start know knows about io characteristics on EC2 instance types i4i, is4gen and im4gn in addition to i3en and i3, allowing more effective IO scheduling even if rpk iotune has not been run. by @travisdowns in [#17220](https://github.com/redpanda-data/redpanda/pull/ 17220)
rpk transform deploy --file now supports https:// URLs by @rockwotj in #16050
rpk transform deploy takes a --file flag to deploy a compiled WebAssembly binary. by @rockwotj in [#15932](https://github.com/redpanda- data/redpanda/pull/15932)
verbose_logging_timeout_max node config, to prevent setting TRACE and DEBUG log level indefinitely by @oleiman in [#15798](https://github. com//pull/15798)
better ERROR|WARN messages for candidate_creation_error by @andijcr in #17090
better control of memory usage in storage layer. by @mmaslankaprv in #16846
ensure only positive values are accepted for log_segment_ms_min/max by @andijcr in [#16513](https://github.com/redpanda-data/redpanda/pull/ 16513)
largely reduced number of health report copies by @mmaslankaprv in #17863
less overhead of health report collection by @mmaslankaprv in #17158
minimising possibility of data loss when using relaxed conistency replication by @mmaslankaprv in #17915
new metric redpanda_cloud_storage_cloud_log_size reports size in bytes of the log in cloud storage by @andijcr in #17445
optimized updating leadership metadata with health reports by @mmaslankaprv in #16512
preventing large allocation in partition balancer code by @mmaslankaprv in #16917
rpk: Remove 10s timeout in rpk profile create by @r-vasquez in #16836
rpk: change hash for backup filenames to use sha256 by @andrewhsu in #17568
rpk: tune script uses sha256sum instead of md5sum by @andrewhsu in #17595
rpk: use go module version of common proto definitions. by @bojand in #17628
skipping overhead of collecting node health report for each node separately. by @mmaslankaprv in [#17715](https://github.com/redpanda-data/ redpanda/pull/17715)
smaller memory footprint when using with large number of topics with small partition count by @mmaslankaprv in [#16247](https://github.com/ /pull/16247)

None

No release notes explicitly specified.

PR #13532 Add small batches nightly regression test by @ballard26
PR #13927 archival: Add offset_range_size method to the storage::log by @Lazin
PR #14599 archival: Start housekeeping jobs after STM sync by @Lazin
PR #14745 storage: assert when appending empty batch by @andrwng
PR #15199 rptest: ignore test_cloud_cache_thrash by @nvartolomei
PR #15271 Fix CI failure in test_concurrent_append_flush by @travisdowns
PR #15287 rptest/scale_test: enable test_many_partitions by @nvartolomei
PR #15407 c/topic_table: do not log duplicated lifecycle marker command by @ mmaslankaprv
PR #15424 Manage STM creation outside of cluster::partition by @mmaslankaprv
PR #15594 removes kics by @wreet
PR #15616 Add at: in Top-N alloc site output by @travisdowns
PR #15669 ci: Add bucket cleanup scripts by @savex
PR #15672 tests: repro for zombie txs after compaction by @andrwng
PR #15677 c/archival_stm: do not reset _last_replicate on timeout by @nvartolomei
PR #15678 c/topics_dispatcher: do not guesstimate leader ids by @mmaslankaprv
PR #15697 cloud_storage: hold gate in hydration by @andrwng
PR #15701 rptest: add test_add_and_decommission() by @andrewhsu
PR #15718 kafka: fix holding of attached_partition by @andrwng
PR #15724 archival: remove outdated codedoc by @nvartolomei
PR #15725 cluster/config_manager: use property's main name in store_delta by @andijcr
PR #15726 transform-sdk/rust: fix publishing by @rockwotj
PR #15730 rptest: fix an always true assertion by @nvartolomei
PR #15731 c/log_eviction_stm: do not take the snapshot if start offset progressed by @mmaslankaprv
PR #15735 security: update supported version by @rockwotj
PR #15736 transform-sdk/go: fix releases by @rockwotj
PR #15737 Fix bad argument passed to kgo-repeater constructor by @graphcareful
PR #15738 c/members_member: don't ignore update_broker_client future by @rockwotj
PR #15739 Increase audit buffer sizes for audit scale test by @graphcareful
PR #15740 cluster_recovery: direct requests to controller leader by @andrwng
PR #15743 cloud_storage_clients: use appropriate beast error for end of stream by @ nvartolomei
PR #15748 cloud_metadata: upload consumer groups in batches by @andrwng
PR #15754 rptest: clean up cluster metadata in CDT by @andrwng
PR #15755 rptest: log error on failure to delete bucket by @andrwng
PR #15765 follow-up: config/config_store_test: ignored_keys test by @andijcr
PR #15778 Fix 128K iobuf zero-copy by @travisdowns
PR #15779 e2e: Basic producer and consumer using Table API by @savex
PR #15804 c/archival_stm: downgrade timeout log to warn by @nvartolomei
PR #15809 rptest: update cloud test suite to subset by @andrewhsu
PR #15813 archival: disable scrubber by default by @nvartolomei
PR #15814 rptest: increase grace period for finalize by @nvartolomei
PR #15815 cluster/config_manager: ensure cleanup of old aliases by @andijcr
PR #15817 audit: Renamed audit log topic by @michael-redpanda
PR #15820 Skip assertion in audit log tests if results beat the baseline by @ graphcareful
PR #15822 test: remove source of false positive failure by @dotnwat
PR #15836 gha: remove invalid permission by @rockwotj
PR #15844 Disable cloud metadata upload and cluster restore in recovery mode by @ztlpn
PR #15852 t/kgo: upgrade kgo to do a full run after /last_pass by @nvartolomei
PR #15853 rptest: allow the new version of xfs/ext4 fs error msg by @nvartolomei
PR #15862 c/s/leader_balancer: prevent oversized alloc by @rockwotj
PR #15870 gh/workflow: add build message in promote trigger by @gousteris
PR #15882 gha: s/git.ref_name/github.ref_name by @rockwotj
PR #15889 utils: Move cstore column and frame implementation to utils by @Lazin
PR #15890 gha: improve backport reviewers by @rockwotj
PR #15903 fix grammar in README by @bachmanity1
PR #15910 cmake: Fix minor typos in doc string by @mfleming
PR #15919 r/offset_translator: remove unsafe bootstrap code by @ztlpn
PR #15924 rm_stm: remove dead code by @bharathv
PR #15935 gha: create stale issue workflow by @rockwotj
PR #15942 gha: mark ci-failures as stale by @rockwotj
PR #15945 c/log_eviction_stm: do not request snapshot if already progressed by @ mmaslankaprv
PR #15948 cloud_storage: Improve scrubber by @Lazin
PR #15951 dt-deps: Bump openjdk and Keycloak by @oleiman
PR #15953 tx_migration: avoid ping pong of requests between brokers by @bharathv
PR #15955 rptest: Basic transaction produce test by @savex
PR #15957 topic_recovery_test: adjust wait condition for finished uploads by @andrwng
PR #15962 dt: Fixed flaky schemas test by @michael-redpanda
PR #15963 tests/si_utils: error message for quiesce_uploads timeouts by @andijcr
PR #15968 build: introduce foundational base library by @dotnwat
PR #15971 rpc: pass requests by value by @rockwotj
PR #15981 k/metadata: guesstimate leader when information is not yet present by @ mmaslankaprv
PR #15982 c/partition_balancer: use full partition move when disk is full by @ mmaslankaprv
PR #15983 utils: allow memory data source to take tmpbuf by @dotnwat
PR #15986 securit/:OIDC: Enable licence check and telemetry by @BenPope
PR #15990 tests/si_utils: assert message for verify_file_layout() by @andijcr
PR #15993 rptest: more typing 1 by @travisdowns
PR #15997 build: expose submodule public headers by @dotnwat
PR #16000 tests: Use multiple topics in OMBValidationTest.test_max_partitions by @ StephanDollberg
PR #16025 tests: delete run script by @rockwotj
PR #16028 rptest: use 10 nodes in OMB validation tests by @travisdowns
PR #16029 features: Update to version 12 by @StephanDollberg
PR #16049 rptest: clean cloudv2 cluster topics in tearDown() by @andrewhsu
PR #16053 build: make random a foundational library by @dotnwat
PR #16055 Controller backend: reconcile each ntp in individual fibers by @ztlpn
PR #16067 Introduced partition shutdown watchdog timer by @mmaslankaprv
PR #16071 archival: avoid division by 0 when computing slow down rate by @nvartolomei
PR #16076 gha: use latest action for stale by @andrewhsu
PR #16078 Fix for CI Failure (Failed to make list_offsets request, unknown broker) in DeleteRecordsTest.test_delete_records_segment_deletion by @graphcareful
PR #16083 Add oversized allocation to bad log lines by @ballard26
PR #16086 tests: Make the OMB validator use result.json by @StephanDollberg
PR #16095 transform-sdk: decouple releases from broker by @rockwotj
PR #16097 rptest: Additional logging audit_log_test infra by @graphcareful
PR #16108 Rename max_client_count to max_connection_count by @travisdowns
PR #16119 rptest: Implement flink service memory autosizing by @savex
PR #16121 hashing: remove unused hashing code by @dotnwat
PR #16123 build: move bytes headers into public include path by @dotnwat
PR #16125 rptest/scale_test: Remove p50 check in audit test by @graphcareful
PR #16126 rptest: bump multiplier to 2.0 for test_max_partitions() by @andrewhsu
PR #16127 rptest: Wait for the group to be in a stable state before starting tests by @graphcareful
PR #16132 rptest: bad field use in test_consume_miss_cache by @travisdowns
PR #16133 rptest: invalid comparison in test_max_connections by @travisdowns
PR #16134 rptest: more typing 2 by @travisdowns
PR #16135 ci: Fix flink install on ARM64 by @savex
PR #16136 OMBSampleConfigurations predicates are static by @travisdowns
PR #16137 tests: update email in setup.py by @andrewhsu
PR #16140 gha: run build-redpanda workflow once per day by @ivotron
PR #16144 build: add container and strings library by @dotnwat
PR #16148 rptest: Transaction workload with scaling in mind by @savex
PR #16152 build: move frag_vec to container library by @dotnwat
PR #16153 Introduce wasm::logger by @oleiman
PR #16157 rptest: reduce HighThroughputTests.msg_size by @andrewhsu
PR #16158 Change metrics to match generate_charts.py by @ballard26
PR #16159 Stabilize OMB regression tests by @ballard26
PR #16181 rptest: log kubectl errors by @travisdowns
PR #16184 rptest: expose health check reason by @travisdowns
PR #16189 archival: apply PR feedback by @dotnwat
PR #16190 Transform logging data model by @oleiman
PR #16210 transform-sdk/rust: borrow output record by @rockwotj
PR #16211 cloud_storage: Fix unit-test timeout on ARM CI by @Lazin
PR #16212 tests/tiered_storage_model_test: temp ok_to_fail by @andijcr
PR #16215 io: add fault injection helpers to persistence layer by @dotnwat
PR #16226 tests/rptest/tests/tiered_storage_model_test: ignore by @andijcr
PR #16229 transform-sdk: experimental C++ support by @rockwotj
PR #16232 Change cloud test hierarchy, introduce cloud test base by @travisdowns
PR #16233 admin api: skip partition info in /brokers end point by @bharathv
PR #16234 ducktape: fix flink URL 404 by @bharathv
PR #16242 tests: updated flink to 1.18.1 by @mmaslankaprv
PR #16243 Node UUID admin by @mmaslankaprv
PR #16244 Make the request context fixture code work with produce requests by @ mfleming
PR #16248 "enable by default spillover manifest" testing followups by @andijcr
PR #16249 ci: Mark test ok_to_fail untill fix released by @savex
PR #16250 rptest: Refactor consumer validation and tune table idle settings by @savex
PR #16253 wasm: transform callback by @rockwotj
PR #16257 container: introduce chunked vector by @rockwotj
PR #16258 gha: don't mark backports as drafts by @rockwotj
PR #16260 lint: fix linter error by @rockwotj
PR #16277 rptest: simplify metric_sum filtering by @andrwng
PR #16283 dt/deps: Use setuptools for confluent-kafka source build by @oleiman
PR #16284 cloud_storage_compaction_test: a couple test fixes by @andrwng
PR #16285 rm_stm: remove mem_state::last_end_tx by @bharathv
PR #16286 rm_stm: remove log_state::prepared by @bharathv
PR #16287 Fixed large allocation in kafka::wait_for_leaders by @mmaslankaprv
PR #16293 labeler: apply wasm label for Redpanda subsystems by @rockwotj
PR #16301 Introduce transform::logging::manager by @oleiman
PR #16302 demote ERROR message to DEBUG for timequerys at the edge of spillover retention by @andijcr
PR #16303 tests: replaced no_timeout with default timeout in raft test by @ mmaslankaprv
PR #16304 Wrapped logging with vlog macro in places that missed it by @mmaslankaprv
PR #16311 resource_mgmt: disable flaky test by @rockwotj
PR #16314 transform-sdk/rust: update rustdocs by @rockwotj
PR #16319 rptest: add ability to restart pods in RedpandaServiceCloud by @andrewhsu
PR #16337 rptest: fix cluster healthy check by @andrewhsu
PR #16347 rptest: Bump audit logging scale test run time by @graphcareful
PR #16353 CDT: fix rpk tune list test in GCP. by @r-vasquez
PR #16359 rptest: clean teleport data dir before use by @andrewhsu
PR #16361 rptest: Fix race condition in CloudRetentionTest by @Lazin
PR #16375 rptest: Create multi-node test with 1k topics and >10M events by @savex
PR #16383 rptest: Add error message to log allow list by @Lazin
PR #16384 rptest: Fix throttling e2e test by @Lazin
PR #16385 cloud_storage_clients/client_pool: handle broken _self_config_barrier by @ andijcr
PR #16386 tests: Add produce microbenchmark by @mfleming
PR #16400 coding-style: propose documentation recommendations by @rockwotj
PR #16427 slash-cmd: fix /backport command by @gousteris
PR #16428 rptest: Fix s3.copy_object when running on GCP by @savex
PR #16429 metrics-mock: initial commit by @travisdowns
PR #16437 io: misc updates and test helpers by @dotnwat
PR #16440 transform: data path for multiple output topics by @rockwotj
PR #16443 config/config_store_test: unit test for property::operator= + watchers by @ andijcr
PR #16454 io: add io queue manager by @dotnwat
PR #16472 rptest: tolerate missing topic metrics after start by @nvartolomei
PR #16478 transform/logging/test: deflake TransformLogManagerTest.LargeBuffer by @ rockwotj
PR #16489 rptest: exclude OMBValidationTest.test_retention by @andrewhsu
PR #16495 tests: disable snapshots when testing partial controller deletion by @ mmaslankaprv
PR #16496 rptest: Ignore error logs generated by anomaly injection by @abhijat
PR #16497 rptest: enable test_add_and_decommission by @andrewhsu
PR #16499 rptest: Fix instance type getter for GCP by @savex
PR #16502 container: chunked_vector improvements by @rockwotj
PR #16505 rptest: use okta auth for teleport by @andrewhsu
PR #16506 Fix missing comma in kubectl.py by @StephanDollberg
PR #16525 rpc: Add config flag to enable/disable compression for replies by @ StephanDollberg
PR #16526 transform: gc committed offsets by @rockwotj
PR #16527 Increase default value of rpc_client_connections_per_peer to 32 by @ ballard26
PR #16538 Introduced cluster::vcluster_id type underlied by XID by @mmaslankaprv
PR #16539 c/namespaced_cache: added implementation of namespaced cache by @ mmaslankaprv
PR #16542 rptest: force cleaning of teleport data dir by @andrewhsu
PR #16548 rpc: Disable compression for internal rpc replies by @StephanDollberg
PR #16554 Rethrow on unknown exceptions in fetch handler by @ballard26
PR #16571 xform/service: Remove exception handling from _log_manager->start by @ oleiman
PR #16577 Set KAFKA_OPTS to empty in OMB by @travisdowns
PR #16580 cloud_storage: update upload_object api by @abhijat
PR #16611 cloud_storage/inventory: Add inventory create-config API for AWS by @abhijat
PR #16614 cloud_storage: Add download_object method for downloading small objects by @abhijat
PR #16617 Flex partition assignment stage 1 by @ztlpn
PR #16633 io: add scheduler by @dotnwat
PR #16636 cst/inv: Create helpers to conditionally create inventory config by @abhijat
PR #16637 Added timeout to gtest_raft_rpunit by @mmaslankaprv
PR #16646 rptest: remove config_file credential source support by @nvartolomei
PR #16653 Increase allowed p75 e2e latency for ts_write OMB test by @ballard26
PR #16656 serde: Add node_health_report ser/de benchmark by @StephanDollberg
PR #16658 Virtual connections in Kafka server by @mmaslankaprv
PR #16662 Implement async_for_each by @travisdowns
PR #16664 vint_bench by @travisdowns
PR #16667 ci: Bump client-swarm to latest commit by @savex
PR #16668 PESDLC-109 rptest: add metrics gathering methods to RedpandaServiceCloud by @andrewhsu
PR #16669 test: retry on readyReplicas test by @dotnwat
PR #16671 test: check for non-empty samples from any core by @dotnwat
PR #16674 Revert "rptest: remove config_file credential source support" by @ nvartolomei
PR #16675 r/tests: removed infinite loop from raft test by @mmaslankaprv
PR #16676 Add forward iterator to async_for_each by @travisdowns
PR #16680 test: wait for cluster to become healthy after setup by @dotnwat
PR #16693 rpk: update help text of decommission-status by @daisukebe
PR #16704 rptest/scale_tests: compute expected time for non-ts workload by @ nvartolomei
PR #16705 http/client: add request interface that takes an iobuf by @dotnwat
PR #16710 tests: Fix missing include in redpanda_cloud_test by @StephanDollberg
PR #16711 c/p_leaders: check if leaders table was modified after async iteration by @ mmaslankaprv
PR #16714 Added redpanda.virtual.cluster.id topic property by @mmaslankaprv
PR #16715 cloud-tests: use a time-based calculation of load by @travisdowns
PR #16718 http/client: add post() method helper by @dotnwat
PR #16720 io: add input stream equality test helper by @dotnwat
PR #16721 storage: move ghost batch generation into log reader by @andrwng
PR #16745 u/named_type: use fmt to format named type by @mmaslankaprv
PR #16750 PESDLC-901 Tests to create 10k+ topics and do checks with low throughput by @savex
PR #16751 http: remove demo by @dotnwat
PR #16763 Introduce RBAC data model by @oleiman
PR #16764 rptest: fix hwm computation in topic_recovery_test by @andrwng
PR #16765 rptest: better health check by @travisdowns
PR #16768 Introduce principal_type::role by @oleiman
PR #16775 tests: refactor consumer group balancing test by @mmaslankaprv
PR #16776 c/leaders: trigger leadership notification when term changes by @ mmaslankaprv
PR #16780 Cloud Storage: Ignore abort_requested_exception on shutdown by @jcipar
PR #16781 RBAC Admin API Swagger and handler stubs by @oleiman
PR #16796 Add support for client_swarm's metrics api to produce_swarm by @ballard26
PR #16798 c/topic_table_probe: use btree_map in topic table probe by @mmaslankaprv
PR #16799 tests: dropped old upgrade tests which validated v22.2 version by @ mmaslankaprv
PR #16817 PESDLC-890 Use timeout when creating topics by @savex
PR #16824 PESDLC-895 Split produce/consume methods by @savex
PR #16825 Crypto lib by @michael-redpanda
PR #16826 rpk: add debug logs to BYOC by @r-vasquez
PR #16835 rpk: replace web UI url for cloud clusters by @r-vasquez
PR #16841 rpk: bump deps and toolchain to 1.22 by @r-vasquez
PR #16843 PESDLC-919 Constant data generation workload by @savex
PR #16845 Remove v23.1 from the PR template. by @piyushredpanda
PR #16847 Raft vote process unification by @mmaslankaprv
PR #16850 Fixed background apply fiber race condition in raft:: state_machine_manager by @mmaslankaprv
PR #16853 cloud_storage: Improve stale_reader test by @Lazin
PR #16857 cmake: upgrade tinygo compiler by @rockwotj
PR #16883 docs: update version in README by @rockwotj
PR #16886 cloud_storage: various non-functional changes by @andrwng
PR #16892 storage: adopt offset_translator by @andrwng
PR #16900 r/tests: added context to visible offset validating test assertion by @ mmaslankaprv
PR #16905 crypto: Updated jwt to use crypto lib by @michael-redpanda
PR #16906 security/license: Use crypto library by @michael-redpanda
PR #16907 Revert "rm_stm/idempotency: fix the producer lock scope" by @bharathv
PR #16909 Add support for cryptolib RNG by @michael-redpanda
PR #16910 Use cryptolib instead of gnutls for HMAC and secure digests by @michael- redpanda
PR #16911 tests: add test for GTEST_SKIP_CORO by @rockwotj
PR #16919 c/topic_table: replaced partition metadata map with chunked_vector by @ mmaslankaprv
PR #16920 k/group_manager: used chunked_vector when cleaning groups by @mmaslankaprv
PR #16927 Add OpenSSL Library Context Service by @michael-redpanda
PR #16929 storage: add missing flush in a test by @nvartolomei
PR #16931 fix for cluster_config_test.py::test_aliasing by @andijcr
PR #16933 gha: allow manually dispatching the open source build by @rockwotj
PR #16936 crypto/cmake: Fixed exact versioning by @michael-redpanda
PR #16943 rptest: Fix cloud_retention_test by @Lazin
PR #16953 wasm: add out_handle by @rockwotj
PR #16954 storage: support dirty entries in batch cache by @nvartolomei
PR #16958 Ensure fragment_vector fragments are always <= 128KiB by @ballard26
PR #16962 cst/inv: locate latest report by @abhijat
PR #16969 tests: Tweak OMBValidationTest.test_max_connections by @StephanDollberg
PR #16970 Prepare leader_balancer for node-local core assignment by @ztlpn
PR #16971 dependencies: Added missing OpenSSL dependencies by @michael-redpanda
PR #16973 rptest: improve data transform tests by @rockwotj
PR #16975 Name all util/mutexes by @travisdowns
PR #16976 increased a timeout for cluster replicas by @rpdevmp
PR #16980 Update client-swarm dependency by @ballard26
PR #16986 storage: space management o11y by @dotnwat
PR #16987 crypto: Move special members after impl definition by @BenPope
PR #16988 cloud: Address some warnings by @BenPope
PR #16989 raft|cluster: Address some warnings by @BenPope
PR #16990 treewide: Address some warnings by @BenPope
PR #16991 dt: Fix flaky TLS metrics test by @oleiman
PR #16992 Shard producer state manager with v_cluster_id by @mmaslankaprv
PR #16994 transform-sdks: introduce integration tests by @rockwotj
PR #17000 ducktape: temp fix for erlang 26.2.3 arm by @andrewhsu
PR #17002 treewide: Reduce header dependencies by @BenPope
PR #17003 CORE-6: Pre-fetching algorithms by @michael-redpanda
PR #17010 Check cloud type and do not run if FMC by @rpdevmp
PR #17021 PESDLC-953 ConsumerSwarm fixes by @savex
PR #17024 ducktape: lint ocsf-server by @andrewhsu
PR #17025 Role based authorization by @oleiman
PR #17026 CORE-2459 Fix UBSAN diagnostics by @nvartolomei
PR #17027 tools/cmake_test: remove accidental env override by @nvartolomei
PR #17028 crypto: Cleanup includes by @BenPope
PR #17029 Add /microbench slash command for running microbenchmarks by @mfleming
PR #17032 storage: assert no inflight writes when closing segment by @nvartolomei
PR #17037 Commented out this log info since it is AWS specific and affects GCP by @ rpdevmp
PR #17039 storage: ensure monotonic stable offset updates by @nvartolomei
PR #17084 io: add pager for append-only file abstraction by @dotnwat
PR #17085 rptest: fix _lifecycle_test naming by @travisdowns
PR #17087 container: add interval_set by @andrwng
PR #17089 admin: implement create role API by @pgellert
PR #17092 Using contiguous_range_map in partition_leaders_table by @mmaslankaprv
PR #17101 cloud_storage_clients: classify request_timeout as retriable by @nvartolomei
PR #17102 PESDLC-978 Add status check when conducting a health assertion of the cluster by @savex
PR #17103 added topic parameter by @rpdevmp
PR #17109 rpk: bump deps and toolchain to 1.22.1 by @r-vasquez
PR #17110 test: Don't run MTT in CDT by @StephanDollberg
PR #17117 rpk profile: a few more fixes by @twmb
PR #17118 PESDLC-983 Trim errors and add Broker exception by @savex
PR #17119 CORE-20: Use a venv to install and use ducktape by @michael-redpanda
PR #17145 ducktape: deflake test_topic_aware_rebalance in debug mode by @bharathv
PR #17152 acl_store: Return prefix matches as a view by @oleiman
PR #17156 compression: Allocate memory for LZ4_compressEnd by @abhijat
PR #17161 c/types: fix typo in operator<< for topic_properties by @nvartolomei
PR #17163 rbac: Add Telemetry and Licence nag by @BenPope
PR #17164 test: Add n2d-standard-32 machinetype by @StephanDollberg
PR #17165 tests: Update OMB version on CDT by @StephanDollberg
PR #17166 admin: implement RBAC List/Get/Rename/Delete APIs by @pgellert
PR #17169 Fixed typo in comment in ntp_archiver by @WillemKauf
PR #17170 rpk profile double pointer fixes by @twmb
PR #17172 PESDLC-986 Introduce randomized name checking mechanism by @savex
PR #17178 PESDLC-985 Raise crash errors on pod restarts by @savex
PR #17181 CORE-7 Add MD5 Support in FIPS mode by @michael-redpanda
PR #17182 container: Introduce chunked_hash_map by @StephanDollberg
PR #17183 Add a way to skip the test based on cloud_type GH:1169 Jira:PESDLC-971 by @ rpdevmp
PR #17187 git: ignore top level data/ dir by @rockwotj
PR #17193 Lint includes by @dotnwat
PR #17194 shard_placement_table: stress test and fixes by @ztlpn
PR #17195 clang-tidy: remove deprecated option by @rockwotj
PR #17196 io/tests: fix bugprone-empty-catch by @rockwotj
PR #17208 wasm: logging tests by @rockwotj
PR #17212 clang-tidy: disable misc-include-cleaner by @rockwotj
PR #17214 CORE-33: Hooked OpenSSL context service into app by @michael-redpanda
PR #17215 write caching - raft - follow up fixes by @bharathv
PR #17219 RBAC End to End Integration Test by @oleiman
PR #17228 rptest: remove vestigial CloudTierName.DOCKER by @travisdowns
PR #17229 tests: allow node ready timeout to be adjusted by @dotnwat
PR #17230 goreleaser: adjust goreleaser to dynamically link rpk on linux by @ andrewhsu
PR #17231 cluster: Documentation for security_frontend.h by @oleiman
PR #17232 PESDL-942 concurrent pods restart by @rpdevmp
PR #17236 CORE-2177: rptest: e2e write caching tests with failure injection by @ nvartolomei
PR #17242 feature_table: remove dependency on cluster/types.h by @dotnwat
PR #17243 CORE-31 dt: Use version of ducktape with updated paramiko by @michael- redpanda
PR #17245 kafka: chunked_vector for config responses by @pgellert
PR #17249 transform-sdk/cpp: add string_view conversion from bytes_view by @rockwotj
PR #17250 PESDLC-995 Create means to search logs in cloud pods by @savex
PR #17260 k/group: recover leader epoch on leader change by @nvartolomei
PR #17262 features: rename node_local_core_assignment by @ztlpn
PR #17351 PESDLC-1009 Bump latency for e2e p99 by @savex
PR #17352 Remove global dependencies from test_utils/random by @dotnwat
PR #17357 storage: coalesce and rename record batch disk serialization methods by @ andrwng
PR #17358 storage/mvlog: introduce batch collection to be used in new log impl by @ andrwng
PR #17359 security: Wire up RBAC role to OCSF user::groups by @BenPope
PR #17372 http: make http module new-style! by @dotnwat
PR #17376 transform/processor test: prevent batching flakes by @rockwotj
PR #17379 cluster/tests: add test case for eviction of the tx coordinator by @andrwng
PR #17381 Offset tracker invariants by @WillemKauf
PR #17382 storage: adopt offset_translator in namespace by @andrwng
PR #17384 storage: add log_reader option to translate offsets by @andrwng
PR #17385 CORE-1722: compression: Use preallocated decompression buffers for lz4 by @ abhijat
PR #17388 serde/rw/sstring: fix header self inclusion by @andijcr
PR #17389 cluster/types: removed redundant operator<< declaration by @andijcr
PR #17391 CORE-14 disable tests that fail in fips by @michael-redpanda
PR #17392 c/test: fix shard_placement_table_test by @rockwotj
PR #17396 licenses: Add unordered_dense license by @StephanDollberg
PR #17399 admin/api-doc: Remove role_member_update def by @oleiman
PR #17401 treewide: address newer clang-tidy warnings by @rockwotj
PR #17403 json: minor clean-ups by @dotnwat
PR #17405 Feat/topic manifest serde by @andijcr
PR #17413 c/partition_balancer: don't copy topic metadata on each partition visit by @ztlpn
PR #17414 RBAC: Fix use-after-free in security::acl_store by @oleiman
PR #17421 c/frag_vector: added get_allocator() method to fragmented vector by @ mmaslankaprv
PR #17423 Fixed node_hash_map caused oversized allocations in cluster module by @ mmaslankaprv
PR #17425 use chunked vector as batches cache in raft::replicate_batcher by @ mmaslankaprv
PR #17427 authorizer: add parens by @travisdowns
PR #17433 tx: fix param ordering in log statement by @nvartolomei
PR #17434 container/fragmented_vector: add comment about resize by @rockwotj
PR #17436 PESDLC-1036 Wait for metrics at startup of producer_swarm by @savex
PR #17437 RBAC: Add ability to delete role ACLs when deleting the role itself in the Admin API by @oleiman
PR #17444 PESDLC-1021 Update cloud cleanup with options by @savex
PR #17446 RBAC: remove update role and friends by @oleiman
PR #17447 security/role: Minor cleanups by @oleiman
PR #17450 io: clang tidy fixes by @dotnwat
PR #17455 cloud_storage: Fix download throttling ducktape test by @Lazin
PR #17457 tests: do not run offsets for leader epoch test in debug mode by @ mmaslankaprv
PR #17463 security/rbac: Add metrics by @BenPope
PR #17464 c/partition: cleaned up files included in partition.h by @mmaslankaprv
PR #17465 PESDLC-1021 Use bucket policy for cleanup by default by @savex
PR #17467 security: Various minor cleanups by @oleiman
PR #17472 rm_stm: do not hold producer lock for the duration of the barrier by @ bharathv
PR #17474 rptest: Fix throttling test by @dotnwat
PR #17475 s/authorizer: Comments about allow_empty_matches by @oleiman
PR #17476 dt/rbac_test: Add RolePersistenceTest by @oleiman
PR #17477 c/replication_monitor: majority replication optimization by @bharathv
PR #17494 rptest: handle errors and retry them in list offsets request by @nvartolomei
PR #17498 Improved validation of Fetch requests when reading from follower by @ mmaslankaprv
PR #17499 archival/tests: add include for archival_metadata_stm by @bharathv
PR #17500 gh: Added workflows to sync GitHub with Jira by @michael-redpanda
PR #17501 Update metrics_reporter to show the number of rbac roles by @oleiman
PR #17512 gha: update lint-golang by @andrewhsu
PR #17515 k/topics: make write caching configs settable at creation by @bharathv
PR #17519 cloud_roles: convert to new-style module by @dotnwat
PR #17520 deps: Add lz4-static for fedora by @abhijat
PR #17525 PESDLC-1021 Add optional 8h delay to bucket cleaning by @savex
PR #17527 Update approach to kafka security for clients by @travisdowns
PR #17528 rptest: delete vestigial datapolicy stuff by @travisdowns
PR #17529 ducktape: improve ubsan violation handling in dt tests by @bharathv
PR #17534 io: fix clang-tidy warning by @rockwotj
PR #17537 serde: support variant by @rockwotj
PR #17544 c/topics_frontend: break the loop when dispatching to current leader by @ mmaslankaprv
PR #17545 CORE-1643 rptest: Increase backoff interval for GCS by @Lazin
PR #17546 Revert "Add a way to skip the test based on cloud_type GH:1169 by @rpdevmp
PR #17549 Convert modules to new-style: ssx, io, syschecks by @dotnwat
PR #17551 Removed adl support for cluster rpc types by @mmaslankaprv
PR #17552 rptest: Improve RRR end to end test by @Lazin
PR #17555 rptest: add ubsan suppressions by @rockwotj
PR #17557 iwyu: add pragma to always keep by @rockwotj
PR #17559 remote: fix exception checking in delete_objects_sequentially by @andrwng
PR #17562 RBAC: Enable the role store to survive a controller snapshot by @oleiman
PR #17563 cluster: avoid throwing on duplicate recovery by @andrwng
PR #17580 gha: lock to goreleaser v1.24.0 by @andrewhsu
PR #17588 rptest: upgrade kgo by @nvartolomei
PR #17592 gh: Updates to Jira issue sync by @michael-redpanda
PR #17594 rptest/transform: add multiple output topics test by @rockwotj
PR #17596 CORE-2055: net: allow transport logger to be customized by @dotnwat
PR #17615 PESDLC-1063 Refactor test_consume_miss_cache by @savex
PR #17617 Add ducktape tests for rpk security role by @r-vasquez
PR #17621 c/controller_snapshot: Add serde envelope for named_role by @oleiman
PR #17624 cluster: create default "Users" role when RBAC is activated by @pgellert
PR #17626 transform: convert to new-style module by @rockwotj
PR #17629 PESDLC-1064 Enable cloud network clean by @savex
PR #17665 PESDLC-1069 Disable decomission tests by @savex
PR #17666 s3_client: add keys to log messages by @andrwng
PR #17671 rptest: write caching tests with failure injection by @nvartolomei
PR #17672 gh: Improve workflow script by @michael-redpanda
PR #17676 goreleaser: remove GOEXPERIMENT=systemcrypto by @andrewhsu
PR #17678 CORE-2056: net: allow conn_quota logger to be configured by @dotnwat
PR #17680 goreleaser: default to CGO_ENABLED=0 allow override by @andrewhsu
PR #17681 transform-sdk/cpp: add string conversion operators by @rockwotj
PR #17685 PESDLC-1084 Add copyright to topic_operations.py by @savex
PR #17686 transform-sdk/tests: add schema registry test by @rockwotj
PR #17687 rpk: use fork of avro by @andrewhsu
PR #17689 rpk: doc updates and small QOL improvements by @r-vasquez
PR #17695 wasm: reset call context on failure by @rockwotj
PR #17700 PESDLC-1095 Fix and check len limits when creating shell by @savex
PR #17701 cmake: upgrade wasmtime by @rockwotj
PR #17702 transform/rpc: prevent cross shard memory writes by @rockwotj
PR #17703 CORE-2061: net: remove rpc logger from server connection by @dotnwat
PR #17704 Topic-aware replica placement in partition_allocator:: reallocate_partition by @ztlpn
PR #17705 CORE-1705 rptest: filter abort strings from GCP headers by @andrwng
PR #17708 tests: wait for partition leaders before querying hwm by @mmaslankaprv
PR #17709 gtest: set death_test_style=threadsafe by @ztlpn
PR #17714 Introduce RedpandaMixedTest by @travisdowns
PR #17716 CORE-1614 storage: Initialize timestamps in the compaction_placeholder by @ Lazin
PR #17717 CORE-2059: net: separate rpc metrics from generic transport by @dotnwat
PR #17721 gha: rpk improvements by @andrewhsu
PR #17725 test/shadow_indexing_compacted_topic_test: factored out rpk.describe_topic by @andijcr
PR #17726 kafka: fix write caching validation during topic creation by @nvartolomei
PR #17729 storage: document semantics of the read interface by @nvartolomei
PR #17730 CORE-2212: test: do not start seastar for --gtest_list_tests by @dotnwat
PR #17773 verifier/tx: explicit flush before aborting transaction by @bharathv
PR #17785 CORE-1752: cst: improved logging by @abhijat
PR #17786 test/compaction_e2e_test: strict check by @andijcr
PR #17795 rptest: mix write caching with disabled batch cache by @nvartolomei
PR #17798 tests: do not upgrade more than one feature version at a time by @ mmaslankaprv
PR #17800 rbac_migrator: suppress error on role_exists by @pgellert
PR #17801 slash-cmds: add /publish-to-install-pack command by @gousteris
PR #17802 c/partition_allocator: fix bench test by @ztlpn
PR #17807 dt: allow cleanup.policy config diff in backport by @pgellert
PR #17811 cluster: add number of transform to metrics reporter by @rockwotj
PR #17820 PESDLC-1121 Use test start time to capture proper logs by @savex
PR #17823 write caching follow ups by @bharathv
PR #17833 CORE-1743 rptest: fix flaky metric check by disabling leader balancer by @ nvartolomei
PR #17836 ducktape: more robust tinygo downloads by @rockwotj
PR #17838 rpk: bump docker version by @r-vasquez
PR #17839 CORE-2243 archival: Start housekeeping jobs only on a leader by @Lazin
PR #17840 CORE-2369: Fix/azure managed identities hns detection by @andijcr
PR #17841 CORE-2347: net: remove dependence on rpc parsing exception type by @dotnwat
PR #17859 transform-sdk: Introduce QuickJS based JS SDK by @rockwotj
PR #17868 CORE-2348 s3_client: fix typo in log statement format specifier by @ nvartolomei
PR #17870 rptest: propagate log level to verifiable services by @nvartolomei
PR #17879 CORE-2365: storage: increase size of offset key map fragment size by @ dotnwat
PR #17890 ducktape: retry tinygo by @rockwotj
PR #17892 CORE-2378 id_allocator: do not forward requests beyond first hop by @ nvartolomei
PR #17893 dt: Updated tinygo to work on Ubuntu 20.04 by @michael-redpanda
PR #17894 tests: Marking test_drain_audit_disabled as ok_to_fail by @graphcareful
PR #17896 Fix KafkaCliTools.produce in cloud tests by @travisdowns
PR #17901 gh: add rpunit issue template by @rockwotj
PR #17906 [CORE-2400] kafka/server: Disable quota balancer by @BenPope
PR #17909 ducktape: add write caching to random_node_ops test by @bharathv
PR #17910 log_manager: gate orphan file cleanup by @bharathv
PR #17912 c/hm_backend: cache the collected report by @mmaslankaprv
PR #17913 CORE-2388 rptest: remove node from started list after kill by @nvartolomei
PR #17919 gh: Add docs team as CODEOWNERS for configs by @michael-redpanda
PR #17926 c/controller_backend: don't require cur_operation after reconciliation step by @ztlpn
PR #17932 rptest: fix test_exceed_broker_limit flake by @travisdowns
PR #17949 cleanup: clang 18 fixes by @rockwotj
PR #17953 CORE-2452 followup for azure managed identities #17840 by @andijcr
PR #17963 ducktape: Update client-swarm deps by @StephanDollberg
PR #17965 transform: start loading VMs after logs are started by @rockwotj
PR #17967 fetch: Avoid formatting client ip and port for each partition by @ StephanDollberg
PR #17968 cloud_roles: fix includes by @rockwotj
PR #17969 CORE-2451 cloud_storage_clients/abs_clients: fix iterator check by @andijcr
PR #17970 wasm: add a README explaining the architecture of the Wasm subsystem by @ rockwotj
PR #17978 treewide: fix oss build with clang-18 by @rockwotj
PR #15996 storage: enable space management by default by @dotnwat
PR #16925 Add an option to specify topic name for nodes_report_cloud_segments by @ rpdevmp
PR #16926 Cluster integration for role_store and related structures. by @oleiman
PR #17387 (fix): Handle EDITOR with options by @kameshsampath
PR #17723 Followup/feat/cloud storage compaction metric by @andijcr
PR #18025 [v24.1.x] CORE-2490 cloud_storage_clients/client_pool: error->warn by @ vbotbuildovich
PR #18041 [v24.1.x] c/controller_backend: try to force-abort reconfiguration only on leaders by @vbotbuildovich
PR #18042 [v24.1.x] CORE-2627 fix AZURE_AUTHORITY_HOST parsing in azure_ask_refresh_impl by @vbotbuildovich
PR #18068 [v24.1.x] storage: add additional logging for deep log_reader recursion by @vbotbuildovich
PR #18070 [v24.1.x] cmake: upgrade wasmtime by @vbotbuildovich
PR #18108 [v24.1.x] wasm/parser: fix global parsing by @vbotbuildovich
PR #18110 [v24.1.x] gh: fix lint-cpp for ubuntu noble by @vbotbuildovich
PR #18117 [v24.1.x] wasm/parser: better global support by @vbotbuildovich
PR #18132 [v24.1.x] cmake: Add support for sccache by @vbotbuildovich
PR #18137 [v24.1.x] tests: adjust time between failures in relaxed consistency test by @vbotbuildovich
PR #18148 [v24.1.x] rpk container: offer port selection on start by @vbotbuildovich

Full Changelog: v23.3.1...v24.1.1