Compared to the previous version 5.3.3, this is a bug-fix release that includes numerous improvements to enhance stability, performance, and correctness. Key highlights include critical dependency upgrades (e.g., Netty, OpenTelemetry), significant stability enhancements like the TimerWheel snapshot mechanism for reliable recovery, and a wide range of bug fixes across modules such as Broker, Proxy, and Client.
Thank you for your attention, If you have any suggestions please reply to this email.
What's Changed
- [ISSUE #9337] Upgrade Netty to 4.1.119.Final by @weihubeats in #9338
- [ISSUE #9253] Make the message visible earlier when response process encounters errors by @redlsz in #9255
- [ISSUE #8651] Fix revive topic process lag metric calculation by @redlsz in #8652
- [ISSUE #8629] Add proxy RocketmqTraffic logger appender to separate traffic logs by @redlsz in #8630
- [ISSUE #8232] Fix the issue that proxy remoting thread pool isolation does not take effect by @redlsz in #8233
- [ISSUE #9414] Updated the Quick Start version in README to the 5.3.3 by @mxsm in #9415
- [ISSUE #9412] Flush bak metrics file before deleting the original file by @GenerousMan in #9413
- [ISSUE #9403] Fix the issue of metadata synchronization failure between master and slave when the topic has attributes by @RongtongJin in #9404
- [ISSUE #9416] Fix batch send messages have the same message id when consumed by @qianye1001 in #9417
- [ISSUE #9379] Set compact timestamp when timeStoreTable empty by @lizhimins in #9419
- [ISSUE #9418] Fix ColumnFamilyHandle not close in PopConsumerRocksdbStore by @lizhimins in #9420
- [ISSUE #9352] Fix wrong value in topic-group cache by @golden-yang in #9424
- [ISSUE #9441] Add file length check in IndexService by @lizhimins in #9442
- [ISSUE #9443] Fix add wrong value to topicGroupTable in ConsumerManager by @yx9o in #9444
- [ISSUE #9254] Add CombineConsumeQueue to support CQ migration by @qianye1001 in #9256
- [ISSUE #9439] Add escape for win in the method returning broker configuration by @Crazylychee in #9440
- [ISSUE #9451] Optimize back pressure default value to 1024. by @GenerousMan in #9452
- [ISSUE #9454] Make sure topic can be deleted when using RocksDB CQ by @qianye1001 in #9457
- Fix IllegalStateException during app shutdown caused by premature shutdown trigger by @hepengju in #9408
- [ISSUE #9335] Use MixAll.ROCKETMQ_HOME_DIR for home directory by @WJ66880 in #9474
- [ISSUE #9481] Fix Issue 4050 didn't fix completely by @WJ66880 in #9482
- [ISSUE #9489] Fix clean pull offset in #removeOffset by @3424672656 in #9490
- [ISSUE #8920] Refactor SSL context loading process to support multiple protocols dynamic loading by @EnableAsync in #9483
- [ISSUE #9491] Remove the volatile declaration for the two variables requestsWrite and requestsRead in the GroupCommitService class by @huanzhikang in #9492
- [ISSUE #9486] Fix missing of valueOfCurrentMinusBorn equals checkImmunityTime by @WJ66880 in #9487
- [ISSUE #9497] Fix IndexOutOfBoundsException in getEarliestMessageTime when running in IPv6 environment by @woaishixiaoxiao in #9498
- [ISSUE #9479] Relax topic length constraint when creating retry and dlq topics by @Allenhuangrui in #9480
- [ISSUE #9476] Set queueId to 0 in DefaultTransactionalMessageCheckListener#toMessageExtBrokerInner to improve performance by @mxsm in #9477
- [ISSUE #9447] Avoiding possible resource leaks in InputStream by @AROP-REPLICATION in #9459
- [ISSUE #9467] Implement paged retrieval of Topic and SubscriptionGroup information by @ltamber in #9468
- [ISSUE #8920] Make TLS certificate watch interval configurable by @EnableAsync in #9513
- fix CONTRIBUTING.md spelling mistake by @datagramsocket in #9511
- [ISSUE #9501] Correcting mismatched comments by @yx9o in #9502
- [ISSUE #9515] Return origin handle to consumer when changeInvisibleDuration encounters flow limit by @qianye1001 in #9516
- [ISSUE #9254] Fix combineCQ extra search commitLog files for recovery by @qianye1001 in #9526
- [ISSUE #9309] Avoid the generation of dirty data in #assignResetOffset by @3424672656 in #9310
- [ISSUE #9530] Reduce unnecessary memory allocation in adaptiveBackOffSpinLock by @3424672656 in #9530
- [ISSUE #9520] Remove unnecessary topicConfig data version change when enable split registration by @RongtongJin in #9521
- [ISSUE #9254] Optimize the logs of the message store by @qianye1001 in #9528
- [ISSUE #9537] MQClientAPIFactory Implement NameServerUpdateCallback interface by @qianye1001 in #9538
- [ISSUE #9544] Must correct file size when init the file segment by @lizhimins in #9545
- [ISSUE #9539] Fix compare policy entry logic in ACL 2.0 by @1782935682 in #9540
- [ISSUE #9534] Remove the meaningless brokerVersion by @RongtongJin in #9533
- [ISSUE #9555] Fix the npe in the hashCode method when subscriptionDataSet is null by @1782935682 in #9556
- [ISSUE #9553] Improve performance by avoiding repeated get(key) by @yx9o in #9554
- Limit group max length to 120. by @lollipopjin in #9563
- [ISSUE #9254] Refactor notifyMessageArriveInBatch in RocksDBConsumeQueueStore to adapt to CombineConsumeQueueStore by @qianye1001 in #9566
- [ISSUE #9572] Unnecessary code comment cleanup by @yx9o in #9573
- Fix typos by @co63oc in #9560
- [ISSUE #9254] Optimize innerConsumeQueueStoreList order in CombineConsumeQueueStore by @qianye1001 in #9599
- [ISSUE #9596] Optimize log in invokeSync when addr is null by @1782935682 in #9597
- [ISSUE #9605] Fixed a potential resource leak by @yx9o in #9606
- [ISSUE #9589] Optimize broker metrics initialization by @qianye1001 in #9598
- [ISSUE #9611] Should exec callback in the Pop based on rocksdb impl by @lizhimins in #9612
- [ISSUE #9600] Remove wrong logic in the callback for sending messages in rpc by @3424672656 in #9608
- [ISSUE #9626] Prevent premature offset commit before consumer record flush by @lizhimins in #9627
- [ISSUE #9542, #9609] Fix bazel CI and reduce dependencies by @xuhongxu96 in #9610
- [ISSUE #9652] Enhance test for MQClientInstance by @GabrielDUX in #9653
- [ISSUE #9648] Fix getOffsetInQueueByTime missing boundaryType in tieredMessageStore by @GabrielDUX in #9649
- Fix start() and shutdown() of DefaultMessagingProcessor by @redlsz in #7838
- [ISSUE #9663] Adopt Maven's revision mechanism (requires Maven >= 3.5.0) by @cvictory in #9665
- [ISSUE #9650] Unified FAQ related URLs by @yx9o in #9651
- [ISSUE #9620] Placeholder replacement string concatenation by @yx9o in #9621
- [ISSUE #9581] Optimize the resource bloat of pollingMap and topicCidMap in LMQ scenarios by @RongtongJin in #9579
- [ISSUE #9764] Fix bazel-compile (ubuntu-latest) error by @yx9o in #9675
- [ISSUE #9666] Add accelerated startup recovery feature for RocksDB store with SYNC_FLUSH by @RongtongJin in #9667
- [ISSUE #9634] Improve BrokerContainer extensibility and module structure by @RongtongJin in #9635
- [ISSUE #9547] Fix sync group if attributes isn't null by @1782935682 in #9679
- [ISSUE #9682] Increase initial delay for scanning inactive brokers to 5000 milliseconds by @mxsm in #9683
- [ISSUE #9680] Improve RocksDB compaction filter factory resource management by @RongtongJin in #9681
- [ISSUE #9677] Resolve metrics static variable conflicts in BrokerContainer mode by @RongtongJin in #9678
- [ISSUE #9695] Not use pull offset when use pop orderly consume by @lizhimins in #9696
- [ISSUE #9693] Add writeWithoutMmap configuration to prevent JVM crash when device becomes read-only by @guyinyou in #9694
- [ISSUE #9701] Synchronize metrics shutdown to prevent JVM crashes during broker shutdown by @guyinyou in #9702
- [ISSUE #9703] Remove pull offset and reset offset when deleting topic by @qianye1001 in #9704
- [ISSUE #9707] Integrate RunningFlags with MappedFile system for better error handling and state management by @guyinyou in #9708
- [ISSUE #9713] Improve data consistency in DefaultMappedFile flush method by @guyinyou in #9714
- [ISSUE #9699] Optimize shutdown process and resource management by @RongtongJin in #9700
- [ISSUE #9705] Improve persist() method reliability to prevent broker startup failure after power outages by @guyinyou in #9706
- [ISSUE #9709] Add enableRunningFlagsInFlush configuration switch for CommitLog by @guyinyou in #9710
- [ISSUE #9716] Refactor: replace RandomAccessFile with FileChannel by @RongtongJin in #9715
- [ISSUE #9717] Fix RaftBrokerHeartBeatManager#scanNotActiveBroker was not actually executed by @redlsz in #9718
- [ISSUE #9721] Fix TimerDequeueGetService thread not exiting after shutdown by @RongtongJin in #9722
- [ISSUE #9689] Fix the issue that master transfer epoch was not updated in time by @redlsz in #9690
- [ISSUE #9719] Fix Static Variable Conflicts in Multi-Broker Metrics Management by @RongtongJin in #9720
- Fix/update metadata bug in v2 config by @cvictory in #9724
- [ISSUE #9726] Abstract StoreMetricsManager interface to support non-DefaultMessageStore implementations by @RongtongJin in #9727
- [ISSUE #9687] Cleanup BrokerHeartbeatManager code by @majialoong in #9688
- [ISSUE #9732] Fix request may still be invoked when local process timeout by @qianye1001 in #9733
- [ISSUE #9658] Replace JUnit 5 imports with JUnit 4 by @yx9o in #9659
- [ISSUE #9743] Fix ReceiptHandleProcessorTest by @yx9o in #9744
- Fix typos and linguistic errors in documentation / hacktoberfest by @survivant in #9748
- fix get dledger mod storepath by @LittleBoy18 in #9746
- [ISSUE #9741] Optimize authentication whitelist lookup efficiency by @yx9o in #9742
- [ISSUE #9739] Revert skipped test cases in macos environment by @lizhimins in #9749
- [ISSUE #9750] Fix NPE when isAsyncSendEnable is false by @RongtongJin in #9751
- [ISSUE #9756] Fix the issue where the port is always 10911 when starting broker with brokerStartup by @RongtongJin in #9755
- [ISSUE #9752] Fix time dequeue latency unit to milliseconds by @dingshuangxi888 in #9753
- [ISSUE #9761] Fix get authentication and authorization metadata provider error in AuthorizationMetadataManagerImpl by @majialoong in #9762
- [ISSUE #9758] Fix resource filter does not take effect in the listAcl by @majialoong in #9759
- [ISSUE #9765] Improve null handling in file operations by @yx9o in #9766
- [ISSUE #9767] Fix incorrect decode method for UnlockBatchRequestBody in DefaultAuthorizationContextBuilder by @majialoong in #9768
- [ISSUE #9769] Add tls.ciphers and tls.protocols in system properties by @carlvine500 in #9770
- [ISSUE #9771] Remove static from transactionalMessageCheckListener executorService to support multi-broker in brokercontainer by @RongtongJin in #9772
- [ISSUE #9763] Fix invalid user disable status check in authorization by @majialoong in #9764
- [ISSUE #9773] Implement Shared RocksDB Instance for Broker Configs by @qianye1001 in #9774
- [ISSUE #9776] Make SharedByteBuffer size configurable via MessageStoreConfig.maxMessageSize by @RongtongJin in #9775
- [ISSUE #9773] Fix core dump when shutdown broker twice by @qianye1001 in #9783
- Correct method name from updateDateVersion to updateDataVersion by @majialoong in #9788
- [ISSUE #9784] Fix the policy comparator to prioritize DENY over ALLOW by @majialoong in #9785
- [ISSUE #9798] Reduce unnecessary thread switching in sending message via proxy by @qianye1001 in #9799
- [ISSUE #9811] Fixed the issue of excessive loop iterations in reput service by @lizhimins in #9812
- [ISSUE #9816] Fix concurrent modify opentelemetry record in calculate consumer lag by @lizhimins in #9817
- [ISSUE #9813] Fix buffer fetch thread pool starvation in tiered storage by @lizhimins in #9818
- Fixed error message of CommandUtil.fetchMasterAddrByBrokerName by @TheRealHaui in #9825
- [ISSUE #9802] Fix the issue of delete logic in tiered storage index service by @lizhimins in #9803
- [ISSUE #9819] Upgrade OpenTelemetry to version 1.44.1 by @RongtongJin in #9820
- [ISSUE #9735] Add TimerWheel snapshot mechanism for reliable recovery by @guyinyou in #9736
- Add query message offset by timestamp to abstract class by @lizhimins in #9836
- [ISSUE #9821] Fix getBrokerConfig no response issue for empty parameters by @yx9o in #9822
- [ISSUE #9794] Change JDK prerequisite from 1.7+ to 1.8+ in documentation by @yx9o in #9795
- fix controller mode scanControllerAddress by @LittleBoy18 in #9845
- [#ISSUE 9841] Improve Resource Management in TimerWheel to Prevent File Handle Leaks by @guyinyou in #9842
- [ISSUE #9632] Fix: Pop Long-polling Not Awakened for V1 Retry Messages by @qianye1001 in #9828
- [ISSUE #9838] IndexStoreService use forceShutdown when disk is not writable by @RongtongJin in #9839
- [ISSUE #9834] Support writeWithoutMmap in IndexStoreFile by @RongtongJin in #9835
- Fix resource leak in IndexStoreFile shutdown by @RongtongJin in #9840
- [ISSUE #9593] Optimize Deflater resource release by @yx9o in #9594
- [ISSUE #9847] Reduce lock contention on the HandleData object to prevent threads from hanging by @qianye1001 in #9848
- [ISSUE #9857] Optimize the performance of the notification by removing unnecessary retry checks by @qianye1001 in #9858
- [ISSUE #9807] Optimize log.warn Time Consumption in the send Method by @yx9o in #9808
- [ISSUE #9737] Fix client memory leak on connection failure by @qianye1001 in #9740
- [ISSUE #9849] Correct ACL cache loader error log in ClusterMetadataService by @majialoong in #9850
- [ISSUE #9741] Optimize authorization whitelist lookup efficiency by @majialoong in #9760
- [ISSUE #9809] Fix NPE in getAcl when subject is null by @majialoong in #9810
- [ISSUE #9789] LitePullConsumer supports manually adding subscription reported in Heartbeat by @qianye1001 in #9790
- [ISSUE #9862] [Bug] Fix the issue of missing bornTime in POP requests when broker ACL is enabled by @dingshuangxi888 in #9863
- [ISSUE #9870] Ensure metadata provider cache executors are shutdown correctly by @majialoong in #9871
- [ISSUE #9779] Fix the promblem of 'DirectBuffer will cause error at 9+ JDK version'. by @sinberCS in #9801
- [ISSUE #9663] Add flatten-maven-plugin plugin by @lizhimins in #9866
- [ISSUE #9816] Fix calculate consumer lag with opentelemetry by @lizhimins in #9873
- [ISSUE #9875] Optimize the RocksDB config shutdown logic when useSingleRocksDBForAllConfigs is set to true to prevent JVM crashes. by @RongtongJin in #9874
- [ISSUE #9734] Prepare to release Apache RocketMQ 5.3.4 by @lizhimins in #9861
New Contributors
- @hepengju made their first contribution in #9408
- @EnableAsync made their first contribution in #9483
- @huanzhikang made their first contribution in #9492
- @woaishixiaoxiao made their first contribution in #9498
- @Allenhuangrui made their first contribution in #9480
- @AROP-REPLICATION made their first contribution in #9459
- @datagramsocket made their first contribution in #9511
- @1782935682 made their first contribution in #9540
- @co63oc made their first contribution in #9560
- @xuhongxu96 made their first contribution in #9610
- @GabrielDUX made their first contribution in #9653
- @cvictory made their first contribution in #9665
- @majialoong made their first contribution in #9688
- @survivant made their first contribution in #9748
- @TheRealHaui made their first contribution in #9825
- @sinberCS made their first contribution in #9801
Full Changelog: rocketmq-all-5.3.3...rocketmq-all-5.3.4