StarRocks/starrocks 3.1.9

Release notes 3.1.9

on GitHub

Release date: March 8, 2024

New Features

• Cloud-native Primary Key tables in shared-data clusters support Size-tiered Compaction to reduce write I/O amplification for the loading of a large number of small-sized files. #41610
• Added the function regexp_extract_all. #42178
• Added the view information_schema.partitions_meta, which records detailed metadata of partitions. #41101
• Added the view sys.fe_memory_usage, which records the memory usage for StarRocks. #41083

Behavior Changes

• The logic of dynamic partitioning is changed. Now partition columns of the DATE type do not support hour-level data. Note that partition columns of the DATETIME type still support hour-level data. #40328
• The user who can refresh materialized views is changed from the root user to the user who creates the materialized views. This change does not affect existing materialized views. #40698
• By default, when comparing columns of constant and string types, StarRocks compares them as strings. Users can use the session variable cbo_eq_base_type to adjust the default rule used for the comparison. For example, users can set cbo_eq_base_type to decimal, and StarRocks then compares the columns as numeric values. #41712

Improvements

• StarRocks supports using the parameter s3_compatible_fs_list to specify which S3-compatible object storage can be accessed via AWS SDK, and supports using the parameter fallback_to_hadoop_fs_list to specify non-S3-compatible object storage that require access via HDFS Schema (this method necessitates the use of vendor-provided JAR packages). #41612
• The compatibility with Trino's SQL statement syntax is optimized to support converting the following functions of Trino: current_catalog, current_schema, to_char, from_hex, to_date, to_timestamp, and index. #41505 #41270 #40838
• A new session variable cbo_materialized_view_rewrite_related_mvs_limit is added to control the maximum number of candidate materialized views allowed during query planning. The default value of this session variable is 64. This session variable helps mitigate the excessive resource consumption caused by a large number of candidate materialized views for a query during the query planning. #39829
• The agg_type of BITMAP-type columns in an Aggregate table can be set to replace_if_not_null to support updates only to a few columns of the table. #42102
• The session variable cbo_eq_base_type is optimized to support specifying the implicit conversion rule applied to the comparison of data that contains both string and numeric data types. By default, such data is compared as strings. #40619
• More DATE-type data (for example, "%Y-%m-%e %H:%i") can be recognized to better support partition expressions for Iceberg tables. #40474
• The JDBC connector supports the TIME data type. #31940
• The path parameter in the SQL statement for creating a file external table supports wildcards (*). However, like the DATA INFILE parameter in the SQL statement for creating a Broker Load job, the path parameter supports using wildcards (*) to match at most one level of directory or file. #40844
• A new internal SQL log file is added to record log data related to statistics and materialized views. #40682

Bug Fixes

Fixed the following issues:

• "Analyze Error" is thrown if inconsistent letter cases are assigned to the names or aliases of tables or views queried in the creation of a Hive view. #40921
• I/O usage reaches the upper limit if persistent indexes are created on Primary Key tables. #39959
• In shared-data clusters, the primary key index directory is deleted every 5 hours. #40745
• After a table for which list partitioning is enabled is truncated or its partitions are truncated, queries based on the partitioning keys of the table return no data. #40495
• After users execute ALTER TABLE COMPACT by hand, the memory usage statistics for compaction operations are abnormal. #41150
• During data migration between clusters, if only some columns are updated in column mode, the destination cluster may crash. #40692
• The SQL blacklist may not take effect if the submitted SQL statement contains multiple spaces or newline characters. #40457