v0.10.2
Release date: November 26, 2024
Versions v0.10.0 and v0.10.1 are deprecated because of critical bugs; please use this version instead.
๐ Highlights
- Vector type: GreptimeDB supports vector data types to optimize edge scenarios, such as IoT in vehicles, enabling efficient storage and computation of vector data for real-time perception and AI applications in smart driving.
- Free index: Inverted indexes are no longer tied to primary keys, allowing users to create them on any column, enhancing query flexibility and efficiency across different scenarios.
- Alter table options: Enhancements include setting TTL for databases and tables, modifying compaction parameters, and enabling/disabling full-text indexes on columns.
- Loki remote write: GreptimeDB now supports the Loki remote write protocol, allowing users to log data in Loki format through Grafana tools.
- Performance Optimization: Up to 10x improvement for queries fetching the latest N records by timestamp (
ORDER BY timestamp DESC LIMIT N
). - Remove the option:
region_engine.mito.max_background_jobs
and add three new options:region_engine.mito.max_background_flushes
,region_engine.mito.max_background_compactions
, andregion_engine.mito.max_background_purges
for more detailed configuration.
Breaking changes
- fix!: replace timeout_millis and connect_timeout_millis with Duration in DatanodeClientOptions by @WenyXu in #4867
- feat!: Divide flush and compaction job pool by @evenyag in #4871
๐ Features
- feat: yields empty batch after reading a range by @evenyag in #4845
- feat: update dashboard to v0.6.0 by @ZonaHex in #4861
- feat: Sort within each PartitionRange by @discord9 in #4847
- feat: Add functionality to the Opentelemetry write interface to extract fields from attr to top-level data. by @paomian in #4859
- feat: Limit CPU in runtime (#3685) by @ActivePeter in #4782
- feat: introduce the
PluginOptions
by @WenyXu in #4835 - feat: add json_path_match udf by @Kev1n8 in #4864
- feat: optimizer rule for windowed sort by @waynexia in #4874
- feat(index): support building inverted index for the field column on Mito by @zhongzc in #4887
- feat: add json datatype for grpc protocol by @WenyXu in #4897
- feat: Support altering table TTL by @v0y4g3r in #4848
- feat: adds the number of rows and index files size to region_statistics table by @killme2008 in #4909
- feat: implement parse_query api by @sunng87 in #4860
- feat: enhance windowed-sort optimizer rule by @waynexia in #4910
- feat: get row group time range from cached metadata by @evenyag in #4869
- feat: simple limit impl in PartSort by @waynexia in #4922
- feat: heartbeat_flush_threshold option by @fengjiachun in #4924
- feat: support to insert json data via grpc protocol by @WenyXu in #4908
- feat: add more geo functions by @sunng87 in #4888
- feat: support filter with windowed sort by @waynexia in #4960
- feat(index): support SQL to specify inverted index columns by @zhongzc in #4929
- feat(puffin): apply range reader by @zhongzc in #4928
- feat: alter fulltext options by @CookiePieWw in #4952
- feat: introduce vector type by @zhongzc in #4964
- feat: refine region state checks and handle stalled requests by @WenyXu in #4971
- feat: support alter twcs compaction options by @lyang24 in #4965
- feat: add distance functions by @zhongzc in #4987
- feat: implement statement/execution timeout session variable by @lyang24 in #4792
- feat: introduce
DynamicTimeoutLayer
by @WenyXu in #5006 - feat: Loki remote write by @shuiyisong in #4941
- feat: make greatest supports timestamp and datetime types by @killme2008 in #5005
- feat: update dashboard to v0.6.1 by @ZonaHex in #5017
- feat: CREATE OR REPLACE FLOW by @discord9 in #5001
- feat: also shutdown gracefully on sigterm on unix by @discord9 in #5023
- feat(vector): remove
simsimd
and usenalgebra
instead by @zhongzc in #5027 - feat: reimplement limit in PartSort to reduce memory footprint by @waynexia in #5018
- feat(vector): add conversion between vector and string by @zhongzc in #5029
- feat: add unset table options support by @WenyXu in #5034
- feat: alter database ttl by @CookiePieWw in #5035
- feat: use cache kv manager for SchemaMetadataManager by @v0y4g3r in #5053
๐ Bug Fixes
- fix: fix broken import by @WenyXu in #4880
- fix: pyo3 ut by @v0y4g3r in #4894
- fix(config): update tracing section headers in example TOML files by @waynexia in #4898
- fix: set transaction variables not working in mysql protocol by @killme2008 in #4912
- fix: prune batches from memtable by time range by @evenyag in #4913
- fix: typo by @killme2008 in #4931
- fix: panic when jsonb corrupted by @CookiePieWw in #4919
- fix: data_length, index_length, table_rows in tables by @killme2008 in #4927
- fix: violations of
elided_named_lifetimes
by @WenyXu in #4936 - fix: database base ttl by @v0y4g3r in #4926
- fix: pprof by @discord9 in #4938
- fix: the region_stats API will return an error in instance test by @linyihai in #4951
- fix: bugs introduced by alter table options by @killme2008 in #4953
- fix: do not pick compacting/expired files by @evenyag in #4955
- fix: round euclidean result in sqlness by @v0y4g3r in #4956
- fix: column already exists by @waynexia in #4961
- fix: json_path_exists null results by @Kev1n8 in #4881
- fix(otlp): replace otlp trace attr type from string to jsonb by @paomian in #4918
- fix: alter table add column id alloc mismatch by @discord9 in #4972
- fix: physical table statistics info by @killme2008 in #4975
- fix: run
install.sh
error by @zyy17 in #4989 - fix: obsolete wal entries while opening a migrated region by @WenyXu in #4993
- fix: ensure Create Or Replace and If Not Exist cannot coexist in create view by @lyang24 in #5003
- fix: correct
unset_maintenance_mode
behavior by @WenyXu in #5009 - fix: distinct respect in range by @discord9 in #5015
- fix: inverted index constraint to be case-insensitive by @zhongzc in #5020
- fix: android build failed due to simsimd by @zhongzc in #5019
- fix: prune memtable/files range independently in each partition by @evenyag in #4998
- fix: find latest window by @v0y4g3r in #5037
- fix: prepare param mismatch by @CookiePieWw in #5025
- fix: prevent metadata region from inheriting database ttl by @waynexia in #5044
- fix(metric-engine): set ttl also on opening metadata regions by @v0y4g3r in #5051
- fix: pass series row selector to file range reader by @evenyag in #5054
๐ Refactor
- refactor: json conversion by @CookiePieWw in #4893
- refactor: make use of the "pre_execute" in sql execution interceptor by @MichaelScofield in #4875
- refactor: simplify WeightedChoose by @WenyXu in #4916
- refactor: refactor alter parser by @CookiePieWw in #4933
- refactor: pass
LogicalPlan
to promql execution interceptor by @MichaelScofield in #4937 - refactor: consolidate
DatanodeClientOptions
by @linyihai in #4966 - refactor: support distinct JSON format and improve type conversions by @WenyXu in #4979
- refactor(mito): tidy memtable stats by @v0y4g3r in #4982
- refactor: use UNSET instead of enable by @CookiePieWw in #4983
- refactor(grafana): update cluster dashboard by @zyy17 in #4980
- refactor: Avoid wrapping Option for CacheManagerRef by @linyihai in #4996
- refactor: introduce
MaintenanceModeManager
by @WenyXu in #4994 - refactor: split up different stmts by @CookiePieWw in #4997
- refactor: unify mysql execute through cli and protocol by @CookiePieWw in #5038
- refactor: expose configs for http clients used in object store by @MichaelScofield in #5041
๐งช Testing
- test: add fuzz test for metric region migration by @WenyXu in #4862
- test: more sqlness tests for flow by @discord9 in #4988
- test: subquery test migrated from duckdb by @CookiePieWw in #4985
- test: reduce round precision to avoid platform diff by @zhongzc in #5013
โ๏ธ Miscellaneous Tasks
- chore: bump greptime-meter by @v0y4g3r in #4858
- chore: make pusher log easy to understand by @fengjiachun in #4841
- chore: better column schema check for flow by @discord9 in #4855
- chore: udapte Rust toolchain to 2024-10-19 by @v0y4g3r in #4857
- chore: add schema urls to otlp logs by @shuiyisong in #4876
- chore: graceful exit on bind fail by @discord9 in #4882
- chore: remove struct size assertion by @v0y4g3r in #4885
- ci: install numpy in CI by @discord9 in #4895
- chore: update proto depend by @discord9 in #4899
- chore: provide more info in check batch message by @evenyag in #4906
- chore: short desc markdown about change log level by @discord9 in #4921
- chore: update default cache size to 1Gib by @killme2008 in #4923
- chore: minor refactor for weighted choose by @fengjiachun in #4917
- chore: fix typos in change log level doc by @WenyXu in #4948
- chore: add json path for pipeline by @paomian in #4925
- chore: paginated query region stats by @fengjiachun in #4942
- chore: update cluster dashboard by @zyy17 in #4995
- chore: update greptime-proto to e1070a by @discord9 in #4992
- chore: rename change to modify by @CookiePieWw in #5000
- chore(cli): set default timeout for cli commands by @WenyXu in #5021
- chore: bump version to 0.10.0 by @zhongzc in #5040
- chore: bump version to v0.10.1 by @zhongzc in #5048
- chore: bump version to v0.10.2 by @zhongzc in #5056
Build
- build(deps): switch to upstream jsonb by @CookiePieWw in #4986
New Contributors
- @linyihai made their first contribution in #4996
- @ActivePeter made their first contribution in #4782
All Contributors
We would like to thank the following contributors from the GreptimeDB community:
@ActivePeter, @CookiePieWw, @Kev1n8, @MichaelScofield, @WenyXu, @ZonaHex, @discord9, @evenyag, @fengjiachun, @killme2008, @linyihai, @lyang24, @paomian, @shuiyisong, @sunng87, @v0y4g3r, @waynexia, @zhongzc, @zyy17