Dear community members,
The past six years have been a long but exciting journey for TDengine, seeing the evolution of our product from a basic TSDB to a full-featured cloud-native solution for time-series data, and the growth of our team from a five-person startup to the 80 members of today.
Throughout our journey, our constant goal has been to provide our community with first-class products and services. Today, on TDengine’s sixth anniversary, it is with much pride that we look back on the evolution of TDengine from 1.0 to 2.0 and 3.0, from the first line of code written by Jeff himself to the hundreds of thousands of lines that now comprise our product, indicating the continuous growth of our ecosystem, the increasing maturity of features such as stream processing and data subscription, the launch of our fully managed cloud service – and, of course, the resolution of many issues brought up by community members like yourself.
And indeed, what gives our team the most satisfaction is the growth of our open-source community over the years, exemplified by our over 21,000 stars on GitHub and more than 250,000 instances installed worldwide. We are delighted that an ever-increasing number of users and developers have chosen to become members of our community and accompany us on our journey, and we would like to take this opportunity to express our sincere gratitude for your support – our greatest motivation to continue our work and accomplish our goals.
Going forward, we intend to continue innovating in the time-series database field and enabling digital transformation across more industries. It is only through your support that we will be able to deepen our understanding of the scenarios and use cases of various industries and to maintain the growth of our product in a way that aligns with the needs of our community. Once again, thank you for being there with us over the past six years, and we hope that we can count on your continuing support in the future.
The TDengine Team, 2023/Jun/6
New Features and Improvements in TDengine 3.0.5.0
- System stability & performance
- Improved system stability under high stress data writing
- Optimized system performance in some query scenarios
- Altering database replicas doesn't block writing by introducing RAFT Learner
- Write driven cache for last() and last_row() to improve the query performance
- Optimized time cost of creating/dropping database
- Log long queries by default for easy debugging
- Controlled meta data cache in taosc library
- dnode can be restored after its data is totally lost (Enterprise only)
- System security
- Privilege control at table level (Enterprise only)
- License key can be updated using SQL command by "root" (Enterprise only)
- Stream processing
- Significantly reduced disk I/O and memory usage
- Stream can be paused/resumed
- Data Subscription
- Consuming progress can be queried
- Consumers can perform seek operation
- Consumers can subscribe supertable with tag filtering
- Consumers can retrieve meta data based on a topic name
- Improved performance
- Others
- Maximum row length is increased to 64KB
- interp() can be used for super table
- Python UDF can support multiple versions with "REPLACE" command
- Partition by and window clause can be followed by "Having" clause
What's Changed
- merge main by @xiao-yu-wang in #20924
- fix: taos shell cloud ux with 3 more stars mask by @sangshuduo in #20912
- docs(taosAdapter): the automatic database creation behavior for the schemaless protocol can be configured by @huskar-t in #20929
- Update index.md to add enableScience option in taos.cfg (english) by @DuanKuanJun in #20932
- Update index.md add enableScience option to taos.cfg by @DuanKuanJun in #20931
- docs/v3.0.3.4 by @xleili in #20935
- docs: support schemaless insert by @AdamEECS in #20944
- docs update offcial doc monitor section by @sunpe in #20949
- merge main by @xiao-yu-wang in #20950
- fix: atoi on int64 config item by @sangshuduo in #20946
- enhance: udf handle expired after 10s by @shenglian-zhou in #20928
- enhance: add udf stub expiration of 10s after creation by @shenglian-zhou in #20876
- fix: illegal accesses by @huolibo in #20960
- feat: the fill value clause supports constant expressions by @xiao-yu-wang in #20961
- docs: fix for monitor doc by @sunpe in #20963
- docs: update title for monitor doc by @sunpe in #20965
- enh: add param for to_unixtimestamp to return timestamp type by @glzhao89 in #20959
- feat: last queries with tags output can be read from cache by @xiao-yu-wang in #20964
- feat: the having clause can be used after the partition by clause or window clause by @xiao-yu-wang in #20981
- enh: adjust size limit of applyQ and negotiationWin by @bgzhao66 in #20990
- enhance: package jdbc-driver for enterprise package by @xleili in #20999
- add learner by @cadem in #20971
- docs: update taoskeeper and monitor docs by @sunpe in #21004
- fix: taosbenchmark escape char for 3.0 by @sangshuduo in #21017
- pause && resume by @54liuyao in #21035
- fix: spool logging msg with pMsg->term in syncNodeOnHeartbeat by @bgzhao66 in #21038
- fix: update commit index from heartbeat on learner in the same way as on follower by @bgzhao66 in #21039
- fix(stream): set the correct start offset for stream task. by @hjxilinx in #21018
- fix/block-check-to-async-check by @cadem in #21045
- enhance: error code refactor by @shenglian-zhou in #21034
- refactor: do some internal refactor and add the sample code. by @hjxilinx in #21053
- enh: support user passwd version by @kailixu in #20828
- remove null pointer by @cadem in #21055
- docs: add docs for
taoskeeper with prometheus
by @sunpe in #21052 - enh: ignore single row null data type validation by @dapan1121 in #21059
- feat: support log slow query by default by @dapan1121 in #21057
- filter voter when agree upon by @cadem in #21073
- change learner config format by @cadem in #21069
- docs: fix tmq create database args by @AdamEECS in #21081
- enh: comment off unused error codes in sync and wal by @bgzhao66 in #21079
- enh: unify error msg for no disk space by @bgzhao66 in #21072
- feature: tag scan group and slimit by @shenglian-zhou in #21066
- docs: update connector matrix for 3.0 by @sangshuduo in #21086
- docs: fix typos and use 3.1.0 in java connector for3.0 by @sangshuduo in #21095
- sim/valgrind: suppress rocks reachables by @stephenkgu in #21091
- Update 14-stream.md by @54liuyao in #21096
- fix(taosAdapter): tmq lift blocking time limit by @huskar-t in #21101
- enhance: enterprise pull jdbc from tag 3.1.0 by @xleili in #21103
- docs:fix compile error by @dingbo8128 in #21108
- docs: release 3.0.4.1 by @xleili in #21109
- docs: release 3.0.4.1 zh by @xleili in #21112
- docs: fix build zh release history failed by @xleili in #21113
- docs(driver): jdbc error code by @huolibo in #21114
- opti:make note more specific by @wangmm0220 in #21084
- opti:make note more specific by @wangmm0220 in #21083
- opti:make note more specific by @wangmm0220 in #21082
- docs: update range for numOfCommitThreads by @sangshuduo in #21122
- docs: update taosdump doc for 3.0 by @sangshuduo in #21130
- docs: python udf docs by @shenglian-zhou in #21138
- enh: support split vgroup by @bgzhao66 in #21107
- docs: add python udf english version by @shenglian-zhou in #21151
- Update 09-udf.md by @DuanKuanJun in #21157
- doc: revise doc structure of UDF by @gccgdb1234 in #21161
- doc: refine the description of TABLE_PREFIX and TABLE_SUFFIX by @gccgdb1234 in #21166
- enh: column/row max length support up to 64K by @kailixu in #20741
- Update 09-udf.md by @DuanKuanJun in #21176
- enh: declare mndSplitVgroup in mndVgroup.h by @bgzhao66 in #21173
- enh: unify error msg for disk space checking in doInitAggInfoSup by @bgzhao66 in #21162
- enh: commit vnode after consolidating alter hash range by @bgzhao66 in #21155
- fix: having clause issues by @dapan1121 in #21168
- balance leader skew by @cadem in #21146
- Update 14-stream.md by @54liuyao in #21182
- feat: process split vgroup msg imp by @bgzhao66 in #21188
- fix: projection merge issue by @dapan1121 in #21191
- Update 14-stream.md by @54liuyao in #21189
- docs: fix example tmp by @AdamEECS in #21197
- transaction multithread by @cadem in #21136
- fix: projection group merge issue by @dapan1121 in #21202
- fix: memory leak by @kailixu in #21207
- fix: count wrong group number issue by @dapan1121 in #21215
- enh(cache/rocks): base for rocks put/get by @stephenkgu in #20996
- docs: add cmd to check status of taosd for mac by @AdamEECS in #21220
- docs: add example and document for rust schemaless insert by @AdamEECS in #21024
- docs(driver): jdbc 3.2.1 document and error code by @huolibo in #21223
- docs(driver): fix format by @huolibo in #21233
- docs: resolve format error by @gccgdb1234 in #21239
- docs: formate and note of java doc by @huolibo in #21235
- docs: fix broken and redirecting links in 3.0 by @danielclow in #20684
- fix(tsdb/cache): primary ts column loading by @stephenkgu in #21234
- fix(cache/get): fix rocks batch writing when storing loading results by @stephenkgu in #21248
- Feat/restore dnode by @cadem in #21247
- fix(tmq): add some logs. by @hjxilinx in #21255
- chore: add libgeos-dev package for geometry pull-request by @sangshuduo in #21256
- feat: support connectors licensing by @kailixu in #21253
- fix(tmq): update the local offset when retrieving offset from vnode. by @hjxilinx in #21260
- doc: remove experimental.enable.snapshot by @gccgdb1234 in #21264
- feat(query): support interp with super table by @glzhao89 in #21144
- Update 04-stable.md by @plum-lihui in #21272
- Update 04-stable.md by @plum-lihui in #21273
- fix: error wording about reconfig by @sangshuduo in #21049
- enh: error code optimization by @kailixu in #21128
- test:add testcases of learner in election by @tomchon in #21261
- invalid message by @cadem in #21274
- fix: kafka connector doc issues by @feici02 in #21277
- docs(driver): fix jdbc error code by @huolibo in #21278
- docs: fix markdown error by @danielclow in #21280
- docs: fix broken links by @sangshuduo in #21288
- merge main into 3.0 branch. by @hjxilinx in #21270
- Enh/rocksdb sstate merge dev3 by @yihaoDeng in #21245
- docs: update readme with libgflags by @sangshuduo in #21293
- docs: update grafana doc for 3.0 by @sangshuduo in #21300
- enh(taosAdapter): update taosAdapter by @huskar-t in #21298
- feat: create dnode limited by grant of cpu cores by @kailixu in #21291
- Correction/纠正 01-data-type.md by @meeki007 in #21313
- Enh/rocksdb sstate merge by @yihaoDeng in #21295
- feature(tmq): add new API to extract offset from result set. by @hjxilinx in #21316
- fix: client hb logic fix and optimization by @kailixu in #21315
- fix: file fd not clear issue by @dapan1121 in #21319
- fix: fix null pointer issue in destroyTimesliceOperator by @glzhao89 in #21308
- pause source task by @54liuyao in #21320
- chore: restore the test cases by @kailixu in #21327
- chore: change rocksdb repo to upstream by @sangshuduo in #21318
- fix: client HB logic by @kailixu in #21334
- fix stream queue mem leak by @54liuyao in #21344
- Update cases.task by @plum-lihui in #21357
- merge main, and do some internal refactor. by @hjxilinx in #21353
- test: add cluster global tdDnodes by @DuanKuanJun in #21317
- feat: add restore command sets to taos-CLI by @DuanKuanJun in #21354
- fix: fill window end issue by @dapan1121 in #21363
- Update container_build.sh by @tomchon in #21370
- Test/td 21161 by @happyguoxy in #21326
- fix pause agg task by @54liuyao in #21367
- state merge into 3.0 by @yihaoDeng in #21364
- restore learner stuck by @cadem in #21350
- refactor: do some internal refactor. by @hjxilinx in #21362
- test: increase timeouts of checking db ready by @tomchon in #21373
- change restore prompt message by @cadem in #21366
- status resp return null when only one ep by @cadem in #21322
- Feat/force drop dnode by @cadem in #21325
- docs: fix zh/02-intro by @sangshuduo in #21380
- doc: refine Kafka source connector configuration parameters by @gccgdb1234 in #21377
- enh(cache/batchread): load columns in one trip by @stephenkgu in #21331
- fix: invalid memory access of buffer after vsnprintf by @wangjiaming0909 in #21348
- doc: refine kafka connector by @gccgdb1234 in #21381
- opt stream ci test by @54liuyao in #21383
- feat: support client meta control by @dapan1121 in #21335
- feat: support update active code by SQL by @kailixu in #21358
- fix: fix interp in nested query and fill value with string issue by @glzhao89 in #21352
- feat(query): interp support ignore null value opition by @glzhao89 in #21329
- fix: fix assert crash in tsort.c by @glzhao89 in #21391
- docs: update wechat tdengine1 by @sangshuduo in #21395
- Refact/fillhistory by @hjxilinx in #21396
- docs: udf requires python 3.7+ by @sangshuduo in #21397
- doc: add td.connect.type by @gccgdb1234 in #21400
- doc: refine td.connect.type by @gccgdb1234 in #21403
- fix: free assignment crash on Windows by @huskar-t in #21402
- add parameter by @yihaoDeng in #21409
- fix(cache/loading): skip partial loading tables by @stephenkgu in #21410
- docs: update docs for tdinsight and monitor by @sunpe in #21408
- Update 01-faq.md by @DuanKuanJun in #21392
- fix: last query taosd crash issue by @dapan1121 in #21415
- Update 01-faq.md by @DuanKuanJun in #21420
- fix(cache/commit): skip batch write if empty by @stephenkgu in #21389
- fix: last(tag) report error issue by @dapan1121 in #21419
- docs: update connect for jdbc version by @sangshuduo in #21426
- fix: tsdb read and append _wstart for tsma by @kailixu in #21413
- enh(taosAdapter): support get assignment and seek by @huskar-t in #21416
- fix(cache/read): set none to null instead of skipping column by @stephenkgu in #21432
- fix(cache): sychronize commit, read and write threads with writebatch by @stephenkgu in #21429
- test:comment unit test catalog in cases by @tomchon in #21435
- test:comment the testcase of tmq by @tomchon in #21425
- Remove taosx, connector, explorer and taoskeeper in cloud tdengine by @wingwing2005 in #21422
- fix: alter dnode option has no effect by @kailixu in #21439
- fix: last(tbname) out of memory issue by @dapan1121 in #21430
- docs: 3.0.4.2 by @xleili in #21443
- build: release ver-3.0.4.2 by @xleili in #21444
- fix: remove taosTools zh duplicate release ver-2.5.0 by @xleili in #21447
- set agg pause status by @54liuyao in #21442
- comfirm msg when add learner by @cadem in #21446
- fix: delete index filter for desc query by @kailixu in #21437
- fix: core dump when select with interval caused by order by @wangjiaming0909 in #21448
- feat:[TD-23117] add schema for ins_topics by @wangmm0220 in #21282
- enh: support config batch rows number when import data from csv file by @dapan1121 in #21440
- Feature/3.0 geometry by @dinglezhang in #21037
- docs: update readme with geos by @sangshuduo in #21453
- docs: remove geometry docs by @sangshuduo in #21454
- enh: stmt insert demo by @huolibo in #21456
- fix: fix count var type error by @glzhao89 in #21455
- skip some errors by @xiaopingcs in #21458
- fix(cache/read): associate tbname with primary ts column by @stephenkgu in #21445
- implement TD-24378 by @wingwing2005 in #21452
- implement TD-24378 by @wingwing2005 in #21451
- fix sma sliding window by @54liuyao in #21462
- test case parameter by @cadem in #21464
- docs: update readme by @sangshuduo in #21473
- feat:add privilege for schemaless by @wangmm0220 in #21470
- enh: tmq example add all properties by @huolibo in #21465
- refactor: remove the module dependency. TD-24279 by @hjxilinx in #21449
- enh(taosAdapter): change taosAdapter to 3.0 branch by @huskar-t in #21482
- enh(taosAdapter): change taosAdapter to main branch by @huskar-t in #21481
- prompt message when 2 mnode by @cadem in #21483
- enhance: python udf sample with numpy by @shenglian-zhou in #21484
- fix: fix timestamp conversion error due to daylight saving time by @glzhao89 in #21467
- fix: fix log not compressed when logKeepDays set to positive value by @glzhao89 in #21469
- fix(query): fix bug in iterating the table list. by @hjxilinx in #21486
- fix(query): fix bug in iterating the table list. by @hjxilinx in #21489
- fix(tmq): set the correct offset rsp when no poll occuring. by @hjxilinx in #21492
- fix pause issue by @54liuyao in #21487
- fix tsdb read error version by @54liuyao in #21478
- fix(cache): fix deleted data by @stephenkgu in #21479
- check offline when force by @cadem in #21490
- fix:memory leak by @wangmm0220 in #21480
- docs: fix typos by @sangshuduo in #21503
- fix(stream): set the skip offset when resume stream task. by @hjxilinx in #21501
- docs: fix udf zh typos by @sangshuduo in #21504
- enhance: scalar filter improvement by @shenglian-zhou in #21405
- doc: refine python UDF user guide by @gccgdb1234 in #21500
- Update 09-udf.md by @gccgdb1234 in #21508
- docs: fix timeline function super table explaination by @glzhao89 in #21491
- fix: sliding can't be 1a issue by @dapan1121 in #21497
- enh: refactor some func names for syncLogReplProcessReply by @bgzhao66 in #21299
- refactor: do some internal refactor. by @hjxilinx in #21511
- docs: update alter database options for WAL_RETENTION_PERIOD etc. by @bgzhao66 in #21518
- fix: TDengine build on macOS arm64 by @huskar-t in #21513
- fix: page buffer cache flush wrong bytes of data into disk by @wangjiaming0909 in #21512
- fix: show create table privilege issue by @dapan1121 in #21519
- check empty dnode by @cadem in #21498
- refactor: do some internal refactor. by @hjxilinx in #21521
- fix(rocks/log): set info log level to warn_level by @stephenkgu in #21517
- fix: if block row num is less than definit min rows, set the bucket index to 0 by @shenglian-zhou in #21510
- change parameter by @yihaoDeng in #21472
- fix: link rocksdb shared library by @hzcheng in #21457
- fix:put poll to push manager if wal not exist when offset is latest by @wangmm0220 in #21499
- Update 01-faq.md by @DuanKuanJun in #21522
- set sched status by @54liuyao in #21516
- fix: disable debug info by @dapan1121 in #21525
- feat(driver): jdbc add tmq seek function by @huolibo in #21279
- docs: refine python connector doc for req_id by @sangshuduo in #21526
- fix(query): invalid buf page by @kailixu in #21514
- add smaBasic.py test case by @DuanKuanJun in #21529
- fix(stream): fix memory leak in handling dispatch msg. TD-24447 by @hjxilinx in #21531
- enh: stmt column length validation by @dapan1121 in #21542
- fix(stream): set the correct rps flag as the dispatch rsp. by @hjxilinx in #21539
- fix sliding window issue by @54liuyao in #21530
- Enh/change rocks param by @yihaoDeng in #21545
- change db param by @yihaoDeng in #21541
- fix(query): check the version range when dump partial rows. #TD-24504 by @hjxilinx in #21551
- feat: support create topic as stable with conditions by @wangmm0220 in #21515
- fix:make version compatible by @wangmm0220 in #21557
- set session update info by @54liuyao in #21559
- reset sream fill block index by @54liuyao in #21558
- set recover scan version by @54liuyao in #21564
- add test case for ci from test task ts_3398 by charles by @flyingangel2013 in #21438
- fix: fix interval operator ensure capacity logic by @glzhao89 in #21556
- other: merge main into 3.0. by @hjxilinx in #21555
- other: merge 3.0 into main. by @hjxilinx in #21570
- reset ud table name by @54liuyao in #21566
- feat: change get taos-tools code with 3.0 branch by @DuanKuanJun in #21573
- feat: get taos-tools code from main branch by @DuanKuanJun in #21574
- change log level by @yihaoDeng in #21553
- git enchance: change goes as static link by @xleili in #21582
- enchance: support libgeos static link & install remove librocksdb by @xleili in #21580
- fix:open task case & modify mqRebVgReq encode/decode style by @wangmm0220 in #21562
- reset unclosed window info by @54liuyao in #21577
- fix: no data after seek by @huskar-t in #21578
- refactor: do some internal refactor. by @hjxilinx in #21554
- fix: no data after seek by @huskar-t in #21579
- other: merge main. by @hjxilinx in #21585
- other: merge 3.0 into main. by @hjxilinx in #21587
- chore: unbind the dependency by @kailixu in #21588
- chore: update docs about support row/columns width up to 64KB by @kailixu in #21611
- fix(cache/binary): fix crash in freeItem by @stephenkgu in #21608
- fix/td-24600 by @yihaoDeng in #21590
- fix: change compile opt by @yihaoDeng in #21609
- docs(driver): jdbc consumer demo by @huolibo in #21618
- test:modify test case for multilevel.py by @jiacy-jcy in #21623
- update release version by @xiaopingcs in #21625
- update taos-tools release version by @xiaopingcs in #21628
- fix stream ci issue by @54liuyao in #21617
- docs: td-24609 by @AdamEECS in #21636
- docs: go connector add tmq assignment and seek by @huskar-t in #21619
- Update 03-package.md by @CubicTec in #21634
- fix(client): remove dependency by @hjxilinx in #21637
- opt stream sliding test by @54liuyao in #21633
- opt stream ci test by @54liuyao in #21630
- fix: rewrite count(*) to count(1) for temp table by @shenglian-zhou in #21606
- fix(os): add -L jemalloc lib path by @stephenkgu in #21642
- fix/TD-24644 by @cadem in #21635
- docs: add version history by @AdamEECS in #21643
- change link opt by @yihaoDeng in #21638
- fix: exclude install and remove lbrocksdb.so by @xleili in #21652
New Contributors
- @meeki007 made their first contribution in #21313
- @dinglezhang made their first contribution in #21037
- @flyingangel2013 made their first contribution in #21438
- @CubicTec made their first contribution in #21634
Full Changelog: ver-3.0.4.2...ver-3.0.5.0