This is the first release under the new brand name
Highlights
- HDFS reader plugin supports reading of binary data type
- HDFS writer plugin supports writing of binary data type
- Rewritten the implementation of ORC and PARQUET for more efficient execution
- Rewrite the string type field splitting method using the database's own functions
- Add support for auto guess splittable column (introduce new configure item
autoPk
) - Bump a dozen of 3rd dependencies to the updated version
该版本主要功能点
- HDFS reader 插件支持读取 binary 数据类型
- HDFS writer 插件支持写入 binary 数据类型,该数据可以从关系型数据库的 Blob 字段获取(Oracle 暂未支持)
- 重写了 ORC 和 PARQUET 文件的读写代码,使用两个文件系统的直接API,提升读写效率
- 重写了字符类型字段的分割函数,利用数据自身的功能提供了更合理的分割范围,而不是之前的随机字符串分割,并支持中文
- 增加了读取关系型数据库表时自动猜测分割字段的功能(默认不启用),详见文档 说明
- 大量第三方依赖包升级到最新版本
Details
c52715e Changes main program to addax.py
288fbdf Add tdengine reader/writer plugins to package #224
6c9f4b4 Bugfixed: exception occurred when move hdfs file from temporary location to destination
e582f76 Use phoenix-core instead of phoenix-client to reduce size
76d3062 Bugfix: string format error
86457ab Add correspond compress suffix
9c6e88f Bump postgresql from 42.2.22 to 42.2.23 (#218)
df8e8be Add correspond compress suffix
e4a04a8 Bump maven-javadoc-plugin from 3.2.0 to 3.3.0 (#211)
83221c3 Bump commons-net from 3.7.2 to 3.8.0 (#212)
7cbf4c6 Bump parquet-avro from 1.11.1 to 1.12.0 (#213)
6264fac Bump httpcore from 4.4.13 to 4.4.14 (#214)
40e6194 Rewrite the string type field splitting method using the database's own functions
18a6c3f Bump kudu-client from 1.8.0 to 1.15.0 (#190)
d8f3295 Bump nexus-staging-maven-plugin from 1.6.7 to 1.6.8 (#210)
91de3c2 Bump commons-lang3 from 3.11 to 3.12.0 (#209)
0e80d62 Bump commons-io from 2.8.0 to 2.10.0 (#208)
68f7e03 Bump snappy-java from 1.1.8 to 1.1.8.4 (#207)
4a638dc Bump taos-jdbcdriver from 2.0.18 to 2.0.32 (#204)
5c25819 Bump fastjson from 1.2.75 to 1.2.76 (#206)
58a8a57 Bump ognl from 3.2.18 to 3.2.21 (#203)
8fc41ad Bump joda-time from 2.9.7 to 2.10.10 (#205)
02b000b Bump maven-dependency-plugin from 3.1.2 to 3.2.0 (#199)
6802862 Bump jedis from 3.4.0 to 3.6.1 (#201)
62ea143 Bump error_prone_core from 2.4.0 to 2.7.1 (#202)
8fee29b Bump maven-gpg-plugin from 1.5 to 3.0.1 (#194)
49342f7 Bump postgresql from 42.2.19 to 42.2.22 (#195)
eb09cad Bump aircompressor from 0.17 to 0.19 (#196)
96c3c85 Bump druid from 1.2.4 to 1.2.6 (#197)
96e7e75 Add support for auto guess splittable column (#198)
920f93c Bump cassandra.jdbc.version from 3.10.2 to 3.11.0 (#192)
44e2d72 Bump avro from 1.10.1 to 1.10.2 (#188)
26d2783 Bump slf4j-api from 1.7.28 to 1.7.31 (#189)
f8d481b Bump json-path from 2.5.0 to 2.6.0 (#191)
b254c18 Bump javadbf from 1.11.2 to 1.13.1 (#186)
238fbe2 Bump fluent-hc from 4.5.12 to 4.5.13 (#187)
9516941 Bump httpcore-nio from 4.4.13 to 4.4.14 (#181)
ca07dca Bump redis-replicator from 3.4.2 to 3.5.4 (#148)
89feeb7 Bump mongo-java-driver from 3.12.7 to 3.12.8 (#142)
0320d50 Add test for decimal type
98286bb Allow all fields accept null value
52ace57 plugin hdfswriter: rewrite orc file schema generate method
ce8cfbf plugin hdfsreader: simplify parquet file reader codes
bd4aa41 Uses consistent variable definitions
0c14f86 plugin: dbffilewriter Add check for column configuration
8401fbc bugfix FileNotFoundException
73e8e03 Support for read/write Parquet file with binary type
1c0b80d Add some test cases
cbe64da plugin hdfsreader: Add support for reading binary data
4e008b9 plugin hdfswriter: Add support for writing binary data with OrcFile