github wgzhao/Addax 4.0.0

latest releases: 4.2.2, 4.2.1, 4.2.0...
3 years ago

This is the first release under the new brand name

Highlights

  1. HDFS reader plugin supports reading of binary data type
  2. HDFS writer plugin supports writing of binary data type
  3. Rewritten the implementation of ORC and PARQUET for more efficient execution
  4. Rewrite the string type field splitting method using the database's own functions
  5. Add support for auto guess splittable column (introduce new configure item autoPk)
  6. Bump a dozen of 3rd dependencies to the updated version

该版本主要功能点

  1. HDFS reader 插件支持读取 binary 数据类型
  2. HDFS writer 插件支持写入 binary 数据类型,该数据可以从关系型数据库的 Blob 字段获取(Oracle 暂未支持)
  3. 重写了 ORC 和 PARQUET 文件的读写代码,使用两个文件系统的直接API,提升读写效率
  4. 重写了字符类型字段的分割函数,利用数据自身的功能提供了更合理的分割范围,而不是之前的随机字符串分割,并支持中文
  5. 增加了读取关系型数据库表时自动猜测分割字段的功能(默认不启用),详见文档 说明
  6. 大量第三方依赖包升级到最新版本

Details

c52715e Changes main program to addax.py
288fbdf Add tdengine reader/writer plugins to package #224
6c9f4b4 Bugfixed: exception occurred when move hdfs file from temporary location to destination
e582f76 Use phoenix-core instead of phoenix-client to reduce size
76d3062 Bugfix: string format error
86457ab Add correspond compress suffix
9c6e88f Bump postgresql from 42.2.22 to 42.2.23 (#218)
df8e8be Add correspond compress suffix
e4a04a8 Bump maven-javadoc-plugin from 3.2.0 to 3.3.0 (#211)
83221c3 Bump commons-net from 3.7.2 to 3.8.0 (#212)
7cbf4c6 Bump parquet-avro from 1.11.1 to 1.12.0 (#213)
6264fac Bump httpcore from 4.4.13 to 4.4.14 (#214)
40e6194 Rewrite the string type field splitting method using the database's own functions
18a6c3f Bump kudu-client from 1.8.0 to 1.15.0 (#190)
d8f3295 Bump nexus-staging-maven-plugin from 1.6.7 to 1.6.8 (#210)
91de3c2 Bump commons-lang3 from 3.11 to 3.12.0 (#209)
0e80d62 Bump commons-io from 2.8.0 to 2.10.0 (#208)
68f7e03 Bump snappy-java from 1.1.8 to 1.1.8.4 (#207)
4a638dc Bump taos-jdbcdriver from 2.0.18 to 2.0.32 (#204)
5c25819 Bump fastjson from 1.2.75 to 1.2.76 (#206)
58a8a57 Bump ognl from 3.2.18 to 3.2.21 (#203)
8fc41ad Bump joda-time from 2.9.7 to 2.10.10 (#205)
02b000b Bump maven-dependency-plugin from 3.1.2 to 3.2.0 (#199)
6802862 Bump jedis from 3.4.0 to 3.6.1 (#201)
62ea143 Bump error_prone_core from 2.4.0 to 2.7.1 (#202)
8fee29b Bump maven-gpg-plugin from 1.5 to 3.0.1 (#194)
49342f7 Bump postgresql from 42.2.19 to 42.2.22 (#195)
eb09cad Bump aircompressor from 0.17 to 0.19 (#196)
96c3c85 Bump druid from 1.2.4 to 1.2.6 (#197)
96e7e75 Add support for auto guess splittable column (#198)
920f93c Bump cassandra.jdbc.version from 3.10.2 to 3.11.0 (#192)
44e2d72 Bump avro from 1.10.1 to 1.10.2 (#188)
26d2783 Bump slf4j-api from 1.7.28 to 1.7.31 (#189)
f8d481b Bump json-path from 2.5.0 to 2.6.0 (#191)
b254c18 Bump javadbf from 1.11.2 to 1.13.1 (#186)
238fbe2 Bump fluent-hc from 4.5.12 to 4.5.13 (#187)
9516941 Bump httpcore-nio from 4.4.13 to 4.4.14 (#181)
ca07dca Bump redis-replicator from 3.4.2 to 3.5.4 (#148)
89feeb7 Bump mongo-java-driver from 3.12.7 to 3.12.8 (#142)
0320d50 Add test for decimal type
98286bb Allow all fields accept null value
52ace57 plugin hdfswriter: rewrite orc file schema generate method
ce8cfbf plugin hdfsreader: simplify parquet file reader codes
bd4aa41 Uses consistent variable definitions
0c14f86 plugin: dbffilewriter Add check for column configuration
8401fbc bugfix FileNotFoundException
73e8e03 Support for read/write Parquet file with binary type
1c0b80d Add some test cases
cbe64da plugin hdfsreader: Add support for reading binary data
4e008b9 plugin hdfswriter: Add support for writing binary data with OrcFile

Don't miss a new Addax release

NewReleases is sending notifications on new releases.