Addax 5.1.0 has released! 🎉

Highlight

1. HDFS Reader Plugin: Complex Data Type Support

Added support for Array and Map data types in ORC and Parquet file formats, enabling seamless read operations for nested and structured data.

2. HDFS Writer Plugin: Expanded Data Compatibility

Enhanced the writer plugin to support Array and Map data types for ORC and Parquet formats, ensuring full compatibility with complex data structures during write operations.

3. Hive Reader Plugin: Complex Data Type Support

Added support for Array and Map data types in ORC and Parquet file formats

4. Optimized Package Size

Streamlined dependencies and removed redundant components, reducing the overall package size by ~40% (from 900MB to 450MB). This improves deployment efficiency and resource utilization.

📜 Changes

• [feature][plugin][hivereader] Add support for ARRAY and Map data type in HiveReader
• [improve][plugin][hdfswriter] Refactor ORC writer for code clarity
• [feature][plugin][hdfsreader] Add support for ARRAY and MAP data types in ORC and Parquet file format (#1240)
• [chore][doc] Refine slogan picture
• [feature][plugin][hdfswriter] Add support for ARRAY and MAP data type in ORC and Parquet format (#1238)
• [improve][plugin][paimonwriter] Add dynamic bucket mode support and optimize write logic
• [improve][docs] Update statsreport.md for clarity and add example service interface
• [improve][plugin][hdfsreader] Optimize dependencies to reduce package size by 50%
• [improve][issue-template] Update default version in NEW-ISSUE.yml to 5.0.0 and add additional version options
• [improve][script] Enhance build-module script to support multiple module builds and improve deployment logic
• [improve][plugin][hdfswriter] Optimize dependencies to reduce package size by 50%
• [improve][lib][rdbms] Refactor update SQL generation for cleaner code and enhanced readability
• [bugfix][plugin][oraclewriter] Remove unnecessary Clob and Blob freeing in PreparedStatement handling
• [improve][plugin][rdbms] Add case-insensitive column matching for update mode
• [chore][plugin][oracle] Bump Oracle JDBC driver from 19.7.0.0 to 21.5.0.0
• [improve][plugin][oraclewriter] Enhance CLOB and BLOB handling in PreparedStatement and add NVARCHAR support
• [feature][lib][rdbms] Revert to use commons-dbcp2 BasicDataSource for better compatibility
• [bugfix][plugin][hivereader] Fix backwards compatibility with older Hive JDBC drivers
• [feature][plugin][hivereader] Optimize dependencies with minimal Hive JDBC imports
• [bugfix][plugin] update MySQL connector dependency from mysql-connector-java to mysql-connector-j
• [improve][core] remove redundant echo statements before executing Addax job
• [chore][core] Bump ch.qos.logback:logback-core and logback-classic from 1.3.12 to 1.3.15
• [refactor][lib][rdbms] replace BasicDataSource with DruidDataSource and update dependency versions in pom.xml
• [improve][core] enhance boolean parsing to accept 'yes' and 'no' strings
• [feature][lib][rdbms] implement efficient string split point calculation for primary keys
• [improve][lib][rdbms] streamline SQL schema handling and improve readability in GetPrimaryKeyUtil
• [feature][plugin][hdfsreader] remove unused Hive dependencies and update exclusions in pom.xml
• [refactor][plugin][hdfsreader] remove unused JobConf and update dependency versions in pom.xml
• [feature][plugin][accesswriter] add file existence check and creation for MDB file
• [chore][plugin][jsonfilereader] use updated json-smart dependency to fix CVE-2023-1370
• [feature][plugin][kafkareader] add maxMessageNumber configuration to KafkaReader for limiting message consumption
• [refactor][plugin][paimonweriter] update pom.xml to use properties for version management of Paimon dependencies
• [refactor][plugin][hivereader] update pom.xml to simplify dependency management and improve version handling
• [refactor][core] remove version numbers and exclusions from dependencies in pom.xml
• [update][3rd] update dependencies in pom.xml to latest versions
• [feature][core] add install_plugins.sh script for plugin installation from Maven Central
• [improve][core] update addax.sh to improve error handling and code readability
• [chore][docs] update README files to remove lite image references and adjust build instructions
• [refactor][core] improved directory structure and module organization (#1224)
• [improve][lib][rdbms] update executeQuery method to handle non-result set queries
• [bugfix][plugin][doriswriter] add missing MySQL jdbc driver
• [maven-release-plugin] prepare for next development iteration