slingdata-io/sling-cli v1.4.17

on GitHub

Sling v1.4.17 (2025-08-19T21:47:49Z)

Database Connections & Support

New Database Support

• Added Exasol connector: New database type with comprehensive bulk import functionality, CSV handling, and timestamp formatting

Connection Improvements

• BigQuery: Improved error handling for dataset access failures (warnings instead of errors)
• Athena: Enhanced workgroup error messaging with specific workgroup names
• Iceberg: Added S3 property pass-through for REST access and filesystem operations
• DuckDB: Enhanced AWS credential chain support with automatic provider detection

Transform System Overhaul

• Major refactor: Complete rewrite of the transform system architecture
  • Replaced legacy Transform with new TransformLegacy for backward compatibility
  • Introduced staged transforms with ParseStageTransforms function
  • Enhanced expression evaluation with record references
• New transform features:
  • Support for multi-stage transformations
  • Cross-column references using record.field_name syntax
  • Conditional logic and calculated fields
  • Enhanced function support (hash, cast, coalesce, etc.)

Data Type Enhancements

Time Data Types

• PostgreSQL TIME: Proper preservation of TIME, TIME WITH TIME ZONE, and TIME(precision) types
• MSSQL UNIQUEIDENTIFIER: Full support for UUID/GUID data type preservation
• Enhanced time handling: Better formatting and timezone support across all databases

Column Type Improvements

• Position handling: Fixed column position assignment when merging new columns
• String type detection: Refined logic for TIME vs STRING type classification
• Arrow format: Enhanced support for Arrow file format detection and streaming

File Processing & Encoding

Encoding Support

• Target encoding: Full implementation of encoding options for file outputs
• Multiple encodings: Support for Latin1, UTF8, Windows1252 encoding in target files
• Source/target distinction: Separate decode (source) and encode (target) handling

File Format Enhancements

• Arrow format: Improved detection and processing of Arrow files vs streams
• CSV improvements: Better handling of special characters and null values
• Parquet optimization: Enhanced reading performance and DuckDB integration

Performance & Optimization

• Debug logging: Converted g.DebugLow to g.Debug across codebase for better performance
• Memory management: Improved handling of large datasets and file processing
• Concurrent processing: Enhanced parallel processing capabilities

API & Pagination

• Fixed API pagination: Corrected pagination logic in API specifications
  • Fixed length(records) to length(response.records)
  • Improved header parsing with proper bracket notation

Testing & Quality Assurance

• New test suites: Added comprehensive tests for:
  • Staged transforms functionality
  • Encoding target options
  • Time data type preservation
  • UNIQUEIDENTIFIER handling
  • Large file ingestion (100K rows)
• Enhanced test coverage: Improved validation and error checking across test cases

Bug Fixes

• CSV string casting: Fixed method calls from CastToString to CastToStringCSV for proper CSV formatting
• Compression handling: Added compression parameter support for Databricks and Snowflake connections
• Error handling: Improved error messages and warnings across various connectors
• API column handling: Fixed column detection and handling in API responses