jqnatividad/qsv 0.89.0

on GitHub

Added

• cat: added new rowskey subcommand. Unlike the existing rows subcommand, it allows far more flexible concatenation of CSV files by row, even if the files have different number of columns and column order. #795
• added jemalloc support. As the current default mimalloc allocator is not supported in some platforms. Also, for certain workloads, jemalloc may be faster. See Memory Allocator for more info #796
• added --no-memcheck and related QSV_NO_MEMORY_CHECK env var. This relaxes the conservative Out-of-Memory prevention heuristic of qsv. See Memory Management for more info #792

Changed

• --version now returns "non-streaming" mode max input file size and detailed memory info. See Version details for more info #780
• exclude: expanded usage text and added 'input parameters' help by @tmtmtmtm in #783
• stats: performance tweaks in 96e8168, 634d42a and 7e148cf
• Use simdutf8 to do SIMD accelerated utf8 validation, replacing problematic utf8 screening. Together with #782, completes utf8 validation revamp. #784
• Bump sysinfo from 0.27.7 to 0.28.0 by @dependabot in #786
• cargo update bump several indirect dependencies
• pin Rust nightly to 2023-02-18

Removed

• Removed patched versions of csv crate optimized for performance. With the release of csv 1.2, switched back to csv crate upstream. #794
• removed utf8 first 8k screening. It was increasing code complexity and not very reliable. #782

Fixed

• dedup: refactored to use iterators to avoid out of bounds errors. f5e547b
• exclude: don't screen for utf8. This bugfix spurred the utf8 validation revamp, where I realized, I just needed to pull out utf8 screening #781
• py: col, not row #793

New Contributors

• @tmtmtmtm made their first contribution in #783

Full Changelog: 0.88.2...0.89.0