jqnatividad/qsv 0.112.0

on GitHub

This is the second in a series of "Giddy-up" 🏇🏽 releases, improving the performance of the following commands:

• stats: by refactoring the code to detect empty cells more efficiently, and by removing
  unnecessary bounds checks in the main compute loop. (~10% performance improvement)
• sample: by refactoring the code to use an index more effectively when available - not only making it faster, but also eliminating the need to load the entire dataset into memory. Also added a --faster option to use a faster random number generator. (~15% performance improvement)
• frequency, schema, search & validate by amortizing/reducing allocations in hot loops
• excel: by refactoring the main hot loop to convert Excel cells more efficiently

The prebuilt binaries are also built with CPU optimizations enabled for x86_64 and Apple Silicon (arm64) architectures.

0.112.0 is also a "Carousel" (i.e. increased usability) 🎠 release featuring new Jupyter notebooks in the contrib/notebooks directory to help users get started with qsv.

• intro-to-count.ipynb by @rzmk
• qsv-describegpt-qa.ipynb by @a5dur

Added

• sqlp: added CASE expression support with Polars 0.32 9d508e6
• sample: added --faster option to use a faster random number generator #1210
• jsonl: added --delimiter option #1205
• excel: added --delimiter option ab73067
• notebook/describegpt: added describegpt QA Jupyter notebook by @a5dur in #1215
• notebook/count: added intro-to-count.ipynb by @rzmk in #1207

Changed

• stats: refactor hot compute function - 35999c5
• stats: faster detection of empty samples b054815 and a7f0836
• sample: major refactor making it faster, but also eliminating need to load the entire dataset into memory when an index is available. #1210
• frequency: refactor primary ftables function 57d660d
• excel: refactor main loop for more performance - 61f227b
• rustfmt: match_block_trailing_comma #1206
• bump MSRV to 1.71.1 1c99364
• apply clippy suggestions #1209
• build(deps): bump tokio from 1.29.1 to 1.30.0 by @dependabot in #1204
• build(deps): bump log from 0.4.19 to 0.4.20 by @dependabot in #1211
• build(deps): bump redis from 0.23.1 to 0.23.2 by @dependabot in #1213
• build(deps): bump tokio from 1.30.0 to 1.31.0 by @dependabot in #1212
• build(deps): bump sysinfo from 0.29.7 to 0.29.8 by @dependabot in #1214
• upgrade to Polars 0.32.0 #1217
• build(deps): bump flate2 from 1.0.26 to 1.0.27 by @dependabot in #1218
• build(deps): bump polars from 0.32.0 to 0.32.1 by @dependabot in #1219
• cargo update bump several indirect dependencies
• pin Rust nightly to 2023-08-13

Removed

• stats: removed Debug derives from structs - 2def136

Fixed

• notebook/count: fix Google Colab link by @rzmk in #1208

Full Changelog: 0.111.0...0.112.0