dathere/qsv 5.1.0

on GitHub

[5.1.0] - 2025-06-17

Highlights

• lens is now colorful by default, with a --monochrome option to turn it off:

   qsv lens /tmp/NYC_311_SR_2010-2020-sample-1M.csv
  

• lens can now have custom prompts with the --prompt option (with support for ANSI escape codes to format the prompt). Meant to be paired with the --echo-column <colname> option, e.g.:

  qsv lens --prompt $'\033[1;5;31mBlinking red, bold text\033[0m' --echo-column 'Unique Key' \
   /tmp/NYC_311_SR_2010-2020-sample-1M.csv
  

• the qsv-stats crate - the underlying engine behind the central stats, frequency and "smart" commands, got a lot of love in this release
• validate got a tad faster while decreasing its memory footprint. The new --no-format-validation option now also allows you to ignore all JSON Schema "format" keywords (e.g. date, email, url, currency, etc.) when validating CSVs.

Added

• lens: add --prompt option, add examples to regex-enabled options #2772
• lens: add --monochrome option, otherwise, columns displayed in different colors #2761
• validate: add --no-format-validation option when in JSON Schema mode #2762
• docs: add shell completions badges by @rzmk in #2760
• feat: added criterion trim algorithm microbenchmarks #2789

Changed

• frequency: performance microoptimizations - use stats cache column cardinality to pre-alloc & size frequency hash tables
• geocode: refactor regex handling for performance & maintainability
• json: preserve key order #2777
• stats: performance microoptimizations - use unwrap_unchecked() instead of just unwrap() in hot sampling functions
• validate: major refactoring for added performance/memory efficiency
• chore: temporarily use qsv-calamine until a new calamine is released #2790
• Bump cpc from 1.9 to 2 #2770
• deps: bump criterion from 0.5 to 0.6 #2791
• deps: use latest csvlens upstream with colorful columnshttps://github.com/dathere/qsv/commit/f2c9322e33a0ac335dafec10a490c871d3de0a6c
• deps: temporarily use qsv-calamine until a new calamine is released #2790
• deps: bump our patched forks of cached, csvs_convert, json-objects-to-csv, jsonschema, localzone, rfd, self_update until PRs are merged or new releases are made
• deps: bump zip from 3 to 4 in 75909d2
• deps: bump polars to 0.48.1 at 49ce57a revision
• build(deps): bump atoi_simd from 0.16.0 to 0.16.1 by @dependabot in #2766
• build(deps): bump bytemuck from 1.23.0 to 1.23.1 by @dependabot in #2778
• build(deps): bump flate2 from 1.1.1 to 1.1.2 by @dependabot in #2781
• build(deps): bump flexi_logger from 0.30.1 to 0.30.2 by @dependabot in #2765
• build(deps): bump flexi_logger from 0.30.2 to 0.31.0 by @dependabot in #2793
• build(deps): bump hashbrown from 0.15.3 to 0.15.4 by @dependabot in #2779
• build(deps): bump libc from 0.2.172 to 0.2.173 by @dependabot in #2787
• build(deps): bump mimalloc from 0.1.46 to 0.1.47 by @dependabot in #2792
• build(deps): bump mlua from 0.10.3 to 0.10.5 by @dependabot in #2758
• build(deps): bump num_cpus from 1.16.0 to 1.17.0 by @dependabot in #2771
• build(deps): bump parking_lot from 0.12.3 to 0.12.4 by @dependabot in #2768
• build(deps): bump pyo3 from 0.25.0 to 0.25.1 by @dependabot in #2785
• deps: upgrade qsv-stats from 0.32 to 0.33, which features major memory and performance optimizations behind the stats & frequency commands #2786
• deps: bump redis from 0.29.5 to 0.32
• build(deps): bump reqwest from 0.12.15 to 0.12.16 by @dependabot in #2764
• build(deps): bump reqwest from 0.12.16 to 0.12.18 by @dependabot in #2767
• build(deps): bump reqwest from 0.12.18 to 0.12.19 by @dependabot in #2773
• build(deps): bump reqwest from 0.12.19 to 0.12.20 by @dependabot in #2782
• build(deps): bump rust_decimal from 1.37.1 to 1.37.2 by @dependabot in #2788
• build(deps): bump smallvec from 1.15.0 to 1.15.1 by @dependabot in #2780
• build(deps): bump sysinfo from 0.35.1 to 0.35.2 by @dependabot in #2774
• build(deps): bump titlecase from 3.5.0 to 3.6.0 by @dependabot in #2775
• build(deps): bump tokio from 1.45.0 to 1.45.1 by @dependabot in #2759
• build(deps): bump uuid from 1.16.0 to 1.17.0 by @dependabot in #2757
• applied select clippy suggestions
• updated indirect dependencies
• set Rust nightly to 2025-05-21, the same nightly Polars uses 872ade1

Fixed:

• fix: frequency recover from non-fatal absence of stats cache, instead of panicking b2821a0
• fix: flaky json tests caused by hardcoding name of intermediate file - 62ca310
• fix: flaky reverse property tests by handling BOM characters cefd490
• fix: util::process_input helper does not honor QSV_SKIP_FORMAT_CHECK when processing dir input #2784

Full Changelog: 5.0.3...5.1.0