jqnatividad/qsv 0.130.0

on GitHub

Following the 0.129.0 release - the largest release to date, 0.130.0 continues to polish qsv as a data-wrangling engine, packing new features, fixes, and improvements, previewing upcoming features in qsv pro 1.0. Here are a few highlights:

Highlights

• Added .ssv (semicolon separated values) automatic support. Semicolon separated values are now automatically detected and supported by qsv. Though not as common as CSV, SSV is used in some regions and industries, so qsv now supports it.
• Added cargo deb compatibility. In preparation for the release of DataPusher+ 1.0, we're now making it easier to upgrade qsvdp so CKAN administrators can install and upgrade it easily using apt-get install qsvdp or apt-get upgrade qsvdp.
  DP+ is our next-gen, high-speed data ingestion tool for CKAN that uses qsv as its analysis engine. Its not only a robust, fast, validating data pump that guarantees high quality data, it also does extended analysis to infer and derive high-quality metadata - what we call "automagical metadata".
• Upgraded to the latest Polars upstream at the py-polars-1.3.0 tag. Polars tops the TPC-H Benchmark and is several orders of magnitude faster than traditional dataframe libraries (cough - 🐼 pandas). qsv proudly rides the 🐻‍❄️ Polars bear to get subsecond response times even with very large datasets!
• qsv v0.130.0 shell completions files are available for download here. With shell completions, pressing tab in a compatible shell provides suggestions for various qsv commands, subcommands, and options that you can choose from. Supported shells include bash, zsh, powershell, fish, nushell, fig, and elvish. View tips on how to install completions for the bash shell here.

Added

• apply: add base62 encode/decode operations #2013
• headers: add --just-count option #2004
• json: add --select option #1990
• searchset: add --not-one flag by @rzmk in #1994
• Added .ssv (semicolon separated values) automatic support #1987
• Added cargo deb compatibility by @tino097 in #1991
• contrib(completions): add --just-count for headers by @rzmk in #2006
• contrib(completions): add --select for json by @rzmk in #1992
• added several benchmarks
• added more tests

Changed

• diff: allow selection of --key and --sort-columns by name, not just by index #2010
• fetch & fetchpost: replace deprecated Redis execute command 75cbe2b
• stats: more intelligent --infer-lenoption c6a0e64
• validate: return delimiter detected upon successful CSV validation #1977
• bump polars to latest upstream at py-polars-1.3.0 tag #2009
• deps: bump csvs_convert from 0.8.12 to 0.8.13 d1d0800
• build(deps): bump cached from 0.52.0 to 0.53.0 by @dependabot in #1983
• build(deps): bump cached from 0.53.0 to 0.53.1 by @dependabot in #1986
• build(deps): bump postgres from 0.19.7 to 0.19.8 by @dependabot in #1985
• build(deps): bump pyo3 from 0.22.1 to 0.22.2 by @dependabot in #1979
• build(deps): bump redis from 0.25.4 to 0.26.0 by @dependabot in #1995
• build(deps): bump serde_json from 1.0.120 to 1.0.121 by @dependabot in #2011
• build(deps): bump simple-expand-tilde from 0.1.7 to 0.4.0 by @dependabot in #1984
• build(deps): bump tokio from 1.38.0 to 1.38.1 by @dependabot in #1973
• build(deps): bump tokio from 1.38.1 to 1.39.1 by @dependabot in #1988
• build(deps): bump xxhash-rust from 0.8.11 to 0.8.12 by @dependabot in #1997
• apply select clippy suggestions
• updated several indirect dependencies
• made various usage text improvements
• pin Rust nightly to 2024-07-26

Fixed

• diff: clarify --key usage examples, resolves #1998 by @rzmk in #2001
• json: refactored so it didn't need to use threads to spawn qsv select to order the columns. Had to do this as sometimes intermediate output was sent to stdout before the final output was ready 0f25def
• py: replace row with col in usage text by @allen-chin in #2008
• reverse: fix indexed bug #2007
• validate: properly auto-detect tab delimiter when file extension is TSV or TAB #1975
• fix panic when process_input helper fn receives unexpected input from stdin 152fec4

Removed

• docs: remove *nix only message for foreach by @rzmk in #1972

New Contributors

• @tino097 made their first contribution in #1991
• @allen-chin made their first contribution in #2008

Full Changelog: 0.129.1...0.130.0

To stay updated with datHere's latest news and updates (including qsv pro, datHere's CKAN DMS, and analyze.dathere.com), subscribe to the newsletter here: dathere.com/newsletter