Happy New Year! 🎉🎉🎉
Here's the first release of 2024, the biggest ever with 280+ commits! qsv 0.120.0 continues to focus on performance, stability and reliability as we continue setting the stage for qsv's big brother - qsv pro.
Apart from wrapping qsv with a User Interface, qsv pro also comes with a retinue of related cloud-based data cleaning, enrichment and enhancement services along with expanded metadata inferencing to make your Data Useful, Usable and Used!
qsv pro draws inspiration from OpenRefine, but reimagined without its file size and speed limitations, with qsv pro having the ability to process multi-gigabyte files in seconds.
It incorporates hard lessons we learned in the past 12 years deploying Data Portals and Data Pipelines to create a new Data/Metadata Wrangling and AI-assisted Data Publishing service that's easy to use for casual Excel users and Data Publishers, yet powerful enough for data scientists and data engineers.
But it's not quite ready for release yet, so stay tuned!
We're now taking signups for a preview release however, so if you're interested, please sign up here!
Excitingly, qsv was also mentioned on Hacker News in this thread Dec 23, 2023! As a result, we're now almost at 2,000+ stars on GitHub from 900 stars on Dec 22! 🎉🎉🎉
Stay tuned for more advancements in 2024 – it's set to be a landmark year for qsv! 🦄🦄🦄
Added
cat: add rowskey --group options; increased perf of rowskey #1508validate: add --trim and --quiet options #1452apply&applydp:operations regex_replacenow supports empty--replacementwith the "<NULL>" special value #1470 and #1471exclude: also consider rows with empty fields #1498extsort: add--tmp-diroption ca1f461
Changed
validate: Faster RFC4180 validation with byterecords and SIMD-accelerated utf8 validation #1440excel: minor performance tweaks #1446apply,applydp,explode,geocode,pseudo: consolidate redundant code and use onereplace_column_valuehelper fn in util.rs #1456excel: bump calamine from 0.22 to 0.23 #1473excel&joinp: use atoi_simd for faster &[u8] to int conversion 9521f3ecat,describegpt,headers,sqlp,to,tojsonl: refactor commands that accept multiple input files to use improved process_input helper #1496fetch&fetchpost: get_response refactor for maintainability and performance #1507luau: replaced --no-colindex option with --colindex option. --col-index slows down processing and is not often used, so make it an option, not the default. a0c8568- make thousands crate optional with apply feature in #1453
- build(deps): bump uuid from 1.6.0 to 1.6.1 by @dependabot in #1430
- build(deps): bump serde from 1.0.192 to 1.0.193 by @dependabot in #1432
- build(deps): bump data-encoding from 2.4.0 to 2.5.0 by @dependabot in #1435
- build(deps): bump mlua from 0.9.1 to 0.9.2 by @dependabot in #1436
- build(deps): bump url from 2.4.1 to 2.5.0 by @dependabot in #1437
- build(deps): bump jql-runner from 7.0.6 to 7.0.7 by @dependabot in #1439
- build(deps): bump jql-runner from 7.0.7 to 7.1.0 by @dependabot in #1447
- build(deps): bump jql-runner from 7.1.0 to 7.1.1 by @dependabot in #1457
- build(deps): bump jql-runner from 7.1.1 to 7.1.2 by @dependabot in #1486
- build(deps): bump hashbrown from 0.14.2 to 0.14.3 by @dependabot in #1441
- build(deps): bump redis from 0.23.3 to 0.23.4 by @dependabot in #1442
- build(deps): bump redis from 0.23.3 to 0.24.0 by @dependabot in #1455
- build(deps): bump atoi_simd from 0.15.3 to 0.15.4 by @dependabot in #1444
- build(deps): bump atoi_simd from 0.15.4 to 0.15.5 by @dependabot in #1445
- build(deps): bump atoi_simd from 0.15.5 to 0.15.6 by @dependabot in #1512
- build(deps): bump actions/setup-python from 4.7.1 to 4.8.0 by @dependabot in #1454
- build(deps): bump actions/setup-python from 4.8.0 to 5.0.0 by @dependabot in #1459
- build(deps): bump actions/stale from 8 to 9 by @dependabot in #1463
- build(deps): bump itoa from 1.0.9 to 1.0.10 by @dependabot in #1464
- build(deps): bump tokio from 1.34.0 to 1.35.0 by @dependabot in #1465
- build(deps): bump tokio from 1.35.0 to 1.35.1 by @dependabot in #1483
- build(deps): bump ryu from 1.0.15 to 1.0.16 by @dependabot in #1466
- build(deps): bump file-format from 0.22.0 to 0.23.0 by @dependabot in #1468
- build(deps): bump github/codeql-action from 2 to 3 by @dependabot in #1476
- build(deps): bump geosuggest-utils from 0.5.1 to 0.5.2 by @dependabot in #1479
- build(deps): bump geosuggest-core from 0.5.1 to 0.5.2 by @dependabot in #1478
- build(deps): bump reqwest from 0.11.22 to 0.11.23 by @dependabot in #1480
- build(deps): bump calamine from 0.23.0 to 0.23.1 by @dependabot in #1481
- build(deps): bump qsv-sniffer from 0.10.0 to 0.10.1 by @dependabot in #1484
- build(deps): bump anyhow from 1.0.75 to 1.0.76 by @dependabot in #1485
- build(deps): bump futures from 0.3.29 to 0.3.30 by @dependabot in #1492
- build(deps): bump futures-util from 0.3.29 to 0.3.30 by @dependabot in #1491
- build(deps): bump crossbeam-channel from 0.5.9 to 0.5.10 by @dependabot in #1490
- build(deps): bump sysinfo from 0.29.10 to 0.29.11 by @dependabot in #1443
- Bump sysinfo from 0.29.11 to 0.30 #1489
- build(deps): bump sysinfo from 0.30.0 to 0.30.1 by @dependabot in #1495
- build(deps): bump sysinfo from 0.30.1 to 0.30.2 by @dependabot in #1504
- build(deps): bump sysinfo from 0.30.2 to 0.30.3 by @dependabot in #1509
- build(deps): bump tabwriter from 1.3.0 to 1.4.0 by @dependabot in #1500
- build(deps): bump tempfile from 3.8.1 to 3.9.0 by @dependabot in #1502
- build(deps): bump qsv_docopt from 1.4.0 to 1.5.0 by @dependabot in #1503
- build(deps): bump ahash from 0.8.6 to 0.8.7 by @dependabot in #1510
- build(deps): bump serde_json from 1.0.108 to 1.0.109 by @dependabot in #1511
- apply select clippy suggestions
- update several indirect dependencies
- pin Rust nightly to 2023-12-23
Fixed
apply: Fix fordynfmtandcalcconvsubcommands not working in release mode #1467luau: fix check for excess mapped columns earlier. Otherwise, we'll get a CSV different field count error db15811
Removed
luau: remove unneeded--jitoption as we precompile luau scripts to bytecode #1438
Full Changelog: 0.119.0...0.120.0