dathere/qsv 16.1.0

on GitHub

[16.1.0] - 2026-02-15 📊 "The Accelerated Civic Intelligence (ACI) Release" 📊

Statistical analysis gets faster and more robust; User & Agent Experience (UAX) improvements keep the CLI parser, docs, shell completions, and MCP tool definitions in sync from a single source; and the qsv MCP Server gets leaner and smarter.

With a properly configured environment, a User can team up with several AI Agents for accelerated analysis of large, real-world, messy data — raw datasets, presentations, reports, spreadsheets, etc. — without uploading it all to the cloud or manually wrangling it into shape first. Analyzing in a few minutes, what would otherwise take a few days, if not a few weeks to compile.

🌟 Major Features

New pragmastat Command

A pragmatic statistical toolkit by @AndreyAkinshin — Compute robust, median-of-pairwise statistics with the Pragmastat library. Designed for messy, heavy-tailed, or outlier-prone data where mean/stddev can mislead. See pragmastat.dev for details on the underlying algorithms and design philosophy.

Frequency Cache System

New --frequency-jsonl option for the frequency command creates a JSONL cache (analogous to stats --stats-jsonl) that accelerates repeated frequency analysis. Uses a hybrid strategy for high-cardinality columns with configurable thresholds.

Improved UAX: Unified Documentation & Shell Completions

A new qsv-docopt parsing system now generates markdown documentation, shell completions, and MCP tool definitions from the same USAGE text that powers qsv's CLI parsing. Everything stays in sync automatically — no more drift between help text, docs, completions and AI tooling.

• --generate-help-md flag produces polished markdown docs with section navigation, emoji legends, clickable URLs, and argument/option tables that are both Human and Agent-friendly.
• Shell completions are now auto-generated, replacing 68 manually maintained completion files.

qsv MCP Server: Leaner Architecture

The qsv_pipeline tool has been removed in favor of direct sequential command execution. In practice, agents were already calling commands one at a time, and removing the pipeline abstraction made the server simpler, more predictable, and easier to debug. Additional MCP improvements include:

• Extended AI agent guidance to take advantage of frequency and stats caches
• Seamless support for Google Gemini CLI thanks to @kulnor's continuing contributions
• Major codebase refactoring: deduplicated helpers, extracted filesystem tools, fixed any types, and various bug fixes

Detailed MCP changes are documented in the MCP CHANGELOG for full details.

Added

• feat: pragmastat command — pragmatic statistical toolkit with parallelism, progress bar, and memcheck (by @AndreyAkinshin)
• feat: frequency --frequency-jsonl — JSONL frequency cache with hybrid strategy for high-cardinality columns
• feat: --generate-help-md flag — auto-generate markdown docs from USAGE text with section navigation, emoji legends, and clickable URLs
• docs: add QSV_FREQ_HIGH_CARD_THRESHOLD and QSV_FREQ_HIGH_CARD_THRESHOLD_PCT env vars

Changed

• perf: stats — skip redundant modes tracking, reduce allocations, optimize cache line layout, deterministic antimode sorting
• perf: pragmastat — reduce redundant computations, add parallelism
• perf: frequency — use sort_unstable_by for faster sorting; parallel computation for high-cardinality columns
• refactor: shell completions auto-generated from USAGE text (removed 68 manual files)
• refactor: describegpt — disambiguate "Other" bucket from literal "Other" in Data Dictionary Examples column
• deps: bump anstream from 0.6.21 to 1.0.0
• deps: bump futures to 0.3.32
• deps: bump jsonschema from 0.41 to 0.42
• deps: bump libc from 0.2.180 to 0.2.181
• deps: bump memmap2 from 0.9.9 to 0.9.10
• deps: bump polars to latest upstream
• deps: bump pyo3 from 0.28.0 to 0.28.1
• deps: bump quickcheck from 1.0.3 to 1.1.0
• deps: bump rand from 0.9 to 0.10, rand_hc to 0.5, rand_xoshiro to 0.8
• deps: bump sysinfo from 0.37.2 to 0.38.2
• deps: bump tempfile from 3.24.0 to 3.25.0
• deps: bump toml from 0.9.12 to 1.0.1
• deps: bump uuid from 1.20.0 to 1.21.0
• deps: bump zmij from 1.0.20 to 1.0.21
• deps: update csv patched fork MSRV to 1.93

Fixed

• fix: frequency — normalize delimiter for cache compatibility; deterministic output with secondary sort key; hybrid cache for high-cardinality columns
• fix: stats — remove unsafe block; deterministic antimode sorting
• fix(help): section detection, acronym casing, and option word-wrap in markdown generation

Removed

• removed 68 manual shell completion files (now auto-generated from USAGE text)

Full Changelog: 16.0.0...16.1.0