New major version with lots of additions:
-
Automatic Immutable Caching - Automatic caching of data files (into
<my-csv-filename>.qsqlfiles), with huge speedups for medium/large files. Enabled through-C readwriteor-C read -
Direct querying of standard sqlite databases - Just use it as a table name in the query. Format is
select ... from <sqlitedb_filename>:::<table_name>, or just<sqlitedb_filename>if the database contains only one table. Multiple separate sqlite databases are fully supported in the same query. -
Direct querying of the
qsqlcache files - The user can query directly from theqsqlfiles, removing the need for the original files. Just useselect ... from <my-csv-filename>.qsql. Please wait for the next version before deciding that you don't need your original files anymore... This version contains major changes and might include subtle bugs in it. -
Revamped
.qrcmechanism - allows opting-in to caching without specifying it in every query. By default, caching is disabled, for backward compatibility and for finding usability issues. -
Save-to-db is now reusable for queries -
--save-db-to-diskoption (-S) has been enhanced to match the new capabilities. You can query the resulting file directly through q, using the method mentioned above (it's just a standard sqlite database).
Breaking Changes:
-
Support for file concatenation (
filenameA+filenameB+...) has been dropped. Glob support (myfilename*.csv) is still supported. -
Some cases of "one column only" warnings have been dropped as well
-
Only python3 is supported from now on - Shouldn't be an issue, since q is a self-contained binary executable which has its own python embedded in it. Internally, q is now packaged with Python 3.8. After everything cools down, I'll probably bump this to 3.9/3.10.
-
Minimal Linux Version Bumped - Works with CentOS 8, Ubuntu 18.04+, Debian 10+. Currently only for x86_64. Depends on glibc version 2.25+. Haven't tested it on other architectures. Issuing other architectures will be possible later on
For details on the changes and the new usage, see here
The version is now built using the awesome pyoxidizer project.