saulpw/visidata v2.-4.0 on GitHub

New Features (v2.-4)

1. reintegrate deferred mods; save with `z Ctrl+S`

First and foremost, the 'defermods' feature has been reintegrated into core VisiData.
It was added in v2.-1, but it caused some headaches, so we separated it out into a plugin for v2.-2, along with vls for the DirSheet.
Then we found that we really missed having the functionality right in vd alongside everything else, and not having to reach for a specialized tool.
@anjakefala got to the bottom of our headaches and did the heavy lifting to make sure that everything works properly now.
So now the DirSheet and SqliteSheets are once again able to 'defer' their modifications and deletions, until saved to the source with z Ctrl+S. Ctrl+S saves the sheet to a regular data file as usual.

2. unfurl a column (`unfurl-col` bound to `zM`)

As suggested by @jsvine, unfurl is a row-wise expansion of a column with containers.

expand-col does a column-wise expansion: 1 column is added for each key/value in the containers (assumes mostly homogenous data)
melt converts non-key columns into rows (on a new melted sheet)
unfurl-col is kind of like a shortcut for an expand followed by a melt (demo on twitter)

In my mind, unfurl is conceptually closer to expand-col, but since it pushes a new sheet, putting it on a Shift+ command feels more natural.
The unfurl-col command feels like a focused melt--melting lists or dicts in the cursor column, instead of the individual values of the non-key columns as melt does.
So it's now bound to z Shift+M.

3. 'i'ncrement and family

Generating a range of numbers turns out to be surprisingly handy.

Previously, if you wanted to make a column with a list of numbers, you'd have to use za to create a column, gs to select all the rows (losing your previous selection) gz= (setcol-range
array languages, the 'increment' operator generates a sequence of integers from whole cloth.

In fact there was a version of this in the rownum plugin, and another version in @jsvine's unfurl plugin.
So we added equivalent commands to stock VisiData.
Because of the tradition of using i-like characters in APL notation ('ι' in APL and 'i.' in J), and the coincidence
that i has remained one of the few unbound keystrokes, I decided to develop this set of increment concepts, and bind them to the i family of commands.
See below for the new commands.

4. cli options improved; set sheet-specific options from CLI

We had a couple of open issues with command line options, and finally I bit the bullet and just wrote a custom cmdline args parser, instead of using Python's argparse. It was actually about the same amount of code, but now the syntax is more regular, more permissive, and can set sheet-specific options multiple times and differently for different sheets.
Thanks to @tsibley for patient testing and precise bug reports.

See /options#cli for details on how the command line interface works.

Also the visidata_dir option (default $HOME/.visidata) can now be set via the environment variable $VD_DIR.
Likewise config can be set via $VD_CONFIG.

5. `fancy_chooser`

Setting the fancy_chooser option to True enables a fancy selection menu for adding aggregators and choosing jointypes.
Help text is included for each item; press Enter to choose a single item, or select multiple rows and press g Enter, or q to abort.
This uses the splitpane functionality from the previous release.

6. column format strings

Python has multiple syntaxes for formatting numbers within strings: %.02f which used with the % format operator, and {:.02f} used with .format().
Which should you use for in the fmtstr attribute on the Columns sheet? A Twitter poll was evenly split, and @tsibley suggested ¿porque no los dos?](https://twitter.com/trs/status/1182390622507827202)
So as of v2.-4, you can use either format mechanism! If you want thousands separators though, you have to use {:,.02f}. See /blog/fmtstr for details.

new and improved loaders

IMAP (email): open an imap:// url and it'll download all your email, which you can then save to .jsonl like I did
basic MySQL loader (requires ..; thanks @ )
basic PDF text extractor (requires pdfminer)
improved loaders: csv hdf5 html http json jsonl pandas pcap postgres xls yaml

Thanks to @ajkerrigan for actively following up on the pandas and json bug reports, and all of his work in making them better.

new commands

z Shift+M unfurl-col
i addcol-range adds a column with integer values reflecting the current row order.
gi setcol-range sets values in the current column for selected rows
zi addcol-range-step adds a column with values
gzi setcol-range-step sets values
x (Canvas) resize-x-input sets x axis dimensions on a plot
y (Canvas) resize-y-input sets y axis dimensions on a plot
(unbound) select-error
(unbound) select-error-col
(unbound) mouse-enable and mouse-disable
(unbound) guard-sheet sets quitguard on current sheet

lots more improvements and bugfixes

And of course there are tons of bugfixes and smaller improvements throughout. For a more complete list, see the CHANGELOG.

saulpw/visidata v2.-4.0 v2.-4 pre-release on GitHub