johnkerl/miller v6.9.0

Nanosecond timestamps, ZSTD compression, improved data-error handling, and more

on GitHub

New features

Support for nanosecond-resolution timestamps:

• Add DSL functions for integer nanoseconds since the epoch by @johnkerl in #1326
• Add %N and %O for strfntime by @johnkerl in #1334
• Add %s format specifier for strftime by @johnkerl in #1335
• Requested on issue #1152 by @derekmahar
• See also https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#time-functions

Statistical functions from the stats verb may now be computed over arbitrary arrays and maps:

• New DSL functions for summary stats over arrays / maps by @johnkerl in #1364
• Requested on issue #1345 by @janxkoci
• See also https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#stats-functions

Additional control over filenames for the split verb:

• Filename options for split by @sloanlance in #1366
• Requested on issue #1365 by @sloanlance

Support for details of data-computation errors beyond the current (error):

• Fatal-on-data-error mlr -x option by @johnkerl in #1373
• See also https://miller.readthedocs.io/en/6.9.0/reference-dsl-errors/#handling-for-data-errors
• Requested on issue #1106 by @honzajde

New verbs and DSL functions:

• New sub, gsub, and ssub verbs by @johnkerl in #1361, requested by @janxkoci on #1356. See also:
  • https://miller.readthedocs.io/en/6.9.0/reference-verbs/#sub
  • https://miller.readthedocs.io/en/6.9.0/reference-verbs/#gsub
  • https://miller.readthedocs.io/en/6.9.0/reference-verbs/#ssub
• New contains DSL function by @johnkerl in #1374
  • https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#contains

Other updates:

• Support ZSTD compression in-process by @johnkerl in #1360, from issue #1342 by @aborruso
  • See also https://miller.readthedocs.io/en/6.9.0/reference-main-compressed-data/
• Support comments in mlr -s files by @johnkerl in #1359, from issue #1343 by @janxkoci
• Add empty-key check to mlr check by @johnkerl in #1330, from issue #1050 by @aborruso

Bug fixes

• Do wildcard globbing on Windows by @johnkerl in #1362, from issue #1341 by @coolbq
• Treat empty like absent in + - * by @johnkerl in #1371, from issue #1001 by @Poshi
• Can't use ${field_name} if it contains UTF-8 characters also encodeable as Latin-1 by @johnkerl in #1363, from issue #1358 by @clemente
• Typofix in uif/uof percentiles by @johnkerl in #1375

Documentation updates

• Update readthedocs notes in the how-to-release page by @johnkerl in #1308
• Fix mlr grep docs re OFS/OPS by @johnkerl in #1309
• Update Fedora link by @bkmgit in #1339
• Small typos in documentation of mlr nest by @johnkerl in #1352, from issue #1350 by @kusalananda

Internal

• Update 2015-era Python sketch to Python 3 by @johnkerl in #1372
• Remove redundant nil check by @Juneezee in #1367
• Bump actions/checkout from 3.5.2 to 3.5.3 by @dependabot in #1319
• Bump github/codeql-action from 2.3.6 to 2.13.4 by @dependabot in #1318
• Bump golang.org/x/term from 0.8.0 to 0.9.0 by @dependabot in #1321
• Bump goreleaser/goreleaser-action from 4.2.0 to 4.3.0 by @dependabot in #1320
• Bump golang.org/x/text from 0.9.0 to 0.10.0 by @dependabot in #1322
• Bump golang.org/x/text from 0.10.0 to 0.11.0 by @dependabot in #1337
• Bump golang.org/x/sys from 0.9.0 to 0.10.0 by @dependabot in #1336
• Bump golang.org/x/term from 0.9.0 to 0.10.0 by @dependabot in #1338
• Bump golang.org/x/sys from 0.10.0 to 0.11.0 by @dependabot in #1347
• Bump golang.org/x/text from 0.11.0 to 0.12.0 by @dependabot in #1349
• Bump actions/setup-go from 4.0.1 to 4.1.0 by @dependabot in #1351
• Bump goreleaser/goreleaser-action from 4.3.0 to 4.4.0 by @dependabot in #1354
• Bump golang.org/x/term from 0.10.0 to 0.11.0 by @dependabot in #1348
• Bump actions/checkout from 3.5.3 to 3.6.0 by @dependabot in #1369

New Contributors

• @bkmgit made their first contribution in #1339
• @Juneezee made their first contribution in #1367
• @sloanlance made their first contribution in #1366

Full Changelog: v6.8.0...v6.9.0