wader/fq v0.10.0 on GitHub

Adds support for various LevelDB formats (thanks @mikez) and Garmin Flexible and Interoperable Data Transfer format (FIT) (thanks @mlofjard).

And as usual some small fixes and dependency updates.

Changes

On macOS fq now reads init script from ~/.config/fq in addition to ~/Library/Application Support/fq. #871
Switch readline module from own fork to https://github.com/ergochat/readline #854
Updated gojq fork. Notable changes from upstream below. #844
- Fix pre-defined variables to be available in initial modules
- Fix object construction with duplicate keys

Format changes

aac_frame Decode instance tag and common window flag. #859

fit Add support for Garmin Flexible and Interoperable Data Transfer decoder. Thanks @mlofjard #874

This format is used by various GPS tracking devices to record location, speed etc.
Example of converting position information to KML:

# to_kml.jq
# convert locations in a fit structure to KML
def to_kml:
  ( [ .data_records[].data_message as {position_lat: $lat, position_long: $long}
    | select($lat and $long)
    | [$long, $lat, 0]
    | join(",")
    ]
  | join(". ")
  | { kml: {"@xmlns":"http://earth.google.com/kml/2.0"
    , Document: {Placemark: {LineString: {coordinates: .}}}}
    }
  | to_xml({indent:2})
  );

Usage:

# -L to add current directory to library path
# -r for raw string output
# 'include "to_ml";' to include to_kml.jq
# to_kml calls to_kml function
$ fq -L . -r 'include "to_kml"; to_kml' file.fit > file.kml

hevc_sps Fix some incorrect profile_tier_level decoding. #829
html Fix issue parsing elements including SOLIDUS "/". #870
- Upstream issue golang/go#63402
mpeg_es Support ES_ID_Inc and decode descriptors for IOD tags
leveldb_descriptor, leveldb_log, leveldb_table Add support for LevelDB. Thanks @mikez #824
- This format is used by many database backends and applications like Google chrome.
pcapng Decode all section headers instead of just the first. #860
png Fix incorrect decoding of type flags. #847
hevc_sps Fix incorrect decoding of profile_tier_level. #829
tls Fix field name typos. #839
mp4
- Don't try decode samples for a track that has an external reference. #834
- Use box structure instead of track id to keep track for sample table data. #833
- ctts box v0 sample offset seems to be signed in practice but not in spec. #832
webp Decode width, height and flags for lossless WebP. #857

Changelog

6c3914a Add DFDL acronym to README.md
43c7693 Update docker-golang to 1.21.5 from 1.21.4
836af78 Update docker-golang to 1.21.6 from 1.21.5
f9e1e75 Update docker-golang to 1.22.0 from 1.21.6
54da5ef Update ergochat-readline to 81f0f2329ad3 from cca60bf24c9f
bed912c Update github-go-version to 1.21.5 from 1.21.4
3f48e3b Update github-go-version to 1.21.6 from 1.21.5
5bfb2bb Update github-go-version to 1.22.0 from 1.21.6
888b0be Update github-golangci-lint to 1.56.0 from 1.55.2
283380b Update github-golangci-lint to 1.56.1 from 1.56.0
be79c19 Update gomod-golang-x-crypto to 0.16.0 from 0.15.0
6aba928 Update gomod-golang-x-crypto to 0.17.0 from 0.16.0
561fed9 Update gomod-golang-x-crypto to 0.18.0 from 0.17.0
b5388ea Update gomod-golang-x-crypto to 0.19.0 from 0.18.0
0200a4e Update gomod-golang-x-net to 0.19.0 from 0.18.0
2e22e1b Update gomod-golang-x-net to 0.20.0 from 0.19.0
40bcb19 Update gomod-golang-x-net to 0.21.0 from 0.20.0
2a1d970 Update gomod-golang-x-term to 0.16.0 from 0.15.0
310c78e Update gomod-golang-x-term to 0.17.0 from 0.16.0
536583c Update make-golangci-lint to 1.56.0 from 1.55.2
5369576 Update make-golangci-lint to 1.56.1 from 1.56.0
e51c746 aac_frame: CPE: Decode instance tag and common window flag
f5f8e93 bson: Fix jq style a bit
d0a1b30 bump readline to 0.1.0-rc1
b56258c bump,readline: Add bump config and update to latest master
0070e38 bump,readline: Skip bump until not rc
63e0aa3 doc: Fix weird wording in README.md
e5fd1eb doc: Generate svgs with new ansisvg
55470de doc: Update docs to include fit format
c47756d doc: Update svgs after ansisvg update
54c6f0c fit: Added support for "invalid" value checking.
46dbf5b fit: Added support for ANT+ FIT format (used by Garmin devices)
6219d57 fit: Added support for dynamic subfields
33e5851 fit: Fix field casing to snake_case. Misc cleanup.
76307e4 fit: Formatted date/time description for timestamp fields
8862280 fit: Made long/lat present as float64
788088f fit: Show crc as hex, lower case "invalid" and some style harmonization
a07ce11 fq: Relicense float80 code to MIT
5829c6b gojq: Update fq fork
d155c80 gojq: Update fq fork
a793d10 goreleaser: Fix deprecated options
919e079 hevc_sps: Fix some incorrect profile_tier_level decoding
69ae765 interp: Fix crash when using line_bytes=0
2176600 interp: Support ~/.config/fq as fallback on macOS
fb910bd ldb: first draft
efc59a8 ldb: uncompression support
41f27a1 leveldb: add torepr for descriptor
2df0f0f leveldb: add log and descriptor decoders
b05aa99 leveldb: address PR comments
2f5f183 leveldb: decode unfragmented .log files further; fix UTF8 decoding
e826f09 leveldb: fix Errorf arguments
4283091 leveldb: fix all.fqtest failures
287ed36 leveldb: fix metaindex keys, refactoring, and jq syntax per PR
8665df5 leveldb: fix table's data blocks' internal keys decoding
08e3d2d leveldb: improve stringify by preallocating result
3a396e1 leveldb: improve log documentation
1ba8dec leveldb: in some properties, change spaces to underscores
e735cea leveldb: propagate error
07ad940 leveldb: rename "suffix" to "sequence_number_suffix"
78a3e94 leveldb: rename functions and add comments
cc0d5a8 leveldb: update docs
fe1099b leveldb: updates per PR comments
0d06e0a mp4: Don't decode samples if track has external data reference
d2c5ce5 mp4: Use box structure instead of track id to keep track for sample table data
aadf26f mp4: ctts v0 sample_offset seems to be signed in practice
fca55b2 mpeg_es: Support ES_ID_Inc and decode descriptors for IOD tags
e3af467 pcapng: Decode all section headers, clenaup descriptions
38b4412 png: Type flags were off-by-one bit
fb1c625 readline,bump: Fix version compare link
41226f4 readline: Switch to ergochat/readline
cd572d3 readline: Update to 0.1.0 and add bump config
7906f33 test: Support to more common -update flag
3b7cc1f tls: Fix field name typos
b0421df webp: Decode width, height and flags for lossless webp