wader/fq v0.13.0 on GitHub

Changes

New format decoders midi, negentropy, tap and txz, see below for details.

Add byte_array bits format. #992

# by default binary values are UTF-8 strings
$ fq -V -n '[1,2,3] | tobytes'
# with -o bits_format=byte_array they are byte arrays instead
"\u0001\u0002\u0003"
$ fq -cV -n -o bits_format=byte_array '[1,2,3] | tobytes'
[1,2,3]

# data is a string
$ fq '.program_headers[] | select(.type=="interp")' /bin/ls
{"align":1,"data":"/lib/ld-musl-x86_64.so.1\u0000", ...}
# data is a byte array
$ fq -cV -o bits_format=byte_array '.program_headers[] | select(.type=="interp")' /bin/ls
{"align":1,"data":[47,108,105,98,47,108,100,45,109,117,115,108,45,120,56,54,95,54,52,46,115,111,46,49,0], ...}

Go 1.22 or later is now required to build. This is to keep up with build requirements of some dependencies. #1011
gojq changes from upstream: #1000
- Implement add/1 function
- Improve time functions to accept fewer element arrays
- Fix uri and urid formats not to convert a space between a plus sign

Format changes

matroska Updated to latest specification. #1010
midi MIDI decoder added by @twystd. #1004
negentropy Negentropy message decoder added by @fiatjaf. #1012
tap and txz TAP and TXZ tape format for ZX Spectrum computers added by @mrcook. #975

Changelog

64b0523 Add PBS tidbit 8 episode
3105b7d Added to probe group
d73c663 Update docker-golang to 1.22.6 from 1.22.5
790aa8a Update docker-golang to 1.23.0 from 1.22.6
ea3294f Update docker-golang to 1.23.1 from 1.23.0
34b2cc3 Update github-go-version to 1.22.6 from 1.22.5
8bb841f Update github-go-version to 1.23.0 from 1.22.6
2c38cf1 Update github-golangci-lint to 1.60.1 from 1.59.1
f9dff5c Update github-golangci-lint to 1.60.2 from 1.60.1
ccf78c7 Update github-golangci-lint to 1.60.3 from 1.60.2
23df925 Update gomod-creasty-defaults to 1.8.0 from 1.7.0
9e29682 Update gomod-ergochat-readline to 0.1.3 from 0.1.2
01b292d Update gomod-golang-x-crypto to 0.26.0 from 0.25.0
da6dd9e Update gomod-golang-x-net to 0.28.0 from 0.27.0
de3fa19 Update make-golangci-lint to 1.60.1 from 1.59.1
f6d0279 Update make-golangci-lint to 1.60.2 from 1.60.1
e01cff0 Update make-golangci-lint to 1.60.3 from 1.60.2
4299883 Update make-golangci-lint to 1.61.0 from 1.60.3
ee404b2 all: re-run the test --update
a241774 all: update tests for tap support
2c0b528 ci: Add CGO_ENABLED=0 to make sure we dont need it
550bcc2 cli: Add go version to version string
a195b3f doc: Regenerate
be9a916 doc: Update formats.svg
db0dfb1 doc: Update formats.svg
66345fe doc: regenerat formats.md for tzx/tap updates
f16f6dc fq: Update golang.org/x/exp and github.com/gomarkdown/markdown
7b6d853 go,lint: Update to go 1.22 and fix some lint warnings
3626787 gojq: Update fq fork
824e51e gojq: Update fq fork
fc74f68 intepr: Revert binary value each
3f3183a intepr: Support each for binary values
7bf1129 interp: Add bits_format=byte_array support
52e6e1b matroska: Update spec
fdde568 midi: (partly) fixed SMPTEOffsetEvent gap
a1385ca midi: Decoded EndOfTrack and ProgramChange events
3276641 midi: Decoded KeySignature, NoteOn and Controller events
109719f midi: Decoded MIDIChannelPrefix, MIDIPort and SequencerSpecific metaevents
cbd4a8a midi: Decoded NoteOff event
0226fc6 midi: Decoded PolyphonicPressure, ChannelPressure and PitchBend MIDI events
b2e71a3 midi: accomodated MIDI event running status
0915f75 midi: added 'data' field to EndOfTrack event
473394b midi: added event type to events to simplify query by event
91fa547 midi: added example queries to the test data
ce02d6e midi: added length field to TimeSignature struct
b5f2bda midi: added localised Makefile (cf. #1004 (comment))
7950dd6 midi: added midi to the TestFormats all.fqtest list (cf. #1004 (comment))
80b9343 midi: added new test and MIDI files
57adef4 midi: added sample queries to midi.md
0e0a669 midi: added tests for format 0, format 1 and format 2 MIDI files
e99d9f6 midi: asserted bytes left for MIDI events
d7ec38a midi: basic test set
a41e123 midi: cleaned up SysExContinuation and SysExEscape logic
a7d0cde midi: cleaned up and simplied event decoding logic
8995496 midi: cleaned up event decoding logic
0b4be89 midi: cleaned up unknown chunk logic
3809ddb midi: combined metaevent status + event bytes
b433998 midi: decoded SysEx events
8b236a1 midi: decoded TimeSignature meta event
f11dfaf midi: decoded TrackName meta event
9099a3e midi: decoded chunk tags as FieldUTF8
4fe27f5 midi: decoded remaining text metaevents
1bdeeb6 midi: decoded tempo metaevent
59b1faa midi: decorated 'delta' field with running tick
4fac4c6 midi: discarded unknown chunks
9dfcb96 midi: experimenting with struct metaevent data
2a65036 midi: fixed fuzzing errors
f1b888b midi: fixed gap in SequencerSpecificEvent
4bb3292 midi: fixed key signature map
f424936 midi: fixed lint warning (cf. #1004 (comment))
2f2070e midi: fixed lint warnings
87c80f5 midi: fixes for PR comments
9f057b6 midi: fixes for PR comments
9c7f7f9 midi: fixes for PR comments:
ad0a06d midi: initial import
2970fb1 midi: lowercased event names
3ed9889 midi: mapped SMPTE framerates
54a0cf1 midi: mapped SysEx event types
34fca40 midi: mapped manufacturer IDs to strings
0ef3304 midi: mapped note numbers to note names
763d142 midi: moved 'dev only' Makefile to workdir and removed it from .gitignore (cf. #1004 (comment))
5c89d7d midi: moved MIDI running status and Casio flag to context struct
c5637f0 midi: partly fixed gaps in SysExMessage
7fd9ad2 midi: removed debug leftovers (cf. #1004 (comment))
462ae15 midi: removed superfluous AssertLeastBytesLeft (cf. #1004 (comment))
17bac77 midi: removed superfluous uint64 cast (cf. #1004 (comment))
dad4a91 midi: replace d.BytesLen(1) with d.U8()
578b7e7 midi: restructured event decoding to decode length and struct fields (cf. #1004 (comment))
befdf1f midi: reworked SysEx events as struct with length field
ea3e089 midi: reworked decoding to expect an MThd header as the first chunk (cf. #1004 (comment))
e940f47 midi: reworked metaevent decoding for PR comments
a337ff0 midi: reworked sysex and metaevent 'struct' events to decode as byte arrays
cee51ec midi: reworked to include delta in event
a3a0a06 midi: simplifed and cleaned up MIDI 'fq' tests:
c8d9397 midi: snake-cased event types and event names (cf. #1004 (comment))
3966d5b midi: tightened up status/event decoding logic (cf. #1004 (comment))
2350afe midi: updated help text to use snake-case event names (cf. #1004 (comment))
49bd7c2 negentropy: add format.
e3c7b1a negentropy: another test from a different source, another doc entry, comments on source of test samples.
3853387 negentropy: const modes, RFC3339 dates and synthetic timestamps.
8f9a005 negentropy: timestamp description as UTC.
113ca63 tap: add DefaultInArg
e526caf tap: add README to tests directory
90de619 tap: add a read_one_block test
e3c3d92 tap: add author to tap.md
633160b tap: handle unoffical block types and minor improvements
e5be55c tap: remove Probe group
7816fe1 tap: remove format.TAP_In and update tzx
e345528 tap: rename block_type to type
d384930 tap: revert using an array of bytes
c93dc8b tap: verify checksum values
81b2304 tzx: Add suport for ZX Spectrum TZX and TAP files
6ac69e2 tzx: add README to tests directory
88b6ded tzx: add author to tzx.md
b27017c tzx: add readme instructions for bits_format=byte_array
fc350f3 tzx: change data fields to array of bytes for easier JSON parsing
d3a5b8d tzx: revert using an array of bytes
ffb5eb3 tzx: use jq-ish values for the mapped symbols in block type 0x18