jqlang/jq jq-1.7rc2 on GitHub

Note that this is a pre-release. Feedback is welcome #2862.

After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release.

Since the last stable release many things have happened:

jq now lives at https://github.com/jqlang
New maintainers, admins, and owners have been recruited.
- A list of current maintainers
NEWS file is replaced by NEWS.md with Markdown format. @wader #2599
CI, scan builds, release, website builds etc now use GitHub actions. @owenthereal @wader @itchyny #2596 #2603 #2620 #2723
Lots of documentation improvements and fixes.
Website updated with new section search box, better section ids for linking, dark mode, etc. @itchyny #2628
Release builds for:
- Linux amd64, arm64, armel, armhf, i386, mips, mips64, mips64el, mips64r6, mips64r6el, mipsel, mipsr6, mipsr6el, powerpc, ppc64el, riscv64 and s390x
- macOS amd64 and arm64
- Windows i386 and amd64
- Docker linux/386, linux/amd64, linux/arm64, linux/mips64le, linux/ppc64le, linux/riscv64 and linux/s390x
- More details see @owenthereal #2665
Docker images are now available from ghcr.io/jqlang/jq instead of Docker Hub. @itchyny #2652 #2686
OSS-fuzz. @DavidKorczynski #2760 #2762

Full commit log can be found at jq-1.6...jq-1.7rc2 but here are some highlights:

CLI changes

Make object key color configurable using JQ_COLORS environment variable. @itchyny @haguenau @ericpruitt #2703

# this would make "field" bold yellow (`1;33`, the last value)
$ JQ_COLORS="0;90:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}'
{
  "field": 123
}

Change the default color of null to Bright Black. @itchyny #2824
Respect NO_COLOR environment variable to disable color output. See https://no-color.org for details. @itchyny #2728
Improved --help output. Now mentions all options and nicer order. @itchyny @wader #2747 #2766 #2799

Fix multiple issues of exit code using --exit-code/-e option. @ryo1kato #1697

# true-ish last output value exits with zero
$ jq -ne true ; echo $?
true
0
# false-ish last output value (false and null) exits with 1
$ jq -ne false ; echo $?
false
1
# no output value exists with 4
$ jq -ne empty ; echo $?
4

Add --binary/-b on Windows for binary output. To get \n instead of \r\n line endings. @nicowilliams 0dab2b1

Add --raw-output0 for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684

# will output a zero byte after each output
$ jq -n --raw-output0 '1,2,3' | xxd
00000000: 3100 3200 3300                           1.2.3.
# can be used with xargs -0
$ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1
a
b
c
$ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '[%s]\n'
[a b c]
[d
e
f]
# can be used with read -d ''
$ while IFS= read -r -d '' json; do
>   jq '.name' <<< "$json"
> done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}')
"a b c"
"d\ne\nf"
# also it's an error to output a string containing a NUL when using NUL separator
$ jq -n --raw-output0 '"\u0000"'
jq: error (at <unknown>): Cannot dump a string containing NUL with --raw-output0 option

Fix assert crash and validate JSON for --jsonarg. @wader #2658
Remove deprecated --argfile option. @itchyny #2768
Enable stack protection. @nicowilliams #2801

Language changes

Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752

# precision is preserved
$ echo '100000000000000000' | jq .
100000000000000000
# comparison respects precision (this is false in JavaScript)
$ jq -n '100000000000000000 < 100000000000000001'
true
# sort/0 works
$ jq -n -c '[100000000000000001, 100000000000000003, 100000000000000004, 100000000000000002] | sort'
[100000000000000001,100000000000000002,100000000000000003,100000000000000004]
# arithmetic operations might truncate (same as JavaScript)
$ jq -n '100000000000000000 + 10'
100000000000000020

Adds new builtin pick(stream) to emit a projection of the input object or array. @pkoppstein #2656 #2779

$ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)'
{
  "a": 1,
  "b": {
    "c": 2
  },
  "x": null
}

Adds new builtin debug(msgs) that works like debug but applies a filter on the input before writing to stderr. @pkoppstein #2710

$ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)'
["DEBUG:","Entering function foo with $x == 1"]
["DEBUG:",2]
3
$ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})'
["DEBUG:",{"a":1,"b":2,"sum":3}]
{
  "a": 1,
  "b": 2,
  "c": 3
}

Adds new builtin scan($re; $flags). Was documented but not implemented. @itchyny #1961

# look for pattern "ab" in "abAB" ignoring casing
$ jq -n '"abAB" | scan("ab"; "i")'
"ab"
"AB"

Adds new builtin abs to get absolute value. This potentially allows the literal value of numbers to be preserved as length and fabs convert to float. @pkoppstein #2767

Allow if without else-branch. When skipped the else-branch will be . (identity). @chancez @wader #1825 #2481

# convert 1 to "one" otherwise keep as is
$ jq -n '1,2 | if . == 1 then "one" end'
"one"
2
# behaves the same as
$ jq -n '1,2 | if . == 1 then "one" else . end'
"one"
2
# also works with elif
$ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end
"one"
"two"
3

Allow use of $binding as key in object literals. @nicowilliams 8ea4a55

$ jq -n '"a" as $key | {$key: 123}'
{
  "a": 123
}
# previously parentheses were needed
$ jq -n '"a" as $key | {($key): 123}'
{
  "a": 123
}

Allow dot between chained indexes when using .["index"] @nicowilliams #1168

$ jq -n '{"a": {"b": 123}} | .a["b"]'
123
# now this also works
$ jq -n '{"a": {"b": 123}} | .a.["b"]'
123

Allow dot for chained value iterator .[], .[]? @wader #2650

$ jq -n '{"a": [123]} | .a[]'
123
# now this also works
$ jq -n '{"a": [123]} | .a.[]'
123

Fix try/catch catches more than it should. @nicowilliams #2750
Speed up and refactor some builtins, also remove scalars_or_empty/0. @muhmuhten #1845
Now halt and halt_error exit immediately instead of continuing to the next input. @emanuele6 #2667
Fix issue converting string to number after previous convert error. @thalman #2400
Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661

Fix deletion using assigning empty against arrays. @itchyny #2133

# now this works as expected, filter out all values over 2 by assigning empty
$ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]'
[1,0]

Allow keywords to be used as binding name in more places. @emanuele6 #2681
Allow using nan as NaN in JSON. @emanuele6 #2712
Expose a module's function names in modulemeta. @mrwilson #2837
Fix contains/1 to handle strings with NUL. @nicowilliams 61cd6db
Fix stderr/0 to output raw text without any decoration. @itchyny #2751
Fix nth/2 to emit empty on index out of range. @itchyny #2674
Fix implode to not assert and instead replace invalid unicode codepoints. @wader #2646
Fix indices/1 and rindex/1 in case of overlapping matches in strings. @emanuele6 #2718
Fix sub/3 to resolve issues involving global search-and-replace (gsub) operations. @pkoppstein #2641
Fix significand/0, gamma/0 and drem/2 to be available on macOS. @itchyny #2756 #2775
Fix empty regular expression matches. @itchyny #2677
Fix overflow exception of the modulo operator. @itchyny #2629
Fix string multiplication by 0 (and less than 1) to emit empty string. @itchyny #2142
Fix segfault when using libjq and threads. @thalman #2546
Fix constant folding of division and reminder with zero divisor. @itchyny #2797
Fix error/0, error/1 to throw null error. @emanuele6 #2823
Simpler and faster transpose. @pkoppstein #2758
Simple and efficient implementation of walk/1. @pkoppstein #2795
Remove deprecated filters leaf_paths, recurse_down. @itchyny #2666

jqlang/jq jq-1.7rc2 jq 1.7rc2 on GitHub

CLI changes

Language changes

jqlang/jq jq-1.7rc2
jq 1.7rc2

on GitHub