Releases: jqlang/jq
jq 1.7.1
Security
- CVE-2023-50246: Fix heap buffer overflow in jvp_literal_number_literal
- CVE-2023-50268: fix stack-buffer-overflow if comparing nan with payload
CLI changes
- Make the default background color more suitable for bright backgrounds. @mjarosie @taoky @nicowilliams @itchyny #2904
- Allow passing the inline jq script after
--
. @emanuele6 #2919 - Restrict systems operations on OpenBSD and remove unused
mkstemp
. @klemensn #2934 - Fix possible uninitialised value dereference if
jq_init()
fails. @emanuele6 @nicowilliams #2935
Language changes
- Simplify
paths/0
andpaths/1
. @asheiduk @emanuele6 #2946 - Reject
U+001F
in string literals. @torsten-schenk @itchyny @wader #2911 - Remove unused nref accumulator in
block_bind_library
. @emanuele6 #2914 - Remove a bunch of unused variables, and useless assignments. @emanuele6 #2914
- main.c: Remove unused EXIT_STATUS_EXACT option. @emanuele6 #2915
- Actually use the number correctly casted from double to int as index. @emanuele6 #2916
- src/builtin.c: remove unnecessary jv_copy-s in type_error/type_error2. @emanuele6 #2937
- Remove undefined behavior caught by LLVM 10 UBSAN. @Gaelan @emanuele6 #2926
- Convert decnum to binary64 (double) instead of decimal64. This makes jq behave like the JSON specification suggests and more similar to other languages. @wader @leonid-s-usov #2949
- Fix memory leaks on invalid input for
ltrimstr/1
andrtrimstr/1
. @emanuele6 #2977 - Fix memory leak on failed get for
setpath/2
. @emanuele6 #2970 - Fix nan from json parsing also for nans with payload that start with 'n'. @emanuele6 #2985
- Allow carriage return characters in comments. @emanuele6 #2942 #2984
Documentation changes
- Generate links in the man page. @emanuele6 #2931
- Standardize arch types to AMD64 & ARM64 from index page download dropdown. @owenthereal #2884
libjq
Build and test changes
- Fix incorrect syntax for checksum file. @kamontat @wader #2899
- Remove
-dirty
version suffix for windows release build. @itchyny #2888 - Make use of
od
in tests more compatible. @nabijaczleweli @emanuele6 @nicowilliams #2922 - Add dependabot. @yeikel #2889
- Extend fuzzing setup to fuzz parser and and JSON serializer. @DavidKorczynski @emanuele6 #2952
- Keep releasing executables with legacy names. @itchyny #2951
New Contributors
- @yeikel made their first contribution in #2889
- @dependabot made their first contribution in #2894
- @kamontat made their first contribution in #2899
- @taoky made their first contribution in #2904
- @tboz203 made their first contribution in #2920
- @nabijaczleweli made their first contribution in #2922
- @klemensn made their first contribution in #2934
- @asheiduk made their first contribution in #2946
- @rockwotj made their first contribution in #2953
- @jesperronn made their first contribution in #2898
Full Changelog: jq-1.7...jq-1.7.1
jq 1.7
After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release. We're very grateful for all the new owners, admins, and maintainers. Special thanks go to Owen Ou (@owenthereal) for pushing to set up a new GitHub organization for jq, Stephen Dolan (@stedolan) for transferring the jq repository to the new organization, @itchyny for doing a great deal of work to get the release done, Mattias Wadman (@wader) and Emanuele Torre (@emanuele6) for many PRs and code reviews. Many others also contributed PRs, issues, and code reviews as well, and you can find their contributions in the Git log and on the closed issues and PRs page.
What's Changed
Since the last stable release many things have happened:
- jq now lives at https://github.com/jqlang
- New maintainers, admins, and owners have been recruited.
- A list of current maintainers
- NEWS file is replaced by NEWS.md with Markdown format. @wader #2599
- CI, scan builds, release, website builds etc now use GitHub actions. @owenthereal @wader @itchyny #2596 #2603 #2620 #2723
- Lots of documentation improvements and fixes.
- Website updated with new section search box, better section ids for linking, dark mode, etc. @itchyny #2628
- Release builds for:
- Linux
amd64
,arm64
,armel
,armhf
,i386
,mips
,mips64
,mips64el
,mips64r6
,mips64r6el
,mipsel
,mipsr6
,mipsr6el
,powerpc
,ppc64el
,riscv64
ands390x
- macOS
amd64
andarm64
- Windows
i386
andamd64
- Docker
linux/386
,linux/amd64
,linux/arm64
,linux/mips64le
,linux/ppc64le
,linux/riscv64
andlinux/s390x
- More details see @owenthereal #2665
- Linux
- Docker images are now available from
ghcr.io/jqlang/jq
instead of Docker Hub. @itchyny #2652 #2686 - OSS-fuzz. @DavidKorczynski #2760 #2762
Full commit log can be found at jq-1.6...jq-1.7 but here are some highlights:
CLI changes
-
Make object key color configurable using
JQ_COLORS
environment variable. @itchyny @haguenau @ericpruitt #2703# this would make "field" bold yellow (`1;33`, the last value) $ JQ_COLORS="0;90:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}' { "field": 123 }
-
Change the default color of null to Bright Black. @itchyny #2824
-
Respect
NO_COLOR
environment variable to disable color output. See https://no-color.org for details. @itchyny #2728 -
Improved
--help
output. Now mentions all options and nicer order. @itchyny @wader #2747 #2766 #2799 -
Fix multiple issues of exit code using
--exit-code
/-e
option. @ryo1kato #1697# true-ish last output value exits with zero $ jq -ne true ; echo $? true 0 # false-ish last output value (false and null) exits with 1 $ jq -ne false ; echo $? false 1 # no output value exists with 4 $ jq -ne empty ; echo $? 4
-
Add
--binary
/-b
on Windows for binary output. To get\n
instead of\r\n
line endings. @nicowilliams 0dab2b1 -
Add
--raw-output0
for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684# will output a zero byte after each output $ jq -n --raw-output0 '1,2,3' | xxd 00000000: 3100 3200 3300 1.2.3. # can be used with xargs -0 $ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1 a b c $ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '[%s]\n' [a b c] [d e f] # can be used with read -d '' $ while IFS= read -r -d '' json; do > jq '.name' <<< "$json" > done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}') "a b c" "d\ne\nf" # also it's an error to output a string containing a NUL when using NUL separator $ jq -n --raw-output0 '"\u0000"' jq: error (at <unknown>): Cannot dump a string containing NUL with --raw-output0 option
-
Fix assert crash and validate JSON for
--jsonarg
. @wader #2658 -
Enable stack protection. @nicowilliams #2801
Language changes
-
Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752
# precision is preserved $ echo '100000000000000000' | jq . 100000000000000000 # comparison respects precision (this is false in JavaScript) $ jq -n '100000000000000000 < 100000000000000001' true # sort/0 works $ jq -n -c '[100000000000000001, 100000000000000003, 100000000000000004, 100000000000000002] | sort' [100000000000000001,100000000000000002,100000000000000003,100000000000000004] # arithmetic operations might truncate (same as JavaScript) $ jq -n '100000000000000000 + 10' 100000000000000020
-
Adds new builtin
pick(stream)
to emit a projection of the input object or array. @pkoppstein #2656 #2779$ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)' { "a": 1, "b": { "c": 2 }, "x": null }
-
Adds new builtin
debug(msgs)
that works likedebug
but applies a filter on the input before writing to stderr. @pkoppstein #2710$ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)' ["DEBUG:","Entering function foo with $x == 1"] ["DEBUG:",2] 3 $ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})' ["DEBUG:",{"a":1,"b":2,"sum":3}] { "a": 1, "b": 2, "c": 3 }
-
Adds new builtin
scan($re; $flags)
. Was documented but not implemented. @itchyny #1961# look for pattern "ab" in "abAB" ignoring casing $ jq -n '"abAB" | scan("ab"; "i")' "ab" "AB"
-
Adds new builtin
abs
to get absolute value. This potentially allows the literal value of numbers to be preserved aslength
andfabs
convert to float. @pkoppstein #2767 -
Allow
if
withoutelse
-branch. When skipped theelse
-branch will be.
(identity). @chancez @wader #1825 #2481# convert 1 to "one" otherwise keep as is $ jq -n '1,2 | if . == 1 then "one" end' "one" 2 # behaves the same as $ jq -n '1,2 | if . == 1 then "one" else . end' "one" 2 # also works with elif $ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end "one" "two" 3
-
Allow use of
$binding
as key in object literals. @nicowilliams 8ea4a55$ jq -n '"a" as $key | {$key: 123}' { "a": 123 } # previously parentheses were needed $ jq -n '"a" as $key | {($key): 123}' { "a": 123 }
-
Allow dot between chained indexes when using
.["index"]
@nicowilliams #1168$ jq -n '{"a": {"b": 123}} | .a["b"]' 123 # now this also works $ jq -n '{"a": {"b": 123}} | .a.["b"]' 123
-
Allow dot for chained value iterator
.[]
,.[]?
@wader #2650$ jq -n '{"a": [123]} | .a[]' 123 # now this also works $ jq -n '{"a": [123]} | .a.[]' 123
-
Fix try/catch catches more than it should. @nicowilliams #2750
-
Speed up and refactor some builtins, also remove
scalars_or_empty/0
. @muhmuhten #1845 -
Now
halt
andhalt_error
exit immediately instead of continuing to the next input. @emanuele6 #2667 -
Fix issue converting string to number after previous convert error. @thalman #2400
-
Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661
-
Fix deletion using assigning empty against arrays. @itchyny #2133
# now this works as expected, filter out all values over 2 by assigning empty $ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]' [1,0]
-
Allow keywords to be used as binding name in more places. @emanuele6 #2681
-
Allow using
nan
as NaN in JSON. @emanuele6 #2712 -
Expose a module's function names in
modulemeta
. @mrwilson #2837 -
Fix
contains/1
to handle strings with NUL. @nicowilliams 61cd6db -
Fix
stderr/0
to output raw text without any decoration. @itchyny #2751 -
Fix
nth/2
to emit empty on index out of range. @itchyny #2674 -
Fix
implode
to not assert and instead replace invalid unicode codepoints. @wader #2646 -
Fix
indices/1
andrindex/1
in case of overlapping matches in strings. @emanuele6 #2718 -
Fix
sub/3
to resolve issues involving global search-and-replace (gsub) operations. @pkoppstein #2641 -
Fix
significand/0
,gamma/0
anddrem/2
to be available on macOS. @itchyny #2756 #2775 -
Fix overflow exception of the modulo operator. @itchyny #2629
-
Fix string multiplication by 0 (and less than 1) to emit empty string. @itchyny #2142
-
Fix constant folding of division and reminder with zero divisor. @itchyny #2797
-
Fix
error/0
,error/1
to throw null error. @emanuele6 #2823 -
Simpler and faster
transpose
. @pkoppstein #2758 -
Simple and efficient implementation of
walk/1
. @pkoppstein #2795 -
Remove deprecated filters
leaf_paths
,recurse_down
. @itchyny #2666
Full Changelog: jq-1.7rc2...jq-1.7
jq 1.7rc2
Note that this is a pre-release. Feedback is welcome #2862.
After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release.
Since the last stable release many things have happened:
- jq now lives at https://github.com/jqlang
- New maintainers, admins, and owners have been recruited.
- A list of current maintainers
- NEWS file is replaced by NEWS.md with Markdown format. @wader #2599
- CI, scan builds, release, website builds etc now use GitHub actions. @owenthereal @wader @itchyny #2596 #2603 #2620 #2723
- Lots of documentation improvements and fixes.
- Website updated with new section search box, better section ids for linking, dark mode, etc. @itchyny #2628
- Release builds for:
- Linux
amd64
,arm64
,armel
,armhf
,i386
,mips
,mips64
,mips64el
,mips64r6
,mips64r6el
,mipsel
,mipsr6
,mipsr6el
,powerpc
,ppc64el
,riscv64
ands390x
- macOS
amd64
andarm64
- Windows
i386
andamd64
- Docker
linux/386
,linux/amd64
,linux/arm64
,linux/mips64le
,linux/ppc64le
,linux/riscv64
andlinux/s390x
- More details see @owenthereal #2665
- Linux
- Docker images are now available from
ghcr.io/jqlang/jq
instead of Docker Hub. @itchyny #2652 #2686 - OSS-fuzz. @DavidKorczynski #2760 #2762
Full commit log can be found at jq-1.6...jq-1.7rc2 but here are some highlights:
CLI changes
-
Make object key color configurable using
JQ_COLORS
environment variable. @itchyny @haguenau @ericpruitt #2703# this would make "field" bold yellow (`1;33`, the last value) $ JQ_COLORS="0;90:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}' { "field": 123 }
-
Change the default color of null to Bright Black. @itchyny #2824
-
Respect
NO_COLOR
environment variable to disable color output. See https://no-color.org for details. @itchyny #2728 -
Improved
--help
output. Now mentions all options and nicer order. @itchyny @wader #2747 #2766 #2799 -
Fix multiple issues of exit code using
--exit-code
/-e
option. @ryo1kato #1697# true-ish last output value exits with zero $ jq -ne true ; echo $? true 0 # false-ish last output value (false and null) exits with 1 $ jq -ne false ; echo $? false 1 # no output value exists with 4 $ jq -ne empty ; echo $? 4
-
Add
--binary
/-b
on Windows for binary output. To get\n
instead of\r\n
line endings. @nicowilliams 0dab2b1 -
Add
--raw-output0
for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684# will output a zero byte after each output $ jq -n --raw-output0 '1,2,3' | xxd 00000000: 3100 3200 3300 1.2.3. # can be used with xargs -0 $ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1 a b c $ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '[%s]\n' [a b c] [d e f] # can be used with read -d '' $ while IFS= read -r -d '' json; do > jq '.name' <<< "$json" > done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}') "a b c" "d\ne\nf" # also it's an error to output a string containing a NUL when using NUL separator $ jq -n --raw-output0 '"\u0000"' jq: error (at <unknown>): Cannot dump a string containing NUL with --raw-output0 option
-
Fix assert crash and validate JSON for
--jsonarg
. @wader #2658 -
Enable stack protection. @nicowilliams #2801
Language changes
-
Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752
# precision is preserved $ echo '100000000000000000' | jq . 100000000000000000 # comparison respects precision (this is false in JavaScript) $ jq -n '100000000000000000 < 100000000000000001' true # sort/0 works $ jq -n -c '[100000000000000001, 100000000000000003, 100000000000000004, 100000000000000002] | sort' [100000000000000001,100000000000000002,100000000000000003,100000000000000004] # arithmetic operations might truncate (same as JavaScript) $ jq -n '100000000000000000 + 10' 100000000000000020
-
Adds new builtin
pick(stream)
to emit a projection of the input object or array. @pkoppstein #2656 #2779$ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)' { "a": 1, "b": { "c": 2 }, "x": null }
-
Adds new builtin
debug(msgs)
that works likedebug
but applies a filter on the input before writing to stderr. @pkoppstein #2710$ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)' ["DEBUG:","Entering function foo with $x == 1"] ["DEBUG:",2] 3 $ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})' ["DEBUG:",{"a":1,"b":2,"sum":3}] { "a": 1, "b": 2, "c": 3 }
-
Adds new builtin
scan($re; $flags)
. Was documented but not implemented. @itchyny #1961# look for pattern "ab" in "abAB" ignoring casing $ jq -n '"abAB" | scan("ab"; "i")' "ab" "AB"
-
Adds new builtin
abs
to get absolute value. This potentially allows the literal value of numbers to be preserved aslength
andfabs
convert to float. @pkoppstein #2767 -
Allow
if
withoutelse
-branch. When skipped theelse
-branch will be.
(identity). @chancez @wader #1825 #2481# convert 1 to "one" otherwise keep as is $ jq -n '1,2 | if . == 1 then "one" end' "one" 2 # behaves the same as $ jq -n '1,2 | if . == 1 then "one" else . end' "one" 2 # also works with elif $ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end "one" "two" 3
-
Allow use of
$binding
as key in object literals. @nicowilliams 8ea4a55$ jq -n '"a" as $key | {$key: 123}' { "a": 123 } # previously parentheses were needed $ jq -n '"a" as $key | {($key): 123}' { "a": 123 }
-
Allow dot between chained indexes when using
.["index"]
@nicowilliams #1168$ jq -n '{"a": {"b": 123}} | .a["b"]' 123 # now this also works $ jq -n '{"a": {"b": 123}} | .a.["b"]' 123
-
Allow dot for chained value iterator
.[]
,.[]?
@wader #2650$ jq -n '{"a": [123]} | .a[]' 123 # now this also works $ jq -n '{"a": [123]} | .a.[]' 123
-
Fix try/catch catches more than it should. @nicowilliams #2750
-
Speed up and refactor some builtins, also remove
scalars_or_empty/0
. @muhmuhten #1845 -
Now
halt
andhalt_error
exit immediately instead of continuing to the next input. @emanuele6 #2667 -
Fix issue converting string to number after previous convert error. @thalman #2400
-
Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661
-
Fix deletion using assigning empty against arrays. @itchyny #2133
# now this works as expected, filter out all values over 2 by assigning empty $ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]' [1,0]
-
Allow keywords to be used as binding name in more places. @emanuele6 #2681
-
Allow using
nan
as NaN in JSON. @emanuele6 #2712 -
Expose a module's function names in
modulemeta
. @mrwilson #2837 -
Fix
contains/1
to handle strings with NUL. @nicowilliams 61cd6db -
Fix
stderr/0
to output raw text without any decoration. @itchyny #2751 -
Fix
nth/2
to emit empty on index out of range. @itchyny #2674 -
Fix
implode
to not assert and instead replace invalid unicode codepoints. @wader #2646 -
Fix
indices/1
andrindex/1
in case of overlapping matches in strings. @emanuele6 #2718 -
Fix
sub/3
to resolve issues involving global search-and-replace (gsub) operations. @pkoppstein #2641 -
Fix
significand/0
,gamma/0
anddrem/2
to be available on macOS. @itchyny #2756 #2775 -
Fix overflow exception of the modulo operator. @itchyny #2629
-
Fix string multiplication by 0 (and less than 1) to emit empty string. @itchyny #2142
-
Fix constant folding of division and reminder with zero divisor. @itchyny #2797
-
Fix
error/0
,error/1
to throw null error. @emanuele6 #2823 -
Simpler and faster
transpose
. @pkoppstein #2758 -
Simple and efficient implementation of
walk/1
. @pkoppstein #2795 -
Remove deprecated filters
leaf_paths
,recurse_down
. @itchyny #2666
jq 1.7rc1
Note that this is a pre-release. Feedback is welcome #2802.
After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release.
Since the last stable release many things have happened:
- jq now lives at https://github.com/jqlang
- New maintainers, admins, and owners have been recruited.
- A list of current maintainers
- CI, scan builds, release builds etc now use GitHub actions. @owenthereal #2596 #2620
- Lots of documentation improvements and fixes.
- Web site updated with new auto complete, better section ids for linking, dark mode, etc. @itchyny #2628
- Release builds for:
- Linux
amd64
,arm64
,armel
,armhf
,i386
,mips
,mips64
,mips64el
,mips64r6
,mips64r6el
,mipsel
,mipsr6
,mipsr6el
,powerpc
,ppc64el
,riscv64
ands390x
- macOS
amd64
andarm64
- Windows
i386
andamd64
- Docker
linux/386
,linux/amd64
,linux/arm64
,linux/mips64le
,linux/ppc64le
,linux/riscv64
andlinux/s390x
- More details see @owenthereal #2665
- Linux
- Docker images are now available from
ghcr.io/jqlang/jq
instead of docker hub. @itchyny #2652 - OSS-fuzz. @DavidKorczynski #2760 #2762
Full commit log can be found at jq-1.6...jq-1.7rc1 but here are some highlights:
CLI changes
-
Make object key color configurable using
JQ_COLORS
environment variable. @itchyny @haguenau @ericpruitt #2703# this would make "field" yellow (33, the last value) $ JQ_COLORS="1;30:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}' { "field": 123 }
-
Respect
NO_COLOR
environment variable to disable color output. See https://no-color.org for details. @itchyny #2728 -
Improved
--help
output. Now mentions all options and nicer order. @itchyny #2747 #2766 -
Last output value can now control exit code using
--exit-code
/-e
. @ryo1kato #1697# true-ish last output value exits with zero $ jq -ne true ; echo $? true 0 # false-ish last output value (false and null) exits with 1 $ jq -ne false ; echo $? false 1 # no output value exists with 4 $ jq -ne empty ; echo $? 4
-
Add
--binary
/-b
on Windows for binary output. To get\n
instead of\r\n
line endings. 0dab2b1 @nicowilliams -
Add
--raw-output0
for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684# will output a zero byte after each output $ jq -n --raw-output0 '1,2,3' | xxd 00000000: 3100 3200 3300 1.2.3. # can be used with xargs -0 $ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1 a b c $ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '%q\n' 'a b c' 'd'$'\n''e'$'\n''f' # can be used with read -d '' $ while IFS= read -r -d '' json; do > jq '.name' <<< "$json" > done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}') "a b c" "d\ne\nf" # also it's an error to output a string containing a NUL when using NUL separator $ jq -n --raw-output0 '"\u0000"' jq: error (at <unknown>): Cannot dump a string containing NUL with --raw-output0 option
-
Fix assert crash and validate JSON for
--jsonarg
. @wader #2658
Language changes
-
Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752
# precision is preserved $ jq -n '100000000000000000' 100000000000000000 # comparison respects precision (this is false in JavaScript) $ jq -n '100000000000000000 < 100000000000000001' true # arithmetic operations might truncate (same as JavaScript) $ jq -n '100000000000000000+10' 100000000000000020
-
Adds new builtin
pick(stream)
to emit a projection of the input object or array. @pkoppstein #2656$ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)' { "a": 1, "b": { "c": 2 }, "x": null }
-
Adds new builtin
debug(msgs)
that works likedebug
but applies a filter on the input before writing to stderr. @pkoppstein #2710$ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)' ["DEBUG:","Entering function foo with $x == 1"] ["DEBUG:",2] 3 $ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})' ["DEBUG:",{"a":1,"b":2,"sum":3}] { "a": 1, "b": 2, "c": 3 }
-
Adds new builtin
scan($re; $flags)
. Was documented but not implemented. @itchyny #1961# look for pattern "ab" in "abAB" ignoring casing $ jq -n '"abAB" | scan("ab"; "i")' "ab" "AB"
-
Adds new builtin
abs
to get absolute value. This potentially allows the literal value of numbers to be preserved aslength
andfabs
convert to float. @pkoppstein #2767 -
Allow
if
withoutelse
-branch. When skipped theelse
-branch will be.
(identity). @chancez @wader #1825 #2481# convert 1 to "one" otherwise keep as is $ jq -n '1,2 | if . == 1 then "one" end' "one" 2 # behaves the same as $ jq -n '1,2 | if . == 1 then "one" else . end' "one" 2 # also works with elif $ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end "one" "two" 3
-
Allow use of
$binding
as key in object literals. 8ea4a55 @nicowilliams$ jq -n '"a" as $key | {$key: 123}' { "a": 123 } # previously parentheses were needed $ jq -n '"a" as $key | {($key): 123}' { "a": 123 }
-
Allow dot between chained indexes when using
.["index"]
@nicowilliams #1168$ jq -n '{"a": {"b": 123}} | .a["b"]' 123 # now this works also $ jq -n '{"a": {"b": 123}} | .a.["b"]' 123
-
Fix try/catch catches more than it should. @nicowilliams #2750
-
Speed up and refactor some builtins, also remove
scalars_or_empty/0
. @muhmuhten #1845 -
Now
halt
andhalt_error
exit immediately instead of continuing to the next input. @emanuele6 #2667 -
Fix issue converting string to number after previous convert error. @thalman #2400
-
Make 0 divided by 0 result in NaN consistently. @itchyny #2253
-
Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661
-
Fix deletion using assigning empty against arrays. @itchyny #2133
# now this works as expected, filter out all values over 2 by assigning empty $ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]' [1,0]
-
Fix
stderr/0
to output raw text without any decoration. @itchyny #2751 -
Fix
nth/2
to emit empty on index out of range. @itchyny #2674 -
Fix
implode
to not assert and instead replace invalid unicode codepoints. @wader #2646 -
Simpler and faster
transpose
. @pkoppstein #2758 -
Allow keywords to be used as binding name in more places. @emanuele6 #2681
-
Allow using
nan
as NaN in JSON. @emanuele6 #2712 -
Fix indices/1 and rindex/1 in case of overlapping matches in strings. @emanuele6 #2718
-
Enable
significand/0
,gamma/0
anddrem/2
on macOS. @itchyny #2756 #2775
jq 1.6
New in this release since 1.5:
- Destructuring Alternation
- New Builtins:
builtins/0
stderr/0
halt/0, halt_error/1
isempty/1
walk/1
utf8bytelength/1
localtime/0, strflocaltime/1
- SQL-style builtins
- and more!
- Add support for ASAN and UBSAN
- Make it easier to use jq with shebangs (8f6f28c)
- Add
$ENV
builtin variable to access environment - Add
JQ_COLORS
env var for configuring the output colors
Bug fixes:
- Calling
jq
without a program argument now always assumes.
for the program, regardless of stdin/stdout. (5fe0536) - Make sorting stable regardless of qsort. (7835a72)
- Adds a local oniguruma submodule and the
./configure --with-oniguruma=builtin
option to make it easier to build with oniguruma support on systems where you can't install system-level libraries. (c6374b6 and 02bad4b) - And much more!
jq 1.5
Thanks to the 20+ developers who have sent us PRs since 1.4, and the many contributors to issues and the wiki.
The manual for jq 1.5 can be found at https://stedolan.github.io/jq/manual/v1.5/
Salient new features since 1.4:
-
regexp support (using Oniguruma)!
-
a proper module system
import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace
and
include "foo/bar"; # import foo/bar.jq's defs into the top-level
-
destructuring syntax (
. as [$first, $second, {$foo, $bar}] | ...
) -
math functions
-
an online streaming parser
-
minimal I/O builtions (
inputs
,debug
)One can now write:
jq -n 'reduce inputs as $i ( ... )'
to reduce inputs in an online way without having to slurp them first! This works with streaming too.
-
try/catch, for catching and handling errors (this makes for a dynamic non-local exit system)
-
a lexical non-local exit system
One can now say
label $foo | ..... | break $foo
where the break causes control to return to the label $foo, which
then producesempty
(backtracks). There's named and anonymous
labels. -
tail call optimization (TCO), which allows efficient recursion in jq
-
a variety of new control structure builtins (e.g.,
while(cond; exp)
,repeat(exp)
,until(cond; next)
), many of which internally use TCO -
an enhanced form of
reduce
:foreach exp as $name (init_exp; update_exp; extract_exp)
-
the ability to read module data files
import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar
-
--argjson var '<JSON text>'
Using --arg var bit me too many times :)
-
--slurpfile var "filename"
Replaces the
--argfile
form (which is now deprecated but remains for backward compatibility). -
support for application/json-seq (RFC7464)
-
a large variety of new utility functions, many being community contributions (e.g.,
bsearch
, for binary searching arrays) -
datetime functions
-
a variety of performance enhancements
-
def($a): ...;
is now allowed as an equivalent ofdef(a): a as $a | ...;
-
test and build improvements, including gcov support
Lastly, don't forget the wiki! The wiki has a lot of new content since 1.4, much of it contributed by the community.
jq 1.5 release candidate 2
Thanks to the 20+ developers who have sent us PRs since 1.4, and the many contributors to issues and the wiki. We're nearing a 1.5 release, finally.
Salient new features since 1.4:
-
regexp support (using Oniguruma)!
-
a proper module system
import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace
and
include "foo/bar"; # import foo/bar.jq's defs into the top-level
-
destructuring syntax (
. as [$first, $second, {$foo, $bar}] | ...
) -
math functions
-
an online streaming parser
-
minimal I/O builtions (
inputs
,debug
)One can now write:
jq -n 'reduce inputs as $i ( ... )'
to reduce inputs in an online way without having to slurp them first! This works with streaming too.
-
try/catch, for catching and handling errors (this makes for a dynamic non-local exit system)
-
a lexical non-local exit system
One can now say
label $foo | ..... | break $foo
where the break causes control to return to the label $foo, which
then producesempty
(backtracks). There's named and anonymous
labels. -
tail call optimization (TCO), which allows efficient recursion in jq
-
a variety of new control structure builtins (e.g.,
while(cond; exp)
,repeat(exp)
,until(cond; next)
), many of which internally use TCO -
an enhanced form of
reduce
:foreach exp as $name (init_exp; update_exp; extract_exp)
-
the ability to read module data files
import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar
-
--argjson var '<JSON text>'
Using --arg var bit me too many times :)
-
--slurpfile var "filename"
Replaces the
--argfile
form (which is now deprecated but remains for backward compatibility). -
support for application/json-seq (RFC7464)
-
a large variety of new utility functions, many being community contributions (e.g.,
bsearch
, for binary searching arrays) -
datetime functions
-
a variety of performance enhancements
-
def($a): ...;
is now allowed as an equivalent ofdef(a): a as $a | ...;
-
test and build improvements, including gcov support
Lastly, don't forget the wiki! The wiki has a lot of new content since 1.4, much of it contributed by the community.
1.5 release candidate 1
Salient new features since 1.4:
-
regexp support (using Oniguruma)
-
an online streaming parser
Included is the ability to control reading of inputs from the jq
program, using the newinput
andinputs
builtins.Finally we can write:
jq -n 'reduce inputs as $i ( ... )' # reduce online!
to reduce inputs without slurping them first. This works with
streaming too. -
try/catch, for catching and handling errors (this makes for a dynamic non-local exit system)
-
a lexical non-local exit system
Using try/catch to break out of control structures was not a good
thing. A lexical mechanism is.You can now say
label $foo | ..... | break $foo
where the break causes control to return to the label $foo, which
then producesempty
(backtracks). There's named and anonymous
labels. -
tail call optimization (TCO), which allows efficient recursion in jq
-
a variety of new control structure builtins (e.g.,
while(cond; exp)
,repeat(exp)
,until(cond; next)
) -
an enhanced form of
reduce
:foreach exp as $name (init_exp; update_exp; extract_exp)
-
a proper module system
import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace
-
the ability to read module data files
import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar
-
--argjson var '<JSON text>'
Using --arg var bit me too many times :)
-
--in-place
/-i
for in-place editing of files -
support for application/json-seq.
-
a variety of new utility functions, many being community contributions
-
a variety of performance enhancements (e.g., constant folding)
-
def($a): ...;
is now allowed as an equivalent ofdef(a): a as $a | ...;
Lastly, don't forget the wiki! It has a lot of new content since 1.4, much of it contributed by the community.
jq 1.4
The manual for jq 1.4 can be found at https://stedolan.github.io/jq/manual/v1.4/
jq 1.3
The manual for jq 1.3 can be found at https://stedolan.github.io/jq/manual/v1.3/