Surprising side effect of (array_expr) as $var for downstream output on non/empty value of array_expr

wass rubleff 0
Viewed 59

I need to extract first object matching my criterium in input stream as a variable (if any), then to process the entire stream.

Turns out, the stream becomes empty if there is no matches:

Input:

{"ID":1}
{"ID":2}

Code resulting in full stream:

[inputs] |
(
  first(.[] | select (.ID==1))
) as $match |
.

Code resulting in empty stream:

[inputs] |
(
  first(.[] | select (.ID==3))
) as $match |
.

(The only difference between snippets is .ID=={1|3})

How can I make the whole inputs available to downstream filter, whether or not there is a match?

jq
3 Answers

The key point here is that first(empty) yields the empty stream. So in your query, first(.[] | select (.ID==3)) as $match has the effect of empty, and thus your program is equivalent to empty | ., which is empty.

Unfortunately, you have not indicated what you are actually trying to do, but chances are you might want to consider something along the lines of:

(first(.[] | select (.ID==3)) // null) as $match`

Since @glennjackman's response has been downvoted, it is perhaps worth emphasizing that the command-line option -n must be specified if inputs is to be relied on for the entire input stream.

peak0

Maybe this example will help to clarify:

0 | (1,2,3) as $x | .

0
0
0

Given a single input followed by an (even unrelated) assignment that unfolds multiple times, you end up having a stream of said multitude of items (for each of which $x is set to one of the assigned values).

Now, replacing (1,2,3) with empty

0 | empty as $x | .

is no exception to that: The assignment unfolds 0 times, so you end up with a stream of 0 items as there is no value $x could have been set to, and in consequence no copy of the input for which $x could hold a value.

Following your example, you could envelop the filter result in something like an array, a structure that can handle any multitude of result items, including 0. In fact, with [inputs] you already had that array but decomposed it with the following .[]. Keep it by using map instead, so your filter can yield either an empty array or one containing the items found.

[inputs]
| map(select (.ID == 1)) as $match
| .   # yields the whole input array with $match set to [{"ID": 1}]

# or

[inputs]
| map(select (.ID == 3)) as $match
| .   # yields the whole input array with $match set to []

Next, when using $match you would presumably only extract the first value $match[0] which evaluates to either the first object found, or null if there was no match (the array was empty). In that case, as there's no need to hold an array with possibly more than just one item in it, you could tweak the assignment from above and directly set the variable to hold the first item or null by moving the array (brackets) outside of the capture:

[inputs]
| map(select (.ID == 1)) as [$match]
| .   # yields the whole input array with $match set to {"ID": 1}

# or

[inputs]
| map(select (.ID == 3)) as [$match]
| .   # yields the whole input array with $match set to null
pmf0

I was just playing with this today. If you want [inputs] to contain all the inputs, even the first one, you either need

jq -n '[inputs] | ...'

or

jq '[.] + [inputs] | ...

or

jq '[., inputs] | ...

or

jq -s '...' # "dot" is an array of all the inputs
glenn jackman0
Related