JQ - What is the data type of comma filter

Question

Question

What is the data type of the JQ comma filter?

Background

I get Cannot iterate over number error when trying to feed the output of the comma filter.

$ printf '{"ika":5,"uni":3}' | jq '.ika, .uni | .'
5
3

$ printf '{"ika":5,"uni":3}' | jq '.ika, .uni | . | add'
jq: error (at :0): Cannot iterate over number (5)

To get around, need to convert to an array explicitly.

$ printf '{"ika":5,"uni":3}' | jq '[ .ika, .uni ] | . | add'
8

Clarifications

According to the JQ manual, the output of a JQ filter is a sequence of whitespace-separated JSON data

However, the output the comma filter (which is stated as one of the Basic filters) seems not to be a sequence of whitespace-separated JSON data.

Basic filters

Comma: ,
If two filters are separated by a comma, then the same input will be fed into both and the two filters’ output value streams will be concatenated in order: first, all of the outputs produced by the left expression, and then all of the outputs produced by the right. For instance, filter .foo, .bar, produces both the “foo” fields and “bar” fields as separate outputs.

JQ Manual

jq filters run on a stream of JSON data. The input to jq is parsed as a sequence of whitespace-separated JSON values which are passed through the provided filter one at a time. The output(s) of the filter are written to standard out, again as a sequence of whitespace-separated JSON data.

It’s important to remember that every filter has an input and an output. Even literals like “hello” or 42 are filters - they take an input but always produce the same literal as output. Operations that combine two filters, like addition, generally feed the same input to both and combine the results.

So what is the output of comma filter? If is it JSON data as the JQ manual says? I assume, if it is to be a JSON, then should it be an array?

[5, 3]

Update

Based on the answers:

In jq expressions such as P|Q, the filter Q processes each item in the stream produced by P one at a time. In the invocation jq P, each item in the STDIN stream is processed by P one at a time.

$ printf '{"ika":5,"uni":3}' | \
jq '.ika, .uni' | \
jq '"JQ is processing the \(input_line_number) th line whose input value is \(.)"'

"JQ is processing the 1 th line whose input value is 5"
"JQ is processing the 2 th line whose input value is 3"

user_997358 · Answer

A comma expression in jq takes the form P, Q and like all jq expressions, produces a stream of JSON values, in this case the stream produced by P followed by the stream produced by Q.

Here "stream" means a sequence of 0 or more JSON values, and is not to be confused with an array of JSON values. A "stream of JSON values" is not itself a JSON value.

If this is unclear, you might be interested in my "Stream-Oriented Introduction to jq"

Elaboration

jq expressions are not themselves "data types", but every jq expression is a filter that accepts as input a stream of JSON values, and produces a stream of JSON values. Using the notation JSON* to mean "a stream of JSON values", it being understood that a stream can conceptually be unbounded in length, we can think of jq expressions as being mappings from JSON* to JSON*.

This is important to understand since all jq expressions must always be understood in terms of their inputs. In the case of P,Q, the important point is that if R is a jq expression, then:

R | (P,Q)

is equivalent to:

(R|P), (R|Q)

(This is a very important point, firstly because it makes jq very powerful - R is in effect duplicated; and secondly because it makes the behavior of jq's pipe quite different from the superficially similar behavior of shell pipes - consider e.g. P | (Q;R) in bash.)

The -n command-line option signals that null should be used as the initial JSON value, so for example:

jq -n 1,2

produces:

1
2

The -s command-line option

An invocation of jq of the form jq P causes each JSON item in STDIN to be presented to the jq filter, P, on a one-at-a-time basis.

By contrast, an invocation of the form jq -s P causes the items in STDIN to be wrapped in a JSON array, which is then presented to P.

`reduce`, `foreach`, `foo(P)`, etc

jq also supports reduction over a stream. This is particularly useful with the built-in filter, inputs, to circumvent the need to use the -s command-line option in order to process a stream of external inputs.

Parameterized filters, such as those of the form foo(P) where P is an arbitrary jq expression, are also supported.

For further details, see the jq manual, the previously mentioned document, and/or other documentation.

user_9447571 · Answer

What is the data type of the JQ comma filter?

There's not really such a thing as a data type returned by the comma filter...

Because it doesn't really group the values in a single container data type. Rather, it sends the values separately through the following filters and reports each output separately at the end.

Perhaps a good way to understand it is seeing this result:

$ printf '{"ika":[4,9],"uni":[1,3,5]}' | jq '.ika, .uni | add'
13
9

The two arrays went through the "add" filter, but each on its own, and the two results reported at the end.

Re-read the docs for comma, you'll see that's what it's saying:

If two filters are separated by a comma, then the same input will be fed into both and the two filters’ output value streams will be concatenated in order: first, all of the outputs produced by the left expression, and then all of the outputs produced by the right.

You can turn them into an array, but you need to do that explicitly. For example:

$ printf '{"ika":[4,9],"uni":[1,3,5]}' | jq '[ .ika, .uni | add ]'
[
  13,
  9
]