Shell jq is adding header to every other line in csv output, not just once at the top

user_7331538 0
Viewed 92

I am extracting keys from a json with this format:

[
 {
   "k1":{"k2":4},
   "k3":{"k2":6}
 }
]
[
 {
   "k1":{"k2":4},
   "k3":{"k2":6}
 }
]

I am using this command ["a","b"], (.[] | [.k1.k2,.k3.k2]) | @csv to extract two keys and adding two headers (a, b) to the csv. But The output to this is a header in every other line:

"\"a\",\"b\""
"4,6"
"\"a\",\"b\""
"4,6"

You can see this in this jq play. How can I change to filter to just put header on the first line?

jq header csv shell json
2 Answers

As you have stream in your json, you need inputs instead of .[]

jq -nr '(["a","b"], (inputs | map(.k1.k2,.k3.k2))) | @csv' input-json
user_21256710

Your input is a stream of JSON documents. Thus, your filter gets executed once for each document, including constructing the headers.

One solution would be to use the --slurp or -s option to read in the stream as an array. This, however, would necessitate to go one level deeper in your iteration (.[][] instead of just .[]):

jq -s '["a","b"], (.[][] | [.k1.k2,.k3.k2]) | @csv'

Demo

Another way would be to use inputs which gives you all input documents at once. This, however, would necessitate to iterate over that (input[] instead of .[]) and use the --null-input or -n flag to avoid swallowing up the first elemment:

jq -n '["a","b"], (inputs[] | [.k1.k2,.k3.k2]) | @csv'

Demo

Note: You also may want to use the --raw-output or -r flag to produce raw text instead of JSON when generating the output with @csv (as I already did in the demos).

user_21584790
Related