Expand large array and select elements in JQ

user_10525932 0
Viewed 91

This may just not be possible due to how conceptually streaming/filtering JSON works, but let's suppose I have something like the following JSON:

[
  {
    "name": "account_1",
    "type": "account"
  },
  {
    "name": "account_2",
    "type": "account"
  },
  {
    "name": "user_1",
    "type": "user"
  },
  {
    "name": "user_2",
    "type": "user"
  }
]

And now I want to print out only the user objects.

I know I can filter to just the streaming type entities with something like this:

cat file.json | jq --stream 'select(.[0][1] == "type" and .[1] == "user" | .)'

Which would produce:

[
  [
    2,
    "type"
  ],
  "user"
]
[
  [
    3,
    "type"
  ],
  "user"
]

Is there any way I can print out the parent objects of those types instead of the type entities? E.g. I'd like to get out:

[
  {
    "name": "user_1",
    "type": "user"
  },
  {
    "name": "user_2",
    "type": "user"
  }
]

Without streaming, this is a pretty straightforward exercise. E.g.:

cat file.json | jq '.[] | select(.type=="user")'

In reality the actual input file is around 5GB, so I need to use streaming input, but I can't seem to get the jq syntax right with --stream enabled. E.g.

cat file.json | jq --stream '.[] | select(.type=="user")'

Produces:

jq: error (at <stdin>:3): Cannot index array with string "type"
jq: error (at <stdin>:5): Cannot index array with string "type"
...

(edited to include desired output)

bigdata jq select json
1 Answers

Just truncate the top-level array.

jq -n --stream 'fromstream(1 | truncate_stream(inputs)) | select(.type == "user")'

Online demo

jqplay does not support the --stream option, so the above demo has the output of --stream as the JSON input.

user_102486780
Related