How to retrieve key and value from JSON file using stream option in jq

If the indexes of the arrays are fixed, you can use this filter:

jq '.A[0].B.C[0].D.applicationNumberText.electronicText' file

and to extract all electronicText fields, you can use this:

jq '.A[].B.C[].D.applicationNumberText.electronicText' file

If you want to use the jq stream fashion (though it's not clear why), you can use this:

jq --stream 'select(.[0]|contains(["electronicText"]))|.[1]//empty'

This will look in the key path for the string electronicText and if found, get its associated value. The //empty is to filter out the path array without value.

I belive what you are looking for is:

. as $inputs | {} | setpath($inputs[0]; $inputs[1]) | .A[]?.B.C[]?.D.applicationNumberText.electronicText // empty

This filter expects --stream as a flag (like your first example), generates a temporal obj with a section of the json, runs the filter based on your second example, and then hides the null results. Note: This based on a example in the cookbook

Let me give you an overview of the things at play in case it is confusing:

1. The --stream flag will make your receive your data differently. You filter will now run multiple times with data in the form of [<path>, <leaf-value>] which is received as the parser traverses your json file, one node at a time.

   I believe you refer to this when you mention using indexes, in this case meanining that you are inspecting the path expression value that the --streaming flag exposes instead of the actual json data.

   Please compare the outputs of the following:

   jq '.A[].B.C[].D.applicationNumberText.electronicText' file.json # outputs 15570075 
   jq --stream '.A[].B.C[].D.applicationNumberText.electronicText' file.json # multiple failures: cannot index array with string "A"
   

2. setpath() receives these path and leaf-value parameters puts it on top of its input object. e.g.

   {} | setpath(["a", 0, "b"], "leaf-value") # returns {"a":[{"b":"leaf-value"}]}
   

   in our case we are generating the following stream of values as the parse visits each node:

   {"A":[{"B":{"C":[{"D":{"applicationNumberText":{"value":"15570075"}}}]}}]}
   {"A":[{"B":{"C":[{"D":{"applicationNumberText":{"electronicText":"15570075"}}}]}}]}
   {"A":[{"B":{"C":[{"D":{"applicationNumberText":{"electronicText":null}}}]}}]}
   {"A":[{"B":{"C":[{"D":{"date":"2018-10-01"}}]}}]}
   {"A":[{"B":{"C":[{"D":{"app":"Utility"}}]}}]}
   

3. Now that the data has a format similar to the json file we can run a more familiar filter.

   .A[]?.B.C[]?.D.applicationNumberText.electronicText
   

   keep in mind that there are no spaces between. We are also using the ? operator on both generator .A[] and .C[] because we are not sure that those fields will be available as arrays at all times

As a final note, you might want to consider building a path array with path() and compare that against the path value that appears in .[0] when using the --stream jq flag

In response to the original question:

jq --stream '
  select(length==2 and .[0][-1]=="electronicText")|.[1]
' input.json
"15570075"

If you want the value as well, then you might wish to consider the following filter:

select(length==2 and .[0][-2]=="applicationNumberText")
| .[0][-1] as $last
| select($last == "electronicText" or $last == "value")
| {($last): .[1]}

which with your sample JSON produces:

{"value":"15570075"}
{"electronicText":"15570075"}

Combining key-value pairs into a single JSON object

One way to construct a "dictionary" is to use inputs in conjunction with the -n command-line option. Simply wrap the above filter in the structure:

 [inputs | ....] | add

where here .... stands for the above filter; and invoke jq with both the -n and --stream options.

If you know the name of the objects of interest, just search for object paths by name.

getpath(paths(objects) | select(.[-1] == "applicationNumberText"))

Streaming the input in should help with efficiency concerns. Just identify the paths you want to keep and truncate the paths of interest.

$ jq -n --stream --arg key 'applicationNumberText' '
fromstream(inputs | truncate_stream2((.[0] | index($key) // empty) + 1))
' input.json

This just checks the path and looks for the "applicationNumberText" key then truncates the path to that value so it can be rebuilt from the stream.

This uses an inverted version of the truncate_stream/1 function which swaps the usual inputs which I find using more intuitive.

def truncate_stream2($count): .[0] |= .[$count:];