Merging a long JSON with another using jq

user_4235798 0
Viewed 631

I need to do some CURLs from which I'm building one output JSON. This is how I managed to perform a merge using a function:

...
ADDITIONALJSONDATA="{\"$DATATYPE\" : "$DATA"}" 
MERGEDENTRY=$(echo $SOURCE | jq --argjson json "$ADDITIONALJSONDATA" '. += $json' | tr -d '\r\n')
 ...

It seems when JSON inside $DATA is big enough then I'm getting Argument list too long error. Is it possible (in a nice way) to treat JSON to merge as a single argument here?

jq bash json
3 Answers

  1. If, as seems to be the case here, you already have $DATA and $DATATYPE, there is no need for ADDITIONALJSONDATA

  2. In general, using tr -d '\r\n' here is very bad practice, e.g. because it could scrunch 1 and 2 together to make 12

Hopefully the following will meet your requirements:

MERGEDENTRY=$(jq -c --arg TYPE "$DATATYPE" --slurpfile A <(printf '%s\n' "$DATA") '
  . += {($TYPE): $A[0]}' <<< "$SOURCE")

(We need to use $A[0] here because "slurping" puts $DATA into an array.)

user_9973580

You could use process substitution and --slurpfile option to solve your issue:

MERGEDENTRY=$(echo "$SOURCE" | jq --slurpfile json <(printf '%s\n' "$ADDITIONALJSONDATA") '. += $json[0]' | tr -d '\r\n')

As per Charles' suggestion, we can simplify further by using <<< instead of echo "$SOURCE" | ...:

MERGEDENTRY=$(jq --slurpfile json <(printf '%s\n' "$ADDITIONALJSONDATA") '. += $json[0]' <<< "$SOURCE" | tr -d '\r\n')
user_68626010

I managed to do it far easier: MERGEDENTRY=${source::-1}","\"$data_type\"":"$data"," And then just keep in mind we need to end JSON payload properly by replacing ' by }.

I can do this as I'm always adding new JSON payload at the end of existing one.

user_42357980
Related