How to group lists in this poorly formatted JSON

user_7076818 0
Viewed 58

I have multiple files of JSON-ish text that I need to merge:

{
  "City1":
  {
    "Coords":
    [
      [10,20,30],
      [11,21,31]
    ]
  },
  "City2":
  {
    "Coords":
    [
      [12,22,32]
    ]
  }
}
{
  "City1":
  {
    "Coords":
    [
      [13,23,33]
    ]
  },
  "City3":
  {
    "Coords":
    [
      [14,24,34],
      [15,25,35]
    ]
  }
}

City1, City2, etc. may extend into the dozens, each with a varying number of coordinates.

I understand that the files are not proper JSON but, when I pass the files through to_entries to correct "CityN" to "key":"CityN", then "Coords" is a level deeper and, in any case, almost everything I try to do with the result of to_entries returns Cannot index array with string "key".

I would like to get back

{
  "City1":
  {
    "Coords":
    [
      [10,20,30],
      [11,21,31],
      [13,23,33]
    ]
  },
  "City2":
  {
    "Coords":
    [
      [12,22,32]
    ]
  },
  "City3":
  {
    "Coords":
    [
      [14,24,34],
      [15,25,35]
    ]
  }
}

but am at a loss for how to do it. Thank you very much for your help.

jq merge json
2 Answers

The following solution is neither elegant nor robust but at least it is straightforward and can easily be robustified. It assumes that there's one object per file, and that jq is invoked with the -s option.

# merge two same-key objects with array-valued keys
# by concatenating the arrays
def merge($b):
  if . == null or . == {} then $b
  else reduce (keys_unsorted[]) as $key (.;
    .[$key] += ($b[$key]) )
  end;

reduce .[] as $cities (null;
   reduce ($cities|keys_unsorted[]) as $city (.;
     .[$city] |= merge($cities[$city]) ))

If your jq has inputs (as it probably does), then it would probably be better to change reduce .[] to reduce inputs, and invoke jq with the -n option instead of -s. One of the advantages of this approach is that it removes the one-object-per-file requirement.

user_9973580

The following works:

jq -s 'map(to_entries)|flatten|group_by(.key)|
    map( { (.[0].key): { "Coords": (reduce .[].value.Coords as $i ([]; . + $i))}} )'

For the purpose of jq your input is a stream of proper JSON objects. jq can deal with such input. I used the -s option to convert the stream to a JSON array.

user_19215460
Related