Update operator yields empty object in JQ

Question

I cannot understand the behaviour of the update operator of jq (version 1.6) shown in the following examples.

Why does example 1 return an updated object, but example 2 and 3 return an empty object or a wrong result?

The difference between the examples is only the calling order of the function to convert a string into a number.

#!/bin/bash
#
# strange behaviour jq

# example 1 - works as expected
jq -n '
  def numberify($x): $x | tonumber? // 0;

  "1" as $stringValue
  | numberify($stringValue) as $intValue
# | { }                                     # version 1: a.b does not exist yet
  | { a: { b: 1 } }                         # version 2: a.b exists already
  | .["a"] |= { b: (.b + $intValue) }
'
# result example 1, version 1 - expected
#   {
#     "a": {
#       "b": 1
#     }
#   }

# result example 1, version 2 - expected
#   {
#     "a": {
#       "b": 2
#     }
#   }


# example 2 - erroneous result
jq -n '
  def numberify($x): $x | tonumber? // 0;

  "1" as $stringValue
# | { }                           # version 1: a.b does not exist yet
  | { a: { b: 1 } }               # version 2: a.b exists already
  | .["a"] |= { b: (.b + numberify($stringValue)) }
'
# result example 2, version 1 - unexpected
#   {}

# result example 2, version 2 - unexpected
#   {}


# example 3 - erroneous result
jq -n '
  def numberify($x): $x | try tonumber catch 0;

  "1" as $stringValue
# | { }                           # version 1: a.b does not exist yet
  | { a: { b: 1 } }               # version 2: a.b exists already
  | .["a"] |= { b: (.b + numberify($stringValue)) }
'
# result example 3, version 1 - unexpected
#   {
#     "a": {
#       "b": 0
#     }
#   }

# result example 3, version 2 - unexpected
#   {
#     "a": {
#       "b": 1
#     }
#   }

@oguzismail That's a good idea to use '+=' instead of '|='. I hadn't thought of it before. Currently, my code with the workaround for the bug looks like this:

  def numberify($x): $x | tonumber? // 0;

  "1" as $sumReqSize
  | "10" as $sumResSize
  | { statistics: { count: 1, sumReqSize: 2, sumResSize: 20 } }
  | [numberify($sumReqSize), numberify($sumResSize)] as $sizes     # workaround for bug
  | .statistics |= {
                     count: (.count + 1),
                     sumReqSize: (.sumReqSize + $sizes[0]),
                     sumResSize: (.sumResSize + $sizes[1])
                   }
'

Following your suggestion it will be more concise and doesn't need the ugly workaround:

  def numberify($x): $x | tonumber? // 0;

  "1" as $sumReqSize
  | "10" as $sumResSize
  | { statistics: { count: 1, sumReqSize: 2, sumResSize: 20 } }
  | .statistics.count += 1
  | .statistics.sumReqSize += numberify($sumReqSize)
  | .statistics.sumResSize += numberify($sumResSize)

user_10248678 · Answer

This is a documented bug in jq 1.6. In this case you can use try-catch instead.

def numberify($x): $x | try tonumber catch 0;

But I don't know if there is a generic way to walk around this issue.

user_997358 · Answer

This is a bug in jq 1.6. One option would be to use an earlier version of jq (e.g. jq 1.5).

Another would be to avoid |= by using = instead, along the lines of:

.a = (.a | ...)

or if the RHS does not actually depend on the LHS (as in your original examples), simply replacing |= by =.

Update operator yields empty object in JQ

2 Answers