Filter JSON using Jmespath and return value if expression exist, if it doesn't return None/Null (python)

user_13995166 0
Viewed 1422

How can I get JMESPath to only return the value in a json if it exists, if it doesn't exist return none/null. I am using JMESPath in a python application, below is an example of a simple JSON data.

{
  "name": "Sarah",
  "region": "south west",
  "age": 21,
  "occupation": "teacher",
  "height": 145,
  "education": "university",
  "favouriteMovie": "matrix",
  "gender": "female",
  "country": "US",
  "level": "medium",
  "tags": [],
  "data": "abc",
  "moreData": "xyz",
  "logging" : {
    "systemLogging" : [ {
      "enabled" : true,
      "example" : [ "this", "is", "an", "example", "array" ]
    } ]
  }
}

For example I want it to check if the key "occupation" contains the word "banker" if it doesn't return null.

In this case if I do jmespath query "occupation == 'banker'" I would get false. However for more complicated jmespath queries like "logging.systemLogging[?enabled == `false`]" this would result in an empty array [] because it doesn't exist, which is what I want.

The reason I want it to return none or null is because in another part of the application (my base class) I have code that checks if the dictionary/json data will return a value or not, this piece of code iterates through an array of dictionaries/ json data like the one above.

One thing I've noticed with JMESPath is that it is inconsistent with its return value. In more complicated dictionaries I am able to achieve what I want but from simple dictionaries I can't, also If you used a methods, e.g starts_with, it returns a boolean but if you just use an expression it returns the value you are looking for if it exists otherwise it will return None or an empty array.

jmespath dictionary python json
2 Answers

This is traditionally accomplished by:

dictionary = json.loads(my_json)
dictionary.get(key, None) # None is the default value that is returned.

That will work if you know the exact structure to expect from the json. Alternatively you can make two calls to JMESpath, using one to try to get the value / None / empty list, and one to run the query you want.

The problem is that JMESpath is trying to answer your query: Does this structure contain this information pattern? It makes sense that the result of such a query should be True/False. If you want to get something like an empty list back, you need to modify your query to ask "Give me back all instances where this structure contains the information I'm looking for" or "Give me back the first instance where this structure contains the information I'm looking for."

user_24392800

Filters in JMESPath do apply to arrays (or list, to speak in Python).
So, indeed, your case is not a really common one.

This said, you can create an array out of a hash (or dictionary, to speak in Python again) using the to_array function.
Then, since you do know you started from a hash, you can select back the first element of the created array, and indeed, if the array ends up being empty, it will return a null.

To me, at least, it looks consistant, an array can be empty [], but an empty object is a null.

To use this trick, though, you will also have to reset the projection you created out of the array, with the pipe expression:

Projections are an important concept in JMESPath. However, there are times when projection semantics are not what you want. A common scenario is when you want to operate of the result of a projection rather than projecting an expression onto each element in the array. For example, the expression people[*].first will give you an array containing the first names of everyone in the people array. What if you wanted the first element in that list? If you tried people[*].first[0] that you just evaluate first[0] for each element in the people array, and because indexing is not defined for strings, the final result would be an empty array, []. To accomplish the desired result, you can use a pipe expression, <expression> | <expression>, to indicate that a projection must stop.

Source: https://jmespath.org/tutorial.html#pipe-expressions

And so, with all this, the expression ends up being:

to_array(@)[?occupation == `banker`]|[0]

Which gives

null

On your example JSON, while the expression

to_array(@)[?occupation == `teacher`]|[0]

Would return your existing object, so:

{
  "name": "Sarah",
  "region": "south west",
  "age": 21,
  "occupation": "teacher",
  "height": 145,
  "education": "university",
  "favouriteMovie": "matrix",
  "gender": "female",
  "country": "US",
  "level": "medium",
  "tags": [],
  "data": "abc",
  "moreData": "xyz",
  "logging": {
    "systemLogging": [
      {
        "enabled": true,
        "example": [
          "this",
          "is",
          "an",
          "example",
          "array"
        ]
      }
    ]
  }
}

And following this trick, all your other test will probably start to work e.g.

  • to_array(@)[?starts_with(occupation, `tea`)]|[0]
    will give you back your object
  • to_array(@)[?starts_with(occupation, `ban`)]|[0]
    will give you a null

And if you only need the value of the occupation property, as you are falling back to a hash now, it is as simple as doing, e.g.

  • to_array(@)[?starts_with(occupation, `tea`)]|[0].occupation
    Which gives 
    "teacher"

  • to_array(@)[?starts_with(occupation, `ban`)]|[0].occupation
    Which gives 
    null
user_21235300
Related