How to encode a dict with numpy.int types into a protobuf well known type Value?

Viewed 313

Consider the following python dict

json_obj = {
    "numpy_integer": np.int32(3),
}

I would like to encode it with protobuf using well known type Value.

proto_value = ParseDict(json_obj, Value())

Results in the following error Value 3 has unexpected type <class 'numpy.int32'>

Is there a way to instruct proto encoder to recognize numpy.int32 as an int. ?


Full Error

$ python3 dummy.py
Traceback (most recent call last):
  File "dummy.py", line 11, in <module>
    proto_value = ParseDict(json_obj, Value())
  File "/usr/local/lib/python3.8/site-packages/google/protobuf/json_format.py", line 454, in ParseDict
    parser.ConvertMessage(js_dict, message)
  File "/usr/local/lib/python3.8/site-packages/google/protobuf/json_format.py", line 483, in ConvertMessage
    methodcaller(_WKTJSONMETHODS[full_name][1], value, message)(self)
  File "/usr/local/lib/python3.8/site-packages/google/protobuf/json_format.py", line 642, in _ConvertValueMessage
    self._ConvertStructMessage(value, message.struct_value)
  File "/usr/local/lib/python3.8/site-packages/google/protobuf/json_format.py", line 675, in _ConvertStructMessage
    self._ConvertValueMessage(value[key], message.fields[key])
  File "/usr/local/lib/python3.8/site-packages/google/protobuf/json_format.py", line 654, in _ConvertValueMessage
    raise ParseError('Value {0} has unexpected type {1}.'.format(
google.protobuf.json_format.ParseError: Value 3 has unexpected type <class 'numpy.int32'>.


Is the descriptor_pool parameter to ParseDict a reasonable direction to investigate ?


One way I could envision this is to convert to json string with a specialized JSONEncoder. But I would rather avoid converting back to string and then to protobuf.

class NumpyAwareEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.int32):
            return int(obj)
        return super().default(obj)

json_string = json.dumps(json_obj, cls=NumpyAwareEncoder)
proto_value_from_string = Parse(json_string, Value())
print(proto_value_from_string)


Full Code:

import json

import numpy as np
from google.protobuf.json_format import Parse
from google.protobuf.json_format import ParseDict
from google.protobuf.struct_pb2 import Value


class NumpyAwareEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.int32):
            return int(obj)
        return super().default(obj)


json_obj = {
    "numpy_integer": np.int32(3),
}


json_string = json.dumps(json_obj, cls=NumpyAwareEncoder)
proto_value_from_string = Parse(json_string, Value())
print(proto_value_from_string)

proto_value_from_dict = ParseDict(json_obj, Value())
print(proto_value_from_dict)

0 Answers
Related