Datatype conversion using Python Marshmallow

user_10133195 0
Viewed 2377

I am trying to use Marshmallow schema to serialize the python object. Below is the schema I have defined for my data.

from marshmallow import Schema, fields

class User:

    def __init__(self, name = None, age = None, is_active = None, details = None):
        self.name = name
        self.age = age
        self.is_active = is_active
        self.details = details

class UserSchema(Schema):
    name = fields.Str()
    age = fields.Int()
    is_active = fields.Bool()
    details = fields.Dict()

The input will be in dictionary format and all the values will be in string. 

user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}

When I try to run the below snippet, values of age and is_active got converted into respective datatype but details remains unchanged.

user_schema = UserSchema()
user_dump_data = user_schema.dump(user_data)
print(user_dump_data)

Output:

{'name': 'xyz', 'is_active': True, 'details': "{'key1':'val1', 'key2':'val2'}", 'age': 20}

I need to serialize the input data into respective datatype I defined in my schema. Is there anything I am doing wrongly? Can anyone guide me how to acheive this using Marshmallow?

I am using 

python 3.6
marshmallow 3.5.1

Edit

The above input data is fetched from HBase. By default HBase stores all its values as bytes and return as bytes. Below is the format I get from HBase

{b'name': b'xyz', b'age': b'20', b'is_active': b'true', b'details': b"{'key1':'val1', 'key2':'val2'}"}

Then I decode this dictionary and pass it to my UserSchema to serialize it to be used in web API.

marshmallow python-3.x python
2 Answers

You're confusing serializing (dumping) and deserializing (loading).

Dumping is going from object form to json-serializable basic python types (using Schema.dump) or json string (using Schema.dumps). Loading is the reverse operation.

Typically, your API loads (and validates) data from the outside world and dumps (without validation) your objects to the outside world.

If your input data is this data and you want to load it into objects, you need to use load, not dump.

user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
user_loaded_data = user_schema.load(user_data)
user = User(**user_loaded_data)

Except if you do so, you'll be caught by another issue. DictField expects data as a dict, not a str. You need to enter

user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details": {'key1':'val1', 'key2':'val2'}}
user_46534850

As Jérôme mentioned you're confusing serializing(dumping) with deserializing(loading). As per your requirement, you should use Schema.load as suggested.

Since, all the input values are expected to be of string type. You can use, pre_load to register a method for pre-processing the data as follows:

from marshmallow import Schema, fields, pre_load

class UserSchema(Schema):
    name = fields.Str()
    age = fields.Int()
    is_active = fields.Bool()
    details = fields.Dict()

    @pre_load
    def pre_process_details(self, data, **kwarg):
        data['details'] = eval(data['details'])
        return data

user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}

user_schema = UserSchema()
user_loaded_data = user_schema.load(user_data)
print(user_loaded_data)

Here, pre_process_details will convert string type to dictionary for correct deserialization.

user_126397910
Related