Pydantic - Print object on ValidationError and remove from list

Raymond C. 0
Viewed 57

Background

I am developing an application which requires a module to validate JSON data. The JSON data provided could have between 500 - 2000 entries. I am unsure if choosing Pydantic is the correct way of achieving the desired outcome (Questions section). Since the application is pretty complicated, I have provided a minimal example.

Questions

  1. When a ValidationError is raised by Pydantic, how do I obtain the object with its values? In the example given {'name': 'FISH', 'is_dry': False, 'price': 3.30} is an invalid item, but the output only indicates the error and not the object name \n Item is not part of inventory and is skipped. (type=value_error). This is not helpful when trying to determine which items has failed the validation. Additionally, I aim to log object information for future referencing.

  2. Does Pydantic provide the functionality to obtain valid and invalid items? As per my example, I have to manually append valid items to a separate list.

    Valid items

    [
    Food(name='XA_APPLE', is_dry=True, price=0.3),
    Food(name='XA_RICE', is_dry=True, price=0.5)
]

    Invalid items

    [
    Food(name='FISH', is_dry=False, price=3.30),
    Food(name='FISH', is_dry=False, price=3.30)
]

  3. Would there be a more pythonic way to rewrite this code using list comprehension with the try block?

    # The following statement uses list comprehension but
# stops executing when an exception is raise as there
# is no try block to catch the exception.
# foods = [Food(**food) for food in foods]

# Hence, I resulted in using the following piece of code.
for food in foods:

    try:
        valid_foods.append(Food(**food))

    except pydantic.ValidationError as err:
        print(err, '-'* 30, sep='\n', end='\n')

Code

Imports

import pydantic

Pydantic model with validator

class Food(pydantic.BaseModel):

    name: str
    is_dry: bool
    price: float

    @pydantic.validator('name')
    def validate_name(cls, value: str) -> str:

        if not value.startswith('XA_'):
            raise ValueError('Item is not part of inventory and is skipped.')

        return value

Load values and validate

# List is to simulate a loaded JSON structure.
foods = [
    {'name': 'XA_APPLE', 'is_dry': True, 'price': 0.30},
    {'name': 'FISH', 'is_dry': False, 'price': 3.30},
    {'name': 'XA_RICE', 'is_dry': True, 'price': 0.50},
    {'name': 'FISH', 'is_dry': False, 'price': 3.30},
]

valid_foods = []

for food in foods:

    try:
        valid_foods.append(Food(**food))

    except pydantic.ValidationError as err:
        print(err, '-'* 30, sep='\n')

print(valid_foods, '-'* 30, sep='\n')

Output

1 validation error for Food
name
  Item is not part of inventory and is skipped. (type=value_error)
------------------------------
1 validation error for Food
name
  Item is not part of inventory and is skipped. (type=value_error)
------------------------------
[Food(name='XA_APPLE', is_dry=True, price=0.3), Food(name='XA_RICE', is_dry=True, price=0.5)]
------------------------------
pydantic validation python-3.x python
1 Answers

You'll need an additional model:

class Foods(BaseModel):

    foods: list[Food]

    @validator('foods', each_item=True)
    def validator_logic(cls, v):
        if v.name.startswith('XA_'):
            return v
        else:
            log.error({'invalid': v})

Pydantic allows recursive models and validators allow validation per item passing the argument each_item, so:

  1. You could use the above validator block code where invalid itens are logged.
  2. I'm unaware of some method to set valid/invalids, I believe you must implement by yourself
  3. No need for that using the model above.
Vinícius OA0
Related