How to deter recursion in Flask JSON output for many-to-many relationships?

user6HFH 0
Viewed 26

The error is clear:

RecursionError: maximum recursion depth exceeded while calling a Python object

A model cycles through its properties, including its relationships, outputs the properties. The relationships have a backref, so it's an endless recursion cycle.

Example

Consider an Author describing its Books. During the formatting (default method), the Author model says, "is the object a Book?" If so, it asks Book to serialize itself. In other examples, the Author might hardcode the Book's key/value pairs instead of asking Book to describe itself. I'd like to avoid that as I want to reduce the amount of awareness one model has of another.

Is there a way to track/pass what level is being called?

What I'd prefer is to track the recursion level, such that

book = Book()
book.to_json

Will display something like

{
  "id": 1,
  "name": "Python on Stack Overflow",
  "authors": [
    {
      "id": 300,
      "name": "Mike",
      "books": [
        { "id": 1, "name": "Python on Stack Overflow", "authors": ["<Author id=300>"] },
        { "id": 2, "name": "The Worst Question Ever Asked", "authors": ["<Author id=100>", "<Author id=200>", "<Author id=300>", "<Author id=400>"] },
        { "id": 3, "name": "The Greatest Question Ever Answered", "authors": ["<Author id=300>", "<Author id=400>"] },
      ]
    },
    ...
  ]
}

Don't ask Book to describe its authors if Book calls Author calling Book (greater than 1 level deep).

Models

Disclaimer: This is a limited example and don't include imports or other attributes, methods, mixing, or functions.

Book.py

# models/book.py

def default(object):
  # format dates
  if isinstance(object, (date, datetime)):
    return object.strftime('%Y-%m-%d %H:%M %z')

  # Call 'Author' to serialize itself
  if object.__class__.__name__ == 'Author':               # <-- one place to be call-aware; `and level==1`
    return object.to_json

  # instance display
  return f'<{object.__class__.__name__} id={object.id}>'


class Book(db.Model):
  id = db.Column(db.Integer, primary_key=True)
  name = db.Column(db.Text, index=True, unique=True, nullable=False)
  authors = db.relationship('Author', secondary=Published.__table__, back_populates='authors')

  @property
  def to_json(self):
    columns = self.keys()

    response = {}
    for column in columns:
      response[column] = getattr(self, column)

    return json.loads(json.dumps(response, default=default))

Author.py

# models/author.py

def default(object):
  # format dates
  if isinstance(object, (date, datetime)):
    return object.strftime('%Y-%m-%d %H:%M %z')

  # Call 'Book' to serialize itself
  if object.__class__.__name__ == 'Book':                 # <-- one place to be call-aware; `and level==1`
    return object.to_json

  # instance display
  return f'<{object.__class__.__name__} id={object.id}>'


class Author(db.Model):
  id = db.Column(db.Integer, primary_key=True)
  name = db.Column(db.Text, index=True, unique=True, nullable=False)
  books = db.relationship('Book', secondary=Published.__table__, back_populates='authors')

  @property
  def to_json(self):
    columns = self.keys()

    response = {}
    for column in columns:
      response[column] = getattr(self, column)

    return json.loads(json.dumps(response, default=default))
flask-sqlalchemy flask python-3.x python
1 Answers

One potential solution is to use a global tracking variable.

recursion_level = None

def default(object):
  global recursion_level

  # format dates
  if isinstance(object, (date, datetime)):
    return object.strftime('%Y-%m-%d %H:%M %z')

  # ask object to serialize itself
  max_recursion = 1
  classes = [ model.class_.__name__ for model in app.db.Model.registry.mappers ]
  if object.__class__.__name__ in classes and recursion_level < max_recursion:
    recursion_level += 1
    json_str = object.to_json  
    recursion_level -= 1
    return json_str

  # instance display
  return f'<{object.__class__.__name__} id={object.id}>'


class Author(db.Model):
  id = db.Column(db.Integer, primary_key=True)
  name = db.Column(db.Text, index=True, unique=True, nullable=False)
  books = db.relationship('Book', secondary=Published.__table__, back_populates='authors')

  @property
  def to_json(self):
    columns = self.keys()

    response = {}
    for column in columns:
      response[column] = getattr(self, column)

    global recursion_level                               # <-- new block
    if recursion_level is None:
      recursion_level = 1
    
    # NOTE: I don't know how to pass `recursion_level` to `default`,
    #   which is why it's a global variable for now
    return json.loads(json.dumps(response, default=default))

Comment:

  1. to_json and default are actually defined in one place on a base model class to keep the code DRY. Try not to be distracted by the placement here.
  2. Even though this answer uses global variables, it is not my preference. Python is supposedly single-threaded so it might be safe enough if not using async, but since I'm new to Python and don't fully understand the call stack or the scoping of globals. I defer to experts to poke the holes.
  3. My preference is to pass a variable to default for the recursion level, used as the recursive terminating condition. I'm not sure how to pass the value in json.dumps
  4. object.id is used in default's instance display output, but because the function may handle multiple classes (and not just Book), those classes may not include an id column. A more robust solution is to survey the primary keys and use those values. Something like: 
    pks = object.__table__.primary_key.columns.values()
pk_pairs = [ f'{pk.name}={object[pk.name]}' for pk in pks ]

return f'<{object.__class__.name} {" ".join(pk_pairs)}>'
    NOTE: this all depends on how much control over your models and priamry keys you have. This could be made even safer, but for the purpose of this demo, this should suffice.
user6HFH0
Related