Design Patterns for Integrating External Data with Django's Database-Backed Models

Viewed 136

I'm building a Django web application that maintains both relational data in a MySQL database as well as loosely coupled (by reference ID) full-text data in ElasticSearch (for optimized search). When a new object is created, I leverage Django signals + Celery tasks to asynchronously index the full-text data into Elasticsearch via elasticsearch-py, which works nicely. I had originally started with more tightly-coupled approaches (haystack and django-elasticsearch-dsl), but I quickly realized that I lost flexibility and decided to de-couple a bit.

Well, now I've come full circle as I begin to develop my views/templates (where I largely leverage Model Querysets and the like) and I'm faced with decisions on how to best "fold-in" my loosely-coupled external data. For example, if my homepage by default displays a list of the 50 most recent objects, but then a user conducts a full-text search, how do I best replace the objects (via Ajax) on the page with the ES search result while also linking corresponding database data? And then revert to the Model Queryset data when a filter is pressed and the search is cleared?

In other words, what are best practice implementations and design patterns for integrating external data with Django ORM/database-backed models? Is it a REST-based API layer that consolidates results from multiple sources? A view function? A classmethod on the Model that can dynamically query Elasticsearch? Something else?

0 Answers
Related