DRF dynamic filtering

For parsing a query string like:

string = "((num_of_pages gt 20) OR (num_of_pages lt 10)) AND (date gt '2016-05-01')"

you can use the pyparsing package (not an expert but very powerful library) with django Q objects:

1. parsing code:

import pyparsing as pp
import operator as op

from django.db.models import Q

word = pp.Word(pp.alphas, pp.alphanums + "_-*'")
operator = pp.oneOf('lt gt eq').setResultsName('operator')
number = pp.pyparsing_common.number()
quoted = pp.quotedString().setParseAction(pp.removeQuotes)
term = (word | number | quoted)

key = term.setResultsName('key')
value = term.setResultsName('value')

group = pp.Group(key + operator + value)

def q_item(item):
    """Helper for create django Q() object"""
    k = f'{item.key}__{item.operator}'
    v = item.value
    return Q(**{k: v})


class BaseBinary:

    def __init__(self, tokens):
        self.args = tokens[0][0::2]

    def __repr__(self):
        return f'{self.__class__.__name__}({self.symbol}):{self.args}'

    def evaluate(self):
        a = q_item(self.args[0]) if not isinstance(self.args[0], BaseBinary) else self.args[0].evaluate()
        b = q_item(self.args[1]) if not isinstance(self.args[1], BaseBinary) else self.args[1].evaluate()
        return self.op(a, b)


class BoolNotOp(BaseBinary):
    symbol = 'NOT'
    op = op.not_

    def __init__(self, tokens):
        super().__init__(tokens)
        self.args = tokens[0][1]

    def evaluate(self):
        a = q_item(self.args) if not isinstance(self.args, BaseBinary) else self.args.evaluate()
        return ~a


class BoolAndOp(BaseBinary):
    symbol = 'AND'
    op = op.and_


class BoolOrOp(BaseBinary):
    symbol = 'OR'
    op = op.or_


expr = pp.infixNotation(group,
                        [('NOT', 1, pp.opAssoc.RIGHT, BoolNotOp),
                         ('AND', 2, pp.opAssoc.LEFT, BoolAndOp),
                         ('OR', 2, pp.opAssoc.LEFT, BoolOrOp)])

Now given a string like:

string = "(date gt '2016-05-01') AND ((num_of_pages gt 20) OR (num_of_pages lt 10))"

to the parser:

parser = expr.parseString(string)[0]
print(parser.evaluate())

give us our Q objects:

(AND: ('date__gt', '2016-05-01'), (OR: ('num_of_pages__gt', 20), ('num_of_pages__lt', 10)))

ready to be filtered

class Book(models.Model):
    title = models.CharField(max_length=200)
    counter = models.PositiveIntegerField(default=0)
    date = models.DateField(auto_now=True)
    num_of_pages = models.PositiveIntegerField(default=0)
    
qs = Book.objects.filter(parser.evaluate())
print(qs.query)

SELECT "core_book"."id", "core_book"."title", "core_book"."counter", "core_book"."date", "core_book"."num_of_pages" FROM "core_book" WHERE ("core_book"."date" > 2016-05-01 AND ("core_book"."num_of_pages" > 20 OR "core_book"."num_of_pages" < 10))

P.S not fully tested.

I currently use the Q object extensively in a project I have that is using the users get parameters to filter a search result.

Here is a snippet

some_initial_query_object = Model.objects.all()

qs_result_dates = []

qs_result_dates.append(
    Q(
        event_date__start_date_time__gte='2021-08-01',
        event_date__start_date_time__lt='2021-09-01' + datetime.timedelta(days=1)
      )
)

some_initial_query_object = some_initial_query_object.filter(qs_result_dates)

In your scenario you can use | for OR and & for AND

Q(date='2016-05-01')
&
Q(number_of_calories__gt=20, number_of_calories__lt=10)

Here is an example of dynamic filtering using DRF by overriding the get_queryset method on the ModelViewSet that I use in all of my projects. Using this method I can leverage the full power of Django-ORM framework from the frontend.

views.py

def BaseAPIView(...):

    ''' base view for other views to inherit '''

    def get_queryset(self):

        queryset = self.queryset

        # get filter request from client:
        filter_string = self.request.query_params.get('filter')

        # apply filters if they are passed in:
        if filters: 
            filter_dictionary = json.loads(filter_string)
            queryset = queryset.filter(**filter_dictionary)

        return queryset

The request url will now look like, for example: my_website.com/api/users?filter={"first_name":"John"}

Which can be built like:

script.js

// using ajax as an example:
var filter = JSON.stringify({
  "first_name" : "John"
});

$.ajax({
   "url" : "my_website.com/api/users?filter=" + filter,
   "type" : "GET",
   ...
});

Some advantages:

• no need to specify which fields can be filtered on each view class
• write it once, use it everywhere
• front end filtering looks exactly like django filtering
• can do the same with exclude

Some disadvantages:

• potential security risks if you want some fields to be non-filterable
• less intuitive front-end code to query a table

Overall, this approach has been far more useful for me than any packages out there.