How to use Django QuerySet.union() in ModelAdmin.formfield_for_manytomany()?

user_4720018 0
Viewed 1991

Not sure what I am doing wrong here:

I tried to use QuerySet.union(), in Django 2.2.10, to combine two querysets (for the same model) inside ModelAdmin.formfield_for_manytomany(). However, when the form is saved, the entire queryset is selected, regardless of the actual selection made.

Please consider the minimal example below, based on the standard Django Article/Publication example.

from django.db import models
from django.contrib import admin


class Publication(models.Model):
    pass


class Article(models.Model):
    publications = models.ManyToManyField(to=Publication, blank=True)


class ArticleAdmin(admin.ModelAdmin):
    def formfield_for_manytomany(self, db_field, request, **kwargs):
        if db_field.name == 'publications':
            # the following query makes no sense, but it shows an attempt to
            # combine two separate QuerySets using QuerySet.union()
            kwargs['queryset'] = Publication.objects.all().union(
                Publication.objects.all())
        return super().formfield_for_manytomany(db_field, request, **kwargs)


admin.site.register(Publication)
admin.site.register(Article, ArticleAdmin)

The initial queryset for the publications field is filtered using formfield_for_manytomany, as described in the docs.

PLEASE NOTE: The actual query in this example makes no sense, it just returns everything, but that's not important: the point is that QuerySet.union() messes up the selection. It works normally if you remove the union().

Here's what happens when I add a new Article in the admin, without selecting any publications:

Before "Save" (nothing selected)

enter image description here

After "Save" (everything is selected)

enter image description here

No matter what I do, all options are automatically selected every time the form is saved.

Am I using QuerySet.union() the wrong way, or is this expected behavior, given the restrictions on querysets returned by QuerySet.union()?

django-queryset django-admin django python
2 Answers

As @tom-carrick pointed out, it appears that a QuerySet returned by QuerySet.union() cannot be filtered. I suppose this is implied by the following excerpt from the documentation:

In addition, only LIMIT, OFFSET, COUNT(*), ORDER BY, and specifying columns (i.e. slicing, count(), order_by(), and values()/values_list()) are allowed on the resulting QuerySet.

If you're using Django 3.0, calling filter() on the result of QuerySet.union() will raise an exception with a pretty clear message:

django.db.utils.NotSupportedError: Calling QuerySet.filter() after union() is not supported.

However, no exception is raised if you're using Django 2.2: In that case it just returns the complete queryset, regardless of the filter arguments. Here's a little test to illustrate that (in Django 2.2):

# using Django 2.2.10
class PublicationTests(TestCase):
    def test_union_filter(self):
        for i in range(2):
            Publication.objects.create()
        queryset_union = Publication.objects.filter(id=1).union(
            Publication.objects.filter(id=2))
        self.assertEqual(2, len(queryset_union))
        for obj in queryset_union.all():
            self.assertIn(obj, queryset_union.filter(id=1))
            self.assertIn(obj, queryset_union.filter())
            self.assertIn(obj, queryset_union.filter(id=0))

So, this must be what happens when we use QuerySet.union() to restrict a queryset in the ModelAdmin: The selection widget works as expected, but when the form is validated, filter() is called on the output of QuerySet.union() (see source for the ModelMultipleChoiceField), and that always returns the complete queryset, regardless of the actual subselection.

Depending on the actual use case, there may be ways around using union(), as explained in tom-carrick's answer.

However, there is at least one way to work around the restrictions imposed by QuerySet.union() in this situation, and that is to create a new queryset from the queryset-union:

Here's a modified version of the ArticleAdmin from the original example:

class ArticleAdmin(admin.ModelAdmin):
    def formfield_for_manytomany(self, db_field, request, **kwargs):
        if db_field.name == 'publications':
            queryset_union = Publication.objects.all().union(
                Publication.objects.all())
            kwargs['queryset'] = Publication.objects.filter(id__in=queryset_union)
        return super().formfield_for_manytomany(db_field, request, **kwargs)

Again, the actual query in this contrived example makes no sense, but that is not important here.

This might not be the most efficient solution in terms of database access.

user_47200180

The problem does seem to be .union(), though I can't figure out why. It seems like a bug, or at least funky behaviour.

Since you don't specify your actual use-case, it's hard to know, but for the example you give you can use the OR operator instead, which will work for that:

class ArticleAdmin(admin.ModelAdmin):
    def formfield_for_manytomany(self, db_field, request, **kwargs):
        if db_field.name == 'publications':
            # the following query makes no sense, but it shows an attempt to
            # combine two separate QuerySets using QuerySet.union()
            kwargs['queryset'] = (
                Publication.objects.filter(id__lt=3)
                | Publication.objects.filter(id__gt=2)
            )
        return super().formfield_for_manytomany(db_field, request, **kwargs)
user_6259100
Related