I am reading this article about chunking a large database operation. I am also using django-import-export and django-import-export-celery in my admin site and I would like to integrate chunking into them.
The problem I have is django-import-export already handles the file import, as well as the whole process of importing, in the background.
I tried using django-import-export's bulk imports, but one of the caveats is:
Bulk operations do not work with many-to-many relationships.
so chunking is what we thought was the alternative. Is it possible to perform chunking inside the django-import-export?
UPDATE - ADDED MODEL:
class Profile(models.Model):
firstname = models.CharField(max_length=200)
lastname = models.CharField(max_length=200, blank=True, null=True)
email = models.EmailField(max_length=200)
associated_issuer = models.ManyToManyField('app.Issuer', related_name='add_issuer', blank=True)
associated_profile = models.ManyToManyField('app.Profile', related_name='add_profile', blank=True)
class Issuer(models.Model):
# personal information
name = models.CharField(max_length=200, blank=True, null=True)
contact_email = models.EmailField(max_length=200, blank=True, null=True)
associated_profile = models.ManyToManyField('app.Profile', related_name='add_profile_2', blank=True)
associated_issuer = models.ManyToManyField('app.Issuer', related_name='add_issuer_2', blank=True)