Bulk create auto fields in Django

user_6896234 0
Viewed 989

I have two models and I want to migrate all data from one to another. For simulating the problem, suppose the models below:

from django.db import models

class Book:
    name = models.CharField(max_length=100)
    is_archived = models.BooleanField(default=False)
    created_at = models.DateTimeField(auto_now_add=True)


class BookArchived:
    name = models.CharField(max_length=100)
    created_at = models.DateTimeField(auto_now_add=True)

The code for migrations is:

book_objs = []
for archived_book in BookArchived.objects.all():
    book_objs.append(Book(name=archived_book.name, is_archived=True, created_at=archived_book.created_at))

Book.objects.bulk_create(book_objs)

The problem with this code is that, although I explicitly set created_at field (created_at=archived_book.created_at), Django inserts all of them with current time.

I know that if pk field is explicitly set (id=archived_book.id), new objects will preserve original timestamps. But there is already overlapping IDs in database, so this approach will create another problem. How can I preserve original created_at values for BookArchived instances?

django python
2 Answers

I found this gem https://stackoverflow.com/a/59898220/519995 which should work, but I haven't tested it myself.


from contextlib import contextmanager

@contextmanager
def suppress_auto_now(model, field_names):
    """
    From https://stackoverflow.com/a/59898220/519995
    idea taken here https://stackoverflow.com/a/35943149/1731460
    """
    fields_state = {}
    for field_name in field_names:
        field = model._meta.get_field(field_name)
        fields_state[field] = {'auto_now': field.auto_now, 'auto_now_add': field.auto_now_add}

    for field in fields_state:
        field.auto_now = False
        field.auto_now_add = False
    try:
        yield
    finally:
        for field, state in fields_state.items():
            field.auto_now = state['auto_now']
            field.auto_now_add = state['auto_now_add']

Use it like this:

with suppress_autotime(Book, ['created_at']):
        Book.objects.bulk_create(book_objs)

NOTE: Do NOT use this context manager in your views/forms or anywhere in your Django app. This context manager alter internal state of field (by temporarily setting auto_now and auto_now_add to False). That will cause Django to not populate these fields with timezone.now() during execution of context manager's body for concurrent requests (ie. same process, different thread). Although this can be used for standalone scripts (ex. management commands, data migration) which are not run in the same process with Django app.

user_5199950

As per a note in the documentation of the DateField:

As currently implemented, setting auto_now or auto_now_add to True will cause the field to have editable=False and blank=True set.

Since editable will be set to False you providing any value to the model will not work. You will need to do this step in multiple migrations.

Firstly have the model without auto_now_add:

class Book(models.Model):
    name = models.CharField(max_length=100)
    is_archived = models.BooleanField(default=False)
    created_at = models.DateTimeField()

Generate a migration for this using python manage.py makemigrations. Next we want to do a data migration [Django docs]. First run python manage.py makemigrations --empty <yourappname> this will create an empty migration which you will edit to copy data from BookArchived to Book. This would look something like:

from django.db import migrations

def copy_legacy_books(apps, schema_editor):
    # We can't import the Person model directly as it may be a newer
    # version than this migration expects. We use the historical version.
    Book = apps.get_model('yourappname', 'Book')
    BookArchived = apps.get_model('yourappname', 'BookArchived')
    book_objs = []
    for archived_book in BookArchived.objects.all():
        book_objs.append(Book(name=archived_book.name, is_archived=True, created_at=archived_book.created_at))
    
    Book.objects.bulk_create(book_objs)

class Migration(migrations.Migration):

    dependencies = [
        ('yourappname', '0001_initial'),
    ]

    operations = [
        migrations.RunPython(copy_legacy_books),
    ]

Now after this simply add the auto_now_add kwarg to your the date-time field and generate another migration using python manage.py makemigrations:

class Book:
    name = models.CharField(max_length=100)
    is_archived = models.BooleanField(default=False)
    created_at = models.DateTimeField(auto_now_add=True)
user_149918640
Related