Django - "psycopg2.errors.ForeignKeyViolation: insert or update on table" when running "python manage.py test", but not when running single test

Admin 1
Viewed 732

So I made another post regarding this issue and thought I'd break it down even more to see if the good people of StackOverflow can figure out what's going on, because I can't. So I generate all this test data below using setUpTestData and things look good. I put the debugger log below to show that the test database is populated with what is expected. When I get to the test test_correct_num_posts_generated the test itself passes, but I get this weird database error. I put the full error output below. I've gotten this over many tests. I heard it could be related to a Django bug where the test DB doesn't get properly torn down. This is a weird issue, because when I run python manage.py test cheers.test.ExploreTest it passes and python manage.py test cheers.test.ExploreTest.test_correct_num_posts_generated it passes, but when I run python manage.py test I get the error shown below. What is going on?

Test.py. PostFactory is a dummy data generator.

@classmethod
    def setUpTestData(cls) -> None:
        cls.num_posts = 45
        cls.health_cate = 'Health'
        cls.fitness_cate = 'Fitness'
        cls.relationship_cate = 'Relationship'
        cls.goal_categories_name_list = [cls.health_cate, cls.fitness_cate, cls.relationship_cate]

        cls.user = create_test_user_in_DB()
        cls.access_token = get_test_user_access_token()
        for name in cls.goal_categories_name_list:
            goal_category_obj = GoalCategory.objects.create(name=name, emoji='Some URL')
            goal_obj = Goal.objects.create(creator=cls.user, goal_category=goal_category_obj)
            if name == cls.relationship_cate:
                cls.relationship_join_goal = JoinGoal.objects.create(joiner=cls.user, goal=goal_obj,
                                                                     status=GoalStatus.ONGOING)
            else:
                JoinGoal.objects.create(joiner=cls.user, goal=goal_obj, status=GoalStatus.ONGOING)

        for i in range(cls.num_posts):
            PostFactory()

Debugger print out at the end of setUpTestData. This all gets printed as well when I get to test_correct_num_posts_generated. >>> is the lines that I input and below is the print out.

>>>User.objects.all()
<QuerySet [<User 6badb4b8-33ba-4bb9-aa9a-2e3afb359960>]>
>>>GoalCategory.objects.all()
<QuerySet [<GoalCategory: GoalCategory object (Health)>, <GoalCategory: GoalCategory object (Fitness)>, <GoalCategory: GoalCategory object (Relationship)>]>
>>>len(Post.objects.all())
45
>>>JoinGoal.objects.all()
<QuerySet [<JoinGoal e7c4e4fa-3592-4ad9-b93a-245694a5e384>, <JoinGoal 7d637523-67db-4c59-aea6-37a8b43f0dd3>, <JoinGoal cbb954b7-332e-47f4-82e6-d2c77d874be5>]>

Failing test in Test.py

def test_correct_num_posts_generated(self):
    self.assertTrue(len(Post.objects.all()), self.num_posts)

self.assertTrue(len(Post.objects.all()), self.num_posts) passes, but this test is throwing the notorious error I mentioned:

======================================================================
ERROR: test_correct_num_posts_generated (cheers.test.ExploreTests.ExploreFeedTest.ExploreFeedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\backends\utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.ForeignKeyViolation: insert or update on table "cheers_post" violates foreign key constraint "cheers_post_join_goal_id_da1e6957_fk_cheers_joingoal_uuid"
DETAIL:  Key (join_goal_id)=(5c05a7d2-ff3c-4f1e-95d9-277ae164dabb) is not present in table "cheers_joingoal".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\test\testcases.py", line 284, in _setup_and_call
    self._post_teardown()
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\test\testcases.py", line 1006, in _post_teardown
    self._fixture_teardown()
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\test\testcases.py", line 1248, in _fixture_teardown
    connections[db_name].check_constraints()
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\backends\postgresql\base.py", line 285, in check_constraints
    cursor.execute('SET CONSTRAINTS ALL IMMEDIATE')
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "C:\ProgramData\Anaconda3\envs\cheers\lib\site-packages\django\db\backends\utils.py", line 82, in _execute
    return self.cursor.execute(sql)
django.db.utils.IntegrityError: insert or update on table "cheers_post" violates foreign key constraint "cheers_post_join_goal_id_da1e6957_fk_cheers_joingoal_uuid"
DETAIL:  Key (join_goal_id)=(5c05a7d2-ff3c-4f1e-95d9-277ae164dabb) is not present in table "cheers_joingoal".
django python
2 Answers

Since test is passing on test cheers.test.ExploreTest but not on test, there is definitely some test from some other package messing up the data.

Not sure if this will solve it, but since you are using Django 3.2, there is a small note in the docs:

Changed in Django 3.2:

Objects assigned to class attributes in setUpTestData() must support creating deep copies with copy.deepcopy() in order to isolate them from alterations performed by each test methods. In previous versions of Django these objects were reused and changes made to them were persisted between test methods.

Copying model instance using copy.deepcopy() doesn't work as expected, additional step is required to create a new instance. I can't find it anywhere in the docs but I used code from this answer multiple times (and it works on Django 3.2 with Postgres):

from copy import deepcopy
old_obj = deepcopy(obj)
old_obj.id = None
old_obj.save()

If this is the case, I'd just go with setUp instead of setUpTestData, because it recreates test database before each test.
It is slower and you might have to change specific data for some tests, but it works great and is easier to maintain.

user_121155120
Admin1
Related