I am using celery in my django project.
2 issues, the autodiscover task doesn't discover my task under robovise/tasks.py so I had to explicitly pass the path of the tasks file in the include argument of Celery instance.
Then when I try calling the task in my django shell I get this error in Celery:
[2021-08-17 16:26:36,312: ERROR/MainProcess] Task robovise.tasks.process_new_users_resp[5763be00-77d4-4705-bb9a-6e06f2b03464] raised unexpected: RuntimeError("Model class robovise.models.RoboProcessedFiles doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.")
Traceback (most recent call last):
File "/Users/cyzanfar/Desktop/beagle/projects/financial_services/env/lib/python3.8/site-packages/celery/app/trace.py", line 450, in trace_task
R = retval = fun(*args, **kwargs)
File "/Users/cyzanfar/Desktop/beagle/projects/financial_services/env/lib/python3.8/site-packages/celery/app/trace.py", line 731, in __protected_call__
return self.run(*args, **kwargs)
File "/Users/cyzanfar/Desktop/beagle/projects/financial_services/financial_services/robovise/tasks.py", line 5, in process_new_users_resp
from robovise.models import RegisteredUser
File "/Users/cyzanfar/Desktop/beagle/projects/financial_services/financial_services/robovise/models.py", line 12, in <module>
class RoboProcessedFiles(models.Model):
File "/Users/cyzanfar/Desktop/beagle/projects/financial_services/env/lib/python3.8/site-packages/django/db/models/base.py", line 113, in __new__
raise RuntimeError(
RuntimeError: Model class robovise.models.RoboProcessedFiles doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
I've seen many post talking about this issue but have not found any solution. It's really strange as I have the same setup in another project and it works fine.
project structure:
├── financial_services
│ ├── __init__.py
│ ├── asgi.py
│ ├── celery.py
│ ├── settings
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── development.py
│ │ └── production.py
│ ├── urls.py
│ └── wsgi.py
├── manage.py
├── requirements.txt
└── robovise
├── __init__.py
├── admin.py
├── apps.py
├── constants.py
├── helpers.py
├── migrations
├── models.py
├── seed.py
├── services.py
├── tasks.py
├── tests.py
└── views.py
celery.py
from __future__ import absolute_import
from celery import Celery
# Set the default Django settings module for the 'celery' program.
# os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'financial_services.settings')
app = Celery('financial_services', include=['robovise.tasks'])
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')
# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
and here is robovise/tasks.py:
from celery import shared_task
@shared_task
def process_new_users_resp():
from robovise.models import RegisteredUser
print("IN TASK!!!")
RegisteredUser.fetch_and_update_resp()
return True
Also my settings.base.py:
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'robovise',
]
/robovise/apps.py
from django.apps import AppConfig
class RoboviseConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'robovise'
/robovise/models.py
from django.db import models
from .services import (
upload_req, read_res, read_primeid,
read_positions, read_costbasis, read_acct_value,
read_non_trade_trn, read_trade_trn
)
from django.db import transaction
from .tasks import process_new_users_resp
class RoboProcessedFiles(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
file_type = models.CharField(max_length=50, blank=False, null=False)
file_name = models.CharField(max_length=255, blank=False, null=False)
@classmethod
def processed_file(cls, file_type):
return RoboProcessedFiles.objects \
.filter(file_type=file_type) \
.order_by('-created_at').first()
...
Starting celery using:
celery -A financial_services worker --pool=gevent --concurrency=500 --loglevel=INFO
and then the python shell where I call a function that calls my task:
python manage.py shell -i ipython --settings=financial_services.settings