I have been working on this issue for about 4 days now, and it's been extremely frustrating. I would greatly appreciate it if somebody would answer this question for me. I am working on a Django Project that requires me to periodically check objects and some files in the media directory and delete them if outdated. After referring to the celery documentation , I set up everything properly (at least I think), and went ahead to run the program. I first ran my project on one terminal window, then ran a redis server using redis-server on another terminal window. Using a third terminal window, I ran the following:
(djangoenv) Daeyongs-Air:Data_Science_Project daeyong$ celery -A Data_Science_Project worker -l info
-------------- celery@Daeyongs-Air.fios-router.home v5.0.5 (singularity)
--- ***** -----
-- ******* ---- macOS-10.16-x86_64-i386-64bit 2021-01-02 02:57:54
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: Data_Science_Project:0x7f9b0e59f1f0
- ** ---------- .> transport: redis://localhost:6379//
- ** ---------- .> results: redis://localhost:6379/
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. Data_Science_Project.celery.test
. task_number_one
[2021-01-02 02:57:55,154: INFO/MainProcess] Connected to redis://localhost:6379//
[2021-01-02 02:57:55,175: INFO/MainProcess] mingle: searching for neighbors
[2021-01-02 02:57:56,203: INFO/MainProcess] mingle: all alone
[2021-01-02 02:57:56,224: WARNING/MainProcess] /Users/daeyong/opt/anaconda3/envs/djangoenv/lib/python3.8/site-packages/celery/fixups/django.py:203: UserWarning: Using settings.DEBUG leads to a memory
leak, never use this setting in production environments!
warnings.warn('''Using settings.DEBUG leads to a memory
[2021-01-02 02:57:56,224: INFO/MainProcess] celery@Daeyongs-Air.fios-router.home ready.
So I assumed that the worker was running properly. For my final step, on a fourth terminal window, I ran the following:
(djangoenv) Daeyongs-Air:Data_Science_Project daeyong$ celery -A Data_Science_Project beat -l info
celery beat v5.0.5 (singularity) is starting.
__ - ... __ - _
LocalTime -> 2021-01-02 03:00:02
Configuration ->
. broker -> redis://localhost:6379//
. loader -> celery.loaders.app.AppLoader
. scheduler -> celery.beat.PersistentScheduler
. db -> celerybeat-schedule
. logfile -> [stderr]@%INFO
. maxinterval -> 5.00 minutes (300s)
[2021-01-02 03:00:02,894: INFO/MainProcess] beat: Starting...
But even after waiting for a several minutes, this is all that showed up instead of a visual depiction of a task being ran. I have tried to look at multiple resources beforehand of course, to no avail. I'll paste the code from the files in question down below
celeryconfig.py:
broker_url = 'redis://localhost:6379'
result_backend = 'redis://localhost:6379'
accept_content = ['application/json']
result_serializer = 'json'
task_serializer = 'json'
timezone = 'UTC'
celery.py:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from . import celeryconfig
from . import settings
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'Data_Science_Project.settings')
app = Celery('Data_Science_Project')
app.config_from_object(celeryconfig)
@app.on_after_configure.connect
def setup_periodic_task(sender, **kwargs):
sender.add_periodic_task(10.0, task_number_one())
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
@app.task()
def test(arg):
print(arg)
tasks.py:
from __future__ import absolute_import, unicode_literals
from celery import shared_task
from celery.utils.log import get_task_logger
from django.utils import timezone
from django.conf import settings
from Data_Science_Web_App.models import TemporaryFile
import datetime
import os
logger = get_task_logger(__name__)
@shared_task(name="task_number_one")
def task_number_one():
logger.info("Task is running")
print("Task is running")
objects = TemporaryFile.objects.all()
for object in objects.iterator():
if object.time_created < datetime.datetime.now()-datetime.timedelta(minutes=1):
self_file = TemporaryFile.objects.get(pk = object.pk)
data = object.data
os.remove(os.path.join(MEDIA_ROOT, data))
os.remove(os.path.join(BASE_DIR, data))
self_file.delete()
init.py (this one is the init file for the project):
from __future__ import absolute_import, unicode_literals
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
__all__ = ['celery_app']
If you want me to show any other files, please comment. Any help would be greatly appreciated!