Celery is not loading tasks from one app out of five

I’m using autodiscover in my celery.py file for gathering tasks. Which up until a recently picked all of the app.tasks.py up, I’m not sure why but my config.tasks.py function are no longer being picked up, but all the other apps are. If I from “config.tasks import *” there are no errors and I can run the tasks manually through shell.

ive tried using force=True on autodiscover which had no effect, and a number of other solutions, none of which seem to have any effect, does anyone have any ideas on what to check next?

Thanks

Structure:

-config
--apps.py
--views.py
--models.py
--tasks.py
-monitoring
--apps.py
--views.py
--models.py
--tasks.py
-app
--apps.py
--settings.py
--celery.py

celery config

from __future__ import absolute_import, unicode_literals
import os, django
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')

django.setup()  # This is key

app = Celery("app")
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()

if __name__ == '__main__':
    app.start()

settings.py

CELERY_TIMEZONE = 'Europe/London'
ENABLE_UTC = True
CELERY_BROKER_URL = 'redis://redis:6379'
CELERY_RESULT_BACKEND = 'redis://redis:6379'
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'
CELERY_TASK_TIME_LIMIT = 540

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.humanize',
    'haystack',
    'django_tables2',
    'django_filters',
    'django_celery_beat',
    'config.apps.ConfigConfig',
    'monitoring.apps.MonitoringConfig',
    'storages',
    'stdimage',
    'simple_history',
    'crispy_forms',
    'rest_framework',
    'rest_framework.authtoken',
    )

tasks.py

from __future__ import absolute_import, unicode_literals
import requests, re, json, ipaddress, time, meraki, logging, random, gzip, diffios
from celery import chord, group
from app.celery import app
from django.conf import settings
from config.models import * 


@app.task
def execute_command(device_id, cmd, user_id):
    ...
    return output

console output

celery_1   |  -------------- celery@3504f0d9bb2e v4.3.1 (rhubarb)
celery_1   | ---- **** -----
celery_1   | --- * ***  * -- Linux-4.19.121-linuxkit-x86_64-with-debian-9.7 2021-02-09 12:14:41
celery_1   | -- * - **** ---
celery_1   | - ** ---------- [config]
celery_1   | - ** ---------- .> app:         app:0x7f82527f04a8
celery_1   | - ** ---------- .> transport:   redis://redis:6379//
celery_1   | - ** ---------- .> results:     redis://redis:6379/
celery_1   | - *** --- * --- .> concurrency: 30 (prefork)
celery_1   | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
celery_1   | --- ***** -----
celery_1   |  -------------- [queues]
celery_1   |                 .> celery           exchange=celery(direct) key=celery
celery_1   |
celery_1   |
celery_1   | [tasks]
celery_1   |   . app_settings.tasks.import_meraki_templates
celery_1   |   . celery.accumulate
celery_1   |   . celery.backend_cleanup
celery_1   |   . celery.chain
celery_1   |   . celery.chord
celery_1   |   . celery.chord_unlock
celery_1   |   . celery.chunks
celery_1   |   . celery.group
celery_1   |   . celery.map
celery_1   |   . celery.starmap
celery_1   |   . app.celery.debug_task
celery_1   |   . monitoring.tasks.link_checks
celery_1   |   . monitoring.tasks.service_checks

Answer

I rebuilt my docker container and now they’re all loading again, not sure what would of caused this but its working!

Advertisement

Answer