Celery jobs not running on heroku (python/django app)

I have a Django app setup with some scheduled tasks. The app is deployed on Heroku with Redis. The task runs if invoked synchronously in the console, or locally when I also have redis and celery running. However, the scheduled jobs are not running on Heroku.

My task:

@shared_task(name="send_emails")
def send_emails():
.....

celery.py:

from __future__ import absolute_import, unicode_literals

import os
from celery import Celery
from celery.schedules import crontab

# set the default Django settings module for the 'celery' program.
# this is also used in manage.py
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'my_app.settings')

# Get the base REDIS URL, default to redis' default
BASE_REDIS_URL = os.environ.get('REDIS_URL', 'redis://localhost:6379')

app = Celery('my_app')

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()

app.conf.broker_url = BASE_REDIS_URL

# this allows you to schedule items in the Django admin.
app.conf.beat_scheduler = 'django_celery_beat.schedulers.DatabaseScheduler'

# These are the scheduled jobs
app.conf.beat_schedule = {
    'send_emails_crontab': {
        'task': 'send_emails',
        'schedule': crontab(hour=9, minute=0),
        'args': (),
    }
}

In Procfile: worker: celery -A my_app worker --beat -S django -l info

I’ve spun up the worker with heroku ps:scale worker=1 -a my-app. I can see the registered tasks under [tasks] in the worker logs. However, the scheduled tasks are not running at their scheduled time. Calling send_emails.delay() in the production console does work. How do I get the worker to stay alive and / or run the job at the scheduled time?

I have a workaround using a command and heroku scheduler. Just unsure if that’s the best way to do it.

Answer

If you’re on free demo, you should know that heroku server sleeps and if your scheduled task becomes due when your server is sleeping, it won’t run.

Advertisement

Answer