openedx.core.lib.celery package

openedx.core.lib.celery package#

Submodules#

openedx.core.lib.celery.routers module#

Custom routers used by both lms and cms when routing tasks to worker queues.

For more, see https://celery.readthedocs.io/en/latest/userguide/routing.html#routers

openedx.core.lib.celery.routers.ensure_queue_env(desired_env)#

Helper method to get the desired type of queue.

If no such queue is defined, default routing logic is used.

openedx.core.lib.celery.routers.route_task(name, args, kwargs, options, task=None, **kw)#

Celery-defined method allowing for custom routing logic.

If None is returned from this method, default routing logic is used.

openedx.core.lib.celery.task_utils module#

Middleware utilities

openedx.core.lib.celery.task_utils.emulate_http_request(site=None, user=None, middleware_classes=None)#

Generate a fake HTTP request and run selected middleware on it.

This is used to enable features that assume they are running as part of an HTTP request handler. Many of these features retrieve the “current” request from a thread local managed by crum. They will make a call like crum.get_current_request() or something similar.

Since some tasks are kicked off by a management commands (which does not have an HTTP request) and then executed in celery workers there is no “current HTTP request”. Instead we just populate the global state that is most commonly used on request objects.

Parameters:

  • site (Site) – The site that this request should emulate. Defaults to None.

  • user (User) – The user that initiated this fake request. Defaults to None

  • middleware_classes (list) – A list of classes that implement Django’s middleware interface. Defaults to [CurrentRequestUserMiddleware, CurrentSiteThemeMiddleware] if None.

Module contents#

Instantiate the singleton Celery instance that is used by either lms or cms.

WARNING: Do not import this module directly!

This module should only be imported by cms.celery and lms.celery, which perform setup in a particular order before and after Celery is instantiated. Otherwise, it might be possible for the Celery singleton to be created without variant- specific configuration.

The module is intended as a way to have a Celery singleton shared between cms and lms code. Not having a guaranteed singleton means that it is possible for each of cms and lms to instantiate Celery, and tasks may be nondeterministically registered to one or the other of the instances and therefore sometimes lost to the running process. The root ``__init__.py``s of cms and lms both ensure that this module is loaded when any cms or lms code runs, effectively using the Python module system as a singleton loader. (This is an incremental improvement over older code, and there is probably a better mechanism to be had.)