openedx.core.djangoapps.crawlers package

Submodules

openedx.core.djangoapps.crawlers.models module

This module handles the detection of crawlers, so that we can handle them appropriately in other parts of the code.

class openedx.core.djangoapps.crawlers.models.CrawlersConfig(*args, **kwargs)

Bases: ConfigurationModel

Configuration for the crawlers django app.

exception DoesNotExist

Bases: ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: MultipleObjectsReturned

change_date

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

changed_by

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

changed_by_id

enabled

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_change_date(*, field=<django.db.models.fields.DateTimeField: change_date>, is_next=True, **kwargs)

get_previous_by_change_date(*, field=<django.db.models.fields.DateTimeField: change_date>, is_next=False, **kwargs)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

classmethod is_crawler(request)

Determine if the request came from a crawler or not.

This method is simplistic and only looks at the user agent header at the moment, but could later be improved to be smarter about detection.

known_user_agents

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

Module contents