lms.djangoapps.experiments package

Contents

lms.djangoapps.experiments package#

Submodules#

lms.djangoapps.experiments.apps module#

class lms.djangoapps.experiments.apps.ExperimentsConfig(app_name, app_module)#

Bases: AppConfig

Application Configuration for experiments.

name = 'lms.djangoapps.experiments'#

lms.djangoapps.experiments.factories module#

lms.djangoapps.experiments.filters module#

Experimentation filters

class lms.djangoapps.experiments.filters.ExperimentDataFilter(data=None, queryset=None, *, request=None, prefix=None)#

Bases: FilterSet

class Meta#

Bases: object

fields = ['experiment_id', 'key']#
model#

alias of ExperimentData

base_filters = {'experiment_id': <django_filters.filters.NumberFilter object>, 'key': <django_filters.filters.CharFilter object>}#
declared_filters = {}#
class lms.djangoapps.experiments.filters.ExperimentKeyValueFilter(data=None, queryset=None, *, request=None, prefix=None)#

Bases: FilterSet

class Meta#

Bases: object

fields = ['experiment_id', 'key']#
model#

alias of ExperimentKeyValue

base_filters = {'experiment_id': <django_filters.filters.NumberFilter object>, 'key': <django_filters.filters.CharFilter object>}#
declared_filters = {}#

lms.djangoapps.experiments.flags module#

Feature flag support for experiments

class lms.djangoapps.experiments.flags.ExperimentWaffleFlag(flag_name, module_name, num_buckets=2, experiment_id=None, use_course_aware_bucketing=True, **kwargs)#

Bases: CourseWaffleFlag

ExperimentWaffleFlag handles logic around experimental bucketing and whitelisting.

You’ll have one main flag that gates the experiment. This allows you to control the scope of your experiment and always provides a quick kill switch.

But you’ll also have smaller related flags that can force bucketing certain users into specific buckets of your experiment. Those can be set using a waffle named like “main_flag.BUCKET_NUM” (e.g. “course_experience.animated_exy.0”) to force users that pass the first main waffle check into a specific bucket experience.

If a user is not forced into a specific bucket by one of the aforementioned smaller flags, then they will be randomly assigned a default bucket based on a consistent hash of:

  • (flag_name, course_key, username) if use_course_aware_bucketing=True, or

  • (flag_name, username) if use_course_aware_bucketing=False.

Note that you may call .get_bucket and .is_enabled without a course_key, in which case: * the smaller flags will be evaluated without course context, and * the default bucket will be evaluated as if use_course_aware_bucketing=False.

You can also control whether the experiment only affects future enrollments by setting an ExperimentKeyValue model object with a key of ‘enrollment_start’ to the date of the first enrollments that should be bucketed.

Bucket 0 is assumed to be the control bucket.

See a HOWTO here: https://openedx.atlassian.net/wiki/spaces/AC/pages/1250623700/Bucketing+users+for+an+experiment

When writing tests involving an ExperimentWaffleFlag you must not use the override_waffle_flag utility. That will only turn the experiment on or off and won’t override bucketing. Instead use override_experiment_waffle_flag function which will do both. Example:

from lms.djangoapps.experiments.testutils import override_experiment_waffle_flag with @override_experiment_waffle_flag(MY_EXPERIMENT_WAFFLE_FLAG, active=True, bucket=1):

…

or as a decorator:

@override_experiment_waffle_flag(MY_EXPERIMENT_WAFFLE_FLAG, active=True, bucket=1) def test_my_experiment(self):

…

get_bucket(course_key=None, track=True)#

Return which bucket number the specified user is in.

The user may be force-bucketed if matching subordinate flags of the form “main_flag.BUCKET_NUM” exist. Otherwise, they will be hashed into a default bucket based on their username, the experiment name, and the course-run key.

If self.use_course_aware_bucketing is False, the course-run key will be omitted from the hashing formula, thus making it so a given user has the same default bucket across all course runs; however, subordinate flags that match the course-run key will still apply.

If course_key argument is omitted altogether, then subordinate flags will be evaluated outside of the course-run context, and the default bucket will be calculated as if self.use_course_aware_bucketing is False.

Finally, Bucket 0 is assumed to be the control bucket and will be returned if the experiment is not enabled for this user and course.

Parameters:

  • course_key (Optional[CourseKey]) – This argument should always be passed in a course-aware context even if course aware bucketing is False.

  • track (bool) – Whether an analytics event should be generated if the user is bucketed for the first time.

Returns: int

is_enabled(course_key=None)#

Return whether the requesting user is in a nonzero bucket for the given course.

See the docstring of .get_bucket for more details.

Parameters:

course_key (Optional[CourseKey])

Returns: bool

is_experiment_on(course_key=None)#

Return whether the overall experiment flag is enabled for this user.

This disregards .bucket_flags.

lms.djangoapps.experiments.models module#

Experimentation models

class lms.djangoapps.experiments.models.ExperimentData(*args, **kwargs)#

Bases: TimeStampedModel

ExperimentData stores user-specific key-values associated with experiments identified by experiment_id. .. no_pii:

exception DoesNotExist#

Bases: ObjectDoesNotExist

exception MultipleObjectsReturned#

Bases: MultipleObjectsReturned

created#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

experiment_id#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*, field=<model_utils.fields.AutoCreatedField: created>, is_next=True, **kwargs)#
get_next_by_modified(*, field=<model_utils.fields.AutoLastModifiedField: modified>, is_next=True, **kwargs)#
get_previous_by_created(*, field=<model_utils.fields.AutoCreatedField: created>, is_next=False, **kwargs)#
get_previous_by_modified(*, field=<model_utils.fields.AutoLastModifiedField: modified>, is_next=False, **kwargs)#
id#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

key#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

modified#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>#
user#

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

user_id#
value#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class lms.djangoapps.experiments.models.ExperimentKeyValue(*args, **kwargs)#

Bases: TimeStampedModel

ExperimentData stores any generic key-value associated with experiments identified by experiment_id.

exception DoesNotExist#

Bases: ObjectDoesNotExist

exception MultipleObjectsReturned#

Bases: MultipleObjectsReturned

created#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

experiment_id#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*, field=<model_utils.fields.AutoCreatedField: created>, is_next=True, **kwargs)#
get_next_by_modified(*, field=<model_utils.fields.AutoLastModifiedField: modified>, is_next=True, **kwargs)#
get_previous_by_created(*, field=<model_utils.fields.AutoCreatedField: created>, is_next=False, **kwargs)#
get_previous_by_modified(*, field=<model_utils.fields.AutoLastModifiedField: modified>, is_next=False, **kwargs)#
history = <django.db.models.manager.HistoryManagerFromHistoricalQuerySet object>#
id#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

key#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

modified#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>#
save_without_historical_record(*args, **kwargs)#

Save the model instance without creating a historical record.

Make sure you know what you’re doing before using this method.

value#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

lms.djangoapps.experiments.permissions module#

Experimentation permissions

class lms.djangoapps.experiments.permissions.IsStaffOrOwner#

Bases: IsStaffOrOwner

Permission that allows access to admin users or the owner of an object. The owner is considered the User object represented by obj.user.

has_permission(request, view)#

Return True if permission is granted, False otherwise.

class lms.djangoapps.experiments.permissions.IsStaffOrReadOnly#

Bases: BasePermission

has_permission(request, view)#

Return True if permission is granted, False otherwise.

class lms.djangoapps.experiments.permissions.IsStaffOrReadOnlyForSelf#

Bases: BasePermission

Grants access to staff or to user reading info about their own user

has_permission(request, view)#

Return True if permission is granted, False otherwise.

lms.djangoapps.experiments.routers module#

Experimentation routers

class lms.djangoapps.experiments.routers.DefaultRouter(*args, **kwargs)#

Bases: DefaultRouter

routes = [('^{prefix}{trailing_slash}$', {'get': 'list', 'post': 'create', 'put': 'create_or_update'}, '{basename}-list', False, {'suffix': 'List'}), ('^{prefix}/{lookup}{trailing_slash}$', '{basename}-list', False, {}), ('^{prefix}/{lookup}{trailing_slash}$', {'delete': 'destroy', 'get': 'retrieve', 'patch': 'partial_update', 'put': 'update'}, '{basename}-detail', True, {'suffix': 'Instance'}), ('^{prefix}/{lookup}{trailing_slash}$', '{basename}-detail', True, {})]#

lms.djangoapps.experiments.serializers module#

Experimentation serializers

class lms.djangoapps.experiments.serializers.ExperimentDataCreateSerializer(*args, **kwargs)#

Bases: ModelSerializer

class Meta#

Bases: object

fields = ('id', 'experiment_id', 'user', 'key', 'value', 'created', 'modified')#
model#

alias of ExperimentData

class lms.djangoapps.experiments.serializers.ExperimentDataSerializer(*args, **kwargs)#

Bases: ModelSerializer

class Meta#

Bases: Meta

read_only_fields = ('user',)#
class lms.djangoapps.experiments.serializers.ExperimentKeyValueSerializer(*args, **kwargs)#

Bases: ModelSerializer

class Meta#

Bases: object

fields = ('id', 'experiment_id', 'key', 'value', 'created', 'modified')#
model#

alias of ExperimentKeyValue

lms.djangoapps.experiments.stable_bucketing module#

An implementation of a stable bucketing algorithm that can be used to reliably group users into experiments.

An implementation of this is available as a standalone command-line tool, scripts/stable_bucketer, which can both validate the bucketing of a username and generate recognizable usernames for particular experiment buckets for testing.

lms.djangoapps.experiments.stable_bucketing.stable_bucketing_hash_group(group_name, group_count, user)#

Return the bucket that a user should be in for a given stable bucketing assignment.

This function has been verified to return the same values as the stable bucketing functions in javascript and the master experiments table.

Parameters:

  • group_name – The name of the grouping/experiment.

  • group_count – How many groups to bucket users into.

  • user – The user being bucketed.

lms.djangoapps.experiments.urls module#

Experimentation URLs

lms.djangoapps.experiments.utils module#

Utilities to facilitate experimentation

For an authenticated user, return a link to allow them to upgrade in the specified course.

Returns the upgrade link and upgrade deadline for a user in a given course given that the user is within the window to upgrade defined by our dynamic pacing feature; otherwise, returns None for both the link and date.

lms.djangoapps.experiments.utils.get_base_experiment_metadata_context(course, user, enrollment, user_enrollments)#

Return a context dictionary with the keys used by dashboard_metadata.html and user_metadata.html

lms.djangoapps.experiments.utils.get_course_entitlement_price_and_sku(course)#

Get the entitlement price and sku from this course. Try to get them from the first non-expired, verified entitlement that has a price and a sku. If that doesn’t work, fall back to the first non-expired, verified course run that has a price and a sku.

lms.djangoapps.experiments.utils.get_dashboard_course_info(user, dashboard_enrollments)#

Given a list of enrollments shown on the dashboard, return a dict of course ids and experiment info for that course

lms.djangoapps.experiments.utils.get_experiment_user_metadata_context(course, user)#

Return a context dictionary with the keys used for Optimizely experiments, exposed via user_metadata.html: view from the DOM in those calling views using: JSON.parse($(“#user-metadata”).text()); Most views call this function with both parameters, but student dashboard has only a user

lms.djangoapps.experiments.utils.get_program_context(course, user_enrollments)#

Return a context dictionary with program information.

lms.djangoapps.experiments.utils.get_program_price_and_skus(courses)#

Get the total program price and purchase skus from these courses in the program

lms.djangoapps.experiments.utils.get_unenrolled_courses(courses, user_enrollments)#

Given a list of courses and a list of user enrollments, return the courses in which the user is not enrolled. Depending on the enrollments that are passed in, this method can be used to determine the courses in a program in which the user has not yet enrolled or the courses in a program for which the user has not yet purchased a certificate.

lms.djangoapps.experiments.utils.is_enrolled_in_all_courses(courses, user_enrollments)#

Determine if the user is enrolled in all of the courses

lms.djangoapps.experiments.utils.is_enrolled_in_course(course, enrollment_course_ids)#

Determine if the user is enrolled in this course

lms.djangoapps.experiments.utils.is_enrolled_in_course_run(course_run, enrollment_course_ids)#

Determine if the user is enrolled in this course run

lms.djangoapps.experiments.views module#

Experimentation views

class lms.djangoapps.experiments.views.ExperimentCrossDomainSessionAuth#

Bases: SessionAuthenticationAllowInactiveUser, SessionAuthenticationCrossDomainCsrf

Session authentication that allows inactive users and cross-domain requests.

class lms.djangoapps.experiments.views.ExperimentDataViewSet(**kwargs)#

Bases: ModelViewSet

authentication_classes = (<class 'edx_rest_framework_extensions.auth.jwt.authentication.JwtAuthentication'>, <class 'lms.djangoapps.experiments.views.ExperimentCrossDomainSessionAuth'>)#
basename = None#
create_or_update(request, *args, **kwargs)#
description = None#
detail = None#
filter_backends = (<class 'django_filters.rest_framework.backends.DjangoFilterBackend'>,)#
filter_queryset(queryset)#

Given a queryset, filter it with whichever filter backend is in use.

You are unlikely to want to override this method, although you may need to call it either from a list view, or from a custom get_object method if you want to apply the configured filtering backend to the default queryset.

filterset_class#

alias of ExperimentDataFilter

get_serializer_class()#

Return the class to use for the serializer. Defaults to using self.serializer_class.

You may want to override this if you need to provide different serializations depending on the incoming request.

(Eg. admins get full serialization, others get basic serialization)

name = None#
permission_classes = (<class 'rest_framework.permissions.IsAuthenticated'>, <class 'lms.djangoapps.experiments.permissions.IsStaffOrOwner'>)#
serializer_class#

alias of ExperimentDataSerializer

suffix = None#
class lms.djangoapps.experiments.views.ExperimentKeyValueViewSet(**kwargs)#

Bases: ModelViewSet

authentication_classes = (<class 'edx_rest_framework_extensions.auth.jwt.authentication.JwtAuthentication'>, <class 'lms.djangoapps.experiments.views.ExperimentCrossDomainSessionAuth'>)#
basename = None#
description = None#
detail = None#
filter_backends = (<class 'django_filters.rest_framework.backends.DjangoFilterBackend'>,)#
filterset_class#

alias of ExperimentKeyValueFilter

name = None#
permission_classes = (<class 'lms.djangoapps.experiments.permissions.IsStaffOrReadOnly'>,)#
serializer_class#

alias of ExperimentKeyValueSerializer

suffix = None#
class lms.djangoapps.experiments.views.UserMetaDataView(**kwargs)#

Bases: APIView

authentication_classes = (<class 'edx_rest_framework_extensions.auth.jwt.authentication.JwtAuthentication'>, <class 'lms.djangoapps.experiments.views.ExperimentCrossDomainSessionAuth'>)#
get(request, course_id=None, username=None)#

Return user-metadata for the given course and user

permission_classes = (<class 'lms.djangoapps.experiments.permissions.IsStaffOrReadOnlyForSelf'>,)#

lms.djangoapps.experiments.views_custom module#

The Discount API Views should return information about discounts that apply to the user and course.

class lms.djangoapps.experiments.views_custom.Rev934(**kwargs)#

Bases: DeveloperErrorViewMixin, APIView

Use Cases

Request upsell information for mobile app users

Example Requests

GET /api/experiments/v0/custom/REV-934/?course_id={course_key_string}

Response Values

Body consists of the following fields:
show_upsell:

whether to show upsell in the moble app in this case

price:

(optional) the price to show if show_upsell is true

basket_url:

(optional) the url to the checkout page with the course’s sku if show_upsell is true

upsell_flag:

(optional) false if the upsell flag is off, not present otherwise

Response:

{ “show_upsell”: true, “price”: “$199”, “basket_url”: “https://ecommerce.edx.org/basket/add?sku=abcdef” }

Parameters:

course_key_string:

The course key that may be upsold

Returns

  • 200 on success with above fields.

  • 401 if there is no user signed in.

Example response: {

“show_upsell”: true, “price”: “$199”, “basket_url”: “https://ecommerce.edx.org/basket/add?sku=abcdef”

}

authentication_classes = (<class 'edx_rest_framework_extensions.auth.jwt.authentication.JwtAuthentication'>, <class 'openedx.core.lib.api.authentication.BearerAuthenticationAllowInactiveUser'>, <class 'edx_rest_framework_extensions.auth.session.authentication.SessionAuthenticationAllowInactiveUser'>)#
get(request)#

Return the if the course should be upsold in the mobile app, if the user has appropriate permissions.

permission_classes = (<class 'openedx.core.lib.api.permissions.ApiKeyHeaderPermissionIsAuthenticated'>,)#

Module contents#

Contents