xAPI Transforms#
Aspects converts raw Open edX tracking event JSON into xAPI for storage and analysis. This conversion process is called “transformation”.
This document covers two ways to customize Aspect’s xAPI transforms:
Transforming a new event#
Events emitted by openedx
packages are transformed by event-routing-backends (ERB), a Django plugin which Aspects installs on Open edX.
Transformers for events emitted by non-openedx packages should be stored close to the code that produces the events, and registered using decorators provided by event-routing-backends. We will use OpenCraft’s completion aggregator as the example for this tutorial, specifically code added pr#205.
xAPI Schema#
To decide on an event’s xAPI schema, consider any similar events already being transformed, and what event data will be useful for analysis or visualization in Aspects.
The schema for a new event must uniquely describe that event. However, it’s also important to be as consistent as possible with existing event schemas so that the event can be processed and used in Aspects in a similar way to other events.
As a reminder, an xAPI statement can be expressed as:
Actor Verbed an Object (within Context).
Actor#
For most events, the default Actor transform is enough:
{
"objectType": "Agent",
"account": {
"homePage": "https://lms.url",
"name": "32e08e30-f8ae-4ce2-94a8-c2bfe38a70cb"
}
}
Here, the actor’s external ID (of type=xapi) is used as the name
field. This external ID can be matched against PII data to access the actor’s name, email, and other profile details.
Verb#
The verb is the primary differentiator between different xAPI events in Open edX. Select a verb that describes the event as concisely and accurately as possible, so that future, similar events can still be discerned.
Where possible:
use verbs from one of the registered xAPI profiles.
avoid re-using verbs that are already in use in Aspects (see ERB’s verb list).
For example, the completion aggregator will emit events when progress has been made on a unit/section/subsection/course, so we could use the verb progressed.
{
"id": "http://adlnet.gov/expapi/verbs/progressed",
"display": {
"en": "progressed"
}
}
Object#
Most events in Open edX are Activities, which look like this:
{
"id": "https://lms.url/block/block-v1:edX+DemoX+Demo_Course+type@video+block@0b9e39477cf34507a7a48f74be381fdd",
"description": {
"type": "block",
"name": {
"en": "Welcome!"
},
}
}
id
should uniquely identify the activitytype
should describe the type of activity, e.g. “unit” or “course”name
should provide human-friendly display name(s) for the activityextensions
can be added to provide any extra data important to the activity
Context#
Most events in Open edX happen on an element within a course, like a block or a discussion forum, and so the “context activity” for the event is the course.
Aspects also uses “extensions” to record extra information, like the transformer code version and the actor’s session ID (if found in the event). These “extensions” can be used to communicate any high-level information that is important for the event record.
For example:
{
"contextActivities": {
"parent": [
{
"id": "https://lms.url/course/course-v1:edX+DemoX+Demo_Course",
"object_type": "Activity",
"definition": {
"type": "course",
"name": {
"en-US": "Demonstration Course"
}
}
}
]
},
"extensions": {
"https://w3id.org/xapi/openedx/extension/transformer-version": "7.2.0",
"https://w3id.org/xapi/openedx/extensions/session-id": "993110e9c27848a545da74a74114158d"
}
}
Result#
Some Open edX events use a “result” stanza that communicates information about the effect that this event had. For example, “problem check” events record whether the problem was answered correctly, and what score the actor received.
For these completion “progressed” events, we would want to store the percent completed, so we use an extension:
{
"completion": false,
"extensions": {
"https://w3id.org/xapi/cmi5/result/extensions/progress": "45"
}
}
Implementation#
Once the xAPI event schema is settled, the implementation should be pretty straightforward using event-routing-backends and TinCan. See pr#205 for full example code.
Create a new transformer class that extends XApiTransformer.
Implement the
get_verb
method, returning your chosen verb URI and its short name.Implement any other custom components by overriding their
get
method.For example, to customize the context activities for your event, override
get_context_activities
.Use the built-in transformer method
get_data
to parse and return data from the original tracking event.Register your transformer class using the
XApiTransformersRegistry.registry
decorator.Use the raw tracking event’s
type
as the parameter to ensure this class is used to transform those type of events.Write data transform tests by subclassing XApiTransformersFixturesTestMixin and adding
raw
JSON andexpected
xAPI event fixture data.
Warning
There can only be one registered xAPI transformer class per tracking event type
.
While it is technically possible to overwrite a registered transformer class with another, this is not recommended
and may have unintended side effects.
Example code#
Here is the full code for the new transformer described in this tutorial.
from tincan import LanguageMap, Result, Verb
from event_routing_backends.processors.xapi.registry import XApiTransformersRegistry
from event_routing_backends.processors.xapi.transformer import XApiTransformer
class ProgressTransformerBase(XApiTransformer):
"""
Transformer for completion-aggregated "progress" events.
Uses the default implementations for `get_actor` and `get_context`.
Expects at these fields to be present in the original tracking event:
{
"data": {
"block_id": "block-v1:...", # block usage key
"percent": "0.123", # percent completed, > 0, < 1.0
}
}
"""
object_type = None
additional_fields = ('result', )
def get_verb(self) -> Verb:
return Verb(
id="http://adlnet.gov/expapi/verbs/progressed",
display=LanguageMap({"en": "progressed"}),
)
def get_object(self) -> Activity:
return Activity(
id=self.get_object_iri("xblock", self.get_data("data.block_id")),
definition=ActivityDefinition(
type=self.object_type,
)
)
def get_result(self) -> Result:
return Result(
completion=self.get_data("data.percent") == 1.0,
score={
"scaled": self.get_data("data.percent") or 0,
},
)
# Register subclasses for each individual event type
@XApiTransformersRegistry.register("openedx.completion_aggregator.progress.chapter")
@XApiTransformersRegistry.register("openedx.completion_aggregator.progress.sequential")
@XApiTransformersRegistry.register("openedx.completion_aggregator.progress.vertical")
class ModuleProgressTransformer
object_type = "http://adlnet.gov/expapi/activities/module"
@XApiTransformersRegistry.register("openedx.completion_aggregator.progress.course")
class CourseProgressTransformer
object_type = "http://adlnet.gov/expapi/activities/course"
Modifying an existing transform#
ERB supports modifying some of its transforms using openedx-filters. See ERB’s xAPI filters for a list of available filters.
Warning
Use xAPI filters with care.
Aspects visualizations depend heavily on ERB’s transforms, so removing or modifying data may cause unexpected issues.
Adding new fields is low risk.
Example code#
The example below shows how to add extra data to an event’s Activity object.
See these xapi filters for more examples.
from openedx_filters import PipelineStep
class XApiContextExtensionsFilter(PipelineStep):
"""This filter adds tags to the object.definition.extensions list for "course graded" events.
How to set:
OPEN_EDX_FILTERS_CONFIG = {
"event_routing_backends.processors.xapi.grading_events.course_graded.get_object": {
"pipeline": ["this_module.this_file.XApiContextExtensionsFilter"],
"fail_silently": False,
},
}
"""
def run_filter(self, transformer, result):
"""Appends the list of block tags to the object's extensions list.
Arguments:
transformer <XApiTransformer>: Transformer instance.
result <Activity>: Target activity for the event.
Returns:
Activity: Modified activity object.
"""
block_id = result.id
tags = get_tags(block_id)
if not result.definition.extensions:
result.definition.extensions = {}
result.definition.extensions["http://id.tincanapi.com/extension/tags"] = [
f"{tag.name}={tag.value}",
for tag in tags
]
return resultevent_routing_backends.processors.xapi.grading_events.course_graded.get_object
References#
event-routing-backends: Django plugin that receives tracking events and transforms them into xAPI
completion aggregator: OpenCraft’s plugin which accumulates block completion up to the enclosing unit/section/subsection/course.
xAPI profiles: registry of xAPI schemas
openedx-filters: Open edX filters library