openedx.core.djangoapps.content.block_structure package

Contents

openedx.core.djangoapps.content.block_structure package#

Subpackages#

Submodules#

openedx.core.djangoapps.content.block_structure.api module#

Higher order functions built on the BlockStructureManager to interact with a django cache.

openedx.core.djangoapps.content.block_structure.api.clear_course_from_cache(course_key)#

A higher order function implemented on top of the block_structure.clear_block_cache function that clears the block structure from the cache for the given course_key.

Note: See Note in get_course_blocks. Even after MA-1604 is implemented, this implementation should still be valid since the entire block structure of the course is cached, even though arbitrary access to an intermediate block will be supported.

openedx.core.djangoapps.content.block_structure.api.get_block_structure_manager(course_key)#

Returns the manager for managing Block Structures for the given course.

openedx.core.djangoapps.content.block_structure.api.get_cache()#

Returns the storage for caching Block Structures.

openedx.core.djangoapps.content.block_structure.api.get_course_in_cache(course_key)#

A higher order function implemented on top of the block_structure.get_collected function that returns the block structure in the cache for the given course_key.

Returns:

BlockStructureBlockData - The collected block structure,

starting at root_block_usage_key.

openedx.core.djangoapps.content.block_structure.api.update_course_in_cache(course_key)#

A higher order function implemented on top of the block_structure.updated_collected function that updates the block structure in the cache for the given course_key.

openedx.core.djangoapps.content.block_structure.apps module#

Configuration for block_structure djangoapp

class openedx.core.djangoapps.content.block_structure.apps.BlockStructureConfig(app_name, app_module)#

Bases: AppConfig

block_structure django app.

name = 'openedx.core.djangoapps.content.block_structure'#
ready()#

Define tasks to perform at app loading time

  • Connect signal handlers

  • Register celery tasks

These happen at import time. Hence the unused imports

openedx.core.djangoapps.content.block_structure.block_structure module#

Module with family of classes for block structures.

BlockStructure - responsible for block existence and relations. BlockStructureBlockData - responsible for block & transformer data. BlockStructureModulestoreData - responsible for xBlock data.

The following internal data structures are implemented:

_BlockRelations - Data structure for a single block’s relations. _BlockData - Data structure for a single block’s data.

class openedx.core.djangoapps.content.block_structure.block_structure.BlockData(usage_key)#

Bases: FieldData

Data structure to encapsulate collected data for a single block.

class_field_names()#

Returns list of names of fields that are defined directly on the class. Can be overridden by subclasses. All other fields are assumed to be stored in the self.fields dict.

property usage_key#
class openedx.core.djangoapps.content.block_structure.block_structure.BlockStructure(root_block_usage_key)#

Bases: object

Base class for a block structure. BlockStructures are constructed using the BlockStructureFactory and then used as the currency across Transformers.

This base class keeps track of the block structure’s root_block_usage_key, the existence of the blocks, and their parents and children relationships (graph nodes and edges).

get_block_keys()#

Returns the block keys in the block structure.

Returns:

iterator(UsageKey) - An iterator of the usage keys of all the blocks in the block structure.

get_children(usage_key)#

Returns the children of the block identified by the given usage_key.

Parameters:

children (usage_key - The usage key of the block whose) – are to be returned.

Returns:

[UsageKey] - A list of usage keys of the block’s children.

get_parents(usage_key)#

Returns the parents of the block identified by the given usage_key.

Parameters:

parents (usage_key - The usage key of the block whose) – are to be returned.

Returns:

[UsageKey] - A list of usage keys of the block’s parents.

post_order_traversal(filter_func=None, start_node=None)#

Performs a post-order sort of the block structure and yields the usage_key of each block as it is encountered.

Parameters:

  • in (See the description)

  • openedx.core.lib.graph_traversals.traverse_post_order.

Returns:

generator - A generator object created from the

traverse_post_order method.

set_root_block(usage_key)#

Sets the given usage key as the new root of the block structure.

Note: This method does not prune the rest of the structure. For performance reasons, it is left to the caller to decide when exactly to prune.

Parameters:

the (usage_key - The usage key of the block that is to be set as) – new root of the block structure.

topological_traversal(filter_func=None, yield_descendants_of_unyielded=False, start_node=None)#

Performs a topological sort of the block structure and yields the usage_key of each block as it is encountered.

Parameters:

  • in (See the description)

  • openedx.core.lib.graph_traversals.traverse_topologically.

Returns:

generator - A generator object created from the

traverse_topologically method.

class openedx.core.djangoapps.content.block_structure.block_structure.BlockStructureBlockData(root_block_usage_key)#

Bases: BlockStructure

Subclass of BlockStructure that is responsible for managing block and transformer data.

VERSION = 2#
copy()#

Returns a new instance of BlockStructureBlockData with a deep-copy of this instance’s contents.

create_removal_filter(removal_condition, keep_descendants=False)#

Returns a filter function that automatically removes blocks that satisfy the removal_condition.

Parameters:

  • removal_condition ((usage_key)->bool) – takes a block’s usage key as input and returns whether or not to remove that block from the block structure.

  • keep_descendants (bool) – remove_block.

create_universal_filter()#

Returns a filter function that always returns True for all blocks.

filter_topological_traversal(filter_func, **kwargs)#

A higher-order function that traverses the block structure using topological sort and applies the given filter.

Parameters:

  • filter_func ((usage_key)->bool) – whether or not to yield the given block key. If None, the True function is assumed.

  • kwargs (dict) – to topological_traversal.

get_transformer_block_data(usage_key, transformer)#

Returns the TransformerData for the given transformer for the block identified by the given usage_key.

Raises KeyError if not found.

Parameters:

  • usage_key (UsageKey) – transformer data is requested.

  • transformer (BlockStructureTransformer) – whose dictionary data is requested.

get_transformer_block_field(usage_key, transformer, key, default=None)#

Returns the value associated with the given key for the given transformer for the block identified by the given usage_key; returns default if not found.

Parameters:

  • usage_key (UsageKey) – transformer data is requested.

  • transformer (BlockStructureTransformer) – whose dictionary data is requested.

  • key (string) – that is requested.

  • default (any type) – entry is not found.

get_transformer_data(transformer, key, default=None)#

Returns the value associated with the given key from the given transformer’s data dictionary; returns default if not found.

Parameters:
get_xblock_field(usage_key, field_name, default=None)#

Returns the collected value of the xBlock field for the requested block for the requested field_name; returns default if not found.

Parameters:

  • usage_key (UsageKey) – field is requested.

  • field_name (string) – requested.

  • default (any type) – not found.

iteritems()#

Returns iterator of (UsageKey, BlockData) pairs for all blocks in the BlockStructure.

itervalues()#

Returns iterator of BlockData for all blocks in the BlockStructure.

override_xblock_field(usage_key, field_name, override_data)#

Set value of the XBlock field for the requested block for the requested field_name;

Parameters:

  • usage_key (UsageKey) – field is requested.

  • field_name (string) – requested.

  • override_data (object)

remove_block(usage_key, keep_descendants)#

Removes the block identified by the usage_key and all of its related data from the block structure. If descendants of the removed block are to be kept, the structure’s relations are updated to reconnect the block’s parents with its children.

Note: While the immediate relations of the block are updated (removed), all descendants of the block will remain in the structure unless the _prune_unreachable method is called.

Parameters:

  • usage_key (UsageKey) – removed.

  • keep_descendants (bool) – relations (graph edges) are updated such that the removed block’s children become children of the removed block’s parents.

remove_block_traversal(removal_condition, keep_descendants=False)#

A higher-order function that traverses the block structure using topological sort and removes all blocks satisfying the given removal_condition.

Parameters:

  • removal_condition ((usage_key)->bool) – takes a block’s usage key as input and returns whether or not to remove that block from the block structure.

  • keep_descendants (bool) – remove_block.

remove_transformer_block_field(usage_key, transformer, key)#

Deletes the given transformer’s entire data dict for the block identified by the given usage_key.

Parameters:

  • usage_key (UsageKey) – transformer data is to be deleted.

  • transformer (BlockStructureTransformer) – whose data entry is to be deleted.

retain_or_remove(block_key, removal_condition, keep_descendants=False)#

Removes the given block if it satisfies the removal_condition. Returns True if the block was retained, and False if the block was removed.

Parameters:

  • block_key (usage_key)

  • removal_condition ((usage_key)->bool) – takes a block’s usage key as input and returns whether or not to remove that block from the block structure.

  • keep_descendants (bool) – remove_block.

set_transformer_block_field(usage_key, transformer, key, value)#

Updates the given transformer’s data dictionary with the given key and value for the block identified by the given usage_key.

Parameters:

  • usage_key (UsageKey) – transformer data is to be updated.

  • transformer (BlockStructureTransformer) – whose data is to be updated.

  • key (string)

  • value (any picklable type) – given key for the given transformer’s data for the requested block.

set_transformer_data(transformer, key, value)#

Updates the given transformer’s data dictionary with the given key and value.

Parameters:

  • transformer (BlockStructureTransformer) – whose data is to be updated.

  • key (string)

  • value (any picklable type) – given key for the given transformer’s data.

class openedx.core.djangoapps.content.block_structure.block_structure.BlockStructureModulestoreData(root_block_usage_key)#

Bases: BlockStructureBlockData

Subclass of BlockStructureBlockData that is responsible for managing xBlocks and corresponding functionality that should only be called during the Collect phase.

Note: Although this class interface uses xBlock terminology, it is designed and implemented generically so it can work with any interface and implementation of an xBlock.

get_xblock(usage_key)#

Returns the instantiated xBlock for the given usage key.

Parameters:

usage_key (UsageKey) – xBlock object is to be returned.

request_xblock_fields(*field_names)#

Records request for collecting data for the given xBlock fields.

A Transformer should call this method when it needs to collect data for a common xBlock field that may also be used by other transformers. This minimizes storage usage across transformers. Contrast this with each transformer collecting the same xBlock data within its own transformer data storage.

Parameters:

field_names (list(string)) – xBlock fields whose values should be collected.

class openedx.core.djangoapps.content.block_structure.block_structure.FieldData#

Bases: object

Data structure to encapsulate collected fields.

class_field_names()#

Returns list of names of fields that are defined directly on the class. Can be overridden by subclasses. All other fields are assumed to be stored in the self.fields dict.

class openedx.core.djangoapps.content.block_structure.block_structure.TransformerData#

Bases: FieldData

Data structure to encapsulate collected data for a transformer.

class openedx.core.djangoapps.content.block_structure.block_structure.TransformerDataMap#

Bases: dict

A map of Transformer name to its corresponding TransformerData. The map can be accessed by the Transformer’s name or the Transformer’s class type.

get_or_create(key)#

Returns the TransformerData associated with the given key. If not found, creates and returns a new TransformerData and maps it to the given key.

openedx.core.djangoapps.content.block_structure.exceptions module#

Application-specific exceptions raised by the block structure framework.

exception openedx.core.djangoapps.content.block_structure.exceptions.BlockStructureException#

Bases: Exception

Base class for all Block Structure framework exceptions.

exception openedx.core.djangoapps.content.block_structure.exceptions.BlockStructureNotFound(root_block_usage_key)#

Bases: BlockStructureException

Exception for when a Block Structure is not found.

exception openedx.core.djangoapps.content.block_structure.exceptions.TransformerDataIncompatible#

Bases: BlockStructureException

Exception for when the version of a Transformer’s data is not compatible with the current version of the Transformer.

exception openedx.core.djangoapps.content.block_structure.exceptions.TransformerException#

Bases: BlockStructureException

Exception class for Transformer related errors.

exception openedx.core.djangoapps.content.block_structure.exceptions.UsageKeyNotInBlockStructure#

Bases: BlockStructureException

Exception for when a usage key is not found within a block structure.

openedx.core.djangoapps.content.block_structure.factory module#

Module for factory class for BlockStructure objects.

class openedx.core.djangoapps.content.block_structure.factory.BlockStructureFactory#

Bases: object

Factory class for BlockStructure objects.

classmethod create_from_modulestore(root_block_usage_key, modulestore)#

Creates and returns a block structure from the modulestore starting at the given root_block_usage_key.

Parameters:

  • root_block_usage_key (UsageKey) – of the block structure that is to be created.

  • modulestore (ModuleStoreRead) – contains the data for the xBlocks within the block structure starting at root_block_usage_key.

Returns:

BlockStructureModulestoreData - The created block structure

with instantiated xBlocks from the given modulestore starting at root_block_usage_key.

Raises:

xmodule.modulestore.exceptions.ItemNotFoundError if a block for – root_block_usage_key is not found in the modulestore.

classmethod create_from_store(root_block_usage_key, block_structure_store)#

Deserializes and returns the block structure starting at root_block_usage_key from the given store, if it’s found in the store.

The given root_block_usage_key must equate the root_block_usage_key previously passed to serialize_to_cache.

Parameters:

  • root_block_usage_key (UsageKey) – of the block structure that is to be deserialized from the given cache.

  • block_structure_store (BlockStructureStore) – store from which the block structure is to be deserialized.

Returns:

BlockStructure - The deserialized block structure starting

at root_block_usage_key, if found in the cache.

Raises:

BlockStructureNotFound - If the root_block_usage_key is not found – in the store.

classmethod create_new(root_block_usage_key, block_relations, transformer_data, block_data_map)#

Returns a new block structure for given the arguments.

openedx.core.djangoapps.content.block_structure.manager module#

Top-level module for the Block Structure framework with a class for managing BlockStructures.

class openedx.core.djangoapps.content.block_structure.manager.BlockStructureManager(root_block_usage_key, modulestore, cache)#

Bases: object

Top-level class for managing Block Structures.

clear()#

Removes data for the block structure associated with the given root block key.

get_collected(user=None)#

Returns the collected Block Structure for the root_block_usage_key, getting block data from the cache and modulestore, as needed.

Details: The cache is updated if needed (if outdated or empty), the modulestore is accessed if needed (at cache miss), and the transformers data is collected if needed.

In the case of a cache miss, the function bypasses runtime access checks for the current user. This is done to prevent inconsistencies in the data, which can occur when certain blocks are inaccessible due to access restrictions.

Returns:

BlockStructureBlockData - A collected block structure,

starting at root_block_usage_key, with collected data from each registered transformer.

get_transformed(transformers, starting_block_usage_key=None, collected_block_structure=None, user=None)#

Returns the transformed Block Structure for the root_block_usage_key, starting at starting_block_usage_key, getting block data from the cache and modulestore, as needed.

Details: Similar to the get_collected method, except the transformers’ transform methods are also called.

Parameters:

  • transformers (BlockStructureTransformers) – transformers to apply.

  • starting_block_usage_key (UsageKey) – in the block structure that is to be transformed. If None, root_block_usage_key is used.

  • collected_block_structure (BlockStructureBlockData) – block structure retrieved from a prior call to get_collected. Can be optionally provided if already available, for optimization.

  • user (django.contrib.auth.models.User) – which the block structure is to be transformed.

Returns:

BlockStructureBlockData - A transformed block structure,

starting at starting_block_usage_key.

update_collected_if_needed()#

The store is updated with newly collected transformers data from the modulestore, only if the data in the store is outdated.

openedx.core.djangoapps.content.block_structure.models module#

Models used by the block structure framework.

class openedx.core.djangoapps.content.block_structure.models.BlockStructureModel(*args, **kwargs)#

Bases: TimeStampedModel

Model for storing Block Structure information.

exception DoesNotExist#

Bases: ObjectDoesNotExist

exception MultipleObjectsReturned#

Bases: MultipleObjectsReturned

UNIQUENESS_FIELDS = ['data_usage_key', 'data_version', 'data_edit_timestamp', 'transformers_schema_version', 'block_structure_schema_version']#
VERSION_FIELDS = ['data_version', 'data_edit_timestamp', 'transformers_schema_version', 'block_structure_schema_version']#
block_structure_schema_version#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

data#

The descriptor for the file attribute on the model instance. Return a FieldFile when accessed so you can write code like:

>>> from myapp.models import MyModel
>>> instance = MyModel.objects.get(pk=1)
>>> instance.file.size

Assign a file object on assignment so you can do:

>>> with open('/path/to/hello.world') as f:
...     instance.file = File(f)
data_edit_timestamp#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

data_usage_key#

DO NOT REUSE THIS CLASS. Provided for backwards compatibility only!

A placeholder class that provides a way to set the attribute on the model.

data_version#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

classmethod get(data_usage_key)#

Returns the entry associated with the given data_usage_key. :raises BlockStructureNotFound if an entry for data_usage_key is not found.:

get_next_by_created(*, field=<model_utils.fields.AutoCreatedField: created>, is_next=True, **kwargs)#
get_next_by_modified(*, field=<model_utils.fields.AutoLastModifiedField: modified>, is_next=True, **kwargs)#
get_previous_by_created(*, field=<model_utils.fields.AutoCreatedField: created>, is_next=False, **kwargs)#
get_previous_by_modified(*, field=<model_utils.fields.AutoLastModifiedField: modified>, is_next=False, **kwargs)#
get_serialized_data()#

Returns the collected data for this instance.

id#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

modified#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>#
transformers_schema_version#

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

classmethod update_or_create(serialized_data, data_usage_key, **kwargs)#

Updates or creates the BlockStructureModel entry for the given data_usage_key in the kwargs, uploading serialized_data as the content data.

class openedx.core.djangoapps.content.block_structure.models.CustomizableFileField(*args, **kwargs)#

Bases: FileField

Subclass of FileField that allows custom settings to not be serialized (hard-coded) in migrations. Otherwise, migrations include optional settings for storage (such as the storage class and bucket name); we don’t want to create new migration files for each configuration change.

deconstruct()#

Return enough information to recreate the field as a 4-tuple:

  • The name of the field on the model, if contribute_to_class() has been run.

  • The import path of the field, including the class, e.g. django.db.models.IntegerField. This should be the most portable version, so less specific may be better.

  • A list of positional arguments.

  • A dict of keyword arguments.

Note that the positional or keyword arguments must contain values of the following types (including inner values of collection types):

  • None, bool, str, int, float, complex, set, frozenset, list, tuple, dict

  • UUID

  • datetime.datetime (naive), datetime.date

  • top-level classes, top-level functions - will be referenced by their full import path

  • Storage instances - these have their own deconstruct() method

This is because the values here must be serialized into a text format (possibly new Python code, possibly JSON) and these are the only types with encoding handlers defined.

There’s no need to return the exact way the field was instantiated this time, just ensure that the resulting field is the same - prefer keyword arguments over positional ones, and omit parameters with their default values.

openedx.core.djangoapps.content.block_structure.signals module#

Signal handlers for invalidating cached data.

openedx.core.djangoapps.content.block_structure.signals.update_block_structure_on_course_publish(sender, course_key, **kwargs)#

Catches the signal that a course has been published in the module store and creates/updates the corresponding cache entry. Ignores publish signals from content libraries.

openedx.core.djangoapps.content.block_structure.store module#

Module for the Storage of BlockStructure objects.

class openedx.core.djangoapps.content.block_structure.store.BlockStructureStore(cache)#

Bases: object

Storage for BlockStructure objects.

add(block_structure)#

Stores and caches a compressed and pickled serialization of the given block structure.

The data stored includes the structure’s block relations, transformer data, and block data.

Parameters:

block_structure (BlockStructure) – that is to be cached and stored.

delete(root_block_usage_key)#

Deletes the block structure for the given root_block_usage_key from the cache and storage.

Parameters:

root_block_usage_key (UsageKey) – of the block structure that is to be removed.

get(root_block_usage_key)#

Deserializes and returns the block structure starting at root_block_usage_key, if found in the cache or storage.

The given root_block_usage_key must equate the root_block_usage_key previously passed to the add method.

Parameters:

root_block_usage_key (UsageKey) – root of the block structure that is to be retrieved from the store.

Returns:

BlockStructure - The deserialized block structure starting at root_block_usage_key, if found.

Raises:

  • BlockStructureNotFound if the root_block_usage_key is not –

  • found. –

is_up_to_date(root_block_usage_key, modulestore)#

Returns whether the data in storage for the given key is already up-to-date with the version in the given modulestore.

openedx.core.djangoapps.content.block_structure.tasks module#

Asynchronous tasks related to the Course Blocks sub-application.

openedx.core.djangoapps.content.block_structure.tasks.block_structure_task(**kwargs)#

Decorator for block structure tasks.

openedx.core.djangoapps.content.block_structure.transformer module#

This module provides the abstract base class for all Block Structure Transformers.

class openedx.core.djangoapps.content.block_structure.transformer.BlockStructureTransformer#

Bases: object

Abstract base class for all block structure transformers.

READ_VERSION = 0#
WRITE_VERSION = 0#
classmethod collect(block_structure)#

Collects and stores any xBlock and modulestore data into the block_structure that’s necessary for later execution of the transformer’s transform method. Transformers should store such data in the block_structure using the following methods:

set_transformer_data set_transformer_block_field request_xblock_fields

Transformers can call block_structure.request_xblock_fields for any common xBlock fields that should be collected by the framework.

Any full block tree traversals should be implemented in this collect phase, leaving the transform phase for fast and direct access to a sub-block. If a block’s transform output is dependent on its ancestors’ data, the ancestor’s data should be percolated down to the descendants. So when a (non-root) block is directly accessed in the transform, all of its relevant data is readily available (without needing to access its ancestors).

Traversals of the block_structure can be implemented using the following methods:

topological_traversal post_order_traversal

Parameters:

block_structure (BlockStructureModulestoreData) – block structure that is to be modified with collected data to be cached for the transformer.

classmethod name()#

Unique identifier for the transformer’s class. It is used to identify the transformer’s cached data. So it should be unique and not conflict with other transformers. Consider using the same name that is used in the Transformer Registry. For example, for Stevedore, it is specified in the package configuration (pyproject.toml).

Once the transformer is in use and its data is cached, do not modify this name value without consideration of backward compatibility with previously collected data.

abstractmethod transform(usage_info, block_structure)#

Transforms the given block_structure for the given usage_info, assuming the block_structure contains cached data from a prior call to the collect method of the latest version of the Transformer.

No access to the modulestore nor instantiation of xBlocks should be performed during the execution of this method. However, accesses to user-specific data (outside of the modulestore and not via xBlocks) is permitted in order to apply the transform for the given usage_info.

Note: The root of the given block_structure is not necessarily the same as the root of the block_structure passed to the prior collect method. The collect method is given the top-most root of the structure, while the transform method may be called upon any sub-structure or even a single block within the originally collected structure.

A Transformer may choose to remove entire sub-structures during the transform method and may do so using the remove_block and filter_with_removal methods.

Amongst the many methods available for a block_structure, the following methods are commonly used during transforms:

get_xblock_field get_transformer_data get_transformer_block_field remove_block_traversal filter_with_removal filter_topological_traversal topological_traversal post_order_traversal

Parameters:

  • usage_info (any negotiated type) – that is passed to the block_structure and forwarded to all requested Transformers in order to apply a usage-specific transform. For example, an instance of usage_info would contain a user object for which the transform should be applied.

  • block_structure (BlockStructureBlockData) – block structure, with already collected data for the transformer, that is to be transformed in place.

class openedx.core.djangoapps.content.block_structure.transformer.FilteringTransformerMixin#

Bases: BlockStructureTransformer

Transformers may optionally choose to implement this mixin if their transform logic can be broken apart into a lambda for optimization of combined tree traversals.

For performance reasons, developers should try to implement this mixin whenever possible - with this alternative, traversal of the entire block structure happens only once for all transformers that implement FilteringTransformerMixin.

transform(usage_info, block_structure)#

By defining this method, FilteringTransformers can be run individually if desired. In normal operations, the filters returned from multiple transform_block_filters calls will be combined and used in a single tree traversal.

abstractmethod transform_block_filters(usage_info, block_structure)#

This is an alternative to the standard transform method.

Returns a list of filter functions to be used for filtering out any unwanted blocks in the given block_structure.

In addition to the commonly used methods listed above, the following methods are commonly used by implementations of transform_block_filters:

create_universal_filter create_removal_filter

Note: Transformers that implement this alternative should be independent of all other registered transformers as they may not be applied in the order in which they were listed in the registry.

Parameters:

  • usage_info (any negotiated type) – that is passed to the block_structure and forwarded to all requested Transformers in order to apply a usage-specific transform. For example, an instance of usage_info would contain a user object for which the transform should be applied.

  • block_structure (BlockStructureBlockData) – block structure, with already collected data for the transformer, that is to be transformed in place.

openedx.core.djangoapps.content.block_structure.transformer.combine_filters(block_structure, filters)#

openedx.core.djangoapps.content.block_structure.transformer_registry module#

Block Structure Transformer Registry implemented using the platform’s PluginManager.

class openedx.core.djangoapps.content.block_structure.transformer_registry.TransformerRegistry#

Bases: PluginManager

Registry for all of the block structure transformers that have been made available.

All block structure transformers should implement BlockStructureTransformer.

NAMESPACE = 'openedx.block_structure_transformer'#
USE_PLUGIN_MANAGER = True#
classmethod find_unregistered(transformers)#

Find and returns the names of all the transformers from the given list that aren’t registered with the platform’s PluginManager.

Parameters:

of (transformers ([BlockStructureTransformer] - List) – transformers to check in the registry.

Returns:

set([string]) - Set of names of a subset of the given

transformers that weren’t found in the registry.

classmethod get_registered_transformers()#

Returns a set of all registered transformers.

Returns:

{BlockStructureTransformer} - All transformers that are

registered with the platform’s PluginManager.

classmethod get_write_version_hash()#

Decorator to cache the result of a function for the life of a process.

If the return value of the function for the provided arguments has not yet been cached, the function will be calculated and cached. If called later with the same arguments, the cached value is returned (not reevaluated). https://wiki.python.org/moin/PythonDecoratorLibrary#Memoize

WARNING: Only use this process_cached decorator for caching data that is constant throughout the lifetime of a gunicorn worker process, is costly to compute, and is required often. Otherwise, it can lead to unwanted memory leakage.

openedx.core.djangoapps.content.block_structure.transformers module#

Module for a collection of BlockStructureTransformers.

class openedx.core.djangoapps.content.block_structure.transformers.BlockStructureTransformers(transformers=None, usage_info=None)#

Bases: object

The BlockStructureTransformers class encapsulates an ordered list of block structure transformers. It uses the Transformer Registry to verify the the registration status of added transformers and to collect their data. It provides aggregate functionality for collection and ordered transformation of the transformers.

Clients are expected to access the list of transformers through the class’ interface rather than directly.

classmethod collect(block_structure)#

Collects data for each registered transformer.

transform(block_structure)#

The given block structure is transformed by each transformer in the collection. Tranformers with filters are combined and run first in a single course tree traversal, then remaining transformers are run in the order that they were added.

classmethod verify_versions(block_structure)#

Returns whether the collected data in the block structure is incompatible with the current version of the registered Transformers.

Raises:

  • TransformerDataIncompatible with information about all outdated –

  • Transformers. –

Module contents#

The block_structure django app provides an extensible framework for caching data of block structures from the modulestore.

Dual-Phase. The framework is meant to be used in 2 phases.

  • Collect Phase (for expensive and full-tree traversals) - In the first phase, the “collect” phase, any and all data from the modulestore should be collected and cached for later access to the block structure. Instantiating any and all xBlocks in the block structure is also done at this phase, since that is also (currently) a costly operation.

    Any full tree traversals should also be done during this phase. For example, if data for a block depends on its parents, the traversal should happen during the collection phase and any required data for the block should be percolated down the tree and stored as aggregate values on the descendants. This allows for faster and direct access to blocks in the Transform phase.

  • Transform Phase (for fast access to blocks) - In the second phase, the “transform” phase, only the previously collected and cached data should be accessed. There should be no access to the modulestore or instantiation of xBlocks in this phase.

To make this framework extensible, the Transformer and Extensibility design patterns are used. This django app only provides the underlying framework for Block Structure Transformers and a Transformer Registry. Clients are expected to provide actual implementations of Transformers or add them to the extensible Registry.

Transformers. As inspired by http://www.ccs.neu.edu/home/riccardo/courses/csu370-fa07/lect18.pdf, a Block Structure Transformer takes in a block structure (or tree) and manipulates the structure and the data of its blocks according to its own requirements. Its output can then be used for further transformations by other transformers down the pipeline.

Note: For performance and space optimization, our implementation differs from the paper in that our transformers mutate the block structure in-place rather than returning a modified copy of it.

Block Structure. The BlockStructure and its family of classes provided with this framework are the base data types for accessing and manipulating block structures. BlockStructures are constructed using the BlockStructureFactory and then used as the currency across Transformers.

Registry. Transformers are registered using the platform’s PluginManager (e.g., Stevedore). This is currently done by updating pyproject.toml. Only registered transformers are called during the Collect Phase. And only registered transformers can be used during the Transform phase. Exceptions to this rule are any nested transformers that are contained within higher-order transformers - as long as the higher-order transformers are registered and appropriately call the contained transformers within them.

Note: A partial subset (as an ordered list) of the registered transformers can be requested during the Transform phase, allowing the client to manipulate exactly which transformers to call.

Links to Other Block Structure Related Documentation:

Contents