8. Project Structure#
Status#
Accepted
Context#
The Aspects Analytics system (Aspects) consists of several pieces of technology working together via some configuration and scripting. Decisions around where to store data and configuration need to be made so that developers understand where to make changes and look for the causes of issues. These decisions can have wide ranging impact on things such as extensibility and configurability of the system as a whole.
This ADR offers guiding principles to address these issues:
The number of plugins and systems in play can make it difficult to know where to look for configuration or add new features.
Initialization steps for the third-party systems are sometimes intricate and dependent on other systems, so Aspects deployment needs to control exactly what happens when.
In the future, Aspects will offer a non-Tutor deployment option for experienced operators.
Decisions#
Aspects repo
This repo (openedx-aspects) remains the primary Aspects repository where the project decisions and documentation are stored.
This repo should also be used to store any common code or configuration between Tutor and non-Tutor Aspects deployments.
Single Aspects Tutor plugin
All of the third-party services that Aspects requires can be deployed and configured by a single tutor-contrib-aspects plugin.
This plugin employs the following structure to delineate the different areas of responsibility:
tutoraspects/
├── patches
└── templates
├── aspects
│ ├── apps
│ │ ├── aspects
│ │ ├── clickhouse
│ │ ├── ralph
│ │ ├── superset
│ │ └── vector
│ ├── build
│ │ ├── aspects
│ │ └── aspects-superset
│ │ └── openedx-assets
│ └── jobs
│ └── init
│ ├── aspects
│ ├── clickhouse
│ ├── lms
│ ├── mysql
│ └── superset
└── openedx-assets
This structure is guided by the following principles:
Configuration or functionality specific to a single service should live under a directory named for that service.
E.g. templates/aspects/apps/clickhouse contains deployment templates related to Clickhouse, and templates/aspects/jobs/init/clickhouse contains the Clickhouse initialization scripts.
Configuration or functionality that spans multiple services should live under an aspects directory.
E.g. templates/aspects/apps/aspects contains Aspects-specific database migrations (Clickhouse migrations handled by Alembic) and dbt profiles (data transformation layer).
Templates for Superset assets (dashboards, charts, etc.) live under the templates/aspects/build/aspects-superset/openedx-assets directory to facilitate localization of this user-facing content.
These asset templates utilize shared, non-localized files stored under templates/openedx-assets.
LMS plugins
Aspects depends on specific versions of a few Django plugins to move events from the LMS into its processing pipeline (currently openedx-event-sink-clickhouse and event-routing-backends). These plugins should live in their own repositories, and be installed to the LMS/CMS as “extra requirements”, using the chosen deployment’s best practices. For example, the Tutor plugin installs these dependencies to the LMS+CMS using the OPENEDX_EXTRA_PIP_REQUIREMENTS variable.
Consequences#
The service-based Tutor plugins (
tutor-contrib-clickhouse
,tutor-contrib-ralph
,tutor-contrib-superset
) are consolidated into tutor-contrib-aspects and archived under openedx-unsupported.
Rejected Alternatives#
Separate plugins for each third-party service
Supercedes ADR 6 Areas of responsibility.