platform_plugin_aspects.management.commands package

Submodules

platform_plugin_aspects.management.commands.dump_data_to_clickhouse module

Management command for exporting the modulestore ClickHouse.

Example usages (see usage for more options):

# Dump all objects published since last dump. # Use connection parameters from settings.EVENT_SINK_CLICKHOUSE_BACKEND_CONFIG: python manage.py cms dump_objects_to_clickhouse –object user_profile

# Specify certain objects instead of dumping all of them. # Use connection parameters from settings.EVENT_SINK_CLICKHOUSE_BACKEND_CONFIG. python manage.py cms dump_objects_to_clickhouse –object user_profile –objects 123 124 125

# Dump a limited number of objects to prevent stress on production systems python manage.py cms dump_objects_to_clickhouse –limit 1000

class platform_plugin_aspects.management.commands.dump_data_to_clickhouse.Command(stdout=None, stderr=None, no_color=False, force_color=False)

Bases: BaseCommand

Dump objects to a ClickHouse instance.

add_arguments(parser)

Entry point for subclassed commands to add custom arguments.

handle(*args, **options)

Iterates through each objects, serializes and saves them to clickhouse.

help = 'Dump objects to a ClickHouse instance.'

platform_plugin_aspects.management.commands.dump_data_to_clickhouse.dump_target_objects_to_clickhouse(sink=None, start_pk=None, object_ids=None, objects_to_skip=None, force=False, limit=None, batch_size=1000, sleep_time=10)

Iterates through a list of objects in the ORN, serializes them to csv, then submits tasks to post them to ClickHouse.

Parameters:

force – serialize the objects even if they’ve been recently serialized

Returns: two lists–one of the objects that had dump jobs queued for them

and one of objects that did not.

platform_plugin_aspects.management.commands.load_test_tracking_events module

Generates tracking events by creating test users and fake activity.

This should never be run on a production server as it will generate a lot of bad data. It is entirely for benchmarking purposes in load test environments. It is also fragile due to reaching into the edx-platform testing internals.

class platform_plugin_aspects.management.commands.load_test_tracking_events.Command(stdout=None, stderr=None, no_color=False, force_color=False)

Bases: BaseCommand

Create tracking log events for load testing purposes.

Example: tutor local run lms ./manage.py lms load_test_tracking_events –sleep_time 0 –tags celery 1worker

add_arguments(parser: Any) → None

Entry point for subclassed commands to add custom arguments.

handle(*args, **options)

Create users and trigger events for them as configured above.

help = 'Create tracking log events for load testing purposes.\n\nExample:\ntutor local run lms ./manage.py lms load_test_tracking_events --sleep_time 0 --tags celery 1worker'

class platform_plugin_aspects.management.commands.load_test_tracking_events.LoadTest(num_users: int, username_prefix: str, tags: List[str])

Bases: object

Runs the load test and reports results to ClickHouse.

ch_runs_table = 'load_test_runs'

create_and_enroll_learners(num_users, username_prefix)

Uses create test users and enroll them in our test course.

create_course() → None

Create a course using the CMS API.

create_user(**user_data)

Create, activate, and return a user using the edx-platform API.

record_end() → None

Send an end event to ClickHouse.

record_start() → None

Send a start event to ClickHouse.

Start and end events are used by the monitor script to know when to begin and end monitoring.

record_to_clickhouse(event_type, extra) → None

Send the run events to ClickHouse.

sent_event_count = 0

trigger_event_and_sleep(sleep_time: float) → None

Cause a tracking log to be emitted and sleep the specified amount of time.

trigger_events(num_events: int, sleep_time: float, run_until_killed: bool) → None

Trigger the appropriate number of events based on configuration.

users = []

platform_plugin_aspects.management.commands.monitor_load_test_tracking module

Monitors the load test tracking script and saves output for later analysis.

class platform_plugin_aspects.management.commands.monitor_load_test_tracking.Command(stdout=None, stderr=None, no_color=False, force_color=False)

Bases: BaseCommand

Dump objects to a ClickHouse instance.

Example: tutor local run lms ./manage.py lms monitor_load_test_tracking –sleep_time 5 –backend redis_bus

add_arguments(parser: Any) → None

Entry point for subclassed commands to add custom arguments.

handle(*_, **options)

Creates users and triggers events for them as configured above.

help = 'Dump objects to a ClickHouse instance.\n\nExample:\ntutor local run lms ./manage.py lms monitor_load_test_tracking --sleep_time 5 --backend redis_bus'

class platform_plugin_aspects.management.commands.monitor_load_test_tracking.Monitor(sleep_time: float, backend: str)

Bases: object

Manages the configuration and state of the load test monitor.

check_for_run_id() → str | None

Return a run id for any unfinished run started in the last minute.

get_celery_stats()

Get the current state of Celery for this iteration.

get_clickhouse_stats()

Get the current state of ClickHouse for this iteration.

get_kafka_bus_stats()

Get the current state of ClickHouse for this iteration.

get_redis_bus_stats()

Get the current state of redis for this iteration.

get_vector_stats()

Get the current state of Vector for this iteration.

run() → None

Wait for a new test to start, then run the monitor until killed.

run_id = None

store_stats(current_stats: dict) → None

Send the results for this iteration to ClickHouse.

test_has_ended() → bool

Return True if the current run has finished.

wait_for_start() → None

Wait for a run to start, tested by checking ClickHouse for new run ids.

Module contents