How-to: Enabling toggles reporting for a new IDA

There are three ways to report on feature toggles in an Independently Deployable Application (IDA).

IDA Toggles List on Readthedocs	Generated documentation based on annotated feature toggles.
Toggles State Endpoint	REST endpoint providing the current state of feature toggles at a given time from a given environment.
Toggles State Reports Spreadsheet	A generated spreadsheet combining data from the static feature toggle annotations and the toggle state data from the Toggles State Endpoint.

IDA Toggles List on Readthedocs

You can extract your toggle annotations from code in order to create documentation on Readthedocs for your IDA.

Here are some helpful examples and resources:

• Sample readthedocs documention for edx-platform feature toggles.

• Based on annotated toggles in the edx-platform codebase.

• See How to: Documenting new feature toggles

Steps required (2-4 hours):

• Copy featurestoggles.rst and configuration for the featuretoggles Sphinx extension from edx-platform.

  • Here is a search for featuretoggles in edx-platform.

Note

You can follow similar steps to use the settings Sphinx extension to publish non-boolean non-toggle setting annotations as well.

Toggles State Endpoint

Add a REST endpoint to your IDA that provides the current state of all your toggles, including all waffle toggles and all boolean Django Settings. This endpoint will be available from all environments, including Production.

Here is an example from edx-platform:

• https://courses.edx.org/api/toggles/v0/state/ (requires Staff access)

Steps required (2-4 hours):

• Add a view wrapping ToggleStateReport().as_dict() for your new toggle state REST endpoint:

  • Include edx-toggles in the base.in requirements file (if not already there).

  • Create a view using the ToggleStateReport class to implement the REST endpoint.

    • Our convention is to use the path /api/toggles/v0/state/.

    • Protect the view as a appropriate, potentially limiting to Staff access.

    • See an example toggles state endpoint view in edx-platform.

      • For most IDAs, just replace report = CourseOverrideToggleStateReport().as_dict() with report = ToggleStateReport().as_dict() from the example code.

    • Add the view to a urls.py like in this example urls.py in edx-platform.

    • (Optional) If your IDA has custom toggle types, you can subclass and override the reporting methods as was done in the edx-platform example toggles state endpoint view.

Toggles State Reports Spreadsheet

Note

Much of this section is specific to edX.org, including private links, but the general approach and some of the scripts are available to the wider Open edX community.

Add an IDA to the spreadsheet which combines data from the static feature toggle annotations and the toggle state data.

See the current Toggle State Reports Spreadsheet (private to edX.org).

Prerequisites:

• Implement the Toggles State Endpoint in your IDA.

Steps required (1 hour):

• Add IDA specification to feature-toggle-report-generator.yml under idas key and each of the ENVIRONMENTS that are applicable.

• Add IDA specification in scripts/configuration.yaml.

How the Toggle State Reports Spreadsheet is generated

This section describes how the Toggle State Reports Spreadsheet is generated. This information is not required knowledge for setting up a new IDA.

Gathering annotation data

See How to: Documenting new feature toggles to understand the toggle annotations that will be gathered. The code_annotations tool is used to collect these annotations into a report.

The steps to collect annotations are automated through a Jenkins job:

• groovy job specification in generate-code-annotation-report

• Jenkins Job folder

• Once the job is done, the annotations data is pushed to an S3 bucket: script to push data to s3 bucket

The job creates files with names like <ida_name>-annotations.yml. The feature toggle report generator will key off the ida_name in the filename in order to be able to link this data to the toggle state data collected.

Gathering state data from HTTP endpoint

Toggle state will be gathered from the new Toggles State Endpoint.

Collection of state data is automated through a jenkins job.

• groovy job specification in gather-${ida['name']}-${environment}-feature-toggle-state job

• Jenkins Job folder

• Once the job is done, the state data is pushed to an S3 bucket: script to push data to s3 bucket

Processing annotation and state data

The annotation data and toggle state data dump should be stored in s3 buckets. The automated publish-feature-toggle-report job (in groovy job specification) pulls the data from s3 buckets and calls feature_toggle_report_generator.py to process the data and output it as a csv file.

As long as the data is structured correctly (specified in README), no action should be necessary for a new IDA.

Publishing data

The toggle csv reports are retained as artifacts in the Jenkins job: publish-feature-toggle-report.

The csv reports are published to a private Google Sheet. See Toggle State Reports Spreadsheet (private to edX.org).