.. _vector:

Vector
******

Vector is lightweight and ultra-fast tool for building observability pipelines.
In the Aspects project, Vector can optionally be used as a replacement for Ralph to
capture xAPI learner statements in the ClickHouse database, and/or as a way to
store raw tracking log statements. It can be used as a general purpose log collector
and forwarder.


Vector Components
#################

Vector consists of the following components:

- `Sources <https://vector.dev/docs/reference/sources/>`_ - Collects data from a source
  and sends it to Vector.
- `Transforms <https://vector.dev/docs/reference/transforms/>`_ - Modifies events as they
  pass through Vector.
- `Sinks <https://vector.dev/docs/reference/sinks/>`_ - Sends events to a destination.

Vector can be deployed in two roles:

- `Agent <https://vector.dev/docs/setup/deployment/roles/#agent>`_ - Collects, transforms,
  and sends data to a destination.
- `Aggregator <https://vector.dev/docs/setup/deployment/roles/#aggregator>`_ - Receives data
  from Vector agents and sends it to a destination.


We use the role `Agent` to collect tracking logs and xAPI events from the lms and send it to
the Clickhouse sink.

For instance, Aspects has Sources configured for xAPI logging statements (optionally
generated by event-routing-backends), and for tracking log statements. Each of these have their
own Transforms, which validate that the log statements are JSON and forward them to Sinks which
store them in ClickHouse tables.

Those tables are controlled by the variables:

.. code-block:: yaml

    ASPECTS_VECTOR_DATABASE: "openedx"
    ASPECTS_VECTOR_RAW_TRACKING_LOGS_TABLE: "_tracking"
    ASPECTS_VECTOR_RAW_XAPI_TABLE: "xapi_events_all"

To learn more about Vector, see the `Vector documentation <https://vector.dev/docs/>`_.