About Saagie Usage Monitoring

The Saagie Usage Monitoring (SUM) add-on is an app based on Grafana that can be used to monitor job, app, and pipeline metrics for your projects.

It is available in the official Saagie repository, and you can install it in any project. However, since the app is designed to display cross-project metrics, Saagie recommends deploying it in a dedicated administration project.

Its installation requires some configuration to work. Your user account must have view rights to at least all projects, and you must set several environment variables. For more information, see the README help file.

Architecture and Operating Mode

SUM is based on the combination of several components, including a Grafana app, a Python job, a Cron, a TTYD or a pgAdmin, and either an internal or external PostgreSQL.

add on sum internal postgresql architecture

Figure 1. SUM with an internal PostgreSQL

add on sum external postgresql architecture

Figure 2. SUM with an external PostgreSQL and a pgAdmin app

The Python job retrieves data from jobs and pipelines and imports it into a PostgreSQL database, initiated at the start of SUM. Data is retrieved through Saagie’s Python API, saagieapi.

Data can also be imported from S3 or HDFS storage, which must be configured with environment variables when installing SUM. For more information, see the README help file.
The Cron runs the Python job to synchronize data. By default, it runs every hour, but you can customize it with the SAAGIE_SUM_CRON environment variable. For more information, see About Cron Customization.
The PostgreSQL database feeds your alerts and dashboards by transmitting the retrieved data to the Grafana.
TTYD and pgAdmin allow you to write your queries.

You can also use the pgAdmin app to look into the PostgreSQL and analyze the data model.

External PostgreSQL Database

To use an external PostgreSQL database, you must define the connection parameters via the corresponding environment variables in Saagie:

SAAGIE_PG_HOST: PostgreSQL host
→ The default value is localhost.
SAAGIE_PG_PORT: PostgreSQL port
→ The default value is 5432.
SAAGIE_PG_USER: PostgreSQL user
→ The default value is supervision_pg_user.
SAAGIE_PG_PASSWORD: PostgreSQL password
→ The default value is ` `.
SAAGIE_PG_DATABASE: PostgreSQL database
→ The default value is supervision_pg_db.

SUM Technology Contexts

SUM has several contexts. Depending on the version of Saagie you have, you will need to select the appropriate SUM technology context.

Installing a context that does not match to your version of Saagie may cause issues in the execution of your app.

For Saagie 2023.05: Chose this context if you have the version 2023.05 of Saagie installed.
For Saagie 2023.04: Chose this context if you have the version 2023.04 of Saagie installed.
For Saagie 2023.03: Chose this context if you have the version 2023.03 of Saagie installed.
For Saagie 2023.02: Chose this context if you have the version 2023.02 of Saagie installed.
For Saagie 3.x: Chose this context if you have a version 3.x of Saagie installed.

To find out which version of Saagie you are working on, click Product info