Saagie Usage Monitoring Default Dashboards
When installing SUM, you must set several environment variables, including the MONITORING_OPT
environment variable.
This variable allows you to choose the default configuration of your dashboards by choosing between the following values:
-
SAAGIE
: Choose this value if you want to monitor only Saagie jobs, apps, and pipelines. This is the default value. -
SAAGIE_AND_DATALAKE
: Choose this value if you want to monitor Saagie jobs, apps, and pipelines, as well as your HDFS data lake. -
SAAGIE_AND_S3
: Choose this value if you want to monitor Saagie jobs, apps, and pipelines, as well as your S3 buckets.
If you choose the SAAGIE
option, you will have the following three dashboards:
If you choose the SAAGIE_AND_DATALAKE
option, you will have the dashboards included with the SAAGIE
option, and additional dashboards with information about your HDFS storage space.
If you choose the SAAGIE_AND_S3
option, you will have the dashboards included with the SAAGIE
option, and an additional dashboard with information about your S3 storage space.
Dashboards are accessible from the Dashboards tab:
Saagie – Job Count
You can see information on the total number of jobs and apps on your Saagie platforms, their distribution and evolution in number per project, and details for each app and job.
Click a legend item of a graph to display only the corresponding information. You can also view multiple items at once by pressing ctrl and clicking the desired legend items. To reset the graph, click a legend item twice. |
Saagie – Jobs Failed Timeline
Click a dot in the graph to display a tooltip with the job instance details.
You can see the job name, the date the job instance failed, its status, and its execution time.
You can click Job to open the instance of the job in Saagie.
Red dots are for failed jobs and yellow dots are for killed jobs. |
Saagie – Jobs and Pipelines
You can see information on the evolution of job status, the jobs and pipelines execution time, the average execution time for each job and its success rate.
|
S3 – Global Usage
You can see information about the total space used by all your S3 buckets, their total number of objects and the size of each bucket. The size of your S3 buckets is presented through a table and a graph.
Data Lake - Disk Space Explorer
You can see information about the size of first level directories in the Folder size
dashboard, and about the number of files per first level directories in the Number of files
dashboard.
Hover over the items of the different graphs to display tooltips with more details. |
Data Lake - Global Usage
This page allows you to see the history of the size of the directories, and the number of files. You can also see information about the average file size, and the storage space used compared to the overall capacity.
-
Grafana documentation: Use dashboards