Prometheus

Saagie uses the monitoring system Prometheus to expose metrics through Saagie's HTTP API.

During the Saagie installation process, this exposure can be enabled or disabled.

Exposed components

Each exposed component is accessible by its own HTTP path.

The list of available paths in Saagie can be found in the prometheus Kubernetes Ingress of the <installationId> namespace. Where <installationId> must be replaced with your installation ID, which must match the prefix you have determined for your DNS entry.

To list the paths, run the following command line:

kubectl -n <installationId> get ingress <ingress_name> -o jsonpath='{range .spec.rules[*].http.paths[*]}{.path}{"\n"}{end}'

Where:

  • Prometheus

  • Prometheus-Kafka

To ingress prometheus, you should receive the following result:

/prometheus/projects-and-jobs/jmx/
/prometheus/conforama/jmx/
/prometheus/platform-k8s-controller/jmx/
/prometheus/scredz/jmx/
/prometheus/auth/jmx/
/prometheus/authentication/jmx/
/prometheus/idmacl/jmx/
/prometheus/profile/jmx/
/prometheus/security/jmx/
/prometheus/settings/jmx/
/prometheus/ingress-controller-apps-bouncer/jmx/
/prometheus/technology-manager/jmx/
/prometheus/governance/jmx/
/prometheus/dataset-access/jmx/
/prometheus/rule-manager/jmx/

To ingress prometheus-kafka, you should receive the following result:

/prometheus/kafka-broker/jmx/0/
/prometheus/kafka-broker/jmx/1/
/prometheus/kafka-broker/jmx/2/
/prometheus/kafka-schema-registry/jmx/
/prometheus/kafka-zookeeper/jmx/0/
/prometheus/kafka-zookeeper/jmx/1/
/prometheus/kafka-zookeeper/jmx/2/
The number of replicas may vary.

Paths follow this model:

/prometheus/<component>/<exporter>/{<replicas>/} (1) (2) (3)

Where:

1 component might have a subcomponent appearing as <component-subcomponent>.
For example, the path /prometheus/ingress-controller-apps-bouncer/jmx/ refers to the component ingress-controller with the subcomponent apps-bouncer.
2 exporter is defined according to the component technology.
3 replicas depend on the component and may or may not be present.

Prometheus Server Configuration

Below is an example of the Prometheus server configuration (prometheus.yml) used to connect to Saagie Prometheus exporters:

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it is Prometheus itself.
scrape_configs:
  - job_name: 'saagie-projects'
    metrics_path: '/prometheus/projects/jmx/'
    scheme: 'https'
    basic_auth:
      username: 'prometheus'
      password: 'password'
    static_configs:
      - targets: ['SAAGIE_HOST']
...

How to Collect Prometheus Metrics?

You can use an HTTP call to collect Prometheus metrics.

The following command line gives an example with the curl tool, using metrics from the projects component:

curl -u <prometheus_user>:<prometheus_password> https://<saagie-host>/prometheus/<component>/<exporter>/{<replicas>/}' (1)

Where:

1 <saagie-host> must be replaced with your Saagie domain and <prometheus_user> and <prometheus_password> with your Prometheus credentials.
For more information about Prometheus, you can refer to the Prometheus documentation.
It is not recommended to make more than one request per endpoint per 30-second period to the Prometheus API.