Downloading and Configuring Saagie

Download the Saagie installer and begin the configuration process, which takes place in your terminal command line. Answer to a series of prompts to generate the .mdl files, which will be used during the installation process.

Default answers are in [brackets].
Before you begin:

  1. Download the latest Saagie installer for Linux.

  2. Configure Saagie by following these steps:

    1. Launch the installer.

    2. Choose the deployment mode.

    3. Configure the cluster settings.

    4. Define the platform URL.

    5. Define the SMTP (Simple Mail Transfer Protocol).

    6. Configure the platform settings.

    7. Define the Kubernetes CIDR (Classless Inter-Domain Routing).

    8. Assign the pods.

    9. Define the access modes.

    10. Configure Prometheus authentication.

    11. Configure the HTTP proxy.

    12. Configure the storage size settings.

    13. Configure EKS (Amazon EKS Only).

    14. Configure ingress.

    15. Set the Docker registry (dedicated mode only)

    16. Configure the technology repository (dedicated mode only)

    17. Retrieve the Saagie file.

    As you work through the configuration process, answer carefully to prompts. It is much easier to make a correction before moving on to the next prompt.

    In case of error, follow these instructions:

    1. Close the installer.

    2. Delete the .mdl file that contains the mistake.

    3. Relaunch the installer.

    4. Continue to follow the prompts.

Launch the Installer

  1. Launch the installer, by running the following command:

    ./bin/saagiectl configure

  2. Answer all the prompts.

    • Each prompt is described in detail in the sections below. Follow along with this document as you complete the configuration process.

    • Each prompt generates a file.

Deployment Mode

settings note a File generated → deploymentmode.mdl

What is your K8s provider ? (valid answers: gke, eks, aks, custom) [custom]:

Where:

  • gke is for Google Cloud.

  • eks is for Amazon Web Services.

  • aks is for Microsoft Azure.

  • custom is for any other type of Kubernetes cluster management.

What is the type of registry? (valid answers: 'OFFICIAL', 'CUSTOM') [OFFICIAL]:

Where:

  • If you choose OFFICIAL, Docker images will be pulled from the Saagie Docker registry.

  • If you choose CUSTOM, Docker images will be pulled from your Docker registry.

What is your installation Id:

Where:

We strongly recommend that you use your company name or a shortened version of it. 
Do you want to isolate execution nodes? (valid answers: 'true' or 'false') [true]

Where:

  • If you answer true, your cluster nodes must be labeled beforehand to dedicate their execution to a specific platform. The isolated mode will allow you to separate the workload between your platforms on dedicated nodes. This will prevent the executions of one platform to intrude on another platform’s resources. For more information, see Node Isolation.

    Node labelling must be done beforehand, when configuring your cluster.

  • If you answer false, your cluster nodes will be undifferentiated. Your workload will be executed wherever there is space. In other words, your Saagie installation and all runs related to your Saagie platform(s) will be executed on any available node in your cluster.

You cannot switch between modes seamlessly. If you choose to configure Saagie to isolate your workload, it is not recommended to reconfigure it in non-isolated mode. If you choose to switch between modes anyway, you will have to manually restart all your apps.

To know if you have configured Saagie in isolated or non-isolated mode, you can run the following command lines:

TOKEN=$(curl -k -X "POST" -H "Content-Type: application/json" -H "Saagie-Realm: $INSTALLATION_ID" "$BASE_URL/authentication/api/open/authenticate" --data '{"login":$LOGIN, "password":$PASSWORD}')
curl -X "GET" -k "$BASE_URL/settings/api/v1/settings" \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

It will return, among others, the nodeIsolation value with NONE in non-isolated mode or PLATFORM_ISOLATED in isolated mode.

 
Authentication Mode (valid answers: LDAP, STANDARD) [STANDARD]:

Where:

  • LDAP is to rely on your corporate LDAP for identity and access management.

  • STANDARD is to rely on Saagie's built-in user management.

Cluster Settings

settings note a File generated → settings.mdl

Metric data is mandatory for billing. It is also sent to the Saagie server. To opt out of sending anonymous data, choose false. (valid answers: 'true' or 'false') [true]:

Where:

  • true is to keep anonymous data tracking.

  • false is to avoid sending anonymous data.

Do you need a custom extra volume for FluentBit ? (valid answers: 'true' or 'false') [false]:

Where:

  • true is to customize the Fluent Bit volume.

  • false is to keep the default volume.

Expose Prometheus endpoint ? (valid answers: 'true' or 'false') [false]:

Where:

  • true is to expose Prometheus metrics via HTTP.

  • false will ask you for your username and password later.

We use Prometheus for monitoring purposes. 
Do you need to define a HTTP/HTTPS proxy ? (valid answers: 'true' or 'false') [false]:

Where:

  • true is to define HTTP/HTTPS proxy.

  • false will ask you to provide HTTP/HTTPS proxy and the related credentials, if any.

Platform URL

settings note a File generated → url.mdl

The answers to these prompts must match what you determined for your DNS entry.

Platform url suffix:

It must be the same suffix you will use for your DNS entry.
For example, workspace.

Platform url domain:

This is the DNS entry under which your Saagie instance will be hosted.
For example, dundermifflin.com.

→ The above sample answers results in the URL dunder-workspace.dundermifflin.com.

SMTP (Simple Mail Transfer Protocol)

settings note a File generated → smtp.mdl

SMTP Host:

This is the IP or DNS name of the SMTP host.
For example, smtp.mailgun.org.

SMTP Port (number required) [25]:

The answer is usually either 25, 465, or 587.

SMTP: Enable authentication (valid answers: 'true' or 'false') [true]:

Where:

  • true is to enable the SMTP authentication, if any.

  • false is to disable the SMTP authentication, if you don’t have any.

SMTP: Transport protocol (valid answers: smtp, smtps) [smtp]:

Choose between smtp or smtps depending on your infrastructure.

SMTP: Enable starttls (valid answers: 'true' or 'false') [true]:

Where:

  • true is to allow the SMTP server to negotiate the use of TLS.

  • false is to prevent the SMTP server to negotiate the use of TLS.

SMTP username:
SMTP password:
Repeat for confirmation:

SMTP username and SMTP password are the credentials of the account from which Saagie emails will be sent.

The password must contain at least eight characters, including upper case (A-Z), lower case (a-z), numbers (0-9), and special characters (!, $, #, %, etc). 
Platform email sender ? (your SMTP gateway must allow this email address as the sender):

This is the email address used to send emails from Saagie.

The email address can be used for job alerts and resetting your password.

Platform Settings

settings note a File generated → platforms.mdl

How many platform(s) do you want to create/configure/install? (number required) [1]: 2

Indicate the number of platforms you want.

What is the Platform name?:

Enter the name of your platform. You can choose one that best suits your needs.

The order in which the platforms are declared during configuration must match the order of the platform IDs you entered in the node pool when configuring your cluster.
The platform name will be displayed in the Platforms menu. 
What is the Platform authorized egress CIDR block? [0.0.0.0/0]:

Define an authorized destination network for platform egress (outgoing) communication from the platform.

Do you want to enable GPU option ? (valid answers: 'true' or 'false') [false]:

Where:

  • true is to enable the GPU option, which is required to run processes on GPU nodes.

  • false is to disable the GPU option.

Do you want to customize the data lake url ? (valid answers: 'true' or 'false') [false]
Custom data lake url: http://www.mydatalakeurl.com

Where:

  • true is to define a custom URL for this platform’s data lake.

  • false is to have an automatically generated URL for this platform’s data lake.

Kubernetes CIDR (Classless Inter-Domain Routing)

settings note a File generated → k8scidr.mdl

K8S CIDR IP Range [0.0.0.0/0]:

Specify the IP address or range to which you are joining your Kubernetes API server. You must use the IP or range of your physical network interface of your master server, not the cluster IP of the Kubernetes service in the default namespace.

Remember to add /32 as a netmask if you specify only one address.

Pods Assignment

Fluent Bit Volume

settings note a File generated → fluentbit.mdl

You will only be asked to answer this prompt if you answered true to the Fluent Bit volume prompt when you configured your cluster settings. 
Fluent Bit volume mountPath:

Access Mode

Depending on your answer to the Authentication Mode prompt when choosing the deployment mode, you will be asked different prompts.

  • If you answered STANDARD beforehand, you will be asked to answer the following prompts:

    • Standard Access

    • Customer Access

    settings note a File generated → keycloakaccess.mdl

    Standard Password:
Repeat for confirmation:

    This is the default user, with administrative rights, that will be used for Saagie internal communication. The password will be the one used by M2M user for internal communication.

    The password must contain all the following character types: upper case (A-Z), lower case (a-z), numbers (0-9), and special characters (!, $, #, %, etc).

    settings note a File generated → customeraccess.mdl

    Customer Password:
Repeat for confirmation:

    This is the default user, with administrative rights, that you will use to connect to the Saagie user interface for the first time. The password will be the one used by the customer_admin user.

  • If you answered LDAP beforehand, you will be asked to answer the following prompts:

    settings note a File generated → ldapaccess.mdl

    LDAP Login:
LDAP Password:
Repeat for confirmation:

    This is the credentials for the User DN that Saagie components will use to communicate with your LDAP service.

    LDAP Admin group:

    This is your administration group name.

LDAP

settings note a File generated → ldap.mdl

LDAP Vendor (valid answers: ldap, ad, other) [ad]:

Where:

  • ldap is for LDAP.

  • ad is for Active Directory.

  • other is for other vendor.

LDAP Host:

This is the IP or hostname of the LDAP server.
For example, ldap.priv.company.com.

LDAP Base DN:

Base DN of the LDAP directory.
For example, dc=company,dc=com.

LDAP User DN [CN=Users]:

This is the prefix where to search for users.

Do not add the base DN here. 
LDAP User Object Classes (expecting a comma-separated list) [person, organizationalperson, user]:

This is the comma-separated list of expected object classes for the user.

LDAP Username Attribute [cn]:

This is the attribute used to identify a user.

LDAP RDN Attribute [cn]:

This is the attribute used for user’s RDN.

LDAP UUID Attribute [objectGUID]:

This is the operational attribute, which is unique in the whole directory.

LDAP Bind DN attribute [cn]:

This is the user and the user’s location in the LDAP directory.

LDAP Group Membership Attribute Type (valid answers: DN, UID) [DN]:

How group members are defined in LDAP directory.

LDAP Group Name Attribute [cn]:

This is the attribute used to identify a group.

LDAP Group DN [ou=Groups]:

This is the directory where to search for groups.

Do not add the base DN here. 
LDAP Group Membership Attribute [member]:

This is the attribute used by a group to declare members.

LDAP Group Object Classes (expecting a comma-separated list) [group]:

This is the comma-separated list of expected object classes for groups.

Prometheus

settings note a File generated → prometheus.mdl

You will only be asked to answer this prompt if you answered true to the Prometheus prompt when you configured your cluster settings. 
Prometheus endpoint HTTP Basic Authentication - Set user name [monitoring]:
Prometheus endpoint HTTP Basic Authentication - Set user password:
Repeat for confirmation:

This is the credentials for Prometheus HTTP monitoring.

HTTP Proxy

settings note a File generated → proxy.mdl

You will only be asked to answer this prompt if you answered true to the HTTP proxy prompt when you configured your cluster settings. 
Do you want to define a HTTP Proxy directive ? (valid answers: 'true' or 'false') [false]:
HTTP Proxy [http://proxy.saagie.com:3128]:
Do you want to define a HTTPS Proxy directive ? (valid answers: 'true' or 'false') [false]:
HTTPS Proxy [https://proxy.saagie.com:3128]:
Do you want to define a NO Proxy directive ? (valid answers: 'true' or 'false') [false]:
No Proxy [saagie.com, 10.0.0.0/8]:

Storage Size Settings

settings note a File generated → settingsservice.mdl

Settings Max storage size for apps (in MB) (number required) [128]:

This is the storage size limit in MB for an app volume.

EKS Configuration (Amazon EKS Only)

settings note a File generated → eksconfig.mdl

For security matter, provide the ARN of the role to assign to the Saagie jobs (see documentation):

This is the ARN of the role you created when your configured your EKS Kubernetes cluster.

Restrict to private network (private network needed on VPC - internal load balancer) (valid answers: 'true' or 'false') [false]:

Answer true if the load balancer for the Saagie frontend should not be exposed to the Internet.

Ingress Configuration

settings note a File generated → ingressconfig.mdl

You will only be asked to answer this prompt if you answered custom to the Kubernetes provider prompt when you configured your deployment mode. 
Does the cluster support load balancer auto-provisioning? (valid answers: 'true' or 'false') [true]:

Where:

  • Answer true if Saagie is deployed on a Kubernetes cluster that supports load balancer auto-provisioning. For more information, see the Kubernetes documentation on the LoadBalancer type.

  • Answer false if Saagie is deployed behind an external load balancer that you must configure. For more information, see the Kubernetes documentation on the NodePort type.

What kind of loadbalancer is in front of k8s cluster? (valid answers: 'L3' or 'L4' or 'L7') [L4]:

Where:

  • Answer L3 if Saagie is deployed behind a network load balancer.

  • Answer L4 if Saagie is deployed behind a TCP load balancer.

  • Answer L7 if Saagie is deployed behind an HTTP load balancer.

You should also configure your cluster to collect users' IP addresses, as Saagie will block them when there have been too many unsuccessful connection attempts.

Docker Registry (Dedicated Mode Only)

settings note a File generated → registry.mdl

Docker registry:

Set the Docker registry used to pull the image inside Kubernetes.

Technology Repository (Dedicated Mode Only)

settings note a File generated → technologiesrepository.mdl

Is the technologies repository an internal one, for offline deployment ? (valid answers: 'true' or 'false') [false]:

Where:

  • true is for an offline cluster where you have provided your technologies.zip file using the saagiectl command.

  • false is for a cluster where you will download the technologies.zip file via a URL.

Url of the technology repository:

This is the URL of the technology repository.

Does the technology repository use a different Docker registry than the product ? (valid answers: 'true' or 'false') [false]:

Where:

  • Answer true if the Docker images for the technology repository are hosted on a different Docker registry than the Docker registry for the product.

  • Answer false if the Docker images for the technology repository and the product are hosted on the same Docker registry.

Docker registry for the technologies repository:

This is the Docker registry of the technology repository.

Saagie File

settings note a File generated → saagie.mdl

The saagie.mdl file is automatically generated and compiles all the information from the configuration process.

If you find an error afterwards, you do not need to delete the saagie.mdl file to fix the error. Follow these instructions:

  1. Close the installer.

  2. Delete the .mdl file that contains the mistake.

  3. Relaunch the installer.

  4. Continue to follow the prompts.

Once you have corrected the error and the new file is generated, the saagie.mdl file is automatically updated.