Using Microsoft Azure Kubernetes Service (AKS)

Use this tutorial to create a Microsoft AKS cluster compatible with Saagie.

Before you begin:

Before creating a new cluster, set up your computer as follows:

Create an AKS account if you do not already have one.
Install and configure the Azure command-line interface (CLI).
Increase your VM-family vCPU quotas to 12.

Saagie, Microsoft AKS, and Kubernetes v1.25.x

Microsoft has forced the transition from AKS to Kubernetes v1.25 on OS cgroup_v2. Although Saagie is compatible with Kubernetes v1.25.x, we will not be natively compatible with AKS because we are still using cgroup_v1.
If you want to use Saagie on an AKS v1.25.x environment, you need to reinforce the cgroups_v1. To do this, refer to Azure’s README.md file on GitHub: Revert Kubernetes 1.25 to cgroup v1.

Creating or Configuring Your Cluster (Isolated Mode)

For more information about the isolated mode, see Node Isolation.

Creating a New Cluster
Configuring an Existing Cluster

Create a new resource group in the same region where you want to create the cluster by running the az command line:
```
az group create --location <location> --name <resource group name> (1)
```
Where:

1 <location> and <resource group name> must be replaced with your location and resource group name.

If you prefer to use the Azure portal, select Create a resource, then search for Resource group, and finally, create your new group.

Create your cluster with the following command line:

    az aks create \
    --network-plugin azure \
    --network-policy calico \
    --kubernetes-version <version> \ (1)
    --node-vm-size Standard_DS3_v2 \
    --node-count 2 \
    --location <location> \ (2)
    --name <cluster name> \ (3)
    --resource-group <resource group name> \ (4)
    --nodepool-labels io.saagie/type=common io.saagie/installationId=<installationId> (5)

    az aks nodepool add \
    --cluster-name <cluster name> \ (3)
    --resource-group <resource group name> \ (4)
    --name platform<platformId> \ (6)
    --node-count 3 \
    --labels io.saagie/type=platform io.saagie/installationId=<installationId> io.saagie/platform-assignable=<platformId> \ (5) (6)

Where:

1 <version> must be replaced with a Kubernetes version that is compatible with Saagie. Current compatible Kubernetes versions are 1.20.x, 1.21.x, 1.23.x, 1.24.x and 1.25.x.

2 <location> must be replaced with your location.

3 <cluster name> must be replaced with the name of your cluster.

4 <resource group name> must be replaced with your resource group name.

5 <installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.

<platformId> must be replaced with the ID of the platform. It is determined during the configuration of your platform. Its value is defined according to the number of platforms and their order, starting from one. You can therefore predict it.

The order in which the platforms are declared during configuration must match the order of the platform IDs you entered here in the node pool. So remember it for later.

Once created, it will take about ten minutes before the cluster is ready to be used.

You can follow the AKS Quickstart as a reference to create your AKS cluster using the Azure CLI. The quick start guide assumes a basic understanding of Kubernetes, so read Kubernetes core concepts for AKS if necessary.

If you are using an existing Microsoft AKS cluster, create your configuration file by running the following command line:
```
az aks get-credentials --resource-group <resource group name> --name <cluster name> (1)
```
Where:

1 <resource group name> and <cluster name> must be replaced with your resource group name and name of your cluster.

Verifying Your Kubernetes Cluster

Run the following command line to verify that you have access to your Kubernetes cluster:

kubectl get nodes

The output of the command should look like the following:

NAME                                           STATUS   ROLES    AGE    VERSION
ip-192-168-15-134.eu-west-1.compute.internal   Ready    <none>   9m8s   v1.13.8-eks-cd3eb0
ip-192-168-35-150.eu-west-1.compute.internal   Ready    <none>   9m3s   v1.13.8-eks-cd3eb0
ip-192-168-88-76.eu-west-1.compute.internal    Ready    <none>   9m7s   v1.13.8-eks-cd3eb0

All nodes must have the ready status.

Creating Storage Classes for Your Saagie Platform

Create storage classes to store data in a non-volatile device during and after the execution of your platform. Storage classes are stored in a file named storage.yml which contains the configuration for your storageClass resources:

common-storageclass: Used to store Saagie data, such as databases.
<installationId>-storageclass: Used to store job data, such as uploaded artifacts.
<installationId>-app-storageclass: Optional storageClass used to store app and job data on a different provisioner.

Create the storage.yml file for your Microsoft Azure Kubernetes cluster.

The following sample storage.yml file for AKS can be customized according to your needs.

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: common-storageclass
parameters:
  cachingmode: ReadOnly
  kind: Managed
  storageaccounttype: Standard_LRS
provisioner: kubernetes.io/azure-disk
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: <installationId>-storageclass (1)
parameters:
  cachingmode: ReadOnly
  kind: Managed
  storageaccounttype: Standard_LRS
provisioner: kubernetes.io/azure-disk

Where:

1	`<installationId>` must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.

To store app data and job data on different provisioners, include the following lines in the same storage.yml file:

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: <installationId>-app-storageclass (1)
parameters: (2)

provisioner: (3)

Where:

1	`<installationId>` must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.
2	The `parameters` value must contain the parameters for app data.
3	The `provisioner` value must indicate your second provisioner used to store app data.

Apply the storage.yml file by running the following command line:
```
kubectl apply -f storage.yml
```
Confirm that the storage classes are available by running the following command line:
```
kubectl get sc
```

Creating the requirements.yml File

All Saagie deployments need the same requirements.yml file regardless of your cloud provider. The requirements.yml file will create two service accounts on the saagie-common namespace:

sa-saagie-deploy with the cluster-admin role
traefik-ingress-controller with its related ClusterRole and ClusterRoleBinding

Create your requirements.yml file as follows:

---
apiVersion: v1
kind: Namespace
metadata:
  name: <installationId>
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-saagie-deploy
  namespace: <installationId>
automountServiceAccountToken: true
imagePullSecrets:
  - name: saagie-docker-config
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-saagie-deploy-crbinding
  namespace: <installationId>
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: sa-saagie-deploy
  namespace: <installationId>
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: <installationId>
imagePullSecrets:
  - name: saagie-docker-config
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-cluster-binding
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: <installationId>
roleRef:
  kind: ClusterRole
  name: traefik-ingress-cluster
  apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-cluster
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - networking.k8s.io
    resources:
    - ingresses/status
    verbs:
    - update
  - apiGroups:
    - traefik.containo.us
    resources:
    - middlewares
    - middlewaretcps
    - ingressroutes
    - traefikservices
    - ingressroutetcps
    - ingressrouteudps
    - tlsoptions
    - tlsstores
    - serverstransports
    verbs:
    - get
    - list
    - watch
  - apiGroups:
    - apiextensions.k8s.io
    resources:
    - customresourcedefinitions
    verbs:
    - create
  - apiGroups:
    - apiextensions.k8s.io
    resourceNames:
    - middlewares.traefik.containo.us
    - middlewaretcps.traefik.containo.us
    - ingressroutes.traefik.containo.us
    - traefikservices.traefik.containo.us
    - ingressroutetcps.traefik.containo.us
    - ingressrouteudps.traefik.containo.us
    - tlsoptions.traefik.containo.us
    - tlsstores.traefik.containo.us
    - serverstransports.traefik.containo.us
    resources:
    - customresourcedefinitions
    verbs:
    - get

Where:

<installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.

Apply your requirements.yml file by running the following command line:
```
kubectl apply -f requirements.yml
```
The output of the command should look like the following:
```
namespace/<installationId> created
serviceaccount/sa-saagie-deploy created
...
```
Where:
- <installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.

Applying or Installing Secret saagie-docker-config

Saagie Docker images are pulled from a private registry that requires credentials. The credentials should have been provided to you.

Apply or install the secret:

Apply: If you receive the credentials in a Kubernetes secret file, apply the secret to your cluster by running the following kubectl command line:
```
kubectl apply -n <installationId> -f saagie-docker-config.yaml (1)
```
Where:

1 <installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.

Install: If you receive a username and password, install the secret on your cluster by running the following kubectl command line:

kubectl create secret docker-registry -n <installationId> saagie-docker-config \ (1)
  --docker-server=<registry server \ (2)
  --docker-username=<username> \ (3)
  --docker-password=<password> (4)

Where:

1	`<installationId>` must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.
2	`<registry server>` must be replaced with the Docker repository hosting Saagie images.
3	`<username>` must be replaced with the username provided to you.
4	`<password>` must be replaced with the password provided to you.

Edit the default service account to reference the saagie-docker-config secret by running the following kubectl command line:
```
kubectl patch serviceaccount -n <installationId> default -p '{"imagePullSecrets":[{"name" : "saagie-docker-config"}]}' (1)
```
Where:

1 <installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.
Confirm that the secret is properly installed by running the following command line:
```
kubectl get secret -n <installationId> (1)
```
Where:

1 <installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.

The output of the command should look like the following:
```
NAME                   TYPE                             DATA   AGE
saagie-docker-config   kubernetes.io/dockerconfigjson   1      2m43s
```

Installing Saagie in Offline Mode

In case your Kubernetes cluster is not connected to the internet, you can install Saagie in an offline mode. To do this, you need to manage your own Docker registry. This registry contains images of the Saagie product and Saagie technologies.

This part will guide you to upload the resources in your registry and install the repository in your cluster.

Saagie gives you the archives of Docker images needed to run your platform, as well as the technologies.

Uploading Docker Images

Before you begin:

To upload the Docker images to your registry, make sure you meet all the following requirements. You must have:

A machine with access to your Docker registry.
The tar archives that are provided by Saagie and that contain the Saagie product and technologies.
The Skopeo command line tool installed on your machine. For more information, see the Git repository dedicated to Skopeo.
The credentials to push the images into the registry, if any.

Run the following command line to decompress the archive:
- Uploading Saagie Product Archive
- Uploading Saagie Technologies Archive
untar xvf <product-tar-archive> (1)
Where:

1 tar archive is the file name of the Saagie product provided by Saagie itself.
untar xvf <technologies-tar-archive> (1)
Where:

1 tar archive is the file name of the Saagie technologies provided by Saagie itself.
OPTIONAL: If you need to require authentication, configure the user and password to connect to your registry using skopeo login. For more information, you can refer to the Git repository dedicated to Skopeo.
Run the following command line in the decompressed archive to start the image upload:
```
./pushall.sh <registry> (1)
```
Where:

1 <registry> is the hostname of your Docker registry.

Installing Technology Repository

The repository containing your technologies must be installed manually in your cluster.

For more information on adding technologies, see our SDK documentation.

Copy the path to the technologies.zip file that contains your technologies.
Run the following saagiectl command line to install the repository in your cluster:
```
./bin/saagiectl upload technologies --file <technologies-file> (1)
```
Where:

1 <technologies-file> must be replaced with the path to your technologies.zip file.

Setting Up SMTP (Simple Mail Transfer Protocol) Requirements

An SMTP server is required to send, receive, and relay outgoing mail between your Saagie platform and users' email addresses. For this reason, Saagie must have access to your SMTP server and be compatible with the following configurations:

SMTP authentication can be anonymous or require authentication.
SMTP transport can be SMTP or SMTPS.
You must have a valid SSL certificate.

Once configured, you will be able to use your user email address to receive status alerts or change and reset the password associated with your Saagie account.

Deploying and Updating Your SSL Certificate

Use this tutorial to deploy and update your SSL certificate to your Kubernetes cluster.

The process for deploying and updating your SSL certificate is the same.
If you need help generating a self-signed certificate, you can check the following tutorials:
- Direct guide: Digital Ocean OpenSSL Essentials
- Detailed guide: Jamie Linux OpenSSL Certificate Authority

Before you begin:

Make sure your SSL certificate is valid by checking the following constraints:

The certificate’s validity date must be correct.
The certificate must include at least the Saagie product URL.
The KeyUsage attribute must include the digitalSignature and keyEncipherment elements.

Open your preferred terminal command.
To deploy (or update) your SSL certificate, run the following command line:
```
kubectl create secret tls saagie-common-tls --cert=cert.pem --key=cert.key -n <installationId> --dry-run=client -o yaml | kubectl apply -f -
```
Where:
- <installationId> must be replaced with your installation ID. It must match the prefix you have determined for your DNS entry.