Create Amazon EKS cluster

This section will help you create a Saagie-compatible Kubernetes cluster using the Amazon Elastic Kubernetes Service (EKS).

1. Prerequisites

You need to set up your computer before creating a new cluster.

You need an Amazon Web Services (AWS) account.
Enable appropriate access keys and secret access keys.
Install kubectl.
Install the Amazon Web Services CLI.
Configure the Amazon Web Services CLI.
Install eksctl.

2. Create or configure cluster

Create new cluster
Configure existing cluster

Refer to the Amazon EKS User Guide to create your EKS cluster.

For more details on access management in Amazon Web Services, see the Amazon Web Services documentation.

Choose the eksctl method.
Confirm that the AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables are defined.

Create a cluster.yml file with the following content:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
 name: <cluster name> (1)
 region: <region> (2)
 version: "<version>" (3)

nodeGroups:
 - name: ng-1
   instanceType: m5.2xlarge
   desiredCapacity: 3

1	Replace `<cluster name>` with the name of your cluster. It must be a string that has letters from a to z (regardless of case) or digit.
2	Replace `<region>` with the region in which the cluster will be used.
3	Replace `<version>` with a Kubernetes version that is compatible with Saagie. Current compatible version is `1.19.x`. Use quotes around the version number as `eksctl` requires a string text and not a float number in the YAML file. Run the command: `eksctl create cluster -f cluster.yml`

If you are using an existing Amazon EKS cluster, create your configuration file by running the following aws command line:
```
aws eks --region <aws region> update-kubeconfig --name <cluster name> (1)
```
1 Replace <aws region> and <cluster name> with your region and cluster name.
Once your configuration file is created, check the connectivity.

Refer to the Kubernetes documentation if needed.

3. Verify your Kubernetes cluster

Run the following command to verify that you have access to your Kubernetes cluster:
```
kubectl get nodes
```
The output should be similar to this:

NAME                                           STATUS   ROLES    AGE    VERSION
ip-192-168-15-134.eu-west-1.compute.internal   Ready    <none>   9m8s   v1.13.8-eks-cd3eb0
ip-192-168-35-150.eu-west-1.compute.internal   Ready    <none>   9m3s   v1.13.8-eks-cd3eb0
ip-192-168-88-76.eu-west-1.compute.internal    Ready    <none>   9m7s   v1.13.8-eks-cd3eb0

All nodes must have the status ready.

4. Install Calico

Calico is a network policy engine for Kubernetes used to implement network segmentation and tenant isolation.

Amazon EKS does not automatically install Calico, which is necessary for your Kubernetes cluster.

Refer to Installing Calico on Amazon EKS as needed.
If you did not install Calico while creating your cluster, run this command now:
```
kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/master/config/v1.3/calico.yaml (1)
```
1 Make sure that the version of Calico is compatible with your cluster.

5. Set up a role for Saagie jobs

The Kubernetes pods responsible for running Saagie jobs use a service account associated with an AWS role, which configures access rights.

If you choose to skip it, note that jobs launched on Saagie may obtain admin rights on the AWS API.

First, choose the AWS policy that meets your needs.
- Example: Jobs won’t need access to AWS resources
  - ARN: arn:aws:iam::aws:policy/AWSDenyAll
- Example: Jobs need read access to S3
  - ARN: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
- Refer to AWS’s user guide to create your own policy.

Then, create the file create-job-role.sh with the following content, defining the variables indicated:

#!/bin/bash
set -e

# Define your variables here. Variables are explained below the code block.
CLUSTER_NAME=<cluster-name> (1)
SAAGIE_PREFIX=<prefix> (2)
ROLE_NAME=<role-name> (3)
AWS_POLICY_ARN=<policy-arn> (4)

ISSUER_URL=$(aws eks describe-cluster \
    --name $CLUSTER_NAME \
    --query cluster.identity.oidc.issuer \
    --output text)
ISSUER_HOSTPATH=$(echo $ISSUER_URL | cut -f 3- -d'/')
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
PROVIDER_ARN="arn:aws:iam::$ACCOUNT_ID:oidc-provider/$ISSUER_HOSTPATH"
cat > saagie-job-trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "$PROVIDER_ARN"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${ISSUER_HOSTPATH}:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "${ISSUER_HOSTPATH}:sub": "system:serviceaccount:${SAAGIE_PREFIX}-project*:*"
        }
      }
    }
  ]
}
EOF
aws iam create-role \
  --role-name $ROLE_NAME \
  --assume-role-policy-document file://saagie-job-trust-policy.json
aws iam update-assume-role-policy \
  --role-name $ROLE_NAME \
  --policy-document file://saagie-job-trust-policy.json
aws iam attach-role-policy \
  --role-name $ROLE_NAME \
  --policy-arn $AWS_POLICY_ARN
aws iam get-role \
  --role-name $ROLE_NAME \
  --query Role.Arn --output text

1 Replace <cluster-name> with the name of your EKS cluster.

Replace <prefix> with the prefix determined for your DNS entry.

In this DNS entry dunder-workspace.dundermifflin.com, the <prefix> is dunder.

3 Replace <role-name> with the name of the role which will be created (for example: saagie_job_role).

Replace <policy-arn> with the ARN of the policy chosen for Saagie jobs.

Make a note of this <prefix>, you will need it in several steps to come.

Make the file executable with the following command:
```
chmod +x create-job-role.sh
```
Launch the creation of the role by executing the script file:
```
./create-job-role.sh
```
After running the script, the ARN of the role you created will be printed in the output.

Note the ARN; you’ll need it when configuring your instance.

6. Create requirements

All Saagie deployments need the same requirements.yml file, regardless of your cloud provider. The requirements.yml file will create the following on your cluster:

Two service accounts on the saagie-common namespace:
- sa-saagie-deploy with cluster-admin role
- traefik-ingress-controller with its related ClusterRole and ClusterRoleBinding

6.1. Create `requirements.yml` file

Create your requirements.yml with the code exactly as it appears below.

---
apiVersion: v1
kind: Namespace
metadata:
  name: saagie-common
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-saagie-deploy
  namespace: saagie-common
automountServiceAccountToken: true
imagePullSecrets:
  - name: saagie-docker-config
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-saagie-deploy-crbinding
  namespace: saagie-common
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: sa-saagie-deploy
  namespace: saagie-common
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: saagie-common
imagePullSecrets:
  - name: saagie-docker-config
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-cluster-binding
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: saagie-common
roleRef:
  kind: ClusterRole
  name: traefik-ingress-cluster
  apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-cluster
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - extensions
    - networking.k8s.io
    resources:
    - ingresses/status
    verbs:
    - update
  - apiGroups:
    - traefik.containo.us
    resources:
    - middlewares
    - ingressroutes
    - traefikservices
    - ingressroutetcps
    - ingressrouteudps
    - tlsoptions
    - tlsstores
    verbs:
    - get
    - list
    - watch
  - apiGroups:
    - apiextensions.k8s.io
    resources:
    - customresourcedefinitions
    verbs:
    - create
  - apiGroups:
    - apiextensions.k8s.io
    resourceNames:
    - middlewares.traefik.containo.us
    - ingressroutes.traefik.containo.us
    - traefikservices.traefik.containo.us
    - ingressroutetcps.traefik.containo.us
    - ingressrouteudps.traefik.containo.us
    - tlsoptions.traefik.containo.us
    - tlsstores.traefik.containo.us
    resources:
    - customresourcedefinitions
    verbs:
    - get
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/cluster-service: "true"
  name: 00-saagie-common-psp
spec:
  allowPrivilegeEscalation: false
  allowedHostPaths:
    - pathPrefix: /etc/machine-id
      readOnly: true
    - pathPrefix: /var/log
      readOnly: true
    - pathPrefix: /var/lib/docker/containers
      readOnly: true
    - pathPrefix: /data/docker/containers
      readOnly: true
  fsGroup:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
    - configMap
    - emptyDir
    - secret
    - persistentVolumeClaim
    - hostPath
    - projected
    - downwardAPI
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/cluster-service: "true"
  name: 00-saagie-project-psp
spec:
  allowPrivilegeEscalation: true
  fsGroup:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
    - configMap
    - emptyDir
    - secret
    - persistentVolumeClaim
    - projected
    - downwardAPI
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/cluster-service: "true"
  name: psp:saagie-common:saagie-common-cluster-psp
rules:
  - apiGroups:
      - policy
    resourceNames:
      - 00-saagie-common-psp
    resources:
      - podsecuritypolicies
    verbs:
      - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/cluster-service: "true"
  name: psp:saagie-common:saagie-project-cluster-psp
rules:
  - apiGroups:
      - policy
    resourceNames:
      - 00-saagie-common-psp
    resources:
      - podsecuritypolicies
    verbs:
      - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp:saagie-common:saagie-deploy-psp-crbinding
  namespace: saagie-common
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:saagie-common:saagie-common-cluster-psp
subjects:
  - kind: Group
    name: system:serviceaccounts:saagie-common

6.2. Apply `requirements.yml` file

Run the following command to apply your requirements.yml file:

kubectl apply -f requirements.yml

The output should be similar to this:

namespace/saagie-common created
serviceaccount/sa-saagie-deploy created
...
rolebinding.rbac.authorization.k8s.io/psp:saagie-commin:saagie-deploy-psp-cribinding created

7. Apply or install secret `saagie-docker-config`

Saagie Docker images are pulled from a private registry that requires credentials. The credentials should have been provided to you.

Apply or install the secret:
- Apply: If you receive the credentials in a Kubernetes secret file, apply the secret to your cluster by running the following kubectl command:
  kubectl apply -n saagie-common -f saagie-docker-config.yaml
- Install: If you receive a username and password, install the secret by running the following kubectl command:
  kubectl create secret docker-registry -n saagie-common saagie-docker-config \ --docker-server=<registry server> \ (1) --docker-username=<username> \ (2) --docker-password=<password> (3)
  1 Replace <registry server> with the Docker repository hosting Saagie images.
  
  2 Replace <username> with the username provided to you.
  
  3 Replace <password> with the password provided to you

Edit the default service account to reference the saagie-docker-config secret by running the following kubectl command.

kubectl patch serviceaccount -n saagie-common default -p '{"imagePullSecrets":[{"name" : "saagie-docker-config"}]}'

Run the following command to confirm that the secret are properly installed:

kubectl get secret -n saagie-common

The output should be similar to this:

NAME                   TYPE                             DATA   AGE
saagie-docker-config   kubernetes.io/dockerconfigjson   1      2m43s

8. Offline cluster specifications

When your Kubernetes cluster is not connected to the Internet, you’ll install Saagie in offline mode.

For offline installation, you need to manage your own Docker registry containing images of the Saagie product as well as Saagie technologies. This section of documentation will guide you through uploading the resources to your registry and installing the repository in your cluster.

Saagie will provide the archives of the Docker images needed to run your platform, as well as the technologies.

8.1. Upload Docker images

To upload the Docker images in your registry, you need:

A machine with access to your Docker registry
tar archives provided by Saagie (Saagie product and Saagie technologies)
Command line tool skopeo installed on your machine (refer to Install Skopeo)
Credentials to push images into the registry (if any)

8.1.1. Upload Saagie product archive

Once you meet these requirements, complete the following steps to upload your Docker images.

First, run the following command to decompress the archive:
```
untar xvf <product-tar-archive> (1)
```
1 The file name of the Saagie product tar archive Saagie provided.
Next, if you need to require authentication, configure the user and password to connect to your registry using skopeo login (refer to Skopeo login).
Finally, run the following command in the decompressed archive to launch the image upload:
```
./pushall.sh <registry> (1)
```
1 <registry> is the hostname of your Docker registry.

8.1.2. Upload Saagie technologies archive

The process is the same as for the Saagie technologies tar archive.

If you configured authentication on this machine when uploading the first tar archive, you will not need to configure it again.

Run the following command to decompress the archive:
```
untar xvf <technologies-tar-archive> (1)
```
1 The file name of the Saagie technologies tar archive Saagie provided.
If you need to require authentication, configure the user and password to connect to your registry using skopeo login (refer to Skopeo login).
Run the following command in the decompressed archive to launch the image upload:
```
./pushall.sh <registry> (1)
```
1 <registry> is the hostname of your Docker registry.

8.2. Install technology repository

The repository containing your technologies needs be installed manually in your cluster.

Refer to our SDK documentation for more information about adding technologies.

Copy the path to the technologies.zip file that contains your technologies.
Run the following saagiectl command to install the repository in your cluster:
```
./bin/saagiectl upload technologies --file <technologies-file> (1)
```
1 Replace <technologies-file> with the path to your technologies.zip file.

9. SMTP requirements

A SMTP Server is mandatory to send, receive, and/or relay outgoing mail between your Saagie platform and users email address.
Therefore Saagie must have access to your SMTP server and is compatible with the following configurations:

SMTP authentication can either be anonymous or require authentication.
SMTP transport can be SMTP or SMTPS.
You must have a valid TLS certificate.

Once configured you will be able use your user email address to gets status alerts or change/reset the password associated with your Saagie account.

10. Install certificate

A TLS certificate is required and needs to be installed in your Kubernetes cluster. Make sure that your TLS certificate is valid for the DNS entry you determined when verifying your prerequisites.

The following instructions will guide you through installing or updating the certificate in your Kubernetes cluster.

10.1. Install certificate

Generate an encoded version of your certificate files.
1. Generate a Base-64 encoded version of the certificate:
  - Linux:
    
    cat server.pem | base64 -w 0
  - macOS:
    
    cat server.pem | base64 | tr -d '\n'
2. Generate a Base-64 encoded version of the key:
  - Linux:
    
    cat server-key.pem | base64 -w 0
  - macOS:
    
    cat server-key.pem | base64 | tr -d '\n'

Create a certificate.yml file with the following content.

apiVersion: v1
kind: Secret
metadata:
  name: saagie-common-tls
  namespace: saagie-common
type: kubernetes.io/tls
data:
  tls.crt: <encoded certificate> (1)
  tls.key: <encoded key> (2)

1	Replace `<encoded certificate>` and
2	`<encoded key>` with the values from your encoded files. Deploy your certificate. kubectl apply -f certificate.yml

10.2. Update certificate

If you are updating your certificate after Saagie has been installed, you will also need to restart the ingress controllers and verify that the new certificate is now being used.

Restart the ingress controllers to take into account the new certificate.

kubectl delete pod -n saagie-common -l "app.kubernetes.io/component=controller,app.kubernetes.io/instance=saagie-common,app.kubernetes.io/name=ingress-nginx"

Verify that the new certificate is being used by running the following command, then checking the server certificate section of the output:
```
curl -k -v -I "https://<prefix>-<suffix>.<domain>" (1)
```
1 Replace <prefix>, <suffix>, and <domain> with the DNS entry components you determined when verifying your prerequisites.

1	Replace `<registry server>` with the Docker repository hosting Saagie images.
2	Replace `<username>` with the username provided to you.
3	Replace `<password>` with the password provided to you

Create Amazon EKS cluster

1. Prerequisites

2. Create or configure cluster

3. Verify your Kubernetes cluster

4. Install Calico

5. Set up a role for Saagie jobs

6. Create requirements

6.1. Create requirements.yml file

6.2. Apply requirements.yml file

7. Apply or install secret saagie-docker-config

8. Offline cluster specifications

8.1. Upload Docker images

8.1.1. Upload Saagie product archive

8.1.2. Upload Saagie technologies archive

8.2. Install technology repository

9. SMTP requirements

10. Install certificate

10.1. Install certificate

10.2. Update certificate

6.1. Create `requirements.yml` file

6.2. Apply `requirements.yml` file

7. Apply or install secret `saagie-docker-config`