About Docker Images Within Saagie

This page describes how Saagie uses Docker images to launch jobs and apps, and how Docker images must be created to work in Saagie.

This documentation assumes a basic understanding of Docker, including how to create a Docker image. For more information about Docker basics, check out the Docker’s official documentation.

How Does Saagie Use Docker Images?

Once a Saagie job or app is done, whether it succeeds or fails, it will not be relaunched.

There are some key differences between Saagie jobs and apps:

  • Jobs work with the command line and packages.

  • Apps expose ports and mount volumes, then share that information with all instances of the app.

Launching Saagie Jobs and Apps

When a Saagie job or app is launched, the following elements are added to the Docker image:

  • Environment variables, previously determined in the project or platform.

  • Configuration files mounted to standard paths, includes data lake configuration files and how tools, such as Kerberos, are configured.

A Saagie job also adds the following to the Docker image:

  • If the chosen technology requires a package, such as a .zip, .py, or .jar, it will be mounted in the job.

    • Packages are uploaded when a job is configured.

    • Saagie will replace {file} with the package absolute path in the command line from the job configuration process.

    • The package is unaltered, for example, it is not decompressed or renamed.

  • The command line is saved in the file main_script and mounted as executable on the path /sandbox/main_script.

A Saagie app also adds the following to the Docker image:

  • Mounted directories corresponding to the volume selected when creating the job.

Jobs and Apps

How to Create Docker Images Compatible With Saagie?

In order for your Docker images to be compatible with Saagie, consider the information below.

For Jobs

When using the COMMAND_LINE feature, use a Dockerfile to create a new Docker image. The Dockerfiles used by Saagie to create our Docker images are the following:

Dockerfile
COPY entrypoint /entrypoint
RUN chmod 755 /entrypoint

WORKDIR /sandbox

ENTRYPOINT  ["bash","/entrypoint"]
entrypoint
#!/bin/bash

set -euo pipefail


if test -f /sandbox/main_script;
then sh /sandbox/main_script;
fi;

The command line is saved in the file /sandbox/main_script.

The ENTRYPOINT for the Docker image will run the bash script entrypoint, which verifies that the /sandbox/main_script file exists.

The Docker image will launch the file and, as a result, run the command line entered by the user.

For more examples of how Saagie uses Docker images, see the Saagie repository for official job examples.

For Apps

There are no guidelines for creating Docker images for Saagie apps.

You can check out these examples on the official Saagie repository:

  • Zeppelin: No modification is required for URL rewriting.

  • Jupyter: Modify the command line for URL rewriting.

  • RStudio: Must use a NGINX server, which then manages URL rewriting.

Useful Links