2021

December 2021

Here are the highlights of new and updated features for this release:

  1. New Features
    The product has been released with the following features:

    • You can create conditional pipelines to run multiple jobs in parallel.

    • Resource allocation quotas can now be defined for each job to control their resource consumption.

    • You can now select time zones when configuring scheduled runs for jobs and pipelines.

    • You can now physically delete projects, jobs, pipelines, environment variables, and Docker credentials from their platforms.

  2. Bug Fixes

    • Fixes have been applied to all Spark runtime contexts available in Saagie repositories.

    • Job, pipeline, and app status updates are now automatic.

  3. User Experience Improvements
    Deprecated technologies are tagged as such in technology repositories and when used by a job.

  4. Saagie’s Technology Repository Updates
    New technology versions have been added.

New Features

Conditional Pipelines

You can now create conditional pipelines: it allows multiple jobs to run in parallel. You can also create conditioned branching on the success or failure status of the previous job.
A single pipeline can now cover multiple execution scenarios, allowing you to more centrally control and monitor your entire workflows.

Conditional pipelines

CPU and RAM Limit Per Job

Resource allocation quotas can now be defined for each job to control their resource consumption.
You can define both guaranteed resources and limits for CPU and RAM usage.
Defining guaranteed resources ensures that this amount of resources will be allocated to the job during its execution. On the other hand, limits prevent the job from over-consuming resources during its execution, which allows for proper sharing of platform resources between jobs.

Resource allocation quotas

Time Zone and Daylight Saving Time (DST) Management in Job and Pipeline Scheduling

You can now select time zones when configuring scheduled runs for jobs and pipelines.
In addition, the time automatically adjusts to DST.

Time zone & Scheduling

Physical Deletion of Projects, Jobs, and Other Assets

You can now physically delete projects, jobs, pipelines, environment variables, and Docker credentials from their platforms.
Previously, these assets were only archived and continued to occupy storage resources, which could lead to saturation issues.

Bug Fixes

Spark Technology Fixes

Fixes have been applied to all Spark runtime contexts available in Saagie repositories.
They solve an issue where the status of a Spark job could appear as Succeeded in a pipeline, even though the job had actually failed.

To apply these fixes:

  1. You have to synchronize your Saagie repositories

  2. Since Spark should now be deployed in client mode, you have to remove the spark-submit --deploy-mode cluster command from the command lines used by your Spark jobs.

Without these two actions, Spark jobs will fail.

Automatic Refresh of Job, Pipeline, and App Status

Job, pipeline, and app status updates are now automatic.
This streamlines the user experience, as previously the status of a running job, pipeline, or app would remain stuck on Queued until the job, pipeline, or app was manually updated and displayed its current status in the user interface.

User Experience Improvements

Technology Deprecation Management in the User Interface

Deprecated technologies are now tagged as such in technology repositories and when used by a job.
Saagie tags obsolete technologies as deprecated in its official repository to indicate that these runtime contexts will soon be removed. Jobs, pipelines, and apps using a deprecated technology will work until further notice, but you will not be able to create new ones.

We recommend you to upgrade your technology to use a more up-to-date runtime context.

Deprecated technology labels

Saagie’s Technology Repository Updates

The following technology versions have been added to the official Saagie technology repository:

Technologies New contexts

Spark

3.1.2 Java/Scala 11
3.1.2 Python 3.7

R

4.1

RStudio

4.1

Do not forget to synchronize your Saagie repositories to keep them up to date.

October 2021

Here are the highlights of new and updated features for this release:

  1. Saagie’s Technology Repository Updates
    New technology versions have been added and others have been deprecated.

Saagie’s Technology Repository Updates

The following technology versions have been added or deprecated in the official Saagie technology repository:

Technologies New contexts Deprecated contexts

Bash

Debian-10-Buster, Debian-10-Buster-aws, Debian-10-Buster-azure, Debian-10-Buster-gcp

Debian-9-stretch, Debian-9-stretch-aws, Debian-9-stretch-azure, Debian-9-stretch-gcp

Python

3.8, 3.9

Talend

Java_11

Java_8

Grafana

8.2

Jupyterlab Notebook

Python 3.8, Python 3.9

MLFlow Server

1.20

Do not forget to synchronize your Saagie repositories to keep them up to date.
Obsolete technologies are tagged as deprecated to indicate that these runtime contexts will be removed.
Jobs, pipelines, and apps using a deprecated technology will work until further notice, but you will not be able to create new ones.
We recommend that you upgrade your technology to use a more up-to-date runtime context.

September 2021

Here are the highlights of new and updated features for this release:

This release focuses on bug fixes, platform stability, security enhancements and user experience improvements.

  1. Sample Project
    A sample project is available when installing a new platform.

  2. Customization of the Platform Sender’s Email
    The default email of the platform sender can be customized.

  3. Saagie’s Technology Repository Updates
    New technology versions have been added and others have been deprecated.

Sample Project

A sample project including three jobs is now available in every new installation of the platform. It helps you to become familiar with our basic concepts when you first log in.

The project and its jobs can be upgraded or deleted at your convenience.

Customization of the Platform Sender’s Email

You can customize the platform sender’s default email noreply@saagie.com when installing or updating your platform.

Saagie’s Technology Repository Updates

The following technology versions have been added or deprecated in the official Saagie technology repository:

Technologies New contexts Deprecated contexts

Java/Scala

7, 8

Python

2.7, 3.5, 3.6

R

3.4.4, 3.5.3, 3.6.2

Spark

2.4.8 (replaces Spark 2.4.5)
3.0.3 (replaces Spark 3.0.0)
3.1.2 Java/Scala 11, 3.1.2 Python 3.7

2.4 Java/Scala 8
2.4 Java/Scala 11
2.4 Python 2.7
2.4 Python 3.5
2.4 Python 3.6
2.4 Python 3.7
3.0 Java/Scala 8
3.0 Python 3.5
3.0 Python 3.6
3.1 Python 3.6
3.1 Java/Scala 8

MLFlow Server

1.16

OrientDB

2.1.25, 2.2.37, 3.0.35

R Studio

3.4.2

Zeppelin

0.7.3

Do not forget to synchronize your Saagie repositories to keep them up to date.
Obsolete technologies are tagged as deprecated to indicate that these runtime contexts will be removed.
Jobs, pipelines, and apps using a deprecated technology will work until further notice, but you will not be able to create new ones.
We recommend that you upgrade your technology to use a more up-to-date runtime context.