2022

September 2022

Here are the highlights of new and updated features for this release:

  1. Product Updates (v3.0.0)
    The v.3.0.0 version of the Saagie product has been released with the following features:

    • The management of apps in Saagie has been overhauled.

    • The external technologies can now be orchestrated in Saagie.

    • The Overview page of jobs and pipelines has been redesigned.

    • The user experience in pipelines has been improved.

    • Information about the product version are now available .

    • Various bugs have been fixed.

  2. Saagie’s Technology Repository Updates
    New external job technologies have been added.

Product Updates (v3.0.0)

Revamp of Apps Management

The underlying architecture of the Docker apps implementation in Saagie apps has been completely overhauled to increase overall stability, provide additional monitoring capabilities, and improve storage volume management.

App enhancements include:

  • A redesign of the Overview page.

  • New pages:

    • The History page, which provides information about history of changes made to your app and about its runtime.

    • The Storage page, which provides information about the persistent storage volumes allocated to your app.

  • New API endpoints dedicated to apps.

As part of the upgrade to this new architecture of apps, all current apps installed in your projects will be automatically migrated. This migration implies the following breaking changes:

  • Apps will be shut down during the migration and each will need to be manually restarted after the upgrade.

  • Instances and logs generated prior to the migration will not be kept.

  • Versions created prior to the migration will be kept but storage volumes linked to a version will need to be manually re-associated in case of a rollback.

  • Apps set up without any exposed port prior to the migration will have an exposed port created with the default value 80.

  • The app IDs, as displayed in their URLs, will change. Everything that points to these URLs will need to be updated accordingly.

  • Any workflows using the Saagie API to point to the now deprecated version of apps will need to be updated to use the new API endpoints dedicated to apps.

External Technology Orchestration

Saagie can now run jobs that trigger tasks executed on external technologies. External jobs leverage the APIs of those external technologies to start and stop tasks, as well as receive status and logs for each running instance.
External jobs are listed like any other job in projects and can be scheduled, versioned, and included in pipelines that combine both embedded and external job technologies.

A first set of external job technologies has been integrated into the official Saagie technology repository:

  • AWS Batch

  • AWS EMR

  • AWS Glue

  • AWS Lambda

External jobs are still in an experimental phase and may have performance issues.
You can also integrate other external job technologies with the Saagie SDK.

Revamp of Job and Pipeline Overviews

The Overview page for jobs and pipelines has been revamped to show you more relevant information at a glance.

The Overview page now displays the latest instance of the job or pipeline with information such as the status of the instance, its start and end times, duration, version, scheduling settings, associated pipelines and logs for jobs, and the status of each job composing the pipeline for pipelines.
Job and pipeline settings are also easily accessible via dedicated buttons in the header of the Overview page.

User Experience Improvements in Pipelines

User experience improvements have been made to the pipeline design interface:

  • All errors in the pipeline design are now displayed at once when saving, saving you time in troubleshooting.

  • Jobs using deleted technologies are now clearly highlighted in the pipeline design, prompting you to upgrade them to current technologies to be able to run the pipeline.

  • Links between jobs now use a clearer color code when running a pipeline, helping you to see the current status of your pipeline at a glance.

Product Information Page

User profile menu A Product Information page has been added to display the currently installed product version and the build IDs of the individual modules composing the Saagie software.
You can access this page by clicking Product info in your user profile menu.
This information will help our customer support team resolve your support requests more efficiently.

Bug Fixes

Fixes have been added to address the following issues:

  • The risk of the Saagie authentication pod running out of memory and crashing has been reduced.

  • In pipelines, the stop job function during a pipeline run now correctly stops the pipeline.

  • In the technology catalog, reverting changes on a repository added through a ZIP archive also reverts its previous name.

  • In the projects settings, the addition of technologies is now correctly saved.

Saagie’s Technology Repository Updates

The following technology versions and external job technologies have been added to the official Saagie technology repository:

Technologies New contexts

Grafana

9

Metabase

0.43

pgAdmin4

6.10

AWS Batch
Badge for EXTERNAL JOB

Job EXPERIMENTAL

AWS EMR
Badge for EXTERNAL JOB

Clone step EXPERIMENTAL

AWS Glue
Badge for EXTERNAL JOB

Crawler EXPERIMENTAL
Job EXPERIMENTAL
Workflow EXPERIMENTAL

AWS Lambda
Badge for EXTERNAL JOB

Functions EXPERIMENTAL
External jobs are still in an experimental phase and may have performance issues.
Do not forget to synchronize your Saagie repositories to keep them up to date.

March 2022

Here are the highlights of new and updated features for this release:

  1. Product Updates (v2.2.1)
    The v.2.2.1 version of the Saagie product has been released with the following features:

    • Breaking changes in the Saagie API behavior have been made.

    • The user experience in pipelines has been improved.

    • Multi-context management is now available for app technologies in the Saagie’s official repository of the technology catalog.

    • Bugs in the pipeline execution have been fixed.

  2. Saagie’s Technology Repository Updates
    New technology versions have been added and others have been deprecated.

Product Updates (v2.2.1)

Saagie API Behavior

Breaking changes have been made to the Saagie GraphQL API.

The following deprecated endpoints have been removed from the https://<your_saagie_host>/projects/api/platform/<your_platform_id>/graphql: endpoint server:

  • technologies(projectId: UUID, jobCategory: String): [Technology!]

  • technology(id: UUID): Technology

  • technologiesVersions(technologyId: UUID!): [TechnologyVersion!]!

These endpoints have been replaced with the following in the https://<your_saagie_host>/gateway/api/graphql: endpoint server:

  • technologiesByIds(ids: [UUID!]!): [Technology!]!

  • technology(id: UUID!): Technology

User Experience Improvements in Pipelines

User experience improvements have been made to the pipeline design interface:

  • Adding jobs to a pipeline now uses smart positioning on the design grid to prevent jobs from stacking on top of each other when adding multiple jobs in a row.

  • You can automatically rearrange jobs and conditions to the most optimized layout based on the defined pipeline structure by clicking the magic wand.
    This saves you from having to manually drag jobs and conditions into the pipeline design grid to arrange them legibly.

  • Error handling has been improved by preventing the creation of inconsistent pipelines starting or ending with a condition.

  • In addition to the error messages displayed when trying to save an invalid pipeline, the jobs and conditions causing the errors are now clearly highlighted on the design grid so you can quickly identify the cause of the issue.

Multi-Context Management of Saagie’s Official Repository App Technologies

Similarly to how job technologies appear in the Saagie repository, all contexts associated with an app technology are now listed under a single entry.

Apps multi-context menu

This improves the readability in the Catalog and Project setting interfaces and allows you to quickly select the app technology context to use when installing an app from the Saagie’s official repository.

Existing apps have been migrated to a multi-context architecture and former mono-context app technologies have been disabled.

Bug Fixes in Pipeline Execution

Fixes have been added to pipelines to solve execution issues:

  • Pipeline instances now take into account downstream jobs when their parent job returns a Succeeded status.

  • The time zone selected when running a scheduled pipeline is now saved at the pipeline creation.

Saagie’s Technology Repository Updates

The following technology versions have been added or deprecated in the official Saagie technology repository:

Technologies New contexts Deprecated contexts

Python

3.10

Jupyter Notebook for Spark

2.4

Jupyter Notebook with Python

3.10

3.6

JupyterLab with Python

3.10

Kibana

7.15.1
Do not forget to synchronize your Saagie repositories to keep them up to date.
Obsolete technologies are tagged as deprecated to indicate that these runtime contexts will be removed.
Jobs, pipelines, and apps using a deprecated technology will work until further notice, but you will not be able to create new ones.
We recommend that you upgrade your technology to use a more up-to-date runtime context.

January 2022

Here are the highlights of new and updated features for this release:

  1. Access to Saagie Known Issues

  2. Saagie’s Technology Repository Updates
    Some technology versions have been deprecated.

Access to Saagie Known Issues

You can now access the public portal for Saagie Cloud known issues.

Saagie’s Technology Repository Updates

The following technology versions have been deprecated in the official Saagie technology repository:

Technologies Deprecated contexts

R

4.0

Kibana

5.6.3, 6.4.3, 6.8.4

MLFlow Server

1.16

RStudio

3.6.2
Obsolete technologies are tagged as deprecated to indicate that these runtime contexts will be removed.
Jobs, pipelines, and apps using a deprecated technology will work until further notice, but you will not be able to create new ones.
We recommend that you upgrade your technology to use a more up-to-date runtime context.