Explanation: Pipelines

1. Library

When you arrive to the pipelines within your project, you’ll land in the project’s pipeline library.

Screenshot of the pipelines library homepage with eight sections outlined

  1. Access project-level environment variables in the secondary navigation panel.

  2. Also in the secondary navigation panel you’ll find all Docker credentials for this project.

  3. The breadcrumbs in the top bar display your current location.

  4. The top bar title is the title of the current page. In this case, you’re on Pipelines.

  5. Also in the top bar are buttons to:

    • Update (refresh) your page

    • Access project settings

    • Create a new pipeline

  6. Use the search bar to search for pipelines by name.

  7. Finally, you’ll see a list of all of the pipelines in this project. Continue reading for more details.

1.1. List of pipelines

The list of pipelines includes several details, with each line describing one pipeline.

A pipeline line includes the following information and options:

Pipeline line details

  1. Button to run the pipeline

  2. Pipeline name

  3. Badge indicating the pipeline’s most recent status

  4. Most recent instance of the pipeline

  5. Button to delete the pipeline

Additionally, there is a small clock icon next to scheduled pipelines. When you hover over the icon, you’ll see when the pipeline is next scheduled to run.

Schedule pipeline icon and message

2. Overview

The overview page is the first page you’ll reach when you navigate to your pipeline details. It displays essential information about the pipeline and links to other important pipeline details.

Screenshot of the pipeline overview homepage with nine sections outlined.

  1. The second-level navigation column has links to the pages for a pipeline’s overview, instances, and versions.

  2. The second-level navigation column displays the pipeline’s current status and when the pipeline last ran. There are also buttons to run or delete your pipeline.

  3. The breadcrumbs in the top bar show which project and pipeline you’re viewing. You can also use the breadcrumbs as navigation.

  4. The top bar title is the title of the current page.

  5. Also in the top bar are buttons to update your pipeline, access pipeline settings, and upgrade your pipeline.

  6. In the body of the page layout, there is a section featuring all pipeline settings.

  7. Also in the body of the page layout is a section featuring the pipeline’s configuration, including the pipeline’s version information as well as the version’s release note (if one was added), creation date, and who created the version.

  8. The final section in the body displays information about the pipeline’s orchestration, notably the order in which jobs in the pipeline will run.

3. Settings

Pipeline settings include the following non-versioned features:

  • Name

  • Description

  • Alerts

  • Run type

3.1. Name

Names are always required.

There are minimal restrictions when naming pipelines:

  • Names cannot exceed 255 characters.

  • You cannot repeat pipeline names within a project.

3.2. Description

Descriptions are optional.

There are no restrictions on descriptions, though it’s good practice to keep them short and informative.

3.3. Alerts

By setting up alerts, you’ll receive an email each time your pipeline’s status changes to a status you selected when creating or modifying the pipeline.

Alerts are sent to an email address. You can either choose an email address from the drop-down menu, or, if the email address you need doesn’t appear, you can enter it. You can have alerts sent to one or several email addresses.

You can receive alerts for one or several of the follow statuses:

  • requested: you’ve asked for the pipeline to run; it is waiting for the resources it needs to be able to launch

  • queued: the pipeline is in line to run

  • running: the pipeline is actively running

  • failed: the pipeline’s run failed

  • killing: Saagie is in the process of stopping the pipeline’s run

  • killed: the pipeline’s run has stopped

  • succeeded: the pipeline ran successfully

3.4. Run types

There are two run types:

  1. Manual run requires you to launch the pipeline manually by selecting the Run button in the Saagie user interface.

  2. Scheduled run launches the pipeline according to the schedule you determine.

You can also launch scheduled runs manually.

3.4.1. Scheduled run modes

If you choose a scheduled run, there are three ways to determine the schedule: simple, shortcut, and expert.

Choosing your time and day

  1. All times in Saagie are in UTC. Be sure to calculate for your time zone.

  2. Many Kubernetes clusters are configured to use a time zone where time changes. This will affect scheduled runs.

    1. Any pipeline scheduled between 12:01 and 12:59 AM in that time zone will not run on the days time changes.

    2. For half the year, all pipeline runs will run one hour earlier or later than selected.

  3. Runs scheduled for the 29th, 30th, or 31st day of the month will not run in months with fewer than 29, 30, or 31 days.
Simple

In simple mode, you control each variable through a user interface. Here are the possible configurations.

  1. minute: every [xx] minutes

  2. hour: every [xx] hours at [xx] past the hour

  3. day: every [xx] days at [xx]:[xx] time

  4. week: every [xx] week on [day of week] at [xx]:[xx] time

  5. month: every [xx] month on the [date of month] at [xx]:[xx] time

  6. year: every [xx] year on the [date of month] of [month] at [xx]:[xx] time

Scheduled run type in simple mode
Shortcut

In shortcut mode, you choose how often you want to run the pipeline. All other settings are automatic.

  1. hourly: every hour, on the hour

  2. daily: every day at midnight UTC

  3. weekly: every Sunday at midnight UTC

  4. monthly: the first day of each month at midnight UTC

  5. annually: January 1 at midnight

Scheduled run type in shortcut mode
Expert

In expert mode, you control each variable (like in simple mode), but you do so using Cron format: [minute] [hour] [day of the month] [month] [day of the week]. For example, by entering 30 9 * * 2 in the cron field, you’ve scheduled your pipeline to run at 9:30 AM UTC every Tuesday.

  1. [minute] 0-59

  2. [hour] 0-23

  3. [day of the month] 1-31

  4. [month] 1-12

  5. [day of the week] 0-6 (Sunday to Saturday)

Scheduled run type in expert mode

4. Upgrade

You can upgrade the pipeline’s configuration using the Upgrade pipeline button. Upgrading your pipeline creates a new version of that pipeline.

The pipeline’s configuration includes the following versioned features:

  • Add jobs to a pipeline

  • Remove jobs from a pipeline

  • Release notes

4.1. Add jobs to a pipeline

You can add jobs anywhere in a pipeline. There are pipeline action buttons on either side of each job in the pipeline orchestration window. Use these buttons to add another job before or after any job in the pipeline.

Add job to a pipeline

Pipelines only run the job version tagged as the current version. If you’d like to confirm the job’s current version:

  1. Select the job in the pipeline orchestration window.

  2. Select the Show Details button.

  3. The job’s overview page will open in a new tab, and you can check the job’s current version as well as view other details about the job.

Refer to job version tags for more information about job versions.

4.2. Remove jobs from a pipeline

You can remove jobs from a pipeline’s current orchestration by selecting the job you’d like to remove in the pipeline orchestration window, then selecting Delete.

However, once a job is a part of a pipeline, the job will always be a part of that pipeline’s history. Removing a job from a pipeline does not delete a pipeline’s instances that include the removed job.

Once a job is a part of any pipeline, it can no longer be moved from one project to another—​even after being removed from the pipeline.

4.3. Release notes

Release notes are optional and can be a good way to keep track of changes from one pipeline version to another. Use this field according to your needs.

5. Instances

An instance of your pipeline is a single run of that pipeline.

When you first arrive on your pipeline’s instances page, you’ll see the information for the most recent pipeline instance:

Pipeline instances information page

5.1. Instance information

  • Current pipeline instance number

  • Status of the most recent pipeline instance

  • Current pipeline version

  • Start time, end time, and duration of the instance

5.2. Orchestration progress

A pipeline’s instance page is where you can see how that pipeline’s orchestration progressed for that specific instance.

Additionally, when a pipeline is running, you can see its progress in real time on this page.

There are several buttons in this section so that you can optimize your view. Use them zoom in and out as well as view this section full-screen.

5.3. List of all instances

The pipeline instances page features the list of instances of that pipeline. Select an instance from the list to view its details.

6. Versions

Whenever your pipeline is upgraded, a new version of that pipeline is created. You can access and run any version of your pipeline from the pipeline versions interface.

When you first arrive on your pipeline’s versions page, you’ll see the following information for the current version:

Pipeline versions information page

6.1. Tags

There are two tags available to assign to pipeline versions:

6.1.1. Current version

By default, the current version of the pipeline is the most recent version.

However, you can tag any version of your pipeline as the current version. You can rollback to an older version of the pipeline to make it the current version.

When you create a new version of the pipeline, it is tagged automatically as the current version.

6.1.2. Major version

Use the major version tag however you see fit.

For example, you might use it to highlight stable versions of the pipeline.

You can remove the major version tag from a version by selecting the Unset as a major version button.

6.2. Version information

Each version page contains all information for that version, including:

  • Release note (if one was included)

  • Date created

  • Creator

6.3. List of all versions

The pipeline versions page features a list of all the versions of that pipeline. Select a version from the list to view its details.