Explanation: Pipelines

1. Library

When you arrive to the pipelines within your project, you’ll land in the project’s pipeline library.

Screenshot of the pipelines library homepage with eight sections outlined

  1. Access project-level environment variables in the secondary navigation panel.

  2. Also in the secondary navigation panel you’ll find all Docker credentials for this project.

  3. The breadcrumbs in the top bar display your current location.

  4. The top bar title is the title of the current page. In this case, you’re on Pipelines.

  5. Also in the top bar are buttons to:

    • Update (refresh) your page

    • Access project settings

    • Create a new pipeline

  6. Use the search bar to search for pipelines by name.

  7. Finally, you’ll see a list of all of the pipelines in this project. Continue reading for more details.

1.1. List of pipelines

The list of pipelines includes several details, with each line describing one pipeline.

A pipeline line includes the following information and options:

Pipeline line details

  1. Button to run the pipeline

  2. Pipeline name

  3. Badge indicating the pipeline’s most recent status

  4. Most recent instance of the pipeline

  5. Button to delete the pipeline

Additionally, there is a small clock icon next to scheduled pipelines. When you hover over the icon, you’ll see when the pipeline is next scheduled to run.

Schedule pipeline icon and message

2. Overview

The overview page is the first page you’ll reach when you navigate to your pipeline details. It displays essential information about the pipeline and links to other important pipeline details.

Screenshot of the pipeline overview homepage with nine sections outlined.

  1. The second-level navigation column has links to the pages for a pipeline’s overview, instances, and versions.

  2. The second-level navigation column displays the pipeline’s current status and when the pipeline last ran. There are also buttons to run or delete your pipeline.

  3. The breadcrumbs in the top bar show which project and pipeline you’re viewing. You can also use the breadcrumbs as navigation.

  4. The top bar title is the title of the current page.

  5. Also in the top bar are buttons to update your pipeline, access pipeline settings, and upgrade your pipeline.

  6. In the body of the page layout, there is a section featuring all pipeline settings.

  7. Also in the body of the page layout is a section featuring the pipeline’s configuration, including the pipeline’s version information as well as the version’s release note (if one was added), creation date, and who created the version.

  8. The final section in the body displays information about the pipeline’s orchestration, notably conditions and branches in which jobs in the pipeline will run.

3. Settings

Pipeline settings include the following non-versioned features:

  • Name

  • Description

  • Alerts

  • Run type

You can also access pipeline settings while editing your pipeline.

3.1. Name

Names are always required.

There are minimal restrictions when naming pipelines:

  • Names cannot exceed 255 characters.

  • You cannot repeat pipeline names within a project.

3.2. Description

Descriptions are optional.

There are no restrictions on descriptions, though it’s good practice to keep them short and informative.

3.3. Alerts

By setting up alerts, you’ll receive an email each time your pipeline’s status changes to a status you selected when creating or modifying the pipeline.

Alerts are sent to an email address. You can either choose an email address from the drop-down menu, or, if the email address you need doesn’t appear, you can enter it. You can have alerts sent to one or several email addresses.

You can receive alerts for one or several of the follow statuses:

  • requested: you’ve asked for the pipeline to run; it is waiting for the resources it needs to be able to launch

  • queued: the pipeline is in line to run

  • running: the pipeline is actively running

  • failed: the pipeline’s run failed

  • killing: Saagie is in the process of stopping the pipeline’s run

  • killed: the pipeline’s run has stopped

  • succeeded: the pipeline ran successfully

  • unknown: the pipeline’s run details are missing

3.4. Run types

There are two run types:

  1. Manual run requires you to launch the pipeline manually by selecting the Run button in the Saagie user interface.

  2. Scheduled run launches the pipeline according to the schedule you determine.

You can also launch scheduled runs manually.

3.4.1. Timezone

Whatever the schedule run mode selected, you can choose which timezone to use to apply on your pipelines.

Choosing your time and day

  1. The default value in the timezone selection is your browser timezone.

  2. The next schedule in overview will be displayed in your browser timezone.

  3. The Daylight Saving Time (DST) is automatically adjusted.

  4. Runs scheduled for the 29th, 30th, or 31st day of the month will not run in months with fewer than 29, 30, or 31 days.
Timezone management

Scheduled run modes

If you choose a scheduled run, there are three ways to determine the schedule: simple, shortcut, and expert.

3.4.2. Simple

In simple mode, you control each variable through a user interface. Here are the possible configurations.

  1. minute: every [xx] minutes

  2. hour: every [xx] hours at [xx] past the hour

  3. day: every [xx] days at [xx]:[xx] time

  4. week: every [xx] week on [day of week] at [xx]:[xx] time

  5. month: every [xx] month on the [date of month] at [xx]:[xx] time

  6. year: every [xx] year on the [date of month] of [month] at [xx]:[xx] time

Scheduled run type in simple mode

3.4.3. Shortcut

In shortcut mode, you choose how often you want to run the pipeline. All other settings are automatic.

  1. hourly: every hour, on the hour

  2. daily: every day at midnight UTC

  3. weekly: every Sunday at midnight UTC

  4. monthly: the first day of each month at midnight UTC

  5. annually: January 1 at midnight

Scheduled run type in shortcut mode

3.4.4. Expert

In expert mode, you control each variable (like in simple mode), but you do so using Cron format: [minute] [hour] [day of the month] [month] [day of the week]. For example, by entering 30 9 * * 2 in the cron field, you’ve scheduled your pipeline to run at 9:30 AM UTC +01:00 Europe/Paris time Tuesday.

  1. [minute] 0-59

  2. [hour] 0-23

  3. [day of the month] 1-31

  4. [month] 1-12

  5. [day of the week] 0-6 (Sunday to Saturday)

Scheduled run type in expert mode

4. Upgrade

You can upgrade the pipeline’s configuration using the Upgrade pipeline button. Upgrading your pipeline creates a new version of that pipeline.

The pipeline’s configuration includes the following versioned features:

  • Design your pipeline

  • Add jobs to a pipeline

  • Remove jobs from a pipeline

  • Add a link

  • Remove a link

  • Add condition to a pipeline

  • Remove condition from a pipeline

  • Release notes

4.1. Design your pipeline

  • You will first land in the full screen pipeline design grid.

  • You can move jobs/conditions anywhere on the design grid.

Upgrade a pipeline

  • The header allows you to manage settings while upgrading your pipeline.

Header features

  1. You can update the pipeline’s name.

  2. You can update the pipeline’s description.

  3. You can manage alerts.

  4. You can manage scheduling.

  5. You can manage release notes.

  6. You can save the pipeline modifications.

  7. You can access to other features: Save & Run

  8. You can quit without saving.

Modifications strictly related to metadata as name, description, alerts and scheduling will not trigger a new pipeline’s version creation.

4.2. Add jobs to a pipeline

You can add jobs anywhere in a pipeline.

  1. Click on the add job button to add another job on the grid then drag it where you want.

    Click Add job

  2. Search and select your job from the list.

    Select your job

  3. Drag your job where you want.

    Drag you job

Pipelines only run the job version tagged as the current version. If you’d like to confirm the job’s current version:

  1. Select the job in the pipeline orchestration window.

  2. Select the See overview button.

    Remove a job

  3. The job’s overview page will open in a new tab, and you can check the job’s current version as well as view other details about the job.

Refer to job version tags for more information about job versions.

4.3. Remove jobs from a pipeline

You can remove jobs from a pipeline’s current orchestration by selecting the job you’d like to remove in the pipeline orchestration window, then selecting Remove button.

Remove a job
You can also use Backspace or Delete keys.

However, once a job is a part of a pipeline, the job will always be a part of that pipeline’s history. Removing a job from a pipeline does not delete the pipeline’s instances that include the removed job.

Once a job is a part of any pipeline, it can no longer be moved from one project to another—​even after being removed from the pipeline.

You can add links between jobs and conditions in a pipeline.

  1. Click and hold on the output of a job/condition to start a link.

  2. Drag the link to the input of another job/condition.

Add a link
You can link as many input or output jobs you want to a job.

4.4.1. Job links behaviours

  • When not linked, jobs run in parallel.

    Parallel jobs

  1. When linking one or more source job(s), to a target job, the target job will be triggered
    only if all source job(s) status is SUCCEEDED.

    Add a links - all succeeded

    1. Example with all source jobs status are SUCCEEDED.

      Add a links

    2. Example with one source job status is FAILED.

      Add a links - some failure

You can remove links between jobs and conditions in a pipeline.

  1. Click on the cross in the middle of the link.

Remove a link

4.6. Add a condition

You can add conditions between jobs in a pipeline. It allows you to choose what following jobs will be executed depending on the status of previous input jobs.

  1. Click on Add condition button.

  2. Drag the condition where you want.

  3. Link jobs outputs to the condition input.

  4. Link condition OK output to the next job to execute if all previous jobs status are succeeded or skipped.

  5. Link condition KO output to the next job to execute if all previous jobs status are neither succeeded nor skipped.

Add condition
You can link as many input or output jobs you want to a condition.

4.6.1. Link condition behaviours

  1. When linking one or more source job(s), to a condition, the 1st OK branch job will be triggered only if all source job(s) status is SUCCEEDED or SKIPPED
    else the 1st KO branch job will be triggered.

    Add a condition - design

    1. Example with all source jobs status are SUCCEEDED.

      Add a links

    2. Example with one source job status is FAILED.

      Add a links - some failure
You cannot link conditions.

4.7. Remove a condition

You can remove conditions from a pipeline’s current orchestration by selecting the condition you’d like to remove in the pipeline orchestration window, then selecting Remove.

Remove condition
You can also use Backspace or Delete keys.

4.8. Alerts

By clicking on the header’s alerts icon, you can also manage alerts from a side panel.

Alerts side panel

4.9. Scheduling

By clicking on the header’s schedule icon, you can also manage the pipeline’s run type from a side panel.

Scheduling side panel

4.10. Release notes

Release notes are optional and can be a good way to keep track of changes from one pipeline version to another. Use this field according to your needs.

Release notesside panel

5. Instances

An instance of your pipeline is a single run of that pipeline.

When you first arrive on your pipeline’s instances page, you’ll see the information for the most recent pipeline instance:

Pipeline instances information page

5.1. Instance information

  • Current pipeline instance number

  • Status of the most recent pipeline instance

  • Current pipeline version

  • Start time, end time, and duration of the instance

5.2. Orchestration progress

A pipeline’s instance page is where you can see how that pipeline’s orchestration progressed for that specific instance.

Additionally, when a pipeline is running, you can see its progress in real time on this page.

There are several buttons in this section so that you can optimize your view. Use them to zoom in and out as well as view this section full-screen.

When a pipeline execution is finished, you can select a job and click:

  1. the See overview button to go the job overview

  2. the Display logs button to see its execution logs

Pipeline instances information page

5.3. List of all instances

The pipeline instances page features the list of instances of that pipeline. Select an instance from the list to view its details.

6. Versions

Whenever your pipeline is upgraded, a new version of that pipeline is created. You can access and run any version of your pipeline from the pipeline versions interface.

When you first arrive on your pipeline’s versions page, you’ll see the following information for the current version:

Pipeline versions information page

6.1. Tags

There are two tags available to assign to pipeline versions:

6.1.1. Current version

By default, the current version of the pipeline is the most recent version.

However, you can tag any version of your pipeline as the current version. You can rollback to an older version of the pipeline to make it the current version.

When you create a new version of the pipeline, it is tagged automatically as the current version.

6.1.2. Major version

Use the major version tag however you see fit.

For example, you might use it to highlight stable versions of the pipeline.

You can remove the major version tag from a version by selecting the Unset as a major version button.

6.2. Version information

Each version page contains all information for that version, including:

  • Release note (if one was included)

  • Date created

  • Creator

6.3. Pipeline design

The pipeline’s version page is where you can see the pipeline’s version design.

6.4. List of all versions

The pipeline versions page features a list of all the versions of that pipeline. Select a version from the list to view its details.