Pipeline Settings

This topic presents the setting options available for your pipeline. It also includes the Icon for environment variables Environment Variables page.

You can access the pipeline settings from the The "Overview" page icon is a square divided into several other squares. Overview page (a) and the pipeline edit Edit mode (b).

Pipeline settings from the "Overview" page.
Figure 1. Pipeline Settings from the Overview Page (a)
Pipeline settings from the edit mode.
Figure 2. Pipeline Settings in the Edit Mode (b)

It includes the following non-versioned and editable features:

1. Name

Names are mandatory. They can be up to 255 characters long and must be unique within a project.

2. Alias

The pipeline alias is unique to each pipeline in a project. It allows you to reference a pipeline in your code with a constant key.

3. Description

The description can be changed from the Overview page only.

Descriptions are optional and not restricted. However, it is a good practice to add them. Keep them short and to the point.

4. Scheduled run Icon for runtime types is a calendar.

There are two types of execution:

  1. The manual run, which requires you to click Run Run to start the pipeline.

  2. The scheduled run, which launches the pipeline according to the schedule you choose. It has three schedule modes: Simple, Shortcut, and Expert.

    A scheduled pipeline can always be run manually.
    • Simple Mode

    • Shortcut Mode

    • Expert Mode

    In Simple mode, you can easily specify variables through the user interface. There are many options.

    Screenshot of the settings for the scheduled run type in simple mode

    In Shortcut mode, you can choose the recurrence of your run on an hourly, daily, weekly, monthly, or annual basis. All other settings are automatic.

    Screenshot of the settings for the scheduled run type in shortcut mode

    In Expert mode, you can specify variables using the Cron format. The Cron time string consists of five values separated by spaces: [minute] [hour] [day of the month] [month] [day of the week]. They are based on the following information:

    Table 1. Cron format
    Descriptor Acceptable values

    Minute

    0 to 59, or *

    Hour

    0 to 23, or *

    Day of the month

    1 to 31, or *

    Month

    1 to 12, or *

    Day of the week

    0 to 7 (0 and 7 both represent Sunday), or *

    The Cron time string must contain entries for each character attribute. If you want to set a value using only minutes, you must have asterisk (*) characters for the other four attributes that you are not configuring.

    Screenshot of the settings for the scheduled run type in expert mode

    Regardless of the Scheduled run type selected, you can specify the time zone for your jobs or pipelines. When specifying its time and day, you should know that:

    • The default value in the time zone selection is your browser’s time zone.

    • The next schedule preview will be displayed in the time zone of your browser.

    • Daylight Saving Time (DST) is automatically adjusted.

    • Runs scheduled for the 29th, 30th, or 31st day of the month will not run in months with less than 29, 30, or 31 days.

      Example 1. French time settings

      UTC time is two hours behind French time, so if you want your job to run at midnight French time, set it to two hours later, that is, at 2 a.m.

    Once you have finished scheduling your run, you will see the summary of your choice written below and the time of the next run.

By default, instances can run at the same time. To prevent this, select the Forbid overlapping scheduled instances option. This means that if your pipeline is still running when the next scheduled run is due, it will be skipped.

For Pipeline embedding Spark jobs.

To prevent collisions due to concurrent executions of one Spark job, the orchestration cannot be scheduled under one hour.

pipeline settings spark

5. Alerts The "Alerts" icon is a bell.

Alerts are optional and can be set to receive an email when the status of your pipeline changes. They can be sent to multiple email addresses to notify you of the following status changes:

Status Description

queued requested Requested

The pipeline’s run has been requested and is being executed.

queued requested Queued

The pipeline is waiting for the necessary resources to be executed.

spinner Running

The pipeline is up and running.

fail Failed

The pipeline has crashed.

stop Stopping

The pipeline is stopping.

stop Stopped

The pipeline has stopped running.

success Succeeded

The pipeline has been successfully executed.

unknown Unknown

The pipeline no longer runs because an error has occurred.

6. Variables settings env vars

Variables are optional and can be used to create environment variables on the fly and to transfer variables between jobs during pipeline execution.

Manage Environment Variables on the Fly

You can create, modify, and delete pipeline environment variables on the fly from the Variables side panel. Changes made from this panel (a) will update the pipeline settings env vars a Environment variables page (b) accordingly.

pipeline settings variables side panel
You do not have to enable the setting for this to work.

Transfer Variables Between Jobs During Pipeline Execution

  • When enabled settings env vars a, it allows you to transfer modified and created variables between jobs during pipeline execution. In addition, you can view a summary table of used, modified, and unchanged variables for each job in the pipeline The "Overview" page icon is a square divided into several other squares. Overview and The "Instances" page icon is three overlapping squares. Instances pages.

    Changing variables in this process is local to the execution of the instance on which the variable was changed. In other words, it does not change the original variable listed on the settings env vars a Environment variables page of the pipeline.

    This feature is based on advanced mechanisms that require more resources that would be wasted if not used efficiently. Therefore, we recommend to disable this option if it has no real use during the execution of the pipeline to have correct optimization.
  • When disabled settings env vars, modification and transfer of variables between jobs during pipeline execution is not allowed. You can still read global, project, and pipeline environment variables, but not modify them.

    • The second part of the The “Overview” page icon is a square divided into several other squares. Overview page indicates whether the setting is enabled or not.

    • Each pipeline instance indicates whether it was executed with the setting enabled or not.