Managing Pipelines

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

Click Create pipeline to create a new pipeline.
A page to create your pipeline opens.

Enter a name, alias, and description for your pipeline.

Construct your pipeline by adding at least two jobs.

Configure settings to schedule your pipeline run, send email alerts, and set environment variables. You can also add a release note.

Click either:

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

You can either:

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

Click a pipeline in the list to open it and access its details.
The pipeline Overview page opens.

Click Instances from the secondary navigation menu.

From the list, select the pipeline instance you want to rerun.

Click the arrow next to the Rerun button to open a context menu, and select the desired run option. You can rerun your pipeline instance with the current environment variable values by selecting Updated, or with its initial environment variable values, by selecting Original.

If you want to save your environment variables, you must enable the Environment variables option.

Then click Rerun to run the selected instance again.

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

Click a pipeline in the list to open it and access its details.
The pipeline Overview page opens.

Click the desired setting from the Overview page (a) or in Edit mode (b) to edit it:

Figure 1. Pipeline Settings from the Overview Page

Figure 2. Pipeline Settings in the Edit Mode

• 1 – Name

• 2 – Alias

• 3 – Description

• 4 – Scheduled Run

• 5 – Email Alerts

• 6 – Variables

Names are mandatory. They can be up to 255 characters long and must be unique within a project.

The pipeline alias is unique to each pipeline in a project. It allows you to reference a pipeline in your code with a constant key.

The Description field is only available on the Overview page.

Descriptions are optional and not restricted. However, it is a good practice to add them. Keep them short and to the point.

Click the switch in the side panel of the setting to enable or disable it.

There are two types of execution:

1. The manual run, which requires you to click Run to start the pipeline.

2. The scheduled run, which launches the pipeline according to the schedule you choose. It has three schedule modes: Simple, Shortcut, and Expert.

   A scheduled pipeline can always be run manually.

   • In Simple mode, you can easily specify variables through the user interface. There are many possibilities.

   • In Shortcut mode, you can choose the recurrence of your run on an hourly, daily, weekly, monthly, or annual basis. All other settings are automatic.

   • In Expert mode, you can specify variables using the Cron format. The Cron time string consists of five values separated by spaces: [minute] [hour] [day of the month] [month] [day of the week].

   Once you have finished scheduling your run, you will see the summary of your choice written below and the time of the next run.

By default, instances can run at the same time. To prevent this, select the Forbid overlapping scheduled instances option. This means that if your pipeline is still running when the next scheduled run is due, it will be skipped.

Click the switch in the side panel of the setting to enable or disable it.

Alerts are optional and can be set to receive an email when the status of your pipeline changes. They can be sent to multiple email addresses to notify you of the following status changes:

Status	Description
Requested	The pipeline’s run has been requested and is being executed.
Queued	The pipeline is waiting for the necessary resources to be executed.
Running	The pipeline is up and running.
Failed	The pipeline has crashed.
Stopping	The pipeline is stopping.
Stopped	The pipeline has stopped running.
Succeeded	The pipeline has been successfully executed.
Unknown	The pipeline no longer runs because an error has occurred.

Click the switch in the side panel of the setting to enable or disable it.

This setting can be used to:

• Manage Environment Variables on the Fly

  You can create, modify, and delete pipeline environment variables on the fly from the Variables side panel. Changes made from this panel will update the pipeline Environment variables page accordingly.

  You do not have to enable the setting for this to work.

  For more information, see Managing Pipeline Environment Variables on the Fly.

• Transfer Variables Between Jobs During Pipeline Execution

  • When enabled , it allows you to transfer modified and created variables between jobs during pipeline execution. In addition, you can view a summary table of used, modified, and unchanged variables for each job in the pipeline Overview and Instances pages.

    This feature relies on advanced mechanisms that require more resources. These resources would be wasted if not used efficiently. Thus, to have correct optimization, we recommend to disable this option if it has no real use during the execution of the pipeline.

  • When disabled , modification and transfer of variables between jobs during pipeline execution is not allowed. You can still read global, project, and pipeline environment variables, but not modify them.

    The second part of the Overview page indicates whether the setting is enabled or not. Each pipeline instance indicates whether it was executed with the setting enabled or not.

  For more information, see Creating and Modifying Variables in a Job Output in a Pipeline.

Saving is automatic. You can just press Enter to confirm the pipeline name change, click anywhere nearby to confirm the pipeline alias and description change, and close the side panel to confirm the scheduled run, email alert, and environment variable changes.

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

Click a pipeline in the list to open it and access its details.
The pipeline Overview page opens.

Click the Variables setting in your pipeline from the pipeline Overview page or from its Edit mode.
A panel opens with the existing variables in a code block.

In the code block, add, modify, or delete variables.

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

Click a pipeline in the list to open it and access its details.
The pipeline Overview page opens.

Click Edit from the Overview page.
The page for editing your pipeline opens.

You can change your pipeline through the Design tab and Code tab. The two tabs are linked and changes made in one will be reflected in the other. Synchronization takes place as soon as you switch from one tab to the other.

• Using the Design tab

• Using the Code tab

• Design your pipeline:
  Move jobs and conditions anywhere on the design grid.

• Add or remove jobs and conditions:

  • Click Add job or Add condition and select an item from the list to add a job or a condition to the pipeline.

  • Click the job or condition card and select Remove to remove it from the pipeline.

• Add or remove links:

  • Link the connectors of a job or condition to create connections between them.

  • Click the small grey cross in the middle of a link to remove the link between jobs or conditions.

Upload your code file in .yaml or .yml format by clicking Import a YAML file.

The Code tab displays your code in read-only mode. The imported file cannot be edited directly on the platform.

You cannot import a file that contains syntax errors. If your file has errors, a pop-up message appears to indicate you the error line(s). In this case, the file is not imported. The pipeline keeps the current pipeline design. Try to correct the errors and upload your file again. Only existing jobs of your project can be referenced in the code file. You can download your pipeline code file by clicking Export pipeline as a YAML file. Click Copy the YAML result to clipboard to copy the code to your clipboard and paste it into your text editor. You can also import .xml or .json files. However, once imported into the platform, it will be automatically converted to YAML.

---

Deploy to see an example explaining the YAML code file for a pipeline

Here is an example of what your pipeline code file should look like:

schema: "1.0"
pipeline:
  name: Pipeline from YAML
  alias: pipeline_from_yaml
  executionVariables: true
  nodes:
    - job:
        alias: Utica_Branch_Sales
        node: node001
        position:
          x: 588
          y: -30
        nextNodes:
          - node007
    - conditionStatus:
        trigger: AllSuccessOrSkipped
        node: node006
        position:
          x: 470
          y: 48
        nextNodesSuccess:
          - node001
        nextNodesFailure:
          - node004
    - conditionExpression:
        expression: true != false
        node: node007
        position:
          x: 930
          y: 50
        nextNodesSuccess:
          - node002
        nextNodesFailure:
          - node005
    - job:
        alias: Albany_Branch_Sales
        node: node002
        position:
          x: 1048
          y: -49.5
        nextNodes: []
    - job:
        alias: Rochester_Branch_Sales
        node: node003
        position:
          x: 112
          y: 32
        nextNodes:
          - node006
    - job:
        alias: Utica_Branch_Sales
        node: node004
        position:
          x: 601
          y: 86
        nextNodes:
          - node007
    - job:
        alias: Albany_Branch_Sales
        node: node005
        position:
          x: 1050
          y: 94
        nextNodes: []yamlCopy

Where:

• schema is the version of the internal schema of the document. This line is automatically added when you import your code file in Saagie.

• pipeline is the instanciation of the pipeline object as a whole. It has no value of its own, but has related sub-attributes.

  • name is the name of your pipeline.

  • alias is the alias of your pipeline.

  • executionVariables is to define the option to modify environment variables at runtime. For more information, see Transfer Variables Between Jobs During Pipeline Execution.

  • nodes is to instanciate the list of job(s) and condition(s) to add to your pipeline. It has no value of its own, but has related sub-attributes.

    • job is to add a job to your pipeline. It has no value of its own, but has related sub-attributes.

      • alias is the alias of an existing job in your project.

      • node is a temporary identifier for your job. It allows you to reference your job in the code file. You can reference it as the next element to run after another job or a condition.

      • position is to define your job position in the graph. Where x is for the horizontal axis and y is for the vertical axis.

      • nextNodes is to define the next job(s) and condition(s) to run after this job. Its value(s) are the node value of the desired job(s) and condition(s).

    • conditionStatus is to add conditions on the status of the jobs.

      • trigger is to define the behavior expected by your condition. It can be AllSuccessOrSkipped, AllSuccess, or AtLeastOneSuccess. For more information, see Conditions on Statuses.

      • node is a temporary identifier for your condition. It allows you to reference your condition in the code file. You can reference it in the nextNodes attribute of a job to make it the next element to run after that job.

      • nextNodesSuccess is to define the element(s) to run if the condition succeeds. Its value(s) are the node value of the desired job(s).

      • nextNodesFailure is to define the element(s) to run if the condition fails. Its value(s) are the node value of the desired job(s).

    • conditionExpression is to add a condition on environment variables.

      • expression is to define the behavior expected by your condition. For more information, see Conditions on Environment Variables.

      • node is a temporary identifier for your condition. It allows you to reference your condition in the code file. You can reference it in the nextNodes attribute of a job to make it the next element to run after that job.

      • nextNodesSuccess is to define the element(s) to run if the condition succeeds. Its value(s) are the node value of the desired job(s).

      • nextNodesFailure is to define the element(s) to run if the condition fails. Its value(s) are the node value of the desired job(s).

Here is the graphic representation of the pipeline code above:

Click either:

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

You can click the kebab menu   Duplicate at the end of the relevant row from the Pipeline page, or from the Overview page of a pipeline.

Figure 3. Duplicate your pipeline from the Pipeline page

Select either:

Click Duplicate to confirm the duplication.
Depending on whether you duplicated from the Pipeline page or from the pipeline Overview page, you are returned to either the pipeline library, or the Overview page of the duplicated pipeline, respectively. In addition, a message appears saying that your pipeline has been successfully duplicated.

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

Click a pipeline in the list to open it and access its details.
The pipeline Overview page opens.

Click either:

You can delete either instances or versions of your pipeline.

• Delete a single instance or version

• Delete a selection of instances or versions

• Delete a selection of instances based on filters

• Delete instances by date

You can delete a single pipeline instance or version by clicking the trash can that appears on the line when you hover over the desired element.

You can delete a selection of pipeline instances or versions by selecting them in the list and clicking Delete n instances/versions at the top of the list.

You will then be able to play with the selector by adding or removing elements with different states.

You can delete a selection of pipeline instances based on status criteria. To do this, select the desired filter and click Delete n instances at the top of the list.

You will then be able to play with the selector by adding or removing elements with different states.

You can delete pipeline instances by date criteria with the All instances older than filter. It will delete all instances and associated logs prior to the selected date.

1. Select the All instances older than filter.

   

   A pop-up window opens with a date picker.

2. Select a date.

3. Click Select instances to confirm your choice.
   The corresponding instances are selected in the list.

   You will then be able to play with the selector by adding or removing elements.

4. Click Delete n instances at the top of the list.

The deletion process is slightly different depending on whether you are deleting instances or versions of the pipeline.

Click  Projects from the primary navigation menu.
By default, the All Projects page opens with the list of existing projects.

Click a project in the list to open it.
By default, the Jobs page opens with the list of existing jobs.

Click Pipelines from the secondary navigation menu to open the project’s pipeline library.

You can click the kebab menu   Move to… at the end of the relevant pipeline row from the Pipelines page, or from the Overview page of a pipeline.

Figure 4. Move a pipeline from the Pipeline page

In Figure 4, the Move to… action is in orange. This means that some issue(s) are preventing your pipeline from being moved. You can still click on it; It will display a panel that lists the errors. The possible issues can be as follows: You cannot move a pipeline that has one or more jobs in use in another pipeline. It can be a job that is used in a different version than the current version of another pipeline. Make a deep duplication of your pipeline and move the duplicated pipeline to bypass this issue. You cannot move a pipeline whose instances have not yet been cleaned from the orchestrator. Wait a moment and try again. You cannot move a running and scheduled pipeline, as well as a pipeline containing running and scheduled jobs. Wait for the end of the execution(s) and try to remove the scheduling settings. Then try again. The job technology that is used in the pipeline must be available in the target project. If it does not, select another project or modify the settings of the target project.

Select the target platform and target project from the list.

Click Move to confirm the move.
You will see a message saying that your pipeline has been successfully moved. You can click Go to target project below the success message to be redirected to the pipeline library of the target project. Jobs, conditions, pipeline environment variables, versions, instances, logs, alerts, and resource settings are all included in the moved pipeline. There is no obfuscation of the data. If there is sensitive data, it will be visible in the target project.