References

Find on this page information to help you create your .json configuration files.

Configuration Files

  • Environment

  • Job

  • Pipeline

You must have at least one your_env_name.json configuration file for your Saagie platform. If you have several platforms, you must have one file per platform.

Each your_env_name.json file in /saagie/envs has the following schema:

{
  "platform_id": "2", (1)
  "project_id": "947de7f2-e482-4668-8221-88149ada2e08", (2)
  "project_name": "TEST - qh prod" (3)
}

Where:

Attribute Description Type

number 1

platform_id
Mandatory

This is the ID of your Saagie platform.

String

number 2

project_id
Mandatory

This is the ID of your Saagie project.

UUID

number 3

project_name
Mandatory

This is the name of your Saagie project.

String

You can have one or more your_job_name.json configuration files. Each job must have its own .json file.

Each your_job_name.json file in /saagie/jobs has the following schema:

{
    "job_name": "2 Clean data", (1)
    "file_path": "dist/2 Clean data.zip", (2)
    "description": "new test 4", (3)
    "category": "Processing", (4)
    "technology": "python", (5)
    "technology_catalog": "Saagie", (6)
    "runtime_version": "3.8", (7)
    "command_line": "python {file} --s3bucket_train_csv '/cleaned-data/train/' --s3bucket_test_csv '/cleaned-data/test/'", (8)
    "extra_technology": "", (9)
    "extra_technology_version": "" (10)
}

Where:

Attribute Description Type

number 1

job_name
Mandatory

This is the name of your job.

String

number 2

file_path
Mandatory

This is the path to the code file of the job.

UUID

number 3

description
Optional

This is the description of your job.

If no value is defined, it will take the value of the previous job. If the value of the previous job is empty, it will remain empty.

String

number 4

category
Mandatory

This is the category of your job. It can be Extraction, Processing or Smart App. For more information, see About Jobs.

It is only required at creation. It can be changed or removed afterwards. If removed, it will take the value of the previous job. If the value of the previous job is empty, it will keep the last recorded value.

String

number 5

technology
Mandatory

This is the name of the technology used by your job.

It is only required at creation. It cannot be changed, but it can be removed from the file afterwards. The original value will always be kept.

String

number 6

technology_catalog
Mandatory

This is the name of the Saagie repository that contains the technology used by the job.

It is only required at creation. It cannot be changed, but it can be removed from the file afterwards. The original value will always be kept.

String

number 7

runtime_version
Mandatory

This is the technology version of the job, or the ID of the technology context.

It is only required at creation. It can be changed or removed afterwards. If removed, it will take the value of the previous job. If the value of the previous job is empty, it will keep the last recorded value.

String

number 8

command_line
Mandatory

This is your job’s command line.

It is only required at creation. It can be changed or removed afterwards. If removed, it will take the value of the previous job. If the value of the previous job is empty, it will keep the last recorded value.

String

number 9

extra_technology
Optional

This is the name of the secondary technology that is required to run a Spark job.

  • This attribute is optional except if you use a Spark job. In this case, it is required.

  • Leave the value blank if not required, or the request will not work.

String

number 10

extra_technology_version
Optional

This is the version of the secondary technology required to run a Spark job.

  • This attribute is optional except if you use a Spark job. In this case, it is required.

  • Leave the value blank if not required, or the request will not work.

String

You can have one or more your_pipeline_name.json configuration files. Each pipeline must have its own .json file.

Each your_pipeline_name.json file in /saagie/pipelines has the following schema:

{
    "file_path": "code/pipelines/imdb_sentiment_dectection_training_pipeline.json" (1)
}

Where:

Attribute Description Type

number 1

file_path
Mandatory

This is the path to the .yaml file of your pipeline.

String