Spinnaker – Using Pipeline Templates

Docs: Enable Pipeline Templates

Mon, 01 Jan 0001 00:00:00 +0000

hal config features edit --pipeline-templates true
hal deploy apply

Docs: Create a Pipeline Template

Mon, 01 Jan 0001 00:00:00 +0000

When you create a pipeline template, you start with the underlying JSON of a pipeline that already resembles the template you want. You turn this JSON into template JSON.

1. Get a pipeline’s JSON

Use spin CLI to get the JSON of an existing pipeline that is close to what your template will be. Alternatively, you can create a new pipeline in the model of your intended template.

Install the spin CLI , if you haven’t already done so.
Get the pipeline.
```
spin pipeline get --name <pipelineName> --application <appName>
```
…where <pipelineName> and <appName> are the names shown for the pipeline in the Deck UI.

This returns the pipeline JSON. You’ll create a pipeline template from this by saving the contents to a file and editing the JSON.

You can also view the pipeline JSON from within the Deck UI, copy it there, and and paste it into a file.

2. Save the pipeline JSON to a file

If you want, you can do this when you first get the pipeline:

spin pipeline get --name <pipelineName> --application <appName> | tee new_template.txt

…and then edit the content in that file. But you can also copy the pipeline JSON and paste it into your favorite editor. The important thing is to wind up with a file that you can make available to spin CLI.

3. Edit the file for template format

Just by adding a few fields, you can turn this pipeline JSON into pipeline-template JSON.

The following is the pipeline-template config format. Note the "pipeline": section; it contains the pipeline JSON, the same as what’s in ordinary pipeline JSON, but referencing any variables that are used. So as you start with a pipeline blob, you move that entire JSON fragment to the pipeline section.

Add a reference to the pipeline templates schema.
```
"schema": "v2",
```

Declare your variables.

Add a variables section, in which to list all the variables that you reference in this template:

"variables": [
  {
    "type": "<type>",
    "defaultValue": <defaultValue>,
    "description": "<some description>",
    "name": "<name of this variable>"
  },
  {
    "type": "<type>",
    "defaultValue": <defaultValue>,
    "description": "<some description>",
    "name": "<name of this variable>"
  }
]

For everything in the pipeline section that will be a variable, replace the value of each item with a SpEL expression that references the variable declared in variables:

${ templateVariables.<varName> }

For example in a non-templated pipeline, the amount of time to wait in a Wait stage would be represented by…

"waitTime": <time>

In our parameterized template, it would be…

"waitTime": "${ templateVariables.timeToWait }",

…where timeToWait is declared in the variables section of the template for this purpose.

Here’s a complete set of pipeline-template JSON, with the schema, variables list, and the pipeline definition:

{
  "schema": "v2", # Reference to the MPTv2 schema
  "variables": [
    {
      "type": "int",
      "defaultValue": 42,
      "description": "The time a wait stage shall pauseth",
      "name": "timeToWait" # This is the name that's referenced in the SpEL expression later
    }
  ],
  "id": "newSpelTemplate", # Main identifier to reference this template from instance
  "protect": false,
  "metadata": {
    "name": "Variable Wait",
    "description": "A demonstrative Wait Pipeline.",
    "owner": "example@example.com",
    "scopes": ["global"]
  },
  "pipeline": { # A "normal" pipeline definition.
    "lastModifiedBy": "anonymous",
    "updateTs": "0",
    "parameterConfig": [],
    "limitConcurrent": true,
    "keepWaitingPipelines": false,
    "description": "",
    "triggers": [],
    "notifications": [],
    "stages": [
      {
        "waitTime": "${ templateVariables.timeToWait }", # Templated field.
        "name": "My Wait Stage",
        "type": "wait",
        "refId": "wait1",
        "requisiteStageRefIds": []
      }
    ]
  }
}

4. Save the template

spin pipeline-templates save --file my_template.txt

…where my_template.txt is the file in which you constructed the template JSON.

spin checks that the file has a reference to the v2 schema and that it has a pipeline section.

Spinnaker uses the value of the id field in the JSON as the name of the pipeline template. That’s the name you use when you reference the template from a pipeline instance , and the name a user looks for when listing pipeline templates. So it’s a good practice to give your templates descriptive names. You might also institute a naming convention for your team or teams that makes it even clearer what each template is meant for.

Next steps

Docs: Inherit from a Template or Override It

Mon, 01 Jan 0001 00:00:00 +0000

In a pipeline that instantiates a template, besides providing values for template variables you can…

Inherit features from the template
Override (add to) inherited features.

Inherit from the template

By default, the pipeline instance inherits the stages, expected artifacts, triggers, parameters, and notifications from the template. It’s possible to opt out of inheriting triggers, parameters, and notifications by including the corresponding string in the exclude element.

For example, the template might have a trigger defined in the triggers element, but you can opt out of inheriting it by including triggers inside the exclude element.

In the pipeline template you are instantiating, examine what’s in the pipeline element to see what you want to use in your pipeline.
```
spin pipeline-template get --id <templateName>
```
In the exclude: [] section of the pipeline JSON, include the key for each element of the template that you want to opt out of inheriting.

For example, to opt out of inheriting triggers defined in pipeline.triggers, include triggers inside the exclude element of the new pipeline:
```
"exclude": ["triggers"]
```
The same goes for other items found inside pipeline.
```
"exclude": ["triggers", "notifications"] # for example
```

Now, all the triggers and notificatons (in this example) defined in the template are excluded from the pipeline instance.

Override the template

To override an element from a pipeline template is to add to that element. You can only override an element if you inherit it, and you can only add to it—you can’t remove or edit individual members.

The stage definitions inside stages are inherited by default. You can add further stages, and the configuration required to wire added stages with inherited stages.

Make sure you’re inheriting from the element or elements you want to override.

See above

Also, as mentioned above, stages defined in the tempate are inherited by default.
Add any triggers, notifications, parameters, etc., inside their respective elements.

These items are now added to those inherited from the template.

Docs: Create a Pipeline from a Template

Mon, 01 Jan 0001 00:00:00 +0000

Given a pipeline template that defines variables to be resolved per pipeline, you can create pipelines that reference that template and that provide values for all of those variables.

Instantiate a pipeline

Get the pipeline template.

spin pipeline-template list

This returns a list of all pipeline templates available in the Spinnaker deployment.
From the list, determine which of the listed templates is the one you want, and get it using the following command:

spin pipeline-template get --id <pipelineTemplateId>

This outputs the JSON content of the template. You can save it in a file, or just examine it so that you know what you’re implementing.

Create a file, which will contain the JSON for the pipeline.

Use the following format:

{
  "schema": "v2",
  "application": "myApp", # Set this to the app you want to create the pipeline in.
  "name": "<pipeline name>", # Pipeline name, remember this for the next part.
  "template": {
    "artifactAccount": "front50ArtifactCredentials", # Static constant
    "reference": "spinnaker://newSpelTemplate", # Reference to the pipeline template we published above. We saved it in Spinnaker, so we prefix the template id with ‘spinnaker://’.
    "type": "front50/pipelineTemplate", # Static constant
  },
  "variables": {
    "timeToWait": 4 # Value for the template variable.
  },
  "exclude": [],
  "triggers": [],
  "parameters": [],
  "notifications": [],
  "description": "",
  "stages": []
}

Make sure the content of the file includes, at the beginning, the schema: v2, reference.

Add a reference to the pipeline template:

In the template element, add a reference to the specific template, using the following format:
```
"template": {
  "artifactAccount": "front50ArtifactCredentials", # Static constant
  "reference": "spinnaker://<templateId>",
  "type": "front50/pipelineTemplate" # Static constant
}
```
Because the template was “saved,” using spin pipeline-template save, it was added to the Spinnaker deployment and is available using spinnaker://.

Provide values for the variables

Note: The variables defined in the pipeline template include default values, so you don’t have to provide a value for every variable defined.

In the pipeline JSON file (in the variables section), list each variable for which you’re providing values, and write that value.

Use the following format:

"variables": {
  "varName": <value>
  "otherVarName": <its_value>
}

You can code each value by hand in the pipeline JSON that you create. You can also generate the JSON and populate the values programatically. For simplicity, this page describes doing it by hand.

Specify inheritance and overrides

Indicate which elements of the template you want to inherit .

By default, the pipeline instance inherits the stages, expected artifacts, triggers, parameters, and notifications from the template. It’s possible to opt out of inheriting triggers, parameters, and notifications by including the corresponding string in the exclude element. For example, the template might have a trigger defined in the triggers element, but you can opt out of inheriting it by including triggers inside the exclude element.
If you want, you can override elements in the template.

Add new stages

Create a new stage by adding the stage spec to the pipeline JSON.

Include an inject element, indicating the stage’s position by identifying which stage this new stage comes after.

In the example below, the new stage wait0 is injected after the wait2 stage.

"stages": [
    {
        "refId": "wait2",
        "type": "wait",
        "config": {
            "waitTime": 67
        }
    },
    {
        "refId": "wait0",
        "inject": {
            "after": ["wait2"]
        },
        "type": "wait",
        "config": {
            "waitTime": 2
        }
    }
]

The config object contains the entire stage config. In this Wait-stage example, config includes only waitTime.

Add a branch to the pipeline

The template you’re using might itself have branches, but if it doesn’t, and you want your pipeline instance to have a branch, inject new stages as described above , but include multiple before or multiple after elements (or both) to describe the graph.

Save the pipeline

spin pipeline save --file <path to pipeline json>

Docs: Visualize a Hydrated Pipeline

Mon, 01 Jan 0001 00:00:00 +0000

You can use spin CLI to visualize a pipeline created from a pipeline template without actually instantiating one.

When you do this, you are creating JSON output that looks like regular pipeline JSON (plus the template reference and hydrated variables), but you’re not creating the actual pipeline in Spinnaker.

Using `spin pt plan`

spin pipeline-templates plan --file <path to pipeline config>

{
  "application": "my-spinnaker-app"
  "stages": [...] # Evaluated pipeline config based on template config values.
}

…where <path to pipeline config> points to the file you created when you instantiated a pipeline based on the template.

Now what do I do with it?

Use the output JSON to visualize what your resulting pipeline config will look like after you instantiate it. When you’re ready, you can save it as a pipeline in Spinnaker:

spin pipeline save --file <path to pipeline config>

Next steps

Create a pipeline from the template

Spinnaker – Using Pipeline Templates

Docs: Enable Pipeline Templates

Docs: Create a Pipeline Template

1. Get a pipeline’s JSON

2. Save the pipeline JSON to a file

3. Edit the file for template format

4. Save the template

Next steps

Docs: Inherit from a Template or Override It

Inherit from the template

Override the template

Docs: Create a Pipeline from a Template

Instantiate a pipeline

Provide values for the variables

Specify inheritance and overrides

Add new stages

Add a branch to the pipeline

Save the pipeline

Docs: Visualize a Hydrated Pipeline

Using spin pt plan

Now what do I do with it?

Next steps

Using `spin pt plan`