Spinnaker – Triggers Overview

Docs: Triggering pipelines with Bitbucket Server

Mon, 01 Jan 0001 00:00:00 +0000

Overview of supported Bitbucket Server events

Spinnaker supports the following Bitbucket Server webhook events:

Declined (pr:declined)
Deleted (pr:deleted)
Merged (pr:merged)
Modified/Source branch updated (pr:from_ref_updated)
Opened (pr:opened)
Push (repo:refs_changed)

Before you begin

You are familiar with creating and triggering webhooks in Bitbucket Server.
You have configured Bitbucket notifications (webhooks) in Spinnaker and enabled the desired webhook events in your Bitbucket repository.

Add a Bitbucket webhook event as a pipeline trigger

In your pipeline, go to Configuration > Automated Triggers.
Configure a new trigger with the following information:
- Type: Git
- Repo Type: bitbucket
- Team or User: enter your Bitbucket team or user name
- Repo name: enter your Bitbucket repo name
- Branch: enter your branch name (optional)
- Artifact Constraints: select a constraint (optional)
- Trigger Enabled: select the box

Troubleshooting

Check the Echo log for webhook activity.

Docs: Triggering pipelines on receiving CDEvents

Mon, 01 Jan 0001 00:00:00 +0000

About CDEvents

CDEvents is a common specification for Continuous Delivery events, it enables interoperability between the CI/CD tools.

CDEvents builds upon CloudEvents by adding CI/CD specific extensions to CloudEvents data object as a payload, more details on types of CDEvents and specification can be found in the Documentation at http://cdevents.dev

In order to programmatically trigger pipelines, you can send a CDEvent via POST request to Spinnaker at a preconfigured endpoint or create a subscription with this endpoint for a specific CDEvent type in the Message Broker . You can use this to trigger pipelines from other CI/CD tools which supports Sending CDEvents through configured Message Broker , or from the command line, or from a third-party system. The data payload will be available in the Pipeline’s execution.

Note: You can trigger multiple pipelines with one CDEvent message.

If you’re triggering to a Spinnaker with authentication, see the instructions here to set up the automated trigger.

Prerequisites

Artifact support enabled .

Adding a CDEvents trigger to a pipeline

Assuming you’ve created a pipeline, under Configuration, select Add Trigger and make its type selector CDEvents.

To assign an endpoint that must be hit, you can provide a value to the Source field as shown here:

Notice that in the above image below the Type dropdown, the endpoint configuration points out that we can send CDEvent to http://localhost:8084/webhooks/cdevents/artifactPublished to trigger the pipeline from localhost. The endpoint depends on how you’ve configured your Spinnaker endpoints – in the above image it is showing a different endpoint where Spinnaker is running.

Keeping track of that endpoint as $ENDPOINT (it will depend on where Spinnaker is installed), and save that pipeline.

You can run the below curl command to send a specific type of CDEvent that is built upon CloudEvent.

curl $ENDPOINT -X POST -H "Ce-Id: 123456789" -H "Ce-Specversion: 1.0" -H "Ce-Type: dev.cdevents.artifact.published.0.1.0" -H "Ce-Source: https://ci-build.est.tech" -H "Content-Type: application/json" -d $CDEVENTDATA

$CDEVENTDATA can be constructed as per CDEvents-spec , to trigger the pipeline with different Payload constraints and HTTP headers can be used to trigger the pipelines with matching Attribute constraints.

Attribute constraints

If you want to trigger the pipeline only when the specific headers are matching from the CDEvents webhook request, you can provide Attribute constraints in the trigger configuration.

For example, If we configure the pipeline as below, The attribute constraints used here ce-type = dev.cdevents.artifact.published.0.1.0 and ce-source = https://ci-build.est.tech from the request.

Payload constraints

If you want to ensure that a pipeline triggers only when a certain payload arrives, you can provide Payload Constraints in the trigger. These are key/value pairs where the key must be found in the incoming payload, and the value must match using regex.

For example, if we configured:

The payload constraints used here context.source = https://ci-build.est.tech, customData.mykey = myvalue, customData.bing = b.*p and subject.type = ARTIFACT."

The following $CDEVENTDATA payload would be accepted:

{
  "context": {
    "id": "123456789",
    "type": "dev.cdevents.artifact.published.0.1.0",
    "source": "https://ci-build.est.tech",
    "version": "0.1.2"
  },
  "customData": {
    "mykey": "myvalue",
    "bing": "boooop",
    "x": ["1","2","3"]
  },
  "customDataContentType": "application/json",
  "subject": {
    "type": "ARTIFACT",
    "content": {
      "pipelineName": "test-1",
      "outcome": "SUCCESS"
    }
  }
}

But this payload would be rejected (pipeline would not trigger):

{
  "context": {
    "id": "123456789",
    "source": "https://other-url.est.tech",
    ...
  },
  "customData": {
    "mykey": "myvalue",
    "x": ["1","2","3"]
  },
  "customDataContentType": "application/json",
  "subject": {
    "type": "PIPELINERUN",
    ...
  }
}

Passing parameters

Say your pipeline accepted some parameters (for example, the desired stack to deploy to), you can make this explicit by adding a pipeline parameter on the same configuration screen as the CDEvents Automated trigger:

For more information on how to use pipeline parameters, see the pipeline expressions guide ."

Warning: there are several reserved parameter keys (names) that cause unexpected behaviour and failures if overwritten by a pipeline parameter definition. See the list of reserved parameter and evaluate variable key names .

If you were to manually execute this pipeline, you would be prompted with the following dialogue:

If instead you were to trigger this pipeline with a CDEvents Webhook request, you could supply each parameter a value inside a key/value map called parameters with in the customData of a $CDEVENTDATA payload. Take the following payload for example:

{
  "context": {
    "id": "5fb38d7d-28dd-47e4-ade2-cb21153cc8dc",
    ...
  },
  "customData": {
    "parameters": {
      "stack": "prod"
    }
  },
  "customDataContentType": "application/json",
  "subject": {
    "type": "ARTIFACT",
    ...
  }
}

Note: If you select the Required checkbox for a parameter without providing a default, the pipeline does not trigger if a parameter is not present. The difference between this and the preconditions covered earlier is that when a precondition isn’t met, Spinnaker doesn’t even try to run the pipeline. However, when a required parameter doesn’t exist, Spinnaker tries and fails to run a pipeline, surfacing a “Failed Execution” in the UI.

Passing artifacts

If your pipeline requires artifacts (for example, a custom artifact stored in a specific location), you can make this explicit by defining an Expected Artifact and assigning it to the CDEvents Webhook as shown below:

In order to run this pipeline, you will need to supply the required artifact in the customData of a $CDEVENTDATA payload under a list of artifacts:

{
  "context": {
    "id": "5fb38d7d-28dd-47e4-ade2-cb21153cc8dc",
    ...
  },
  "customData": {
    "artifacts": [
      {
        "type": "custom/object",
        "name": "image-cdevents",
        "version": "1.0.1",
        "location": "localhost:5000/cdevent/image-cdevents:v1.0.1"
        "reference": "custom/object/artifact"
      }
    ]
  },
  "customDataContentType": "application/json",
  "subject": {
    "type": "ARTIFACT",
    ...
  }
}

Docs: Receiving artifacts from GCS

Mon, 01 Jan 0001 00:00:00 +0000

This functionality uses Google’s Pub/Sub system for delivering messages to Spinnaker and must be configured to send messages to Spinnaker’s event bus as shown below.

Prerequisites

If you or your Spinnaker admin have already configured Spinnaker to listen to Pub/Sub messages from the GCS bucket you plan to publish objects to, you can skip this section. One Pub/Sub subscription can be used to trigger as many independent Spinnaker pipelines as needed.

You need the following:

A billing enabled Google Cloud Platform (GCP) project .

This will be referred to as $PROJECT from now on.
gcloud . Make sure to run gcloud auth login if you have installed gcloud for the first time.
A running Spinnaker instance . This guide shows you how to configure an existing one to accept GCS messages and download the files referenced by the messages in your pipelines.
Artifact support enabled .

At this point, we will configure Pub/Sub and a GCS artifact account. The Pub/Sub messages will be received by Spinnaker whenever a file is uploaded or changed, and the artifact account will allow you to download these where necessary.

1. Configure Google Pub/Sub for GCS

Follow the Pub/Sub configuration , in particular, pay attention to the GCS section since this is where we’ll be publishing our files to.

2. Configure a GCS artifact account

Follow the GCS artifact configuration .

3. Apply your configuration changes

Once the Pub/Sub and artifact changes have been made deploy them to Spinnaker .

Using GCS artifacts in pipelines

We will need either an existing or a new pipeline that we want to be triggered on changes to GCS artifacts. If you do not have a pipeline, create one as shown below.

You can create and edit pipelines in the Pipelines tab of Spinnaker

Configure the GCS trigger

Let’s add a Pub/Sub trigger to run our pipeline.

Next, we must configure the trigger:

Field	Value
Type	“Pub/Sub”
Pub/Sub System Type	“Google”
Subscription Name	Depends on your Pub/Sub configuration
Attribute Constraints	Must be configured to include the pair `eventType`:`OBJECT_FINALIZE` (See the docs )
Expected Artifacts	Must reference the artifact defined previously

Configure the GCS artifact

Now we need to declare that the Pub/Sub trigger expects that a specific artifact matching some criteria is available before the pipeline starts executing. In doing so, you guarantee that an artifact matching your description is present in the pipeline’s execution context. If no artifact for this description is present, the pipeline won’t start.

To configure the artifact, go to the Artifact Constraints dropdown for the Pub/Sub trigger configuration, and select “Define a new artifact…” to bring up the Expected Artifact form.

Enter a Display Name, or leave the autogenerated default. In the Account dropdown, select your GCS account. Enter the fully qualified GCS path in the Object path field. Note: this path can be a regular expression. You can, for example, set the object path to be gs://${BUCKET}/folder/.* to trigger on any change to an object inside folder in your ${BUCKET}.

${BUCKET} is a placeholder for the GCS bucket name that you have configured to receive Pub/Sub messages from above.

Test the pipeline

If you upload a file to a path matching your configured Object path, the pipeline should execute. If it doesn’t, you can start by checking the logs in the Echo service.

Docs: Receiving artifacts from GitHub

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This functionality uses GitHub Webhooks for delivering messages to Spinnaker and must be configured to send messages to Spinnaker’s event bus as shown below.

Prerequisites

If you (or your Spinnaker admin) have already configured Spinnaker to listen to a GitHub webhooks from the repository you plan to publish commits to, you can skip this section.

You need the following:

A GitHub repository either under your user or in an organization or user’s account that you have permission to publish commits to.

This will be referred to as $ORGANIZATION/$REPOSITORY from now on (for example, spinnaker/clouddriver).
A running Spinnaker instance . This guide shows you how to update it to accept messages from GitHub.
Artifact support enabled .

At this point, we will configure GitHub webhooks and a GitHub artifact account. The webhook will be received by Spinnaker whenever a commit is made, and the artifact account will allow you to download any pertinent files.

1. Configure GitHub webhooks

Follow the GitHub webhook configuration .

2. Configure a GitHub artifact account

Follow the GitHub artifact configuration .

3. Apply your configuration changes

Once the artifact changes have been made deploy them to Spinnaker .

Using GitHub artifacts in pipelines

We will need either an existing or a new pipeline that we want to be triggered on changes to GitHub artifacts. If you do not have a pipeline, create one as shown below.

You can create and edit pipelines in the Pipelines tab of Spinnaker

Configure the GitHub trigger

Let’s add a Git trigger to run our pipeline. To configure the trigger:

Field	Value
Type	“Git”
Repo Type	“GitHub”
Organization or User	`$ORGANIZATION` from above
Project	`$REPOSITORY` from above
Branch	(optional) Can be used (via regular expressions) to describe which branches to listen to changes one
Secret	(optional) Strongly encouraged It must match the secret provided to the webhook configuration

Configure the GitHub artifact

Now we need to declare that the GitHub trigger expects that a specific artifact matching some criteria is available before the pipeline starts executing. In doing so, you guarantee that an artifact matching your description is present in the pipeline’s execution context. If no artifact for this description is present, the pipeline won’t start.

To configure the artifact, go to the Artifact Constraints dropdown for the GitHub trigger configuration, and select “Define a new artifact…” to bring up the Expected Artifact form.

Enter the artifact information:

Enter a Display Name, or leave the autogenerated default.
In the Account dropdown, select your GitHub account.
Enter a value in the File path field.

☞ Note: This path can be a regular expression. You can, for example, set the object path to be folder/.*\.yml to trigger on any change to a YAML file inside folder in your repository.

Test the pipeline

If you add or modify a file matching your expected artifact to the configured repository, it should execute. If it doesn’t, you can start by checking the logs of the Echo service.

Docs: Triggering pipelines with Jenkins

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Set up Jenkins as a continuous integration system in your Spinnaker deployment.
Enable artifact support .

Adding a Jenkins trigger

Create a pipeline .
In the Configuration stage of your new pipeline, add a trigger .
Select Jenkins from the Type menu, which brings up the following screen:
Select a Jenkins master from the Master drop-down menu, then select a job from the Job drop-down.
Add a property file, if desired. See the property files section of the Pipeline Expression Guide for more information about how to specify and use property files.

Build Artifacts

Jenkins has the ability to save files from a build as a build artifacts. These objects could be compiled binaries or kubernetes manifests or any other artifact created as part of the build process.

Best practice dictates that these artifacts should be uploaded to a dedicated artifact repository such as GCR, DockerHub, Artifactory, Maven etc, however sometimes it’s easier to store the artifact as part of the build output.

Here is an example of a Jenkins pipeline that stores artifacts

stages {
  stage('Generate Kubernetes Manifests') {
    steps {
    sh '''
    npm install
    npm run import
    npm run build
    echo "BUILD=${BUILD_NUMBER}" > build.properties
    '''
    }
  }
}
post {
  always {
    archiveArtifacts artifacts: 'dist/*.yaml', fingerprint: true
    archiveArtifacts artifacts: 'build.properties', fingerprint: true
  }
}

This job will generate the following artifacts

The line echo "BUILD=${BUILD_NUMBER}" > build.properties adds a build number field to the build.properties which can be used later to generate the URL used to download the artifact using HTTP.

The build.properties artifact should be added to the Jenkins trigger like this

Finally to add the Jenkins build artifact as a Spinnaker artifact, do the following

Create a HTTP File artifact account for Jenkins using the Jenkins base URL and credentials that has permissions to access the build artifact
Right click on the build artifact displayed in the Jenkins Build Output above. This will give you the full URL for the build artifact - something like this https://{jenkins-url}/job/DevToolStack/90/artifact/cdk8s/dist/terraform-operator.k8s.yaml
Add a new HTTP file artifact to the pipeline replacing the build number with the BUILD field from the properties using the following variable - ${trigger.properties['BUILD']} like this

Docs: Triggering pipelines with Artifactory

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to add a JFrog Artifactory trigger to your pipeline.

Currently, the Artifactory trigger only works with Maven artifacts.

Prerequisites

Configure artifactory. Add the following to igor-local.yml for triggering/monitoring of repos:

artifactory:
  enabled: true
  searches:
  - name: dev
    username: user
    password: password
    accessToken: accessToken

More configuration properties are available in the ArtifactorySearch class.

A Maven account configured in Spinnaker clouddriver-local.yml (used for pulling a file from maven):

artifacts:
  maven: 
    enabled: true
    accounts:
    - name: maven-central
      repositoryUrl: https://repo1.maven.org/maven2/

Configuration properties are available in the MavenArtifactAccount . Maven artifacts do not support a lot of features at this time. For additional options, please open PRs to add additional capabilities.

Artifact support enabled .

Adding an Artifactory Trigger

Create a pipeline.
In the Configuration stage of your new pipeline, add a trigger.
In the Type menu, select Artifactory. This brings up the following screen:
In the Artifactory Name menu, select an Artifactory search.
In the Artifact Constraints menu, select “Define a new artifact. This brings up the following screen:
Enter a name in the Display Name field or leave the autogenerated default.
In the Account menu, select a Maven account.
In the Maven Coordinate field, enter the Maven coordinates of the artifact.
Click Save Artifact.

Docs: Triggering on Pub/Sub Messages

Mon, 01 Jan 0001 00:00:00 +0000

In order to programmatically trigger pipelines, you can configure Spinnaker to subscribe and listen to a Pub/Sub topic and push messages to the configured topic. This can be used to trigger pipelines during CI jobs, from the command line, or from a third-party system. The message payload will be available in the Pipeline’s execution.

Note: It’s possible to configure multiple pipelines to trigger off of a single Pub/Sub message.

Only Google and AWS Pub/Sub systems are supported. See the detailed instructions for Google Pub/Sub and AWS SQS/SNS .

Prerequisites

Artifact support enabled .

Adding a Pub/Sub trigger to a pipeline

Assuming you have created a pipeline, under Configuration, select Add Trigger and make its type selector Pub/Sub.

To select a Pub/Sub subscription to trigger from, select values for Pub/Sub System Type and Subscription Name. Note that the subscription must be configured before it is available to select in the UI.

Payload constraints

If you want to ensure that a Pub/Sub trigger only fires when a certain message payload arrives, you can provide Payload Constraints in the trigger. These are key/value pairs where the key must be found in the incoming message payload, and the value must match using regex.

For example, if we had configured:

For clarity, the constraints are mykey = myvalue and bing = b.*p.

The following message payload would be accepted:

{
  "mykey": "myvalue",
  "bing": "boooop",
  "x": ["1", "2", "3"]
}

But this message payload would be rejected (pipeline would not trigger):

{
  "mykey": "myvalue",
  "x": ["1", "2", "3"]
}

Passing parameters

For more information on how to use pipeline parameters, see the pipeline expressions guide .

Warning: there are several reserved parameter keys (names) that cause unexpected behavior and failures if overwritten by a pipeline parameter definition. See the list of reserved parameter and evaluate variable key names .

If you were to manually execute this pipeline, you would be prompted with the following dialogue:

If instead you were to trigger this pipeline with a Pub/Sub Trigger, you could supply each parameter a value inside a key/value map called parameters in the message body. Take the following message payload for example:

{
  "parameters": {
    "stack": "prod"
  }
}

☞ Note: If you had selected the Required checkbox for a parameter without providing a default, the pipeline doesn’t trigger if a parameter is not present. The difference between this and the preconditions covered earlier is that when a precondition isn’t met, Spinnaker will not even try to run the pipeline. However, when a required parameter doesn’t exist, Spinnaker will try and fail to run a pipeline, surfacing a “Failed Execution” in the UI.

Passing artifacts

If your pipeline requires artifacts (for example, a Kubernetes manifest file stored in GCS), you can make this explicit by defining an Expected Artifact and assigning it to the Pub/Sub Trigger. Under Artifact Constraints, select Define a new artifact… and supply the artifact details, as shown below:

In order for this to work, you need to supply the required artifact in the Pub/Sub message payload, and configure Spinnaker so that it can translate the Pub/Sub payload into a Spinnaker artifact.

If you’re using GCR, you can use the --message-format flag and Spinnaker will translate the payload automatically:

hal config pubsub google subscription edit my-gcr-subscription \
  --message-format GCR

Otherwise, you need to supply a translation template so that Spinnaker can translate the Pub/Sub payload into a Spinnaker artifact. To do this, create a Jinja template . Note that the output of the Jinja transform must be a JSON list of Spinnaker artifacts. The translation template itself can be any valid Jinja transform.

Use the following hal command to tell Spinnaker how to find the template:

hal config pubsub google subscription edit my-gcs-subscription \
  --template-path /path/to/jinja/template

Example

Let’s say you have a message payload containing the following structure:

{
  ...
  "location": "gs://jtk54-artifacts/manifest.yml"
  ...
}

You can use the following Jinja template to translate the above into the Spinnaker artifact format:

[
  {
    "type": "gcs/object", # static type.
    "reference": "{{"{{ location "}}}}", # 'location' in the Pub/Sub payload.
  }
]

Docs: Triggering on Webhooks

Mon, 01 Jan 0001 00:00:00 +0000

In order to programmatically trigger pipelines, you can send a POST call to Spinnaker at a preconfigured endpoint. You can use this to trigger pipelines when a CI job finishes, from the command line, or from a third-party system. The payload, whether it is one you are able to write or it is provided for you, will be available in the Pipeline’s execution.

Note: You can configure multiple pipelines to trigger off of a single webhook.

If you’re triggering from a GitHub webhook, see the instructions here to set up that webhook.

If you’re triggering to a Spinnaker with authentication, see the instructions here to set up the automated trigger.

Prerequisites

Artifact support enabled .

Adding a webhook trigger to a pipeline

Assuming you’ve created a pipeline, under Configuration, select Add Trigger and make its type selector Webhook.

To assign an endpoint that must be hit, you can provide a value to the Source field as shown here:

Notice that in the above image below the Type dropdown, the webhook configuration points out that we can hit http://localhost:8084/webhooks/webhook/demo to trigger the pipeline. The endpoint depends on how you’ve configured your Spinnaker endpoints – if you’re running on a different endpoint, for example https://api.spinnaker-prod.net, that’ll be shown instead.

Keeping track of that endpoint as $ENDPOINT (it will depend on where Spinnaker is installed), save that pipeline, and run:

curl $ENDPOINT -X POST -H "content-type: application/json" -d "{ }"

Payload constraints

If you want to ensure that a webhook only triggers when a certain payload arrives, you can provide Payload Constraints in the trigger. These are key/value pairs where the key must be found in the incoming payload, and the value must match using regex.

For example, if we configured:

For clarity, the constraints are mykey = myvalue and bing = b.*p.

The following payload would be accepted:

{
  "mykey": "myvalue",
  "bing": "boooop",
  "x": ["1", "2", "3"]
}

But this payload would be rejected (pipeline would not trigger):

{
  "mykey": "myvalue",
  "x": ["1", "2", "3"]
}

Passing parameters

For more information on how to use pipeline parameters, see the pipeline expressions guide .

Warning: there are several reserved parameter keys (names) that cause unexpected behavior and failures if overwritten by a pipeline parameter definition. See the list of reserved parameter and evaluate variable key names .

If you were to manually execute this pipeline, you would be prompted with the following dialogue:

If instead you were to trigger this pipeline with a Webhook, you could supply each parameter a value inside a key/value map called parameters. Take the following payload for example:

{
  "parameters": {
    "stack": "prod"
  }
}

Note: If you select the Required checkbox for a parameter without providing a default, the pipeline does not trigger if a parameter is not present. The difference between this and the preconditions covered earlier is that when a precondition isn’t met, Spinnaker doesn’t even try to run the pipeline. However, when a required parameter doesn’t exist, Spinnaker tries and fails to run a pipeline, surfacing a “Failed Execution” in the UI.

Passing artifacts

In order to run this pipeline, you will need to supply the required artifact in your payload under a list of artifacts:

{
  "artifacts": [
    {
      "type": "gcs/object",
      "name": "manifest.yml",
      "reference": "gs://lw-artifacts/manifest.yml"
    }
  ]
}