Spinnaker – AWS EC2 Server Group Launch Settings

Docs: AWS EC2 Launch Templates Setup

Mon, 01 Jan 0001 00:00:00 +0000

Please note that you should only proceed with this if you have AWS EC2 configured as a cloud provider. These features require 1.24 (although some features were launched in previous releases leading up to 1.24).

AWS uses launch templates to specify instance configuration information. Launch templates are the successor of launch configurations. This means that any new instance configuration feature from AWS will only be supported by launch templates.

Spinnaker still supports launch configurations for backwards compatibility, but recommends enabling launch templates to access any new features that AWS adds.

Setup Steps

This section summarizes the steps required to set up launch templates if you are new to using AWS in Spinnaker or if you have already been using AWS as one of your cloud providers.

New to AWS

If you are new to Spinnaker or even just new to AWS in Spinnaker, we recommend immediately enabling launch template support for all applications.

Update your Clouddriver configuration file, usually clouddriver.yml, to enable launch templates for all applications.

  aws.features.launch-templates.enabled: true
  aws.features.launch-templates.all-applications.enabled: true

Read through the available launch template supported features to determine which features make sense for your users.

Update AWS settings in settings-local.js of Deck to enable launch templates and the features you identified. Ensure that enableLaunchTemplates is true.

window.spinnakerSettings.providers.aws.serverGroups = {
    enableLaunchTemplates: true,
    enableIPv6: true,
    enableIMDSv2: true,
    enableCpuCredits: true,
};

Current AWS User

If you already use AWS as a cloud provider in Spinnaker, we recommend migrating to launch templates. Since there may be pre-existing dependencies on launch configurations, we have created some rollout configurations you can utilize for testing and/or migration.

Update clouddriver.yml. This step can be repeated as needed throughout your rollout. This is an example config where launch templates is rolled out to two applications in production and all of the test account. It also excludes one application completely:
```
  aws.features.launch-templates.enabled: true
  aws.features.launch-templates.allowed-applications: "myapp:prod:us-east-1,anotherapp:prod:us-east-1"
  aws.features.launch-templates.allowed-accounts: "test"
  aws.features.launch-templates.excluded-applications: "dangerousapp"
```
Review the rollout configurations and determine which of these you can temporarily utilize for your rollout. If you do not need to rollout, stop here and follow the new AWS users steps instead.
Read through the available launch template supported features to determine which features make sense for your users.

Update AWS settings in settings-local.js of Deck to enable launch templates and the features you identified. Ensure that enableLaunchTemplates is true.

// enable launch templates for AWS
window.spinnakerSettings.providers.aws.serverGroups = {
    enableLaunchTemplates: true,
    enableIPv6: true,
    enableIMDSv2: true,
    enableCpuCredits: true,
};

When you are ready for a complete rollout, enable launch templates for all applications and clean up rollout config in clouddriver.yml.
```
aws.features.launch-templates.enabled: true
aws.features.launch-templates.all-applications.enabled: true
```

Rollout Configuration

If you already use AWS, then your applications may have some dependencies on launch configurations that prevent simple feature enabling. The configuration options below were created to aid with testing or a rollout period. Feel free to use whatever combination is best for you. If you would prefer to skip a rollout, use the configuration in New to AWS .

Config	Type	Description	Example
allowed-applications	String	A comma-separated list of one or more allowed applications scoped by account-region pairs ("app:account:region"). This helps with preliminary controlled testing on a handful of applications.	"testapp:prod:us-east-1"
allowed-accounts-regions	String	A comma-separated list of allowed account-region pairs. This is good for incrementally rolling out to regions within accounts.	"test:us-east-1"
allowed-accounts	String	A comma-separated list of allowed accounts. This is good for incrementally rolling out launch templates from test to production accounts.	"test"
excluded-accounts	String	A comma-separated list of accounts to exclude from rollout.	"prod"
excluded-applications	String	A comma-separated list of applications to exclude from rollout. This helps prevent any edge cases from delaying a wide rollout..	"myapp1,myapp2"
all-applications.enabled	Boolean	Allows launch templates on any application, except for those that have been excluded. This will override any of the allowed lists, and widely rollout launch templates.	true

Feature Configuration

Learn more about the feature set along with sample API requests here .

Docs: AWS EC2 Launch Templates

Mon, 01 Jan 0001 00:00:00 +0000

Launch Template Support must be enabled in order to use the features described in this page. Follow guidelines here to enable the feature set.

Note: The features supported in the UI (Deck) come with tool tips that can help you learn the new features. Consider trying them out in Deck, especially if they are new to you.

Feature Configuration

Once launch templates support is enabled in Clouddriver, a new set of features are unlocked. Some of them are tied to the EC2 launch template directly like IMDSV2 and others require a launch template to be used in order to support features like SpotAllocationStrategy ( MixedInstancesPolicy features).

Review the sections below for details about the features and sample use cases along with Spinnaker API requests.

Launch Template Only Parameters

Review the sections below to determine which features you want to enable in the Spinnaker UI / Deck. Users will see enabled features as options when configuring a Server Group.

Feature	Description	Deck Setting	Clouddriver API Request Parameter	Release Version	Default value in Spinnaker
IPv6	ASGs associate an IPv6 address to their instances.	enableIPv6	associateIPv6Address	v1.21	no default
IMDSv2	Helps mitigate AWS credential theft from the exploitation of SSRF vulnerabilities in web applications. This is only supported by modern SDKs. Learn more from AWS.	enableIMDSv2	requireIMDSv2	v1.21	false
CreditSpecification	The credit option for CPU usage of the instance. Valid for burstable performance EC2 instances only. Learn more from AWS.	enableCpuCredits	unlimitedCpuCredits	v1.24	default false used only when all instance types in request support bursting.

Additional Features

Enabling launch template support also unlocks additional features offered by Amazon EC2 Auto Scaling that facilitate diversified server groups:

Multiple instance types
Multiple purchase options (On-Demand / Spot) with proportion control
Allocation strategies
Capacity rebalancing

Follow AWS recommended best practices for EC2 Spot to improve your experience with EC2 Spot.

Motivation to diversify instances in your Server Group:

Reduce costs by diversifying instances across purchase options and Spot allocation strategies. See AWS docs to learn more about how to use Spot .
Reduce costs and optimize resources with instance weighting.
Enhance availability by deploying your application across multiple instance types running in multiple Availability Zones.
Enhance availability by proactively augmenting your fleet with a new Spot instance from an optimal pool, before a running instance is interrupted by EC2, by enabling capacity rebalance .
Reduce probability of InsufficientInstanceCapacity exceptions with flexible instance configuration.

Using one or more parameters in the table below will automatically create Server Groups with instances distribution configuration specified, and will use AWS defaults for the parameters not specified. Note that a number of these parameters complement each other. So, combining them can greatly enhance your AWS experience.

Category	Feature / Clouddriver API Request Parameter	Description	AWS Default	Clouddriver Release Version	Deck Setting	Deck Release Version
Multiple instance types with instance weighting and priority, with launch template overrides. Properties overridden will replace the same properties in launch template. AWS docs: Instance Weighting	launchTemplateOverridesForInstanceType.instanceType	Instance Type to override	no default	v1.26	Instance Types / LaunchTemplate -> Instance Types	v1.28
launchTemplateOverridesForInstanceType.weightedCapacity	The number of capacity units for the specified instance type in terms of virtual CPUs, memory, storage, throughput, or other relative performance characteristic. The capacity units count toward the desired capacity.	1 i.e. if no weighted capacity is specified, all instance types (large or small) are treated as the same weight.
launchTemplateOverridesForInstanceType.priority	Optional priority for instance type. Lower the number, higher the priority. If unset, the launch template override has the lowest priority. The order of instance types in the list of launch template overrides sent to AWS is set from highest to lowest priority. Valid values: integer > 0.	first to last in list	v1.27
On-Demand AWS docs: Instances Distribution	onDemandAllocationStrategy	Indicates how to allocate instance types to fulfill On-Demand capacity. The only valid value is prioritized. This strategy uses the order of instance types in the launch template overrides to define the launch priority of each instance type. The first instance type in the list is prioritized higher than the last.	prioritized	v1.26	Instance Types -> Instances Distribution / Instances Distribution
onDemandBaseCapacity	The minimum amount of the Server Group's capacity that must be fulfilled by On-Demand Instances. NOTE: If weights are specified for the instance types in the overrides, set the value of OnDemandBaseCapacity in terms of the number of capacity units, and not number of instances.	0
onDemandPercentageAboveBaseCapacity	The percentages of On-Demand Instances and Spot Instances for additional capacity beyond OnDemandBaseCapacity.	100 i.e. only On-Demand instances
Spot AWS docs: * Instance Distribution * ASG Capacity Rebalance * EC2 Instance Rebalance Recommendations	spotAllocationStrategy	Indicates how to allocate instances across Spot Instance pools. 2 strategies: 1) capacity-optimized (recommended): instances launched using Spot pools that are optimally chosen based on the available Spot capacity. 2) lowest-price: instances launched using Spot pools with the lowest price, and evenly allocated across the number of Spot pools specified in spotInstancePools.	lowest-price
spotInstancePools	The number of Spot Instance pools across which to allocate Spot Instances. The Spot pools are determined from the different instance types in the overrides. Only applicable with 'lowest-price' spotAllocationStrategy.	2
spotPrice	The maximum price per unit hour that the user is willing to pay for a Spot Instance.	On-Demand price for the configuration
capacityRebalance	Enable to allow Amazon EC2 Auto Scaling to attempt proactive replacement of Spot Instances in the server group that have received a rebalance recommendation (i.e. at elevated risk of interruption), before they are interrupted by AWS EC2. Note: Enabling this feature could exceed the server group's max capacity for a brief period of time, leading to higher costs. Learn more in AWS docs.	false i.e. disabled	v1.27	Advanced Settings -> capcity rebalance	v1.27

Use Cases & Sample API Requests

Create a Server Group with launch template

After enabling the launch template feature set in Clouddriver and/or Deck, set setLaunchTemplate to true in order to indicate Spinnaker to create your Server Group with an EC2 launch template.

curl -H 'Content-Type: application/json' -d '{ "job": [
  {
    "type": "createServerGroup",
    "cloudProvider": "aws",
    "account": "my-aws-account",
    "application": "myAwsApp",
    "stack": "myStack",
    "credentials": "my-aws-account",
    "subnetType": "public-subnet",
    "availabilityZones": {"us-west-1": ["us-west-1a","us-west-1b","us-west-1c"]},
    "amiName": "ami-12345",
    "capacity": {"desired": 3,"max": 5,"min": 3},
    "iamRole":"MyInstanceProfile",
    "instanceType":"t3.large",
    "setLaunchTemplate": true,
    "requireIMDSv2": true,
    "unlimitedCpuCredits": true
  }], "application": "myAwsApp", "description": "Create New Server Group in cluster myAwsApp"}' -X POST http://my-spinnaker-gate:8084/tasks

Let’s say, the Server Group created was named myAwsApp-myStack-v005. It is backed by EC2 launch template with IMDSv2 enabled and unlimited CPU credits.

Convert a Server Group with launch template to use mixed instances policy, with multiple instance types and capacity weighting

The Spinnaker operation, modifyServerGroupLaunchTemplate/ updateLaunchTemplate also supports updating a Server Group backed by launch template to use mixed instances policy when one or more mixed instances policy parameters listed above is specified.

curl -H 'Content-Type: application/json' -d '{ "job": [ 
  {
    "type": "updateLaunchTemplate",
    "cloudProvider": "aws",
    "account": "my-aws-account",
    "application": "myAwsApp",
    "stack": "myStack",
    "credentials": "my-aws-account",
    "region": "eu-central-1",
    "asgName": "myAwsApp-myStack-v005",
    "launchTemplateOverridesForInstanceType": [
      {"instanceType":"t2.large","weightedCapacity":"1"},
      {"instanceType":"t3.large","weightedCapacity":"1"},
      {"instanceType":"t2.xlarge","weightedCapacity":"2"},
      {"instanceType":"t3.xlarge","weightedCapacity":"2"}] 
  }], "application": "myAwsApp", "description": "Modify Server Group in cluster myAwsApp"}' -X POST http://my-spinnaker-gate:8084/tasks

Note: instance weighting is requested by vCPUs.

AWS docs:

Diversify instances in a Server Group across purchase options (On-Demand and Spot), multiple instance types with priority assignment:

curl -H 'Content-Type: application/json' -d '{ "job": [ 
  {
    "type": "createServerGroup",
    "cloudProvider": "aws",
    "account": "my-aws-account",
    "application": "myAwsApp",
    "stack": "myStack",
    "credentials": "my-aws-account",
    "availabilityZones": {"us-west-1": ["us-west-1a","us-west-1b","us-west-1c"]},
    "amiName": "ami-12345",
    "capacity": {"desired": 3,"max": 5,"min": 3},
    "iamRole":"MyInstanceProfile",
    "instanceType":"m4.large",
    "setLaunchTemplate": true,
    "onDemandBaseCapacity":1,
    "onDemandPercentageAboveBaseCapacity":50,
    "spotAllocationStrategy":"capacity-optimized",
    "launchTemplateOverridesForInstanceType": [
      {"instanceType":"m5.large","weightedCapacity":"1","priority": 2},
      {"instanceType":"m5.xlarge","weightedCapacity":"2","priority": 1}] 
  }], "application": "myAwsApp", "description": "Create New Server Group in cluster myAwsApp"}' -X POST http://my-spinnaker-gate:8084/tasks

See capacity type for instances in Deck:

AWS docs:

Create a Server Group with AWS recommended best practices for EC2 Spot

The best practices followed in the example:

use multiple instance types
use capacity-optimized Spot allocation strategy to indicate AWS to provision instances from the most-available Spot capacity pools
use default maximum price for Spot i.e. On-Demand price
mix in previous generation instance types
use proactive capacity rebalancing (works best with capacity-optimized spotAllocationStrategy)

curl -H 'Content-Type: application/json' -d '{ "job": [ 
  {
    "type": "createServerGroup",
    "cloudProvider": "aws",
    "account": "my-aws-account",
    "application": "myAwsApp",
    "stack": "myStack",
    "credentials": "my-aws-account",
    "subnetType": "public-subnet",
    "availabilityZones": {"us-west-1": ["us-west-1a","us-west-1b","us-west-1c"]},
    "amiName": "ami-12345",
    "capacity": {"desired": 3,"max": 5,"min": 3},
    "iamRole":"MyInstanceProfile",
    "instanceType":"m4.large",
    "setLaunchTemplate": true,
    "onDemandPercentageAboveBaseCapacity":50,
    "onDemandBaseCapacity":1,
    "spotAllocationStrategy":"capacity-optimized",
    "capacityRebalance": true,
    "launchTemplateOverridesForInstanceType": [
      {"instanceType":"m4.large","weightedCapacity":"1"},
      {"instanceType":"m5.large","weightedCapacity":"1"},
      {"instanceType":"m4.xlarge","weightedCapacity":"2"},
      {"instanceType":"m5.xlarge","weightedCapacity":"2"}] 
  }], "application": "myAwsApp", "description": "Create New Server Group in cluster myAwsApp"}' -X POST http://my-spinnaker-gate:8084/tasks

Note: instance weighting requested by vCPUs

AWS docs: