Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

Spacelift EC2 Private Worker Pool Blueprint

This blueprint deploys a Spacelift private worker pool on AWS EC2 using an Auto Scaling Group. It uses the official terraform-aws-spacelift-workerpool-on-ec2 module.

Architecture

Blueprint Input
    │
    ▼
[Admin Stack] (administrative: true)
    ├── Creates Spacelift Worker Pool
    ├── Generates credentials (token + private key)
    ├── Creates API Key (if autoscaling enabled)
    ├── Creates secure context with credentials
    └── Creates child stack:
        │
        ▼
    [Tofu Stack]
        ├── Creates Security Group
        └── Deploys EC2 Auto Scaling Group
            └── Workers connect to Spacelift

Prerequisites

1. VCS Integration (GitHub/GitLab/Bitbucket)

You must have a VCS provider connected to Spacelift and this repository accessible. The blueprint will automatically use the repository it's created from for child stacks.

2. AWS Cloud Integration

You must have an AWS Cloud Integration configured in Spacelift. The integration needs permissions to create:

  • EC2 instances
  • Auto Scaling Groups
  • Launch Templates
  • Security Groups
  • IAM roles (for the worker instances)
  • Secrets Manager secrets (for credential storage)
  • Lambda functions (if autoscaling is enabled)

Blueprint Inputs

Input Description Required Default
aws_integration_id Spacelift AWS Integration ID Yes -
aws_default_region AWS region for deployment Yes us-east-1
space_id Spacelift space for resources No root
worker_pool_name Name for the worker pool No ec2-workers
subnet_ids Comma-separated subnet IDs Yes -
instance_type EC2 instance type No t3.medium
min_size Minimum ASG instances No 1
max_size Maximum ASG instances No 5
enable_autoscaling Enable queue-based autoscaling No false
use_spot_instances Use Spot instances No false
poweroff_delay Idle worker shutdown delay (seconds) No 15
run_tag Unique deployment identifier No Auto-generated
stack_prefix Optional stack name prefix No -
repo_branch Git branch to use No main

Usage

  1. Create AWS Integration in Spacelift if you haven't already
  2. Note your VPC and Subnet IDs where workers should be deployed
  3. Create a new stack from this blueprint in Spacelift
  4. Fill in the required inputs:
    • AWS Integration ID
    • VPC ID
    • Subnet IDs (comma-separated for multiple AZs)
  5. Trigger the admin stack - it will automatically create and deploy the tofu stack

What Gets Created

In Spacelift

  • Worker Pool with auto-generated credentials
  • API Key (if autoscaling enabled)
  • Secure Context for credential storage
  • Admin Stack (orchestration)
  • Tofu Stack (infrastructure)

In AWS

  • Security Group for worker instances
  • EC2 Auto Scaling Group
  • Launch Template
  • IAM Role for workers
  • Secrets Manager secret for credentials
  • Lambda function (if autoscaling enabled)

Features

Autoscaling (Optional)

When enable_autoscaling is set to true:

  • A Lambda function monitors the Spacelift run queue
  • Workers scale up when runs are waiting
  • Workers scale down when idle
  • Requires API key (created automatically)

Spot Instances (Optional)

When use_spot_instances is set to true:

  • Uses EC2 Spot instances for significant cost savings
  • Workers may be interrupted with 2-minute warning
  • Recommended for non-critical workloads

Project Structure

.
├── blueprints/
│   └── ec2-worker-pool.yaml    # Blueprint definition
├── stacks/
│   ├── admin/                  # Admin stack (orchestration)
│   │   ├── main.tf            # Child stack creation, contexts
│   │   ├── variables.tf       # Input variables
│   │   └── worker_pool.tf     # Worker pool & credentials
│   └── tofu/                  # OpenTofu stack (infrastructure)
│       ├── main.tf            # EC2 module deployment
│       └── variables.tf       # Infrastructure variables
└── README.md

Cleanup

To destroy all resources:

  1. Delete the Tofu stack in Spacelift (or trigger destroy)
  2. Delete the Admin stack in Spacelift (or trigger destroy)
  3. The worker pool and API key will be cleaned up with the admin stack

Troubleshooting

Workers not connecting

  • Check the Security Group allows outbound HTTPS (443)
  • Verify the subnets have internet access (NAT Gateway or public subnets)
  • Check CloudWatch logs for the worker instances

Autoscaling not working

  • Ensure enable_autoscaling is set to true
  • Verify the API key was created successfully
  • Check Lambda function logs in CloudWatch

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages