How to Integrate Pulumi with GitHub Actions

Pulumi is one of the best infrastructure-as-code (IaC) tools available today, letting you leverage different programming languages for managing your infrastructure. As with every IaC tool, you need to leverage continuous integration/continuous deployment (CI/CD) pipelines to streamline the integration and deployment workflows for your infrastructure.

We will cover:

1. What is Pulumi?
2. What is GitHub Actions?
3. Why Pulumi and GitHub Actions?
4. Key benefits of using Pulumi with GitHub Actions
5. Setting up Pulumi with GitHub Actions
6. Building a Pulumi pipeline with Spacelift

Pulumi is an open-source IaC tool that allows developers to define and manage cloud infrastructure using programming languages such as JavaScript, TypeScript, Python, Go, and .NET. It enables developers to apply familiar practices such as loops, functions, and classes to define cloud resources, and it supports a wide range of cloud providers, including AWS, Microsoft Azure, Google Cloud, Kubernetes, and others, facilitating the creation, deployment, and management of cloud infrastructure.

GitHub Actions is a CI/CD platform that enables developers to automate their workflows directly within GitHub. With GitHub Actions, you can set up workflows to build, test, and deploy your code, based on events within your GitHub repositories, such as pull requests, merges, and even manual interventions.

Apart from defining your custom tasks, GitHub Actions leverages a vast ecosystem of reusable tasks called actions that are built and supported by the community and GitHub itself, allowing for highly customizable pipelines that can integrate with various tools and services.

GitHub Actions is a CI/CD service that enables you to automate your build, test, and deployment pipelines within GitHub. In recent years, GitHub Actions has become increasingly popular, and many custom-made actions can be easily leveraged for several use cases. Combining it with Pulumi can significantly enhance your cloud infrastructure management, making it more automated, reproducible, and error-free.

Check out also: How to manage Terraform with GitHub Actions

Using Pulumi with GitHub Actions provides the following benefits:

• Automated Infrastructure Deployment –  Automate your cloud infrastructure deployments directly from your CI/CD pipeline, reducing manual effort and the potential for error.
• Version Controlled Infrastructure – Manage your IaC, allowing you to track changes, review history, and easily revert when necessary.
• Enhanced Collaboration – Increase the collaboration inside the team by easily reviewing infrastructure changes in pull requests.
• Streamlined Workflow – Keep your infrastructure and application code in sync, ensuring that your deployments are consistent and reliable.

Here are some limitations when leveraging the integration:

• Complex workflow configuration – Setting up a workflow for Pulumi in GitHub Actions can be complex, and you will need to have a good understanding of both tools to build it properly.
• Concurrency and state management – Pulumi relies on state files to manage and track the state of cloud resources. When using Pulumi with GitHub Actions in a team environment, concurrent executions might lead to conflicts or race conditions if not carefully managed, especially in scenarios where multiple team members trigger deployments simultaneously.
• Hard to share secrets, files, and configurations between multiple workflows – This can be challenging and requires careful handling to avoid exposing sensitive data.
• Hard to share outputs between multiple workflows – It’s very hard to share outputs from multiple repositories defined for your Pulumi workflows.

After demonstrating how to configure Pulumi with GitHub Actions, we will present an alternative that addresses all these limitations – Spacelift. With Spacelift, you won’t need to define the workflow configuration for running Pulumi; the issue related to race conditions won’t exist due to stack locking; sharing secrets, files, and configurations will be solved with contexts; and sharing outputs between workflows will be solved easily with stack dependencies.

To set up Pulumi with GitHub Actions, we will need to take the following steps:

1. Create a GitHub repository and clone it.
2. Prepare the Pulumi code.
3. Prepare the GitHub actions workflows.
4. Configure the GitHub environment for manual approvals.
5. Configure the GitHub secrets.
6. Push the configuration to GitHub.
7. Run the pipelines.

1. Create a GitHub repository and clone it

Go to your GitHub account and create a repository:

In the owner tab, you can select your account, or your organization –  in my case, I’ve selected an organization, left the repository as public, and I’ve added a README.md file.

After you have selected everything, click on Create Repository.

To clone the repository, you simply need to run:

You can get the repo_link by clicking on the code tab inside your newly created repository:

2. Prepare the Pulumi code

I will create an AWS Pulumi project using Python, which will create an EC2 instance and a security group, and will attach the security group to the EC2 instance.

To generate the boilerplate files for your Pulumi configuration, run the following command inside your repository:

This will generate the following files:

We will make changes only in the __main__.py file for now. When this file is generated, by default it will have a configuration that creates an S3 bucket and exports the bucket name and the bucket id:

As mentioned before, we want to create an EC2 instance, a security group, and an attachment. We will need to modify the code, remove the import to S3, and import the class that takes care of EC2 resources:

Now that we’ve handled the imports, let’s create a security group:

If you are accustomed to OpenTofu or Terraform, you will see the syntax doesn’t feel that much different, although we are using a programming language here.

Next, let’s create an EC2 instance with this security group attached to it:

As you can see, the resources are linked in a pretty straightforward way. You just have to specify the other object you have created and select one of its attributes.

Now, let’s export the instance ID and its public IP:

The configuration is now ready, and we can go on to the next step.

3. Prepare the GitHub Actions workflows

First, we need to create a .github/workflows directory. In this folder, we will add our workflows.

For this example, we will prepare two GitHub actions workflows:

• Workflow 1: plan + manual intervention for apply
• Workflow 2: manually run destroy

Workflow 1: plan + manual intervention for apply

In the first part, we will add a name to the workflow, specify when to run it, and declare a couple of environment variables that we will need for authenticating to AWS:

You don’t necessarily require a session token, but I always try to use short-lived credentials, even if this requires extra work.

Next, we will define the job that takes care of the plan:

Let’s break down the above configuration:

• First, we define what will be our runner and what is the working directory we will use.
• Then we check out the code.
• After that, we install Pulumi and the Pulumi AWS provider.
• We want to leverage S3 for state management, so then we log in to the bucket (this bucket should exist in your AWS account).
• In the next step, we create the Pulumi stack if it doesn’t exist – this step can fail if the stack exists.
• Next, we select the Pulumi stack if it exists.
• Finally, we ensure that we are using AWS KMS secret provider (this KMS key has to exist) and run a Pulumi preview, which is pretty similar to a tofu/terraform plan.

Now let’s take a look at the apply job:

As you can see, most of the steps are the same — installing Pulumi and Pulumi AWS, stack selection, login to the S3 bucket, and changing the secrets provider to AWS.

Here are three very important differences:

• A “needs” option ensures this job runs only if the previous job has finished successfully.
• In addition, we leverage the “environment” option for adding manual approval, only after we are happy with the plan – I’ve added the name production, but you can use any name you want.
• Finally, we’ve changed the Pulumi command to pulumi up, which is similar to a tofu/terraform apply.

This workflow is now complete, so we can take a look at the second one.

Workflow 2: manually run destroy

Whenever you are doing IaC, you need to have a method for creating your infrastructure, as well as a method for destroying your infrastructure.

The above workflow is pretty similar to the ones that do plan and apply, but there are two notable differences:

• This workflow will always run manually.
• The command, in the end, is a pulumi down, which is pretty similar to tofu/terraform destroy.

4. Configure the GitHub environment for manual approvals

On your repository, go to Settings → Environments and select New environment.

Add a name for your environment; it should be the same one you used in the GitHub Actions workflow. In my case, this will be production.

Next, click on Required reviewers, and add the person(s) that you want to be able to deploy your Pulumi code.

5. Configure the GitHub secrets

We now need to define the GitHub Actions secrets. For that, we need to go to Settings → Secrets and Variables and select Actions. Then, to add a secret you need to select the New Repository secret option and provide the values specific to your account.

6. Push the configurations to GitHub

As for any git provider, you need to add/commit/push your changes for them to be available:

7. Run the pipelines

As we did the push, and the first workflow has an “on: push” option on it, the pipeline will be triggered automatically:

This is the plan output:

Now, we have reached the deploy job, but this cannot continue until we approve it:

After you click on Approve and deploy, the code is applied successfully:

Now, let’s take a look at the destroy workflow. Because we’ve added the “on: workflow_dispatch”, this will only run when we manually invoke it. For that, we need to select the workflow and click on Run workflow.

After a couple of seconds, we can see that all resources have been destroyed:

We now have a working GitHub Actions pipeline for Pulumi, and if you don’t want to follow all of these steps to configure everything, you can take the code from here, create a repo based on it, and just add your environment and environment secrets.

Now that we’ve seen the lengthy process for configuring a GitHub Actions workflow for Pulumi, let’s take a look at how simple it is with Spacelift and the advantages of leveraging it.

Let’s reuse the Python code, that we’ve previously created for spawning an ec2 instance and security group. I have the repository in place, so I’ll go to my Spacelift account, and select Create a Stack:

Add a name, select a space, and add labels and a description if desired.

In the next step, select your VCS provider, repository, and the directory from where to run Pulumi commands:

In the following step, select Pulumi as your IaC management tool and provide a Login URL (we will use the same S3 bucket we’ve previously used) and a stack name:

Next, you will need to select a runner image that has Pulumi installed, one example is: public.ecr.aws/spacelift/runner-pulumi-base-alpine:latest

Next, I will also attach a Cloud Integration for Dynamic credentials:

Now we can simply create the stack.

In the Environment tab, define a variable called AWS_REGION in which we will specify the region where we want our code to be deployed:

Going back to the stack view, we can trigger a run. As I didn’t select the AutoDeploy option in the view where I’ve added the runner image, this run will do a pulumi preview, then wait for my input to apply the changes:

I confirmed the run, and the changes have been successfully applied.

Tearing down everything, couldn’t be simpler – you just need to go to tasks, and run a pulumi down –yes:

Everything was destroyed successfully.

Although GitHub Actions is very powerful for many use cases, there are simpler and less error-prone ways to manage infrastructure. 

If you are searching for the right product to manage your OpenTofu, Terraform, Pulumi, Ansible, CloudFormation, Kubernetes, and Terragrunt, Spacelift is the answer.

You can create a free account today, or book a demo with one of our engineers to learn more.