Terraform is one of the most popular tools for deploying and managing infrastructure as code (IaC). As it continues to push the boundaries of IaC, web-based repository hosting platforms like GitHub are being used widely for storing Terraform code and other artifacts.
In this article, we will explore some of the efficient ways to manage GitHub repositories with Terraform.
We will cover:
GitHub provides a web-based centralized platform to store, manage, and share code. With its user-friendly interface and collaborative tools, it has become a popular choice for version control and project management.
Challenges in managing GitHub repositories and projects include:
- Access control, particularly when the number of contributors and repositories grows.
- Security concerns such as exposure of secrets, unauthorized access, data leaks, and breaches.
- The time-consuming and error-prone nature of managing multiple repositories and projects.
- Ensuring consistency across repositories and projects for compliance and collaboration.
- The tedious and error-prone nature of managing GitHub manually.
- The complexity of scaling and automating GitHub management without the right tools and processes.
Terraform has the perfect solution — codify, version, automate, reuse, audit, and release. By integrating GitHub with Terraform, we can manage our GitHub organizations, repositories, branches, teams, projects, and actions more efficiently.
For example, a team leader can use Terraform to grant or revoke access to specific repositories, ensuring compliance and reducing human error. With Terraform, we can use a single configuration file to manage access control for all of our repositories, making it much easier to ensure consistency and security across your organization.
For help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, you can rely on Spacelift instead of using a static pair on your local machine. It supports policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. One of the things we’re most proud of at Spacelift is the deep integration with everyone’s favorite version control system — GitHub. You can check it for free, by creating a trial account.
Before we can start managing GitHub with Terraform, we need to set up the Terraform GitHub provider. To do this, we’ll need to provide our GitHub personal access token and set up the provider configuration.
The following code snippet shows how to set up the GitHub provider in Terraform.
provider "github" {
token = "your_personal_access_token"
owner = "your_org_name"
}
resource "github_repository" "example" {
name = "repo0088"
description = "This is my Github repository"
visibility = "private"
auto_init = true
}The token attribute is the GitHub account’s personal access token. The owner attribute is the given name of the organization. In my case, the name of my organization is “letsdote-ch”. We can also source these values using environment variables. For this post, the token must have admin: org, delete_repo, and repo permissions.
We will use the same provider details to perform various GitHub management activities with Terraform.
You can also check out how to implement GitLab CI/CD pipeline with Terraform.
To manage GitHub repositories with Terraform, we can use the github_repository resource. This resource allows us to create, modify, or delete GitHub repositories programmatically.
Creating a new repository
The following code snippet shows how to create a new GitHub repository with Terraform.
resource "github_repository" "example" {
name = "repo0088"
description = "This is my Github repository"
visibility = "private"
auto_init = true
}Next, run terraform plan and terraform apply to confirm if this created a new repository on GitHub or not. Below is the terminal output for the apply command.
terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# github_repository.example will be created
+ resource "github_repository" "example" {
+ allow_auto_merge = false
+ allow_merge_commit = true
+ allow_rebase_merge = true
+ allow_squash_merge = true
+ archived = false
+ auto_init = true
+ default_branch = (known after apply)
+ delete_branch_on_merge = false
+ description = "This is my Github repository"
+ etag = (known after apply)
+ full_name = (known after apply)
+ git_clone_url = (known after apply)
+ html_url = (known after apply)
+ http_clone_url = (known after apply)
+ id = (known after apply)
+ merge_commit_message = "PR_TITLE"
+ merge_commit_title = "MERGE_MESSAGE"
+ name = "repo0088"
+ node_id = (known after apply)
+ private = (known after apply)
+ repo_id = (known after apply)
+ squash_merge_commit_message = "COMMIT_MESSAGES"
+ squash_merge_commit_title = "COMMIT_OR_PR_TITLE"
+ ssh_clone_url = (known after apply)
+ svn_url = (known after apply)
+ visibility = "private"
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
github_repository.example: Creating...
github_repository.example: Creation complete after 6s [id=repo0088]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Verify the operation by visiting the organization on GitHub and searching for the “repo0088” repository, as shown in the screenshot below:
Importing existing repositories
Instead of creating a new repository, we can import and manage existing GitHub repositories with Terraform using the terraform import command.
To do this, either select an existing repository from the organization’s GitHub account, or create one for testing purposes. We have created a repository named “repo0099-existing” for the sake of this example.
To import this repository under Terraform management, create the corresponding IaC configuration as shown below.
terraform import github_repository.existing_repo repo0099-existing
github_repository.existing_repo: Importing from ID "repo0099-existing"...
github_repository.existing_repo: Import prepared!
Prepared github_repository for import
github_repository.existing_repo: Refreshing state... [id=repo0099-existing]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.As indicated in the console output, we have successfully imported the existing repository from GitHub to Terraform management.
Note: For more details on how to use the import command, refer to Importing Existing Infrastructure into Terraform blog.
To manage GitHub branches with Terraform, use the github_branch and github_branch_protection resources. These resources allow us to create or manage branches and enforce branch protection rules, such as requiring pull request reviews, requiring status checks, and restricting who can push to a branch.
Creating a new branch
To create a branch for a repository, add the following resource block in the Terraform configuration file. It refers to the repository object name and creates the development branch.
resource "github_branch" "development" {
repository = github_repository.example.name
branch = "development"
}Run the terraform apply command and check if the branch is successfully created on the target repository.
This is confirmed by the screenshot below:
Importing existing branches
We can use the import command to import existing GitHub branches that we want to manage with Terraform.
To do this, first, create a branch in one of the repositories using the GitHub web UI. You can also use this if you are working with existing repositories and branches.
For example, we would like to import the staging branch for repository repo0099-existing.
The overall import process is the same — create the corresponding configuration in Terraform Infrastructure as Code (as shown below) and then run the import command.
resource "github_branch" "existing_staging" {
repository = github_repository.existing_repo.name
branch = "staging"
}The following terminal output shows how the import command imports the staging branch of a GitHub repository into Terraform:
terraform import github_branch.existing_staging repo0099-existing:staging
github_branch.existing_staging: Importing from ID "repo0099-existing:staging"...
github_branch.existing_staging: Import prepared!
Prepared github_branch for import
github_branch.existing_staging: Refreshing state... [id=repo0099-existing:staging]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.In GitHub, an issue is a feature that is used to track enhancements, bugs, or any other tasks within a project. Issues allow users to collaborate with the project contributors. Anyone who has access to the GitHub repos can access and open the GitHub issues.
We can also link one issue with other issues or pull requests. This makes it easy to track down all the different bugs, tasks, and enhancements within a project.
Creating a new GitHub issue
To manage GitHub issues with Terraform, use the github_issue resource. This resource allows us to create, modify, or delete GitHub issues with code.
The following code snippet shows how to create a new GitHub issue with Terraform:
resource "github_issue" "first_issue" {
repository = github_repository.example.name
title = "First Issue"
body = "This is the first issue."
}The plan command output on the terminal should highlight that one issue will be added to the selected repository as shown below.
terraform plan
github_repository.existing_repo: Refreshing state... [id=repo0099-existing]
github_repository.example: Refreshing state... [id=repo0088]
github_branch.development: Refreshing state... [id=repo0088:development]
github_branch.existing_staging: Refreshing state... [id=repo0099-existing:staging]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# github_issue.first_issue will be created
+ resource "github_issue" "first_issue" {
+ body = "This is the first issue."
+ etag = (known after apply)
+ id = (known after apply)
+ issue_id = (known after apply)
+ number = (known after apply)
+ repository = "repo0088"
+ title = "First Issue"
}
Plan: 1 to add, 0 to change, 0 to destroy.Before running the apply command, make sure the ‘issues’ feature is enabled on the target repository. This feature is enabled by navigating to “Settings > General > Features.” The “Issues” checkbox is shown in the screenshot below:
We must also update the corresponding resource configuration of the repository in the IaC code.
The resource block for our example repository now has an additional attribute named “has_issues” set as true, as shown below:
resource "github_repository" "example" {
name = "repo0088"
has_issues = true
description = "This is my Github repository"
visibility = "private"
auto_init = true
}If this is enabled, run the apply command and observe the successful terminal output.
You can verify in the GitHub UI that the issue was successfully created.
GitHub Actions enable the creation of several automated workflows, which are triggered through several events in a GitHub repository. They contain one or more jobs that perform stepwise individual tasks. So we can use Actions to perform a wide variety of Tasks, such as building applications, running tests, and automated code reviews. GitHub provides a library containing all pre-built Actions that we can use as-is or customize to suit our personal needs.
Novibet is in the cloud, and everything is provisioned through Terraform, which the team previously managed using GitHub Actions. However, as the organization scaled, managing Novibet’s IaC through a generic CI/CD platform began to stretch the capabilities of both the tool and the DevOps team. The Spacelift platform has enabled the team to deploy faster and with greater control as they move toward a platform engineering mindset and enable autonomy with guardrails.
To manage GitHub Actions with Terraform, use the github_actions_secret resource. This resource allows us to manage the secrets used in GitHub Actions. The following code shows how to create a new Github Actions secret with Terraform:
resource "github_actions_secret" "my_secret" {
repository = github_repository.example.name
secret_name = "MY_SECRET"
plaintext_value = "my-secret-value"
}Run the terraform plan and apply commands to store this secret in the target GitHub repo.
Look at the GitHub UI to see if the secret was successfully created.
Similarly, we can manage other GitHub Action resources using Terraform provider integrations/github.
Note: The integrations/github provider currently does not support any resource for creating the Actions. The workflow for GitHub Actions has to be defined in a separate YAML file (typically named .github.yml). The same file can then be pushed to the repository using null_resource resource and local-exec provisioner.
Read more about managing Terraform with GitHub Actions.
Terraform is a powerful tool that can simplify the management of GitHub resources like projects, repositories, actions, issues, and branches. With Terraform’s IaC approach, we can create, import, and modify various GitHub resources according to the requirements. Terraform’s declarative syntax can manage the complexity of modern computing.
And don’t forget to explore how Spacelift makes it easy to work with Terraform.
Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.
Automate Terraform deployments with Spacelift
Automate your infrastructure provisioning and build more complex workflows based on Terraform using policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and many more.
