Terraform is one of the most in-demand open-source tools for deploying and managing infrastructure as code (IaC). As it continues to push the boundaries of IaC, web-based repository hosting platforms like GitHub are being used widely for storing Terraform code and other artifacts.
This blog aims to showcase some of the efficient ways to manage GitHub repositories with Terraform.
We will cover:
GitHub provides a web-based centralized platform to store, manage, and share code. With its user-friendly interface and collaborative tools, GitHub has become a popular choice for version control and project management.
The challenges in managing GitHub repositories and projects include:Â
- Access control, particularly when the number of contributors and repositories grows.
- Security concerns such as exposure of secrets, unauthorized access, data leaks, and breaches are prevalent.
- Managing multiple repositories and projects can be time-consuming and error-prone.
- Ensuring consistency across repositories and projects is important for compliance and collaboration.
- Manual management of GitHub can be tedious and prone to human error.
- Scaling and automating GitHub management can be difficult without the right tools and processes in place.
Terraform, on the other hand, has the perfect solution – codify, version, automate, reuse, audit, and release. By integrating GitHub with Terraform, we can manage our GitHub organizations, repositories, branches, teams, projects, and actions more efficiently.
For instance, a team leader can use Terraform to grant or revoke access to specific repositories, ensuring compliance and reducing human error. With Terraform, we can use a single configuration file to manage access control for all of our repositories, making it much easier to ensure consistency and security across your organization.
Then if you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, check out Spacelift. It supports policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. One of the things we’re most proud of at Spacelift is the deep integration with everyone’s favorite version control system – GitHub. You can check it for free, by creating a trial account.
Before we can start managing GitHub with Terraform, we need to set up the GitHub provider. To do this, we’ll need to provide our GitHub personal access token and set up the provider configuration.
The following code snippet shows how to set up the GitHub provider in Terraform.
provider "github" {
token = "your_personal_access_token"
owner = "your_org_name"
}
resource "github_repository" "example" {
name = "repo0088"
description = "This is my Github repository"
visibility = "private"
auto_init = true
}The token attribute is the GitHub accountâs personal access token. The owner attribute is the given name of the organization. In my case, the name of my organization is âletsdote-châ.
We can also source these values using environment variables. For this post, the token must have admin: org, delete_repo, and repo permissions.
We will use the same provider details to perform various GitHub management activities with Terraform.
Check out also how to implement GitLab CI/CD pipeline with Terraform.
To manage GitHub repositories with Terraform, we can use the github_repository resource. This resource allows us to create, modify, or delete GitHub repositories programmatically.
Creating a New Repository
The following code snippet shows how to create a new GitHub repository with Terraform.
resource "github_repository" "example" {
name = "repo0088"
description = "This is my Github repository"
visibility = "private"
auto_init = true
}Next, run terraform plan and terraform apply to confirm if this created a new repository on GitHub or not. Below is the terminal output for the apply command.
terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# github_repository.example will be created
+ resource "github_repository" "example" {
+ allow_auto_merge = false
+ allow_merge_commit = true
+ allow_rebase_merge = true
+ allow_squash_merge = true
+ archived = false
+ auto_init = true
+ default_branch = (known after apply)
+ delete_branch_on_merge = false
+ description = "This is my Github repository"
+ etag = (known after apply)
+ full_name = (known after apply)
+ git_clone_url = (known after apply)
+ html_url = (known after apply)
+ http_clone_url = (known after apply)
+ id = (known after apply)
+ merge_commit_message = "PR_TITLE"
+ merge_commit_title = "MERGE_MESSAGE"
+ name = "repo0088"
+ node_id = (known after apply)
+ private = (known after apply)
+ repo_id = (known after apply)
+ squash_merge_commit_message = "COMMIT_MESSAGES"
+ squash_merge_commit_title = "COMMIT_OR_PR_TITLE"
+ ssh_clone_url = (known after apply)
+ svn_url = (known after apply)
+ visibility = "private"
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
github_repository.example: Creating...
github_repository.example: Creation complete after 6s [id=repo0088]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Verify the operation by visiting the organization on GitHub and searching for the ârepo0088â repository, as shown in the screenshot below.
Importing Existing Repositories
Instead of creating a new repository, we can import and manage existing GitHub repositories with Terraform. For that, we will use the terraform import command.
To do this, either select an existing repository from the organizationâs GitHub account, or create one for testing purposes. We have created a repository named ârepo0099-existingâ for the sake of this example.
To import this repo under Terraform management, first, create the corresponding IaC configuration as shown below.
terraform import github_repository.existing_repo repo0099-existing
github_repository.existing_repo: Importing from ID "repo0099-existing"...
github_repository.existing_repo: Import prepared!
Prepared github_repository for import
github_repository.existing_repo: Refreshing state... [id=repo0099-existing]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
As indicated in the console output, we have successfully imported the existing repository from GitHub to Terraform management.
Note: For more details about how to carefully use the import command, refer to Importing Existing Infrastructure into Terraform blog.
Use the github_branch and github_branch_protection resources to manage GitHub branches with Terraform. These resources allow us to create or manage branches and enforce branch protection rules, such as requiring pull request reviews, requiring status checks, and restricting who can push to a branch.Â
Creating New Branch
To create a branch for a repository, add the following resource block in the Terraform configuration file. It refers to the repository object name and creates the development branch on the same.
resource "github_branch" "development" {
repository = github_repository.example.name
branch = "development"
}Run the terraform apply command and check if the branch is successfully created on the target repository.
This is confirmed by the screenshot below.
Importing Existing Branches
We can import existing GitHub branches that we want to manage with Terraform using the import command.
To do this, we first create a branch in one of the repositories using the GitHub web UI. You can also use it if you are working with existing repositories and branches.
For example, we would like to import the staging branch for repository repo0099-existing.
The overall import process is the same – create the corresponding configuration in Terraform Infrastructure as Code (as shown below) and then run the import command.Â
resource "github_branch" "existing_staging" {
repository = github_repository.existing_repo.name
branch = "staging"
}The following terminal output shows how the import command imports the staging branch of a GitHub repository into Terraform.
terraform import github_branch.existing_staging repo0099-existing:staging
github_branch.existing_staging: Importing from ID "repo0099-existing:staging"...
github_branch.existing_staging: Import prepared!
Prepared github_branch for import
github_branch.existing_staging: Refreshing state... [id=repo0099-existing:staging]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
In GitHub, an issue is a feature that is used to track enhancements, bugs, or any other tasks within a project. Issues allow users to collaborate with the project contributors. Anyone who has access to the GitHub repos can access and open the Issues.
We can also link one Issue with other Issues or Pull Requests. This makes it easy to track down all the different bugs, tasks, and enhancements within a project.
Creating a New GitHub Issue
To manage GitHub issues with Terraform, we can use the github_issue resource. This resource allows us to create, modify, or delete GitHub issues with code.Â
The following code snippet shows how to create a new GitHub issue with Terraform.
resource "github_issue" "first_issue" {
repository = github_repository.example.name
title = "First Issue"
body = "This is the first issue."
}The plan command output on the terminal should highlight that one issue will be added to the selected repository as shown below.
terraform plan
github_repository.existing_repo: Refreshing state... [id=repo0099-existing]
github_repository.example: Refreshing state... [id=repo0088]
github_branch.development: Refreshing state... [id=repo0088:development]
github_branch.existing_staging: Refreshing state... [id=repo0099-existing:staging]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# github_issue.first_issue will be created
+ resource "github_issue" "first_issue" {
+ body = "This is the first issue."
+ etag = (known after apply)
+ id = (known after apply)
+ issue_id = (known after apply)
+ number = (known after apply)
+ repository = "repo0088"
+ title = "First Issue"
}
Plan: 1 to add, 0 to change, 0 to destroy.Before running the apply command, make sure the âissuesâ feature is enabled on the target repository. This feature is enabled by navigating to âSettings > General > Featuresâ, and the âIssuesâ checkbox is shown in the screenshot below.
Also, we have to update the corresponding resource configuration of the repository in the IaC code.
As shown below, the resource block for our example repository now has an additional attribute named âhas_issuesâ with the value as true.
resource "github_repository" "example" {
name = "repo0088"
has_issues = true
description = "This is my Github repository"
visibility = "private"
auto_init = true
}If this is enabled, run the apply command and observe the terminal output to be successful.
Verify from the GitHub UI that the issue is successfully created.Â
In GitHub, Actions enable the creation of several automated workflows, which are triggered through several events in a GitHub repository.
GitHub Actions contain one or more jobs that perform stepwise individual tasks. So we can use Actions to perform a wide variety of Tasks, such as building applications, running tests, and automated code reviews.
GitHub provides a library containing all pre-built Actions that we can use as-is or customize to suit our personal needs. To manage GitHub actions with Terraform, we can use the github_actions_secret resource. This resource allows us to manage the secrets used in GitHub Actions.Â
The following code showcases how to create a new Github Actions secret with Terraform.
resource "github_actions_secret" "my_secret" {
repository = github_repository.example.name
secret_name = "MY_SECRET"
plaintext_value = "my-secret-value"
}Run the terraform plan and apply commands to store this secret into the target GitHub repo.
Take a look in the GitHub UI to see if the secret was successfully created.
Similarly there are other GitHub Action resources which we can manage using Terraform provider integrations/github.
Note that the integrations/github provider currently does not support any resource for creating the Actions. The workflow for GitHub Actions has to be defined in a separate YAML file (typically named .github.yml). The same file can then be used to be pushed to the repository using null_resource resource and local-exec provisioner.
Read more about managing Terraform with GitHub Actions.
Terraform is a powerful tool that can simplify the management of GitHub resources like projects, repositories, actions, issues, and branches. With Terraform’s Infrastructure as Code approach, we can create, import, and modify various GitHub resources according to the requirements. Terraform’s declarative syntax is capable of managing the complexity of modern computing.
And don’t forget to explore how Spacelift makes it easy to work with Terraform.
Manage Terraform Better with Spacelift
Build more complex workflows based on Terraform using policy as code, programmatic configuration, context sharing, drift detection, resource visualization and many more.
