Terraform State - CI/CD

What

How do I manage the state for Terraform between builds and releases.

Why

The state file in Terraform keeps track of the state of the infrastructure. I have experimented with some of the marketplace Terraform tasks for Azure DevOps and found that even though they supposedly use the Azure Storage remote backend they didnt seem to work reliably in Azure Devops pipelines and my infrastructure was often planned to be created when it should be updated.

To get around this problem I decided to manage the state file myself and follow the same process that I was using for local development and use the default Terraform approach which just uses a file called Terraform.tfstate which is in the local directory. I decided to use blob storate to store this file between pipeline executions.

How

Below is a picture which shows the DevOps pipeline tasks. The process we use is:

1. Download state file from blob storage
2. Execute Terraform (which will update the state file)
3. Upload the updated state file back to Azure blob storage

Download State File

In this task we will use the Azure CLI task and the CLI to download the state file from blob storage. We will allow this task to continue on error incase the file isnt there.

The below script shows the cli code to do this:

az storage blob download --container-name terraform-build --name terraform.tfstate --file $(terraform_files_directory)\terraform.tfstate --connection-string $(terraform_state_connection)

The YAML for this task is below:

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI - Download Terraform State File'
  inputs:
    azureSubscription: 'TBC'
    scriptLocation: inlineScript
    inlineScript: 'az storage blob download --container-name terraform-build --name terraform.tfstate --file $(terraform_files_directory)\terraform.tfstate --connection-string $(terraform_state_connection)'
    addSpnToEnvironment: true
  continueOnError: true

Upload State File

After Terraform is complete we will upload the state file back to storage so that it can be used in the next build.

Below is the Azure CLI code to do this:

az storage blob upload --container-name $(terraform_state_container) --name terraform.tfstate --file $(terraform_files_directory)\terraform.tfstate --connection-string $(terraform_state_connection)

The YAML for the task is below:

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI - Upload Terraform State'
  inputs:
    azureSubscription: 'TBC
    scriptLocation: inlineScript
    inlineScript: 'az storage blob upload --container-name $(terraform_state_container) --name terraform.tfstate --file $(terraform_files_directory)\terraform.tfstate --connection-string $(terraform_state_connection)'
    addSpnToEnvironment: true