Terraform Task Group

One of the features of Azure DevOps that is really cool is Task Groups. A task group allows you to define a group of tasks which can be made into a reusable unit. Lets say your unit includes 5 tasks. You can then reference this task group in a build or release pipeline and it will automatically run all 5 tasks at execution time rather than you having to add all 5 individual tasks to every pipeline.

The task group also allows you to pass parameters from your pipeline to the task group.

In this case I have created a task group which encapsulates everything I want to do to run Terraform in a pipeline and it becomes a reusable widget so any pipeline I want to make run Terraform I just use the task group and pass the following parameters to it:

• base_directory_path = The root directory of the solution. I use this to list the files in directory so i can see the structure for troubleshooting. I only use this for diagnostics and it is not essential for Terraform
• terraform_files_directory = The path where the .tf files are located
• terraform_state_connection = A connection for a storage account used to store the state file
• terraform_state_container = A container name where the state file is stored

The Terraform files that are ran will be able to interact with the pipeline variables at execution time so your Terraform files do not care that they are in a task group.

Using the task in a Pipeline

The below picture shows all you need to do so that you can configure a pipeline to use the Terraform Task. You can also use it in the release and build pipelines.

The Terraform Task Group

If we now look at more detail of the individual elements of the Task Group.

Step 1 - Download Terraform State File

The first task will use the Azure CLI to download a state file from a storage container and it will save it in the directory where the .tf files are on the build agent.

The below picture shows the task configuration.

The below code shows the yaml for this task.

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI - Download Terraform State File'
  inputs:
    azureSubscription: '[TBC]'
    scriptLocation: inlineScript
    inlineScript: 'az storage blob download --container-name $(terraform_state_container) --name terraform.tfstate --file $(terraform_files_directory)\terraform.tfstate --connection-string $(terraform_state_connection)'
    addSpnToEnvironment: true
  continueOnError: true

Step 2 - Run Terraform

The run terraform task will execute a bunch of terraform commands. Remember that at this stage on the build agent we have our .tf and .tfvars files in a directory. We have also copied down our tfstate file to the build agent too. The commants we will run are just the same as running the commands locally on your dev machine like discussed previously on the local development section. Below shows the commands we will execute.

Note we are also using the Azure CLI task which allows us to execute inline script.

REM Initializes Terraform and sets up providers
terraform init

REM Validates Terraform files
terraform validate

REM Refreshes the Terraform state from existing infrastructure
terraform refresh

REM Works out the Terraform plan
terraform plan

REM Applies the Terraform script
terraform apply -auto-approve

REM Check the output variables
terraform output

REM Exports Terraform Output Variables to a file
terraform output -json > outputs.json

The below picture shows what the task looks like.

Below is the yaml for the task

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI - Run Terraform'
  inputs:
    azureSubscription: '[TBC]'
    scriptLocation: inlineScript
    inlineScript: |
     REM Initializes Terraform and sets up providers
     terraform init
     
     REM Validates Terraform files
     terraform validate
     
     REM Refreshes the Terraform state from existing infrastructure
     terraform refresh
     
     REM Works out the Terraform plan
     terraform plan
     
     REM Applies the Terraform script
     terraform apply -auto-approve
     REM  -detailed-exitcode 
     
     REM Check the output variables
     terraform output
     
     REM Exports Terraform Output Variables to a file
     terraform output -json > outputs.json
    workingDirectory: '$(terraform_files_directory)'

At the end of the task execution there is a file called outputs.json on the build agent which contains any output variables which we can use to copy Terraform output variables to the pipeline if we want to.

Step 3 - Show all Build Variables

Next up we use the Display All Variables task. This allows me to check the variables in the pipeline for diagnostics purposes.

steps:
- task: dutchworkz.DisplayAllVariables.DisplayAllVariables.DutchWorkzToolsAllVariables@1
  displayName: 'Show all build variables in build output.'

Step 4 - List Files

Next we also have another diagnostics task. This one is called VSTS-Tools List Files. This allows me to check what files are in a given directory. I am passing the $(base_directory_path) variable to it.

Below is a picture showing the configuration of the task.

Below is the yaml for the task.

steps:
- task: moonspace-labs-llc.vsts-tools-build-extensions.vsts-ListFiles.VSTSToolsListFiles@1
  displayName: 'VSTS-Tools List Files '
  inputs:
    rootdir: '$(base_directory_path)'

Step 5 - Upload Terraform State File

Finally in the task group we will upload the updated terraform state file back to blob storage so that it can be used in the next build. To do this we will use the Azure CLI task and execute "az blob upload"

The below picture shows the configuration of this task.

Below is the yaml for this task

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI - Upload Terraform State'
  inputs:
    azureSubscription: '[TBC]'
    scriptLocation: inlineScript
    inlineScript: 'az storage blob upload --container-name $(terraform_state_container) --name terraform.tfstate --file $(terraform_files_directory)\terraform.tfstate --connection-string $(terraform_state_connection)'
    addSpnToEnvironment: true

More Info

• More info on managing Terraform State in an Azure DevOps Pipeline - Click Here
• More info on how to connect Terraform to Azure when running in a DevOps pipeline - Click Here