Terraform - Promote Variables to the DevOps Pipeline Context

What

How can I use a Terraform variable in the Azure DevOps Pipeline.

Why

In the build and deployment process I will create a Service Bus namespace. I will use properties to pass the values to create the Service Bus instance. The problem is however that when I create an access rule for the queue I need to be able to get the key generated by Service Bus (or connection string) so I can use this later in the process as an Arm Parameter for my Logic App deployment.

How

Step 1 - Create a Terraform Output Variable

In this step I want to create the right configuration of terraform to create the output variables.

Below in the snippet I have created a data source which will point to the Azure Service Bus authorization rule that I have created for the Logic App. This will allow me to reference this later.

data "azurerm_servicebus_namespace_authorization_rule" "logicapp" {
  name                = "LogicApp"
  namespace_name      = "${var.serviceBus_name}"
  resource_group_name = "${var.resourceGroupName}"
}

Below I have create an output variable in terraform. It will point to the data source I have created above and will output the authorization rules connection string. When I run Terraform locally or on the devops instance I will see the variable appear so I can see the value.

output "ServiceBusLogicAppConnectionString" {
  value = "${data.azurerm_servicebus_namespace_authorization_rule.logicapp.primary_connection_string}"
}

Next I need to output the variables to a file on the build agent. This will allow me to access them later in a different task. The below snippet shows how to do this.

REM Exports Terraform Output Variables to a file
terraform output -json > outputs.json

On my local machine I can just run the above from the command line when running Terraform. On the DevOps process I can use the AzureCli task that I am using to run Terraform.

The YAML from the pipeline is shown below.

#Your build pipeline references an undefined variable named ‘terraform_files_directory’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI - Run Terraform'
  inputs:
    azureSubscription: 'TBC'
    scriptLocation: inlineScript
    inlineScript: |
     REM Initializes Terraform and sets up providers
     terraform init
     
     REM Validates Terraform files
     terraform validate
     
     REM Refreshes the Terraform state from existing infrastructure
     terraform refresh
     
     REM Works out the Terraform plan
     terraform plan
     
     REM Applies the Terraform script
     terraform apply -auto-approve
     
     REM Check the output variables
     terraform output
     
     REM Exports Terraform Output Variables to a file
     terraform output -json > outputs.json
    workingDirectory: '$(terraform_files_directory)'

Step 2 - Promote the Terraform variables as Pipeline variables

In the Terraform output file there is a variable called ServiceBusLogicAppConnectionString. I will read the json from the file output earlier to a json object in Powershell using the following snippet.

$file = Get-Content -Path '$(Build.ArtifactStagingDirectory)\Terraform\outputs.json'
Write-Host 'File Content:'
Write-Host $file

Write-Host "Convert terrafrom output from $jsonPath to devops vars."

$json = $file | ConvertFrom-Json
Write-Host "Json Object:"
Write-Host $json

The above snippet gives me a $json variable with a property for each output variable from the Terraform file.

I will use PowerShell to promote this using the below code snippet which allowes you to promote any value to the pipeline. The snippet will promote a property on a json object in powershell called ServiceBusLogicAppConnectionString as a pipeline variable called ServiceBusLogicAppConnectionString.

Write-Host "##vso[task.setvariable variable=ServiceBusLogicAppConnectionString]$($json.ServiceBusLogicAppConnectionString.value)"

I will put the whole thing together in an AzureCli (v2) task in Azure Devops which will use the YAML below.

#Your build pipeline references an undefined variable named ‘$json.ServiceBusLogicAppConnectionString.value’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972

steps:
- task: AzureCLI@2
  displayName: 'Azure CLI - Promote Terraform Variables'
  inputs:
    azureSubscription: '[TBC]'
    scriptType: ps
    scriptLocation: inlineScript
    inlineScript: |
     $file = Get-Content -Path '$(Build.ArtifactStagingDirectory)\Terraform\outputs.json'
     Write-Host 'File Content:'
     Write-Host $file
     
     Write-Host "Convert terrafrom output from $jsonPath to devops vars."
     
     $json = $file | ConvertFrom-Json
     Write-Host "Json Object:"
     Write-Host $json
     
     Write-Host "##vso[task.setvariable variable=ServiceBusLogicAppConnectionString]$($json.ServiceBusLogicAppConnectionString.value)"

Step 3 - Use the Variable in the ARM Deployment

At this point I can reference the variable in the ARM deployment task just like any other DevOps variable.