More info on Terraform files - Local Development

In this section I will attempt to explain a bit more about some of the Terraform files. In my solution directory I have a bunch of files using either the .tf or .tfvars extensions. They are Terraform files and when you run Terraform it will inspect these files, workout the dependancy order and then perform the actions you are telling it to do.

One of the great things with Terraform is I can split my infrastructure code across multiple files without having to worry about what happens in what order too much. This allows me to have 1 file for each type of Azure resource plus a couple of extra files. I found this to be a really good way to make the scripts managable. As you can see in the below picture its pretty easy to workout which file manages which resource.

The below table will explain what some of the other files are used for.

Terraform State

One discussion point is the Terraform state file. You need to decide how your team will work and manage the state file accordingly. I have decided that the team will work on this solution and share an Azure Resource Group for "local development". This means I can check in the state file and each developer will share the state file to keep the infrastructure in sync.

An alternative way might be for each developer to have their own resource group or subscription. The terraform files should work fine for this scenario but you might need to change the settings for each developer so there are no resource conflicts around things like function app name if there is a setting which needs to be unique

Next I thought it might be beneficial to talk through some of the files to discuss what they are doing.

Main

The code below for the main file is registering some of the providers ill use in Terraform. If is also registering the azurerm_client_config data source. This data source allows me to access the Azure Subscription id which I may use elsewhere in the scripts. I am not explicitly setting the subscription id in terraform, it is set when I log into the Azure CLI locally or via the Service Principal in Azure DevOps.

I also create my resource group here if it doesnt exist too

provider "local" {
	
}

provider "null" {
	
}

provider "random" {
	version = "=2.2.0"
}

provider "azurerm" {
  # Whilst version is optional, we /strongly recommend/ using it to pin the version of the Provider being used
  #version = "=1.28.0"
  version="=1.34.0"
}

resource "random_id" "randomString" {
  byte_length = 8
}


data "azurerm_client_config" "current" {}

resource "azurerm_resource_group" "myResourceGroup" {
  name     = "${var.resourceGroupName}"
  location = "North Europe"
}

#The rest of the process should be resolved from the other .tf files

Service Bus

In the service bus file I perform the following actions:

• Setup a service bus namespace
• Register 2 queues
• Setup some namespace level authorization rules for my Logic App and Function App
• Setup a data source to allow me to export the connection string for Logic App as an output variable for use later

# Service Bus
# ===============================================

#1 Setup Namespace
#2 Setup queues
#3 Setup topics
#4 Setup subscriptions
#5 Setup subscription rules
#6 Setup namespace authorization
#7 Setup entity authorization

#Namespace
#=========
resource "azurerm_servicebus_namespace" "myServiceBus" {
  name                = "${var.serviceBus_name}"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  sku                 = "Standard"

  tags = {
    source = "terraform"
  }
}

#Queues
#=========
resource "azurerm_servicebus_queue" "onBoardingRequestQueue" {
  name                = "OnBoardingRequests"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  namespace_name      = "${azurerm_servicebus_namespace.myServiceBus.name}"

  enable_partitioning = true
}

resource "azurerm_servicebus_queue" "emailReadQueue" {
  name                = "ReadEmails"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  namespace_name      = "${azurerm_servicebus_namespace.myServiceBus.name}"

  enable_partitioning = true
}

#Namespace Authorization
#=======================
resource "azurerm_servicebus_namespace_authorization_rule" "logicAppServiceBusRule" {
  name                = "LogicApp"
  namespace_name      = "${azurerm_servicebus_namespace.myServiceBus.name}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"

  listen = true
  send   = true
  manage = false
}

resource "azurerm_servicebus_namespace_authorization_rule" "functionAppServiceBusRule" {
  name                = "FunctionApp"
  namespace_name      = "${azurerm_servicebus_namespace.myServiceBus.name}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"

  listen = false
  send   = true
  manage = false
}

resource "azurerm_servicebus_namespace_authorization_rule" "serviceBusExplorerServiceBusRule" {
  name                = "ServiceBusExplorer"
  namespace_name      = "${azurerm_servicebus_namespace.myServiceBus.name}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"

  listen = true
  send   = true
  manage = true
}

data "azurerm_servicebus_namespace_authorization_rule" "logicapp" {
  name                = "LogicApp"
  namespace_name      = "${var.serviceBus_name}"
  resource_group_name = "${var.resourceGroupName}"

  depends_on = ["azurerm_servicebus_namespace_authorization_rule.logicAppServiceBusRule"]
}

One interesting point in ths file is that I dont need to export the connection string for the Function App. I will be setting up the function app settings in Terraform because then tend to be a lot more stable and change quite in frequently so I can just cross reference the Service Bus authorization rule when setting up the Function App.

Function App

In the Terraform configuration for setting up the function app I will do the following tasks:

• Setup an AppInsights instance
• Setup a Function App Plan which is a consumption plan
• Setup a storage account to be used by the Function App (note i use a random number as part of the storage account name)
• Setup the function app
• Add the connection string from the Service Bus rule we created earlier as a connection string
• Add the instrumentation key from the AppInsights instance as an app setting on the function app
• Add the connection details for the storage account to the function app

# Azure Function for Helper API for use in Logic Apps
# ===================================================

#1 Setup the Function Hosting Plans
#Then for each app
#1 Setup App Insights for the Function App(s)
#2 Setup Storage for the Function App(s)
#3 Setup the Function App

#Hosting Plans
#=============

resource "azurerm_app_service_plan" "myFunctionPlan" {
  name                = "FunctionAppPlan"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  kind                = "FunctionApp"

  sku {
    tier = "Dynamic"
    size = "Y1"
  }
}

#Function App 1
#==============

#1 Setup App Insights
resource "azurerm_application_insights" "myFunctionAppInsights" {
  name                = "NCL-StudentOnboarding-Helper-API"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  application_type    = "web"
}

#2 Setup Storage
resource "azurerm_storage_account" "myFunctionStorage" {
  name                     = "fappst${lower(random_id.randomString.hex)}"
  resource_group_name      = "${azurerm_resource_group.myResourceGroup.name}"
  location                 = "${azurerm_resource_group.myResourceGroup.location}"
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

#3 Setup Function App
resource "azurerm_function_app" "myFunction" {
  name                      = "${var.functionAppName}"
  location                  = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name       = "${azurerm_resource_group.myResourceGroup.name}"
  app_service_plan_id       = "${azurerm_app_service_plan.myFunctionPlan.id}"
  storage_connection_string = "${azurerm_storage_account.myFunctionStorage.primary_connection_string}"
  https_only				= true
  version					= "~2"

  app_settings = {
    APPINSIGHTS_INSTRUMENTATIONKEY = "${azurerm_application_insights.myFunctionAppInsights.instrumentation_key}",	
	FUNCTIONS_WORKER_RUNTIME       = "dotnet",
	STORAGE_ACCOUNT_NAME		   = "${azurerm_storage_account.myFunctionStorage.name}",
	SERVICEBUS_CONNECTION		   = "${azurerm_servicebus_namespace_authorization_rule.functionAppServiceBusRule.primary_connection_string}"
  }

#  connection_string {
#    name  = "SERVICEBUS_CONNECTION"
#	type  = "Custom"
#    value = "${azurerm_servicebus_namespace_authorization_rule.functionAppServiceBusRule.primary_connection_string}"
#  }


}

APIM

In the APIM Terraform file I will not be creating the APIM instance because of the issue I mentioned previously about consumption tier. I will be doing the following actions:

• Adding an AppInsights Instance to the APIM instance as a logger
• Registering the Azure Functions App as a backend for the API
• Register a product with APIM
• Create an API and import the API definition file
• Import the API policy file and apply it to the API
• Register the API with the product

There are a lot more things you can do with APIM and Terraform but for my usecase this is what I needed and I have slightly simplified the script below so its easier for the reader to understand covering just 1 API setup even though my solution had 2 API's in it.

#APIM
#====================================================


#AppInsights for APIM
#====================

resource "azurerm_application_insights" "myApimAppInsights" {
  name                = "${var.apim_name}"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  application_type    = "web"
}

#API Backends (Downstream Services we comsume)
#=============================================

resource "azurerm_api_management_backend" "helperFunctions" {
  name                = "HelperFunctions"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  api_management_name = "${var.apim_name}"
  protocol            = "http"
  url                 = "https://${var.functionAppName}.azurewebsites.net"
  resource_id          = "https://management.azure.com/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${azurerm_resource_group.myResourceGroup.name}/providers/Microsoft.Web/sites/${var.functionAppName}"
}


#Products
#========

resource "azurerm_api_management_product" "My-API-Product" {
  product_id            = "My-API"
  api_management_name   = "${var.apim_name}"
  resource_group_name   = "${azurerm_resource_group.myResourceGroup.name}"
  display_name          = "My-API"
  subscription_required = true
  approval_required     = true
  published             = true
  subscriptions_limit   = 10
}


#API
#===

resource "azurerm_api_management_api" "My-API" {
  name                = "My-API"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  api_management_name = "${var.apim_name}"
  revision            = "1"
  display_name        = "My-API"
  path                = "My-API/DoSomething"
  protocols           = ["https"]

  import {
    content_format = "swagger-json"
    
	content_value  = "${file("${var.apim_My_filename}")}"
  }
}

#API Policy
#==========

resource "azurerm_api_management_api_policy" "My-API" {
  api_name            = "${azurerm_api_management_api.My-API.name}"
  api_management_name = "${var.apim_name}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"

  xml_content = "${file("${var.apim_My-API_policy_filename}")}"
}

#API Operation Policy
#====================


#API Product Policy
#==================


#Product APIs
#============

resource "azurerm_api_management_product_api" "My-API-Product" {
  api_name            = "${azurerm_api_management_api.My-API.name}"
  product_id          = "${azurerm_api_management_product.My-API-Product.product_id}"
  api_management_name = "${var.apim_name}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
}

Logic App

In the below script I am simply creating 3 empty Logic Apps.

#Logic Apps
# =========

#1 Setup empty logic app templates

#Empty Template Logic Apps
#=========================

resource "azurerm_logic_app_workflow" "LogicApp1" {
  name                = "LogicApp1"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
}

resource "azurerm_logic_app_workflow" "LogicApp2" {
  name                = "LogicApp2"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
}

resource "azurerm_logic_app_workflow" "LogicApp3" {
  name                = "LogicApp3"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
}

SQL Azure DB

In the Terraform config for the SQL Azure DB I have performed the following tasks:

• Setup a logical server for the SQL Azure DB and provided an admin users details
• Created a database and set its price plan etc
• Setup a firewall rule to allow Azure Services to connect to it

I can also perform many other tasks but for this simple demo this is what we have created.

# Azure SQLDB for use helping with data transform
# ===============================================

resource "azurerm_sql_server" "integrationDBServer" {
  name                         = "${var.sql_server_name}"
  resource_group_name          = "${azurerm_resource_group.myResourceGroup.name}"
  location                     = "${azurerm_resource_group.myResourceGroup.location}"
  version                      = "12.0"
  administrator_login          = "${var.sql_administrator_login}"
  administrator_login_password = "${var.sql_administrator_login_password}"
}

resource "azurerm_sql_database" "integrationDB" {
  name                = "${var.sql_database_name}"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  location            = "${azurerm_resource_group.myResourceGroup.location}"
  server_name         = "${azurerm_sql_server.integrationDBServer.name}"
  edition             = "Standard"
  requested_service_objective_name = "S0"
  
}

# Using the ip range below will allow access to the SQLDB by Azure Services
resource "azurerm_sql_firewall_rule" "allow_all_azure_ips" {
  name                = "AllowAllAzureIps"
  resource_group_name = "${azurerm_resource_group.myResourceGroup.name}"
  server_name         = "${azurerm_sql_server.integrationDBServer.name}"
  start_ip_address    = "0.0.0.0"
  end_ip_address      = "0.0.0.0"
}