This article describes how a Terraform plan can read configuration data at plan and application time from local or remote JSON documents to provide dynamic, easily and centrally maintainable settings that guide the provisioning process.
Terraform variables are static: they are assigned using their default value (which cannot use function calls or references to other variables) or by overrides at startup time (from command line parameter or environment variable)
To use dynamic values – derived at run time using function calls and variable references – Terraform provides locals. These can be referred to using local.<name of local> in the same way as variables are used: var.<name of variable>. Locals are available only in the module in which they are defined.
Locals can easily be assigned values using functions in Terraform – functions for example for looking up values from a Map, for turning a JSON formatted string into a Map and for reading the content from a file into a string. With these functions at our disposal, it becomes easy to create a Terraform configuration that is driven in part by the actual contents of a JSON document outside the configuration. We will look first at the local file case, then at a situation with a file accessible through a URL.
The sources used in this articles are available on GitHub.
File variables.tf is the pivotal one:
The file variables.tf defines a variable environment, defined as “dev” and possibly overridden at run time. It also defines a few locals that are assigned files at run time:
The filename is constructed using the environment variable The resulting file is a json file that contains environment specific values. The content of that file is loaded into local json_string and converted to map environmentsettings. Finally, the values we care about – api_a_endpoint, api_b_endpoint and scaleFactor – are defined using the values looked up in the map.
File resources.tf defines four output-resources that print the respective values of the variable and the three locals.
Run this Terraform configuration using:
terraform init
terraform apply
The output is:
The Terraform configuration is processed and uses the values defined in the local JSON file dev-env-variables.json.
Next run
terraform apply -var environment=prod
and find this output
In the second case, the variable environment gets the value prod and this influences the local filename’s value. The configuration settings are read from the other JSON document – prod-env-variables.json.
The file from which the map is created does not have to be a local file, it can also be a document available at some accessible URL. For example: https://raw.githubusercontent.com/lucasjellema/oci-terraform-composites/main/lookup_%20variables_in_json/prod-env-variables.json.
WIth some small changes, we can make the Terraform configuration dynamically decide whether to retrieve the configuration data from the local file or from the remote document,
In variables.tf:
The reference to data.http.downloaded_document.response_body is linked to the new entry in the datasources.tf file:
which uses the Terraform http provider to download a document using an HTTP GET request.
With
terraform apply -var environment=dev -var configuration_origin=remote
we can apply the Terraform configuration, using the remote document with configuration data.
. After making two changes in the local configuration data document:
we can verify that this call
terraform apply -var environment=prod -var configuration_origin=local
reads the local document:
Note: I used ChatGPT while writing this article,. It gave me a number of suggestions for defining variables using functions (not possible in Terraform) as well as other code snippets that proved incorrect.
A small excerpt from our conversation:
I am baffled by how ChatGPT responds, but underwhelmed about the quality of the (f)actual information and suggestions. I have seen many clearly incorrect code samples. I have asked where it found those, but these questions have gone unanswered.
Summary
We do not want to have to update our Terraform definiti0ns – treated as environment independent code, stored in source control – with each change in the environment or new and temporary insight into management settings. By externalizing the dynamic and environment specific values – we achieve some of that ambition. Managing the settings and configuration data becomes the next challenge of course. Use of Key Vault, Secret Store, Configuration Datastore and similar tools seem in order for many situations. A key or configuration data entry could be a JSON document that provides multiple key/value pairs. Or we can simply store a JSON document alongside the Terraform configuration or in a location that is accessible over HTTP from where the Terraform plan is applied.
This article showed a straightforward route to externalizing settings from the Terraform configuration by using both a local and remote JSON document.
