Terraform Modules, For-Each, File Set on Oracle Cloud

The creation of cloud resources on Oracle Cloud Infrastructure is ideally done as Infra as Code using Terraform (plans). No hands need to touch the console, no permissions need to be given to individual users and all actions can be previewed, tested and easily repeated. The support for Terraform in OCI was introduced and discussed in my previous article – with a an example of the creation of an Object Storage Bucket and a file in that bucket through Terraform.

In this article I will take a small additional step by introducing a module, the file-set function and the for-each operator to upload to the Object Storage Bucket all files in the specified directory (and possibly the nested subdirectories).

A Terraform Module is a reusable, standalone component – a group of one or more resources and other Terraform objects in a directory. This is a very loose definition – that works for me at least.

Here is a very simple example of a module:

Directory module-object (the name is randomly created) contains a file (object.tf). That name object.tf too is randomly picked – but the extension .tf is not (it is the extension Terraformre cognizes).

The file object.tf defines an object in OCI Object Storage as well as a number of variables required for managing that object resource.

This module can be used (it feels like the module is called, but that is not really a proper way of looking at it) one or multiple times from other Terraform plans, using the a module object:

The module shown here references the directory that contains the .tf file(s) that make up the module and provides values for the variables required by the module.

When we run the plan (terraform apply) the bucket is created (if it not already exists) and the object is created inside it.

It is quite simple to “call” the same module multiple times. A straightforward, almost naive approach, is like this:

When this plan is applied, one bucket and two files are created:

And the result is as expected:

It is more elegant I would say to define a variable that holds a list of all objects to create. This can be done as shown here:

The variable documents is a map that contains entries for every document that should be created as object in the OCI Bucket:

The module element uses the for-each entry to loop over all entries in the map-variable documents. The module is activated for each map-entry, passing the properties from the map entry as inputs for the variables defined in the module.

In the module I have made a small change to cater for situations where the content is provided directly and where a source url is provided to download the content from. The (module) file object.tf now looks as follows:

The variables source_url and content are mutually exclusive: only one will have a value. Or at least: when source_url has a value, the value in variable content will be ignored. When source_url has a value, than data source data.http.downloaded_document will be ‘activated’ and the value of the resource property content is taken as the result from this data source.

Running this plan has the desired effect:

and the console tells the story:

Upload all files in a directory

The next challenge I want to address is: upload all files in a local directory to the bucket on Object Storage. Using the module to create a single object in a bucket and the Terraform function fileset that returns the relative paths for all files in a directory (that satisfy a filter condition, possibly including files in nested directories).

The directory to practice on:

contains just two file.

Fairly small changes are required in the module (object.tf)

The variable source_file is introduced. If it has a value, it takes precedence over source_url and content (that decision is somewhat arbitrary). Note the use of of null in the ternary expressions. This has the effect of canceling the property altogether (for example when source_file has a value, the resource property source is set and the property content is not (because it is nullified by the ternary expression).

The resource property source (described in the documentation) takes an absolute path to a file on the local system to read the contents of the object from.

When we apply the plan again, the bucket now holds two additional file at seemingly a nested level:

Upload (all files in) GitHub Repository to Object Storage Bucket

To make things a little bit more exciting, let’s clone a Git repository and then bring its entire contents into the bucket on OCI Object Storage.

A small change in the variables.tf – add a variable for the file system directory to take the contents from:

And small changes in the file storageBucketandObject.tf:

Now clone a (small) GitHub repo (github.com/mathiasbynens/small)

git clone https://github.com/mathiasbynens/small

then use Terraform plan to upload all files cloned from this directory to the bucket in OCI:

terraform apply -var source_directory=’/home/jellema/tf/oci-terraform-composites/createStorageBucketAndUploadObjects/small’

The files in the GitHub repo are all created as objects in the bucket – as shown in the screenshot.

Resources

Sources in GitHub for this article

What Are Terraform Modules and How Do They Work? – https://www.freecodecamp.org/news/terraform-modules-explained/

Manage Similar Resources with For Each – https://developer.hashicorp.com/terraform/tutorials/configuration-language/for-each

Terraform Documentation fileset Function – https://developer.hashicorp.com/terraform/language/functions/fileset

StackOveflow – Iterate through map for file function Terraform – https://stackoverflow.com/questions/58033043/iterate-through-map-for-file-function

The for_each Meta-Argument – Terraform Docs – https://developer.hashicorp.com/terraform/language/meta-arguments/for_each

Working with Cloud Shell – OCI D0cs

OCI Provider for Terraform – Introduction (OCI Docs)

OCI Provider for Terraform – Object Storage –Object – https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/objectstorage_object

OCI Provider for Terraform – Reference (Terraform Docs)

Using Terraform variables (and providing values at runtime)

the Terraform http provider

File Samples https://filesamples.com/formats/txt