The creation of cloud resources on Oracle Cloud Infrastructure is ideally done as Infra as Code using Terraform (plans). No hands need to touch the console, no permissions need to be given to individual users and all actions can be previewed, tested and easily repeated. OCI has deep support for Terraform – from creating, updating and removing resources based on Terraform resources (through the OCI Provider for Terraform) to the creation, management and execution of Stacks (predefined collections of Terraform based OCI resources, defined as resources in their own right) and the creation of Terraform resource definitions from existing OCI resources using resource discovery.
Terraform can most easily be used from OCI Cloud Shell – the predefined VM available through a web browser-based terminal accessible from the Oracle Cloud Console. Cloud Shell is free to use (within monthly tenancy limits), and provides access to a Linux shell, with a pre-authenticated Oracle Cloud Infrastructure CLI, and other useful tools including Terraform with the OCI Provider for Terraform. Cloud Shell falls under “instance principal authorization” – this means among other things that we can run Terraform with OCI Provider without configuration of OCI configuration van Private Key pem file or other forms of authentication. We therefore do not have to install anything in order to get going.
In this article I will keep things simple: focus on the creation of an Object Storage Bucket and a file in that bucket through Terraform. Once the Terraform definitions are available, I can apply them as often as I like to produce the same result in the designated compartment.
In a subsequent article I will turn this mini-constellation into a Stack that can be reused by myself and others – without having to know the Terraform resources involved.
Terraform Definitions for Object Storage Bucket and File
There seem to be several ideas and conventions about how to organize Terraform definition files. I am trying to tag along to what seems most common. I am not claiming to have the definitive truth about this. I will create four files:
- provider.tf – configuration of the OCI Provider (and invoking Instance Principal Authorization for it)
- datasources.tf – definitions of “query objects” or data sources that provide data from OCI to be used when creating resources on OCI (this file does not lead to manipulation of any resources, only to querying data)
- variables.tf – definitions of variables used in other files; for example the names of the bucket and object to be created are defined as variable in this file. Note that using command line parameters or environment variables, the values for variables can be specified (overruled) at runtime time
- storageBucketandObject.tf – definitions of the actual resource to be maintained (the Bucket and Object on Object Storage); this file uses data sources and variables. It also reports some of its results using output definitions
Note: Terraform does not care about these individual files. Or their names. We might as well have defined everything in a single file with a very weird name. Terraform is oblivious: it simply loads all definition from all files with a .tf extension in the indicated directory into memory and starts processing them. The file structure is purely for human consumption – for readability and maintainability and perhaps some reuse.
Note: sources described here are available here on GitHub.
The file that describes the actual resources to be managed by Terraform (storageBucketandObject.tf) looks like this:
The first resource is of type oci_objectstorage_bucket – a type the OCI Provider for Terraform recognizes and knows how to deal with. The name of the resource definition is the_bucket. That is not the name of the bucket to be created – it is an internal Terraform label for this block of code. The name of the bucket meanwhile is taken from the variable var.bucket_name – defined in variables.tf. The compartment where the bucket is to be managed is also taken from a variable (var,compartment_id). The namespace is provided through a data source (in file datasources.tf).
The second resource defined here is the object that is created inside the bucket. The reference to the containing bucket is defined through a reference to the resource the_bucket (as in oci_objectstorage_bucket.the_bucket.name). This reference also establishes a dependency for the_object_in_bucket on the_bucket in the logic of Terraform. It will work on the bucket resource first before turning its attention to the object.
The name and content type for the object are taken from variables, the namespace from the data source and the content from another data source: data.http.downloaded_document.response_body.
Time to take a look at that data source definition – in file datasources.tf:
The data source downloaded_document uses the Terraform http provider – a utility provider for interacting with generic HTTP servers. The provider is used to retrieve the content of a document at a given URL. Note: At present this resource can only retrieve data from URLs that respond with text/* or application/json content types, and expects the result to be UTF-8 encoded regardless of the returned content type header.
The result of the GET request is available as data.http.downloaded_document.response_body. And that is the reference used in the definition of the resource oci_objectstorage_object.the_object_in_bucket.
The variables.tf file will not come as a huge surprise: it defines the variables referred to in the files discussed just now:
The variables tenancy_ocid and compartment_id are fairly common in OCI Terraform scripts. They specify the tenancy and the compartment in which resources are managed.
The variable bucket_name is used for the name of the bucket manipulated by this whole Terraform configuration. The variable source_url contains a URL for the document that is retrieved to provide the content for the object created in te bucket on OCI Object Storage. The value is set here to the URL on GitHub of a CSV document that contains book records. The variable content_type indicates exactly that and is used when creating the object in the bucket; it is set to text/csv – and it can be overruled using command line parameters or environment variables when the Terraform plan is applied. The object_name variable specifies the name to use for the object to manage.
Finally file provider.tf defines the OCI Provider that Terraform should enlist when processing these resource definitions.
The provider leverages InstancePrinicipal authorization – when it runs on OCI Cloud Shell.
With these files in place, we can try things out. Clone the repository locally:
Then, initialize Terraform:
terraform init
and the final output:
Next, try out if the plan is correct – without actually manipulating any resources
terraform plan
The output from the plan action is looking good. Bucket to be created, object to be created with content from downloaded CSV file. We can be satisfied.
Now we feel confident that it will work, we can take the next step and really apply the plan:
terraform apply
After entering yes to confirm that we want Terraform to make these changes real, the resource provisioning takes place:
Of course we can check in the console if the bucket and the object have really been created:
Click on the three dots for the object, then click on Download in the popup menu and finally click on the downloaded file to check if it is a CSV document with book records:
This proves that running the Terraform Infrastructure as Code plan has created the bucket and put the object in it with its content downloaded from the file on GitHub.
Note: A Terraform plan describes the desired situation. It does not contain instructions to to perform actions. It is assumed that the providers used for processing the resources know which actions to perform in order to achieve the desired situation. That also means that if the desired situation is already in place, then apply the plan again will not change anything. If we immediately run terraform apply again, no changes will be made:
If you want to create an object in Object Storage based on a different URL and using a different name for the object and perhaps even for the bucket – you can of course change the values in variables.tf. Or you can use the tf-files as they are right now and pass overriding values on the command line – like is demonstrated here:
terraform apply -var bucket_name=’extra_special_bucket’ -var object_name=’sample_text.txt’ -var source_url=’https://filesamples.com/samples/document/txt/sample3.txt’ -var content_type=’text/plain’
Type yes to go ahead and the bucket and object created earlier are removed and a new bucket and object are created (instead).
In the console the result looks like this:
Resources
Sources in GitHub for this article
Working with Cloud Shell – OCI D0cs
OCI Provider for Terraform – Introduction (OCI Docs)
OCI Provider for Terraform – Reference (Terraform Docs)
Using Terraform variables (and providing values at runtime)
File Samples https://filesamples.com/formats/txt
