Interacting with OCI Object Storage using Go SDK (first steps)

Lucas Jellema

I am still fairly new at working with Go. I have the ambition of working with and perhaps even on several open source projects that all use Go as their implementation language. It seems like the number of open source projects that use Go is increasing [rapidly]. It seems that I need to get acquainted more with Go and I need to find out how to interact from Go with Oracle Cloud Infrastructure (OCI). (I intend to add features to open source products that leverage OCI services).

image

In this article a quick introduction into interaction with the OCI Object Storage Service for creating and removing a bucket and an object in that object from a simple Go program that uses the OCI Go SDK. My environment is Ubuntu 20.4 on WSL2 on Windows 10,  although that should not matter much.

The steps I go through in this article:

  • install a Go environment
  • create a Go application to connect to OCI using the OCI SDK
  • interact with OCI Object Storage

Install Go

Using this article:  Install Go on Ubuntu – https://buildvirtual.net/how-to-upgrade-go-on-ubuntu/  – I quickly got the current version of Go removed and the latest version of Go installed.

image

Go get the URL for the latest release: https://go.dev/dl/

image

Download using wget and untar

image

Then move the extracted directory go to the /usr/local directory:

image

I tried to run go at this point, but I was too early. First, a small edit of .profile is in order: add the definition of environment variable GOPATH and add $GOPATH to the PATH environment variable.

image

Once the change is made to .profile and the change has been applied with source ~/,profile the environment is once again ready to run go, the latest version.

    Create a Go application to connect to OCI using the OCI SDK

    On the Linux command line, I create a new directory – oci-client – and with “code .” I launch VS Code on this directory.

    image

    I can edit code in VS Code and run the code in the context of the Ubuntu environment with the Go runtime set up. Let’s create the simplest of Go applications.

    SNAGHTML2c210431

    And run it.

    No interaction with OCI yet, but a running Go application is a nice start.

    Now create go.mod and add this contents:

    module oci-client

    go 1.16

    require github.com/oracle/oci-go-sdk/v54 v54.0.0

    Then run:  “go mod tidy” (see docs for help): “go mod tidy ensures that the go.mod file matches the source code in the module. It adds any missing module requirements necessary to build the current module’s packages and dependencies, and it removes requirements on modules that don’t provide any relevant packages. It also adds any missing entries to go.sum and removes unnecessary entries. “

    Define constants in the application for the tenancy OCID, the user OCID, the region and the fingerprint (the usual entries in the oci config file):

    image

    Import these packages:

    image

    Extend the application to the following listing – with functions initializeConfigurationProvider (to contact OCI) and listDomains (to interact with the identity service APIs

    image

    Finally create a file – for example called ppk (putty private key) – that contains the private key for an OCI account in PEM format.

    SNAGHTML2c3107ae

    WIth this in place, I can run the application.

    The first action is to export an environment variable with the contents of the file ppk as its value:

    export pem=$(cat ./ppk)

    This makes the private key accessible to the Go application, using os.Getenv(“pem”).

    image

    And there is contact between my locally running Go application and the OCI.

    Interact from Go Application with OCI Object Storage

      Time now to introduce the Object Storage APIs in the SDK (see docs here). These will allow us to create  buckets and objects in those buckets from the Go application. And that is a capability I will need when I implement a custom Dapr state component leveraging OCI Object Storage – as is my ambition.

      Let’s run the Go application I have crafted:

      The result of running this code is the creation of a Bucket:

      image

      and a new file with the specified contents in this bucket:

      image

      Inspect the contents by downloading and view the object to my local file system:

      image

      So my Go application works with OCI Object Storage.

      The main function in the application orchestrates the moves:

      image

      The configurationProvider contains the OCI connection details (such as OCIDs for tenancy and user, the fingerprint and private key). This provider is used to create the objectStorageClient – which is used for all subsequent interactions with the OCI ObjectStorage service.

      The steps subsequently taken:

      • determine the namespace for the tenancy
      • ensure that a bucket with the specified name exists – check if it is there already and if it is not, then create it
      • put an object with the specified name and contents in the bucket (overwrite the object that may already be there under that name)
      • defer the statement that will remove the object again
      • retrieve the object that was just created and write it contents to the output
      • sleep for one minute and afterwards execute all deferred statements (which means that the object is removed)

      This code is not robust, it does hardly any error checking and handling. It most certainly cannot be used in anything like a production environment. However, it does illustrate how the ObjectStorage service is interacted with through the Go SDK and under the right circumstances, it certainly does the job of creating the bucket and the object, reading the object and removing it.

      Let’s look at the code in a little more detail: this code for initializing the OCI Configuration Provider is generic across all Go applications that interact with whatever OCI services. It uses common OCI configuration details – commonly found in the config file in directory .oci when using the OCI CLI – to create signed HTTP requests to the OCI REST APIs

      image

      The code that checks if a bucket already exists – by getting it and checking on status code 404 (not found) and creating the bucket if it was not found – is show below. Note that the getBucketRequest struct expects pointers to strings for namespacename and bucketname (see docs), hence the &namespace and &name to produces the addresses for these string values (one of the intricacies of Go I am still getting used to)

      image

      With the Bucket established, creating the object is done as follows:

      image

      This is quite straightforward code – except for the way in which the string that contains the object;s content is turned into a reader object (or at least to me that looks a bit special).

      To retrieve an object from the OCI Object Storage service is more or less the reverse operation from putObject:

      image

      This function returns the contents of the object as a string – assuming that it is at all meaningful to turn this object into a string. Note that the response object returns pointers to ContentLength, ContentType and Content. The asterisk (*response.Content:Length) is used to retrieve the actual value (at the pointer’s location).

      Finally to delete the object – the simplest of the operations:

      image

      The full code for my OCI ObjectStorage sample client is shown below

      Resources

      Package Index: https://docs.oracle.com/en-us/iaas/tools/go/54.0.0/index.html#pkg-index

      Configuring Go SDK: https://github.com/oracle/oci-go-sdk/blob/master/README.md#configuring 

      Article DEVELOPING MICROSERVICES WITH OCI SDKS AVOIDING TO INCLUDE THE PRIVATE KEY FILE IN THE CONTAINER IMAGE | EXAMPLE IN GO by Javier Mugueta –  https://javiermugueta.blog/2021/04/26/developing-microservices-with-oci-sdks-avoiding-to-include-the-private-key-file-in-the-container-image-example-in-go/

      OCI Object Storage REST API – https://docs.oracle.com/en-us/iaas/api/#/en/objectstorage/20160918/Bucket/CreateBucket 

      OCI Go SDK – generated client for GetObject – https://docs.oracle.com/en-us/iaas/tools/go-sdk-examples/54.0.0/objectstorage/GetObject.go.html 

      OCI Go SDK – ObjectStorage Package Index – https://docs.oracle.com/en-us/iaas/tools/go/54.0.0/objectstorage/index.htm

      Leave a Reply

      This site uses Akismet to reduce spam. Learn how your comment data is processed.

      Next Post

      Personal link shortener in AWS

      I like to do presentations about a lot of topics. Most of these presentations are recorded and the link to that recording is then shared to people who couldn’t attend the presentation live. Up to now I used bitly.com to shorten my URLs. Last time that I used bitly I […]
      %d bloggers like this: