Save File to Oracle Cloud Infrastructure Object Storage from Node through REST API

Oracle Cloud Infrastructure Object Storage is a public cloud service for storing data to which you need fast, immediate, and frequent access. As Oracle states: “The Oracle Cloud Infrastructure Object Storage service is an internet-scale, high-performance storage platform that offers reliable and cost-efficient data durability. The Object Storage service can store an unlimited amount of unstructured data of any content type, including analytic data and rich content, like images and videos.”

The Object Storage service – like all OCI services – can be accessed from the command line, through the console and via REST APIs. In this article I will discuss creating a file in OCI Object Storage from a Node(JS) application through these REST APIs.

The code for this article is on GitHub: https://github.com/lucasjellema/oci-objectstorage-nodejs-upload-file 

Create bucket on Object Storage

The first thing I need to do is create a bucket on Object Storage to hold the files I will create from my Node application. From the doc: “A logical container for storing objects. Users or systems create buckets as needed within a region. A bucket is associated with a single compartment that has policies  that determine what actions a user can perform on a bucket and on all the objects in the bucket.”

Log in to the OCI Console and navigate to Object Storage:

On the Object Storage page, select the compartment (root compartment in this case) and click on Create Bucket:

Then specify the Bucket details:

Click Create Bucket and the bucket is … created.

Inspect the bucket details:

We can now add files, set policies and perform other operations.

Node application creating files in Object Storage Bucket

The OCI Object Storage REST API offers a set of Object Storage and Archive Storage APIs for managing buckets, objects, and related resources. It can be invoked from any HTTP Client. The main challenge is in the way the request needs to be signed. Details on the Object Storage REST API can be found here: https://docs.cloud.oracle.com/iaas/api/#/en/objectstorage/20160918/.

I made grateful use of a GitHub Repo by Christopher Beck (GitHub Repo OCI-Rest-APIs-nodejs by Christopher Beck with foundation for invoking many OCI REST APIs from NodeJS – I have used crucial elements from this example) who published a NodeJS equivalent of the REST APIs for integration from NodeJS applications. His repo is a work in progress – but already a very valuable one. If you intend to do more interactions with various OCI services from Node applications, I strongly recommend you to make use of his project. Note: he does not provide a package.json – only a package-lock,json – which I found a little disappointing because I needed his code and add on to it my own application.

In my case – I lifted a few key elements from Christopher’s repo. The main challenge I needed to overcome was with the signing of the requests in case of a passphrase protected private key. Due to a bug in http-signature (see this article) you need to make a few manual fixes in file node_modules\http-signature\lib\signer.js (after you run npm install) in order to make signing work with a private key file that is protected with a passphrase.

The steps to get the code in Node repo to work:

1. Clone this Repository

2. Run npm install

<3. Make Fixes in Node Module HTTP-Signature (node_modules\http-signature\lib\signer.js ): At line 293 I inserted the following: assert.optionalString(options.passphrase, ‘options.passphrase’); At line 363 (formerly line 362) I modifed the line to the following to propagate the options object: key = sshpk.parsePrivateKey(options.key, ‘auto’, options);
4. Copy your Private Key file into the project: At one point, you have probably used openssl genrsa to generate a private key (protected with a passphrase) resulting in a *.pem file. Please copy this file into the project. In the next step, you have to set the relative location of this file in the configuration.js file – property privateKeyFile.
5. Logging – The code is configured to use the Papertrail service for logging. You will need to update the logger.js file if you do not want to use Papertrail – but a different logging service or just a local one.
6. Configure your environment parameters in configuration.js – File configuration.js contains all environment specific settings for your Cloud Tenancy, Compartment, User, Public Fingerprint and Private Key. Update this file with your own settings. 

Now run the code:

node saveObject.js

The relevant line that is executed:

runShippingExtractionJob(“BrandNewFile.json”, { content: “My very very special Content”, moreContent: “Something completely different”, value: 34 })

The logging is produced – to Console, File and Papertrail in my case – and the file object is created in the Object Storage Bucket:

Note: this call can be made multiple times, even if the file already exists. It will simply be overwritten.

Resources

The code for this article is on GitHub: https://github.com/lucasjellema/oci-objectstorage-nodejs-upload-file 

Fix for http-signature to wok with passphrase

GitHub Repo OCI-Rest-APIs-nodejs by Christopher Beck with foundation for invoking many OCI REST APIs from NodeJS – I have used crucial elements from this example

Papertrail Logging Service

Medium Article: Logging in NodeJS using Papertrail

OCI Object Storage Service Documentation

Logging with winston npm module: https://github.com/winstonjs/winston