Get going with automated CI/CD on OCI in Visual Builder Studio

Lucas Jellema

The Oracle Developer Cloud Service was probably the very first service on Oracle Cloud, as early as 2015 if I remember correctly. This service has been repositioned and relabeled as Visual Builder Studio. This service supports agile project management, source code control, artifact management, automated builds, software quality checks, CI/CD pipelines and other actions to take code from the programming phase to subsequent stages. Visual Builder Studio is the Oracle counter offer to the Azure DevOps service that seems to have become more or less an industry standard.

image

It is available for free for anyone who is a paid Oracle Cloud customer for any other service. Customers do have to pay for the compute instances that are engaged for builds and other actions .

image

I wanted to try out this revamped service. In this article I share my first steps to get going – including a simple pipeline for testing (with Jest) and code checking (with SonarQube) a NodeJS application. The first challenge I had to overcome was to even find where and how to get an instance going of the Visual Builder Studio; that was not so easy – and the documentation was not correct on this part.

Steps:

  1. Create an instance of Visual Builder Studio
  2. Configure a Git repository (in this case a public repository on GitHub)
  3. Create a Build VM Template for Node JS applications
  4. Create a Job for building a NodeJS application – including retrieval of sources from a Git repository, checking the code quality, unit testing the code and building an artifact

Create an Instance of Visual Builder Studio

There is no entry for Visual Builder Studio in the OCI Console. It took me a while to figure this out. There is of course a way to create an instance – that eventually I managed to find and leverage.

It starts with the Service User Console that can be opened from the Profile menu in the OCI Console:

image

This console is the frontend of the Classic Cloud – or at least that is what it reminds me of. On this console, find the Visual Builder Studio service:

image

and click on this service tile.

This page opens. Click on the button to create a new instance.

image

Provide the requested details for this new instance:

imagethen press Next. An overview is presented. If you like what you see, then press Create.

image

The instance creation is now in progress.

image

After a few minutes, the instance has been prepared

image

and it can be used. Click on the instance name to access the instance. Note: I have not been able to find a way access the VB Studio instance directly from the OCI Console.

Here is the welcome screen in my brand new VB Studio instance:

image

I am invited to set up my OCI account so that is what I will do next. Note: I have first created an OCI Compartment for resources that I will be using (and billing for and reporting on) specifically for VB Studio:

image

Then I proceed to set up the OCI connection in VB Studio. Note: the fact that this is needed at all is another indication that VB Studio is still not fully integrated in OCI.

image

Click on connect.

Provide OCIDs for tenancy and user, indicate the region and provide a primary key for this user. Enter the id of the compartment to use (for me the especially created compartment) and the storage namespace in OCI Object Storage.

image

Validate the connection details. Then press Save.

image

Click on the Projects tab. Then click on Create to create the new project.

image

Provide name and details for the new project.

image

Click on Next.

Select the Wiki Markup style. Then press Finish.

image

The Project is then initialized.

image

This will take 20-40 seconds.

We can now get going for real – in the project environment that is now available to us. We can create issues and boards. We can configure Git repositories and Docker image registries. Refine the Maven and Gradle repositories and the NPM registry. And define Jobs and Pipelines to automate build, delivery and deployment actions.

Configure a Git repository in the VB Studio project

    VB Studio provides its own Git repository infrastructure. We can create repositories in the context of a project. Alternatively, we can make use of external Git repositories. These can be mirrored into an internal (read only) Git repo inside VB Studio, or in build jobs we can refer directly to the URL of a public Git repository. When a repo is mirrored, VB Studio will periodically poll the external repository and synchronizing recent changes to the local mirror. Unfortunately, VB Studio cannot currently be triggered through WebHooks to immediately synchronize upon a commit.

    In this article, I will work with a public GitHub repo and not make use of mirroring: https://github.com/lucasjellema/function-for-real-ogbemea2020 .

    Create a Build VM Template for a NodeJS application (to use for instantiating a build server)

    Before you can create a Build step that uses Node.js, you must create a Build VM template that includes the Node.js software and add a Build VM that uses that Build VM template. The template can be created from scratch or software can be added to an existing template.

    When you create a Build VM template, VB Studio adds Java, and some required software packages to it. These default software packages are called Required Build VM Components. If you need more software packages in the VM template, you can add the packages from the VB Studio Software Catalog.

    Select the Organization tab. Then select the Virtual Machine Templates tab. And click on Create Template.

    image

    Enter the name and a description of the new template:

    image

    Press Create.

    The new VM Template is created – with default software assigned to it.

    image

    Press Configure Software to open the page where specific software required for building the application can be added to the template.

    image

    Add the Node.js 14 software package.

    image

    Then press Done.

    See this piece of documentation on managing Build VM Template.

    Add a Build VM for Building Node.JS applications

    To add a Build VM, you specify:

    1. The Build VM template for Node.js applications

    When a Build VM starts, VB Studio installs the operating system and the Node.js 14 software I’ve defined in the template on the VM.

    2. VM’s OCI region

    Oracle Cloud Infrastructure is hosted in regions and availability domains. A region is a localized geographic area, and an availability domain is one or more data centers located within a region. Remember, a Build VM is a VM on OCI Compute. To learn more about regions and availability domains, see Regions and Availability Domains.

    3. VM’s shape

    A shape is a template that determines the number of CPUs, amount of memory, and other resources allocated to a newly created instance. To learn more about shapes, see VM Shapes.

    4. VM’s VCN

    Build VMs can run in the VB Studio’s default VCN or in a custom VCN. A VCN is a software-defined network that you set up in Oracle Cloud Infrastructure data centers in a particular region. To find out more about VCNs and subnets, see VCNs and Subnets.

    Open the Virtual Machines tab.image

    In the popup, select the OL7 for NodeJS template. All default values are fine: Quantity, Region, Shape and VCN Selection. This will use the VCN that was created when the OCI Connection was set up for this VB Studio instance.

    image

    Press Add. The VM is now added to the pool of build machines that this VB Studio instance can pick from.

    image

    The VM will be started when a job is scheduled for execution. This will take some time. You can also start the VM explicitly.

    image

    Configure SonarQube Server

    I would like to make use of SonarQube for scanning the code and assessing its quality. VB Studio does not provide a SonarQube server. We need to make sure we have our own SonarQube server running somewhere; if we have the URL and other details, we can register this server with VB Studio and use it in our build jobs.

    Note: in my earlier article I demonstrated how to run a SonarQube Server in a Docker Container on an OCI Always Free VM.

    On the Project Administration page,

    image

    click on Builds. Then select the SonarQube Server tab. Enter the details for the SonarQube Server.

    image

    Create a Job for building a NodeJS application

    Creating a Job (definition) is done from the Builds tab.

    image

    Just click on Create Job.

    Enter the name of the Job (in my case Build Probe because this job will build my Probe function) and select the Build VM Template to use: the new OL7 for NodeJS template.

    image

    Click on Create. Next, click on Configure. On the Job Configuration pane select the Git tab and add a Git repository.

    image

    The form appears for configuring a Git repository.

    image

    This configuration step took me quite some time. The URL for the remote GitHub repo can be taken from the Code | Clone HTTPS URL – and should include the .git extension.

    image

    The default value for the Branch or Tag field in Visual Builder Studio is master.  However, in my case (as in most I think) the main branch is called main. You need to ensure the correct value for this field is set.

    On the tab Before Build, we can define various actions to execute before the actual build job is started – but after the sources have been fetched from the Git repository. Here I have configured two of these: Configure SonarQube Server and perform Dependency Vulnerability Analysis. The former makes sure that environment variables are set with the SonarQube Server details and the latter inspects dependencies – such as NPM modules in my Node application – for known vulnerabilities. See documentation.

    image

    Next, open the Steps tab. Here we can define the actual actions to be performed during the build.

    I will create a step of type (UNIX/Linux) Shell Script to work with NPM to test, sonar scan and possibly inspect and manipulate (ESlint for example) in other ways

    image

    The shell actions are fairly simple:

    cd probe # change into the directory that contains the resources for the Probe application

    npm install # install all npm dependencies
    npm run test  # run the test (through Jest) for the application
    npm run sonar  # run the sonar scan and publish both the scan and test results to SonarQube server

    image

    Finally, to finalize the definition of the job, open the After Build tab.

    We can define several types of actions to have Visual Builder Studio take after the build steps are done. I was hoping that VB Studio could publish the SonarQube scan results to the server – but it seems not to be able to do so for a NodeJS application.  The publication to SonarQube Server is now done by the npm run sonar step in the Shell Script. I am still trying to find a way to have the Jest test results published in JUnit format. Hopefully I can then make use of the JUnit Publisher action available in the After Build tab.

    image

    One other common After Build action is Artifact Archiver. This is used to bundle build products into an archive that is retained in the job run. Archived artifacts can be downloaded manually or used as input for other jobs.

    Run the Job

    The Job can be automatically triggered by changes in the sources.; I did not configure that for this job. Alternatively, a job can be triggered manually.

    image

    Simply click on the Build Now button to start the job.

    When the  button is clicked, the job is scheduled for execution. An executor is looked for – a build VM that is running and available to work on the job. If you are in luck, the job can be picked up right away. If not, the job will have to wait.

    Here is an overview of a number of executions of the job Build Probe.

    image

    Not all runs were successful (color of the bars and status icons). Not all took the same time (height of the bars).

    We can inspect the details of a job run by clicking on the job identifier link.

    image

    I can drill down on the Build Log to take a look at the step by step detailed output from the job.

    SNAGHTML392a683

    Links will be enabled to for example artifacts, JUnit test reports, SonarQube and Vulnerabilities if these are applicable.

    In this case, unfortunately VB Studio does not know about the SonarQube Scan and Reporting that my Job has performed, so no link is available to the SonarQube Server, even though the results of the Job have been published and can be inspected on the SonarQube Dashboard.

    image

    and specifically the test results:

    SNAGHTML3972777

    I am not sure why the Artifact is not available. There is no reference in the build log to the post build action to archive the build results into an Artifact, and there is no link to the Artifact. I also do not know why the Vulnerabilities link is not enabled – even if there are no vulnerabilities, the scan was performed and the result should be reported.

    Conclusion

    Visual Builder Studio is a fairly intuitive environment. It was not too hard for me to get started. With a code base on GitHub, creating a Job to test and QA my application was fairly simple. However, it seems that VB Studio is much more geared towards Java (Maven and Gradle) applications and is not [yet] up to speed with my Node application. Getting SonarQube results, Test Reports or Artifacts to download seems not possible at this moment.

    My next step will probably be an attempt to build and deploy Function to OCI, using the Fn CLI. Since this CLI is explicitly available for creating Job Steps, I have some faith that this will go fairly smoothly. I probably will create a Pipeline with at least two jobs – one for build (test, QA, build container image for function) and one for delivery (push to Container Image Registry) and one for deploy (roll out function to live FaaS environment on OCI, run test invocation).

    Resources

    Oracle Cloud Documentation on creation of an OCI based instance of Visual Builder Studio.

    Documentation on building NodeJS applications

    YouTube Playlist on Visual Builder Studio: https://www.youtube.com/playlist?list=PLPIzp-E1msrYDyzVHxCcGHZ3KQg6cx3ox

    Leave a Reply

    This site uses Akismet to reduce spam. Learn how your comment data is processed.