The Oracle Developer Cloud Service was probably the very first service on Oracle Cloud, as early as 2015 if I remember correctly. This service has been repositioned and relabeled as Visual Builder Studio. This service supports agile project management, source code control, artifact management, automated builds, software quality checks, CI/CD pipelines and other actions to take code from the programming phase to subsequent stages. Visual Builder Studio is the Oracle counter offer to the Azure DevOps service that seems to have become more or less an industry standard.
It is available for free for anyone who is a paid Oracle Cloud customer for any other service. Customers do have to pay for the compute instances that are engaged for builds and other actions .
I wanted to try out this revamped service. In this article I share my first steps to get going – including a simple pipeline for testing (with Jest) and code checking (with SonarQube) a NodeJS application. The first challenge I had to overcome was to even find where and how to get an instance going of the Visual Builder Studio; that was not so easy – and the documentation was not correct on this part.
Steps:
- Create an instance of Visual Builder Studio
- Configure a Git repository (in this case a public repository on GitHub)
- Create a Build VM Template for Node JS applications
- Create a Job for building a NodeJS application – including retrieval of sources from a Git repository, checking the code quality, unit testing the code and building an artifact
Create an Instance of Visual Builder Studio
There is no entry for Visual Builder Studio in the OCI Console. It took me a while to figure this out. There is of course a way to create an instance – that eventually I managed to find and leverage.
It starts with the Service User Console that can be opened from the Profile menu in the OCI Console:
This console is the frontend of the Classic Cloud – or at least that is what it reminds me of. On this console, find the Visual Builder Studio service:
and click on this service tile.
This page opens. Click on the button to create a new instance.
Provide the requested details for this new instance:
then press Next. An overview is presented. If you like what you see, then press Create.
The instance creation is now in progress.
After a few minutes, the instance has been prepared
and it can be used. Click on the instance name to access the instance. Note: I have not been able to find a way access the VB Studio instance directly from the OCI Console.
Here is the welcome screen in my brand new VB Studio instance:
I am invited to set up my OCI account so that is what I will do next. Note: I have first created an OCI Compartment for resources that I will be using (and billing for and reporting on) specifically for VB Studio:
Then I proceed to set up the OCI connection in VB Studio. Note: the fact that this is needed at all is another indication that VB Studio is still not fully integrated in OCI.
Click on connect.
Provide OCIDs for tenancy and user, indicate the region and provide a primary key for this user. Enter the id of the compartment to use (for me the especially created compartment) and the storage namespace in OCI Object Storage.
Validate the connection details. Then press Save.
Click on the Projects tab. Then click on Create to create the new project.
Provide name and details for the new project.
Click on Next.
Select the Wiki Markup style. Then press Finish.
The Project is then initialized.
This will take 20-40 seconds.
We can now get going for real – in the project environment that is now available to us. We can create issues and boards. We can configure Git repositories and Docker image registries. Refine the Maven and Gradle repositories and the NPM registry. And define Jobs and Pipelines to automate build, delivery and deployment actions.
Configure a Git repository in the VB Studio project
VB Studio provides its own Git repository infrastructure. We can create repositories in the context of a project. Alternatively, we can make use of external Git repositories. These can be mirrored into an internal (read only) Git repo inside VB Studio, or in build jobs we can refer directly to the URL of a public Git repository. When a repo is mirrored, VB Studio will periodically poll the external repository and synchronizing recent changes to the local mirror. Unfortunately, VB Studio cannot currently be triggered through WebHooks to immediately synchronize upon a commit.
In this article, I will work with a public GitHub repo and not make use of mirroring: https://github.com/lucasjellema/function-for-real-ogbemea2020 .
Create a Build VM Template for a NodeJS application (to use for instantiating a build server)
Before you can create a Build step that uses Node.js, you must create a Build VM template that includes the Node.js software and add a Build VM that uses that Build VM template. The template can be created from scratch or software can be added to an existing template.
When you create a Build VM template, VB Studio adds Java, and some required software packages to it. These default software packages are called Required Build VM Components. If you need more software packages in the VM template, you can add the packages from the VB Studio Software Catalog.
Select the Organization tab. Then select the Virtual Machine Templates tab. And click on Create Template.
Enter the name and a description of the new template:
Press Create.
The new VM Template is created – with default software assigned to it.
Press Configure Software to open the page where specific software required for building the application can be added to the template.
Add the Node.js 14 software package.
Then press Done.
See this piece of documentation on managing Build VM Template.
Add a Build VM for Building Node.JS applications
To add a Build VM, you specify:
1. The Build VM template for Node.js applications
When a Build VM starts, VB Studio installs the operating system and the Node.js 14 software I’ve defined in the template on the VM.
2. VM’s OCI region
Oracle Cloud Infrastructure is hosted in regions and availability domains. A region is a localized geographic area, and an availability domain is one or more data centers located within a region. Remember, a Build VM is a VM on OCI Compute. To learn more about regions and availability domains, see Regions and Availability Domains.
3. VM’s shape
A shape is a template that determines the number of CPUs, amount of memory, and other resources allocated to a newly created instance. To learn more about shapes, see VM Shapes.
4. VM’s VCN
Build VMs can run in the VB Studio’s default VCN or in a custom VCN. A VCN is a software-defined network that you set up in Oracle Cloud Infrastructure data centers in a particular region. To find out more about VCNs and subnets, see VCNs and Subnets.
Open the Virtual Machines tab.
In the popup, select the OL7 for NodeJS template. All default values are fine: Quantity, Region, Shape and VCN Selection. This will use the VCN that was created when the OCI Connection was set up for this VB Studio instance.
Press Add. The VM is now added to the pool of build machines that this VB Studio instance can pick from.
The VM will be started when a job is scheduled for execution. This will take some time. You can also start the VM explicitly.
Configure SonarQube Server
I would like to make use of SonarQube for scanning the code and assessing its quality. VB Studio does not provide a SonarQube server. We need to make sure we have our own SonarQube server running somewhere; if we have the URL and other details, we can register this server with VB Studio and use it in our build jobs.
Note: in my earlier article I demonstrated how to run a SonarQube Server in a Docker Container on an OCI Always Free VM.
On the Project Administration page,
click on Builds. Then select the SonarQube Server tab. Enter the details for the SonarQube Server.
Create a Job for building a NodeJS application
Creating a Job (definition) is done from the Builds tab.
Just click on Create Job.
Enter the name of the Job (in my case Build Probe because this job will build my Probe function) and select the Build VM Template to use: the new OL7 for NodeJS template.
Click on Create. Next, click on Configure. On the Job Configuration pane select the Git tab and add a Git repository.
The form appears for configuring a Git repository.
This configuration step took me quite some time. The URL for the remote GitHub repo can be taken from the Code | Clone HTTPS URL – and should include the .git extension.
The default value for the Branch or Tag field in Visual Builder Studio is master. However, in my case (as in most I think) the main branch is called main. You need to ensure the correct value for this field is set.
On the tab Before Build, we can define various actions to execute before the actual build job is started – but after the sources have been fetched from the Git repository. Here I have configured two of these: Configure SonarQube Server and perform Dependency Vulnerability Analysis. The former makes sure that environment variables are set with the SonarQube Server details and the latter inspects dependencies – such as NPM modules in my Node application – for known vulnerabilities. See documentation.
Next, open the Steps tab. Here we can define the actual actions to be performed during the build.
I will create a step of type (UNIX/Linux) Shell Script to work with NPM to test, sonar scan and possibly inspect and manipulate (ESlint for example) in other ways
The shell actions are fairly simple:
cd probe # change into the directory that contains the resources for the Probe application
npm install # install all npm dependencies
npm run test # run the test (through Jest) for the application
npm run sonar # run the sonar scan and publish both the scan and test results to SonarQube server
Finally, to finalize the definition of the job, open the After Build tab.
We can define several types of actions to have Visual Builder Studio take after the build steps are done. I was hoping that VB Studio could publish the SonarQube scan results to the server – but it seems not to be able to do so for a NodeJS application. The publication to SonarQube Server is now done by the npm run sonar step in the Shell Script. I am still trying to find a way to have the Jest test results published in JUnit format. Hopefully I can then make use of the JUnit Publisher action available in the After Build tab.
One other common After Build action is Artifact Archiver. This is used to bundle build products into an archive that is retained in the job run. Archived artifacts can be downloaded manually or used as input for other jobs.
Run the Job
The Job can be automatically triggered by changes in the sources.; I did not configure that for this job. Alternatively, a job can be triggered manually.
Simply click on the Build Now button to start the job.
When the button is clicked, the job is scheduled for execution. An executor is looked for – a build VM that is running and available to work on the job. If you are in luck, the job can be picked up right away. If not, the job will have to wait.
Here is an overview of a number of executions of the job Build Probe.
Not all runs were successful (color of the bars and status icons). Not all took the same time (height of the bars).
We can inspect the details of a job run by clicking on the job identifier link.
I can drill down on the Build Log to take a look at the step by step detailed output from the job.
Links will be enabled to for example artifacts, JUnit test reports, SonarQube and Vulnerabilities if these are applicable.
In this case, unfortunately VB Studio does not know about the SonarQube Scan and Reporting that my Job has performed, so no link is available to the SonarQube Server, even though the results of the Job have been published and can be inspected on the SonarQube Dashboard.
and specifically the test results:
I am not sure why the Artifact is not available. There is no reference in the build log to the post build action to archive the build results into an Artifact, and there is no link to the Artifact. I also do not know why the Vulnerabilities link is not enabled – even if there are no vulnerabilities, the scan was performed and the result should be reported.
Conclusion
Visual Builder Studio is a fairly intuitive environment. It was not too hard for me to get started. With a code base on GitHub, creating a Job to test and QA my application was fairly simple. However, it seems that VB Studio is much more geared towards Java (Maven and Gradle) applications and is not [yet] up to speed with my Node application. Getting SonarQube results, Test Reports or Artifacts to download seems not possible at this moment.
My next step will probably be an attempt to build and deploy Function to OCI, using the Fn CLI. Since this CLI is explicitly available for creating Job Steps, I have some faith that this will go fairly smoothly. I probably will create a Pipeline with at least two jobs – one for build (test, QA, build container image for function) and one for delivery (push to Container Image Registry) and one for deploy (roll out function to live FaaS environment on OCI, run test invocation).
Resources
Oracle Cloud Documentation on creation of an OCI based instance of Visual Builder Studio.
Documentation on building NodeJS applications
YouTube Playlist on Visual Builder Studio: https://www.youtube.com/playlist?list=PLPIzp-E1msrYDyzVHxCcGHZ3KQg6cx3ox
