Deployment is a term used frequently and often casually in software engineering. A term that for a long time was a bit cryptic to me. What exactly is deployment and what steps does it entail? Why is it apparently so hard? What help do deployment tools offer? And what is this continuous deployment – and how does that relate to continuous integration (CI/CD)?
In its essence (software) deployment is the activity that takes software to a specific runtime environment and makes it run in that environment. An important part of deployment therefore is handing the software to whatever runtime will run that software – Linux, Kubernetes, Application Server, Web Server, Java Virtual Machine, Serverless Functions framework – along with instructions for running it. Getting access to that runtime environment is therefore a required step in deployment. Software needs to be tailored to the environment in which it is running. Credentials, end points, thresholds, directory locations are examples of such environment specific run time values that need to be provided at deployment time. Along with the hand over of the software artifact to the runtime environment, the deployment process will provide the configuration required to run the artefact in the environment. One could argue that preparation of the runtime environment itself – for example the installation of required libraries or activation of specific features in the runtime platform – is to some extent part of the deployment process
The final step in the deployment process is verification of the success. Is the deployed application actually running (as expected)? And if not: act upon that conclusion.
The next figure shows an overview of the deployment process:
- Get access to the target environment – in a way that allows preparation, installation and configuration; this requires credentials and privileges
- Acquire the software artifacts to be deployed, typically from some form of artifact repository (where it was deposited by a build or CI process)
- Get hold of the configuration values that need to be “injected” into the runtime – for example from a secrets store
- Move the software artefact to the target runtime environment – possibly after first preparing the runtime and along with runtime configuration settings
- Smoke test – verify the success of the deployment by checking the deployed application can be accessed in the expected way (and responds in the expected manner); if the smoke test fails, at the very least a notification can be produced to which human or automated actors and respond.
The ability to perform deployment through an automated process is an important one for mature DevOps teams. Automation brings speed, low effort repeatability and high degree of control and traceability. No hands touching runtime (especially production) environments – and therefore no spread of production environment credentials – is an important risk reduction strategy. Automation also means that anyone on the team can do deployment – simply by triggering it – and the dependency on “that guy” is reduced (to configuring the automated low-code style deployment workflow rather than manually executing deployments)
Deployment on OCI with the OCI DevOps service
After this brief generic introduction of software deployment, let’s turn our attention to OCI (Oracle Cloud Infrastructure). This introduction certainly applies to OCI – where the target runtime environment for example is:
- OCI Compute VM with any type of runtime platform such as Java VM, Python, .NET, Node or just plain Linux
- OKE – OCI Kubernetes Engine onto which Pods with Container Images can be deployed
- OCI Functions – the serverless function framework to which new versions of functions can be deployed
OCI has other types of applications and runtimes – such as Visual Builder, APEX, Autonomous Database, Integration Cloud, Data Integration Cloud and many more. The OCI DevOps service discussed in this article does not cover those target services.
Software engineering and support for DevOps-operations have had a long history on Oracle Cloud. One of the very first services on Oracle Cloud (Classic) was the Developer Cloud Service. This later evolved into today’s Visual Builder Studio – that despite its name is very much an option for development teams that do not do Visual Builder applications but are looking for a broad and integrated toolset for the software development life cycle. Oracle made some small waves when it acquired Wercker and provided a container pipelines service (that seems to have faded into the background). And in the Summer of 2021, the DevOps cloud service has been introduced – with its goal stated as “The goal of this service is to enable customers to easily develop, build, and deploy workloads at scale. The DevOps deployment pipeline reduces change-driven errors and decreases the time customers spend on deploying releases.”
The OCI DevOps service at the time of writing (mid October 2021) provides an Artifact Repository and Deployment Pipelines that know how to interact with the runtime environments OKE, Oracle Functions and Compute VM. Near the end of this month, the DevOps service will be extended with support for the build phase, with (git style) Code Repositories, build pipelines and build servers and integration with Artifact Repository and triggers to kick off Deployment Pipelines.
The integration that the deployment pipelines have with OCI Services is the main distinguishing features compared to other deployment mechanisms you can use. OCI DevOps interacts with OCI platform services such as Logging and Notification and it reads Secrets from OCI Vault. Deployment Pipelines can invoke OCI Functions to perform special tasks as part of the deployment – including the provisioning or reshaping of cloud resources. Deployment Pipelines know how to interact with OKE and with OCI Functions. Deployment Pipelines can also easily interact with Compute VMs: OCI Compute VMs are typically setup with the OCI Cloud Agent. OCI DevOps leverages this cloud agent to be able to have commands executed inside the VM without having to open up an SSH connection into the VM (the agent reaches out from within the VM to learn is anything is expected of it). Note: OCI DevOps Deployment Pipelines can work across OCI Regions. From a single Deployment Pipeline, deployments can be executed in multiple regions – in parallel or sequentially.
- Get access to the target environment – Deployment Pipelines work through a Service Principal – an OCI Dynamic Group that has access privileges inherited from OCI Security Policies. No credentials need to be included in Pipelines – the pipelines need to be in a group with sufficient privileges for the actions the pipeline performs
- Acquire the software artifacts to be deployed – from the OCI Artifact Registry (for any type of artifact) or from the OCI Container Registry for Container Images
- Get hold of the configuration values that need to be “injected” into the
runtime – parameters sin container pipelines can be directly mapped to secrets in OCI Vault – as well as be defined at pipeline or pipeline execution level. These parameters are available as dynamic values in the steps defined in the deployment pipeline – for example to be written to configuration files (Kubernetes manifest files), be mapped to function parameters and be used in shell commands - Move the software artefact to the target runtime environment – Deployment Pipelines can have specific steps for deployment of a Function, a Container Image to OKE and deployment to a Compute Instance; the latter allows the most freedom (obviously) to copy files and execute shell scripts that may include stopping and starting local services
- Smoke test – verify the success of the deployment by checking the
deployed application can be accessed in the expected way: Deployment Pipelines can invoke OCI Functions that perform any activity that a Function can perform. Functions can be used to verify whether the deployed artifact can be accessed (through an internal or external endpoint); the function can also inspect the status of cloud resources through the OCI APIs. The return status of the function can be used to determine the overall success of the deployment pipeline.
Step 0 involves an optional function call (or multiple calls). The Function can for example be used to provision OCI Cloud Resources (through the Resource Manager using Terraform plans or through the specific resource API) that are required for the deployment to be performed.
Note: all actions performed by a Deployment Pipeline fall under the OCI Security Policy Framework. No credentials need to be included in the pipelines – all required privileges are provided through regular OCI policies. The pipeline is made member of a dynamic group. Policies allow specific actions to be performed on specific OCI resources such as Functions, OKE Clusters, Compute VMs and these policies can be associated with a dynamic group. The policies can be defined in a very fine grained manner (per OCI resource) or more coarse grained (such as all OKE Clusters in a compartment, all Functions with a specific Tag).
Using OCI DevOps is free. Running the Deployment Pipelines is not charged to you. Of course the runtimes to which the applications are deployed need to be paid for and the storage required for the Artifact Registry as well as the Container Registry is charged to your cloud account as well.
Note: a Jenkins Plugin for OCI DevOps Deployment is available to trigger a Deployment Pipeline as part of a Jenkins build. When the Jenkins build comes to the point where software that was built, tested, QA-ed and registered in an artifact repository is ready for actual rollout on OCI, the plugin allows easy triggering of the deployment pipeline set up in OCI DevOps. Existing (Jenkins) Build Pipelines can be integrated in this way with the OCI native benefits offered by OCI DevOps.
Summary
The OCI DevOps service is very convenient for any team that deploys applications to OCI runtimes OKE, Compute Instance or Functions. The native integration of OCI DevOps Deployment pipelines with the respective runtimes as well as the OCI security model – leveraging policies and not requiring explicit credentials to connect – and the platform services of Vault, Functions (for any custom management task around the deployment), Logging and Notification makes automated deployment into OCI quite structured and straightforward.
The deployment pipelines are a little rudimentary at the time of writing. For example, testing for a condition (is a target available)is not really supported – so logic such as “waiting until condition is met”, executing unless (if then else ) is not really possible in the deployment pipelines. It is also a little annoying that a function can only be deployed from the pipeline if it already exists. I am also not too happy with the current state of the documentation: first time users need more guidance than is currently available. Two examples: The section on Traffic Shift for example is really unclear to me – I have no clue what this is about. The description of calling a function in an Integration Stage is also quite low on detail. I assume that we can pass parameters to the function – but if indeed and how is not clear.
I am sure we will see a rapid evolution of the OCI DevOps service – beginning in a few weeks’ time with Code Repositories and Build Pipelines. I have learned that the team has a full backlog and an aggressive delivery objective – so I will stay tuned for more features as well as more extensive supporting resources.
Resources
Blog article on the initial launch of OCI DevOps
Build a Continuous Deployment Pipeline using Oracle Cloud Infrastructure DevOps service – OCI Architecture Center
Building and deploying applications using OCI DevOps service – 7 min YouTube Video introducing the Build Pipelines https://www.youtube.com/watch?v=J_WhCbX8c4M (October 2021)
Automated Software Deployments using OCI DevOps Service – 49 min YouTube Video Introducing OCI DevOps Service https://www.youtube.com/watch?v=x0N_TnerKsY (September 2021)
