As part of the operations around Oracle Cloud Infrastructure, you may want to run periodic jobs. To make backups, clean up log files and other temporary assets, process a batch of new employees to onboard, move files across servers or regions etc. However, Oracle Cloud Infrastructure does not provide an out of the box service for this – something like Azure Batch for example. To run a job or workflow on OCI, there are a few options – such as create a Compute Instance or a Container Instance, use Data Flow or Process Automation. However, none of these quite fit the bill and/or are too much hassle. There is a pretty good solution though. A serverless service that comes with its own preconfigured image that is fully managed and only runs when the job is executed. A service that can run jobs from within the cloud using exactly those access privileges that are provided to it. That comes configured with OCI CLI, Helm, Git, Bash, FN Cli (for OCI Functions), Kubectl, Terraform, SQLcl and Python and JDK (Java). A service that allows jobs to be parametrized and to be easily triggered. A service that can easily make use of artifacts in OCI Container Registry or OCI Artifact Repository.
That service is OCI DevOps and more specifically: OCI DevOps Deployment Pipeline Shell (stage).
Shell stage allows you to run custom commands as part of a deployment pipeline. A command specification defines the steps that you want to run in the shell stage. This command spec is run on a fully managed container instance hosted in the selected compartment and subnet in your tenancy. (you pay for the container resources during the execution time of the shell stage; the composition of this shell runtime is described in the documentation). Note: a deployment pipeline is not associated with a schedule. It can be triggered through a notification (for example from a Build Pipeline) and it can be invoked manually and through an OCI API call or a OCI CLI action. The scheduling part of running administrative jobs is still to be resolved (note: using a health check that triggers on a predefined schedule we can improvise some sort of scheduling mechanism; a small container instance with a cron scheduler or Quartz configuration is another option to produce the triggers for jobs defined in deployment pipelines.
A job can now be created in these steps:
- Create Notification Topic – the DevOps Project requires one
- Create a DevOps Project
- Create a Deployment Pipeline
- Create a dynamic group – for all deployment pipelines in the compartment or in the project or only one specific pipeline
- Create policies that allow this dynamic group to run a container and use the network in the compartment
- Add a Shell Stage to the Deployment Pipeline (with a command specification)
- Run the Deployment Pipeline – a manual test run and subsequently automated runs
Create Notification Topic
Create a DevOps Project
Enable logging:
Create a Deployment Pipeline
I then tried to add stage of type Shell:
but then into this warning:
From the instruction I realized that I need the deployment pipeline to have permission to run a container.
Before I can continue with the stage, I first need to define the group and the policies: Configuration of the network setup as well as the policies is important to allow the runner to access the resources it needs to work on. For a Shell Stage, we have to select a Virtual Cloud Network (VCN) and regional subnet (public or private subnet). When the command is run, a virtual network interface card (VNIC) is created in the selected subnet. Ensure that the subnet has a network path to the OCI Container Registry. If any command needs to manage other OCI resources, then this must also be handled to ensure that both the network path and policy are set up correctly.
NOTE: it turns out this error is far more wide spread. It occurs for any type of stage I try to add and even for editing existing stages that I have created without any problem.
Perhaps I can create a stage through the OCI CLI and bypass this issue?
Two days later, I am completely stuck. I cannot add or even edit stages in Deployment Pipelines. And I do not find easy examples of creating a shell stage through either Terraform, the OCI API or the OCI CLI. I am ready to give up trying for myself. I will describe what the further steps should have been. And I may try again at a later date. And I may try with a OCI DevOps Build Pipeline instead. It seems less appropriate but it can do a similar job.
Create a dynamic group – for all deployment pipelines in the compartment or in the project or only one specific pipeline
Rule for creating the Dynamic Group
All {resource.compartment.id = ‘ocid1.compartment.oc1..aaaaaaaab5vkkrodyutgwtqjoivjodwew4qgd3vo674fmajk2oq6h2eagefq’, Any {resource.type = ‘devopsdeploypipeline’, resource.type = ‘devopsbuildpipeline’, resource.type = ‘devopsrepository’, resource.type = ‘devopsconnection’, resource.type = ‘devopstrigger’}}
Create policies that allow this dynamic group to run a container and use the network in the compartment
This list of policies to provide permissions to the dynamic group
Allow dynamic-group fotomapp-deployment-pipelines to manage compute-container-instances in compartment id ocid1.compartment.oc1..2eagefq
Allow dynamic-group fotomapp-deployment-pipelines to manage compute-containers in compartment id ocid1.compartment.oc1..2eagefq
Allow dynamic-group fotomapp-deployment-pipelines to use vnics in compartment id ocid1.compartment.oc1..2eagefq
Allow dynamic-group fotomapp-deployment-pipelines to use subnets in compartment id ocid1.compartment.oc1..2eagefq
Allow dynamic-group fotomapp-deployment-pipelines to use dhcp-options in compartment id ocid1.compartment.oc1..2eagefq
Add a Shell Stage to the Deployment Pipeline (with a command specification)
See for example https://github.com/oracle-devrel/oci-devops-examples/tree/main/oci-deployment-examples/oci-deployment-shell-stages with an overview of how this should work.
An example of the most basic command specification that only write a single message that ends up in the logging
Run the Deployment Pipeline – a manual test run and subsequently automated runs
Trigger the Deployment pipeline and see that output is produced in the logging. See for example https://github.com/oracle-devrel/oci-devops-examples/tree/main/oci-deployment-examples/oci-deployment-shell-stages with an overview of how this should work.
NEXT STEP
In order to allow a deployment pipeline to do its job, it needs to have the proper privileges. These are assigned in a somewhat indirect way through what is called a service principal. A service principal in OCI is essentially an identity that lets services access resources securely. In the context of DevOps, the service principal allows the shell action in a deployment pipeline to authenticate with OCI and perform (only) allowed actions with OCI services. A Dynamic Group is defined in OCI as collection of one or more services such as Deployment Pipelines (or Build Pipelines, Functions or Compute Instances). Subsequently, policies can be defined that grant the dynamic group permissions on OCI resources.
In this example, I want the job performed by the Deployment Pipeline (‘s shell stage) to remove all files in a bucket on OCI Object Storage. Therefore, I need a dynamic group that contains the pipeline and I need to create a policy that grants permission to delete files in this bucket to the dynamic group.
The command specification for the shell stage would be as simple as
<given my problems with the creation of even the most basic shell stage (or any other stage) I cannot continue and create the stage; so for now this is the end of this story>
Resources
OCI Documentation on OCI DevOps – https://docs.oracle.com/en-us/iaas/Content/devops/using/home.htm
OCI Documentation on Deployment Pipeline – https://docs.oracle.com/en-us/iaas/Content/devops/using/deployment_pipelines.htm and on Shell Stage: https://docs.oracle.com/en-us/iaas/Content/devops/using/shell_stage.htm
Introduction to OCI DevOps (my article from November 2021) – https://technology.amis.nl/cloud/oci-devops-free-automated-cloud-native-application-deployment-to-oracle-cloud/
OCI DevOps Deployment Pipeline for Functions–automation on Oracle Cloud (my article, November 2021) – https://technology.amis.nl/cloud/oci-devops-deployment-pipeline-for-functions-automation-on-oracle-cloud/
OCI DevOps – Shell Container runtime image: https://docs.oracle.com/en-us/iaas/Content/devops/using/shellruntime_details.htm
Introduction of Shell Stage in OCI DevOps Deployment Pipeline (my article from February 2023): https://technology.amis.nl/continuous-delivery/deployment/news-flash-oci-devops-deployment-runner-for-shell-stage/
Oracle Tutorial- Streamline Frontend CI/CD with OCI DevOps Service and OCI Object Storage – https://docs.oracle.com/en/learn/deploy-static-website-using-devops/index.html#introduction
