“Continuous Delivery (CD) is a software engineering approach in which teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time.” (from here)

Software artefacts are developed using a build pipeline. This pipeline consists of several steps to provide quick feedback on software quality by means of code quality checks, automated tests, test coverage checks, etc. When the software is done (adhering to a Definition of Done (DoD)), it is released. This release as a whole is then tested and promoted until it reaches a production environment. In the meantime, work on a next release has already started. This process is shown in the below image. This is a minimal example (especially on the test part). In the below image, you can see there are 3 releases in progress. This is only in order to illustrate the process. You should of course try to limit the number of releases which are not in production to reduce the overhead of fixes on those releases.

Automation of the release process is often a challenge. Why is this difficult? One of the reasons is the identification of what should be in a release. This is especially so when the process of creating a release is not automated. There is a transition phase (in the image between Test phase 2 and Test phase 3) when the unit (artefact) build pipeline stops and when the release as a whole continues to be promoted through the environments. In the image of the process above, you can easily identify where you can automate the construction of a release; at the end of the unit build pipeline. This is where you can identify the unit which has been approved by means of the different test phases / quality checks in the unit build pipeline and you know the release the unit has to be put in to be propagated  as a whole. Why not add the unit to the release in an automated fashion there?

Automation

Artifact repository, Maven POM and dependencies

A common practice is to put artifacts in an artifact repository. Artifact repositories are often Maven compliant (i.e. Nexus, Artifactory) so you can identify your artifact with Maven GAV coordinates (groupId, artifactId, version). What better way to describe a collection of artifacts which can be identified with GAV attributes than a Maven POM file? You can define your artifacts as dependencies in your POM file. The Maven assembly plugin can then download those artifacts and put them in a specific structure to allow easy deployment. In case a dependency is already there, you want to update the version in the POM. If it is not there, you want to add it.

Automating adding dependencies to a POM

Below is a short Python (2.7) script to add dependencies to a POM file if the dependency is not there yet or to update the version of the dependency if it already is there.

import os  
import xml.etree.ElementTree as ET  
import xml.dom.minidom as minidom  
import sys,re  
import argparse  
  
#script updates a pom.xml file with a specific artifactid/groupid/version/type/classifier dependency  
#if the dependency is already there, the version is checked and updated if needed  
#if the dependency is not there, it is added  
#the comparison of dependencies is based on artifactid/groupid/type (and optionally classifier). other fields are ignored  
#the pom file should be in UTF-8  
  
#set the default namespace of the pom.xml file  
pom_ns = dict(pom='http://maven.apache.org/POM/4.0.0')  
ET.register_namespace('',pom_ns.get('pom'))  
  
#parse the arguments  
parser = argparse.ArgumentParser(description='Update pom.xml file with dependency')  
parser.add_argument('pomlocation', help='Location on the filesystem of the pom.xml file to update')  
parser.add_argument('artifactid', help='ArtifactId of the artifact to update')  
parser.add_argument('groupid', help='GroupId of the artifact to update')  
parser.add_argument('version', help='Version of the artifact to update')  
parser.add_argument('type', help='Type of the artifact to update')  
parser.add_argument('--classifier', help='Classifier of the artifact to update',default=None)  
args = parser.parse_args()  
  
pomlocation=args.pomlocation  
artifactid=args.artifactid  
groupid=args.groupid  
version=args.version  
type=args.type  
classifier=args.classifier  
  
#read a file and return a ElementTree  
def get_tree_from_xmlfile(filename):  
  if os.path.isfile(filename):  
    tree = ET.parse(filename)  
    return tree  
  else:  
    raise Exception('Error opening '+filename)  
  
#obtain a specific element from an ElementTree based on an xpath  
def get_xpath_element_from_tree(tree,xpath,namespaces):  
  return tree.find(xpath, namespaces)  
  
#returns the content of an element as a string  
def element_to_str(element):  
  return ET.tostring(element, encoding='utf8', method='xml')  
  
#returns an ElementTree as a pretty printed string  
def elementtree_to_str(et):  
  root=et.getroot()  
  ugly_xml = ET.tostring(root, encoding='utf8', method='xml')  
  dom=minidom.parseString(ugly_xml)  
  prettyXML=dom.toprettyxml('\t','\n','utf8')  
  trails=re.compile(r'\s+\n')  
  prettyXML=re.sub(trails,"\n",prettyXML)  
  return prettyXML  
  
#creates an Element object with artifactId, groupId, version, type, classifier elements (used to append a new dependency). classifier is left out if None  
def create_dependency(param_groupid,param_artifactid,param_version,param_type,param_classifier):  
  dependency_element = ET.Element("dependency")  
  groupid_element = ET.Element("groupId")  
  groupid_element.text = param_groupid  
  dependency_element.append(groupid_element)  
  artifactid_element = ET.Element("artifactId")  
  artifactid_element.text = param_artifactid  
  dependency_element.append(artifactid_element)  
  version_element = ET.Element("version")  
  version_element.text = param_version  
  dependency_element.append(version_element)  
  type_element = ET.Element("type")  
  type_element.text = param_type  
  dependency_element.append(type_element)  
  if param_classifier is not None:  
    classifier_element = ET.Element("classifier")  
    classifier_element.text = param_classifier  
    dependency_element.append(classifier_element)  
  return dependency_element  
  
#adds a dependency element to a pom ElementTree. the dependency element can be created with create_dependency   
def add_dependency(pom_et,dependency_element):  
  pom_et.find('pom:dependencies',pom_ns).append(dependency_element)  
  return pom_et  
  
#update the version of a dependency in the pom ElementTree if it is already present. else adds the dependency  
#returns the updated ElementTree and a boolean indicating if the pom ElementTree has been updated  
def merge_dependency(pom_et,param_artifactid,param_groupid,param_type,param_version,param_classifier):  
  artifactfound=False  
  pom_et_changed=False  
  for dependency_element in pom_et.findall('pom:dependencies/pom:dependency',pom_ns):  
    checkgroupid = get_xpath_element_from_tree(dependency_element,'pom:groupId',pom_ns).text  
    checkartifactid = get_xpath_element_from_tree(dependency_element,'pom:artifactId',pom_ns).text  
    checktype = get_xpath_element_from_tree(dependency_element,'pom:type',pom_ns).text  
    if param_classifier is not None:  
      checkclassifier_el = get_xpath_element_from_tree(dependency_element,'pom:classifier',pom_ns)  
      if checkclassifier_el is not None:  
        checkclassifier=checkclassifier_el.text  
      else:  
        checkclassifier=None  
    else:  
      checkclassifier = None  
    if (checkgroupid == param_groupid and checkartifactid == param_artifactid and checktype == param_type and (checkclassifier == param_classifier or param_classifier is None)):  
      artifactfound=True  
      print 'Artifact found in '+pomlocation  
       pomversion=dependency_element.find('pom:version',pom_ns).text  
      if pomversion != param_version:  
        print "Artifact has different version in "+pomlocation+". Updating"  
        dependency_element.find('pom:version',pom_ns).text=param_version  
        pom_et_changed=True  
      else:  
        print "Artifact already in "+pomlocation+" with correct version. Update not needed"  
  if not artifactfound:  
    print 'Artifact not found in pom. Adding'  
    dependency_element = create_dependency(param_groupid,param_artifactid,param_version,param_type,param_classifier)  
    pom_et = add_dependency(pom_et,dependency_element)  
    pom_et_changed=True  
  return pom_et,pom_et_changed  
  
#read the file at the pomlocation parameter  
pom_et = get_tree_from_xmlfile(pomlocation)  
  
#merge the dependency into the obtained ElementTree  
pom_et,pom_et_changed=merge_dependency(pom_et,artifactid,groupid,type,version,classifier)  
  
#overwrite the pomlocation if it has been changed   
if pom_et_changed:  
  print "Overwriting "+pomlocation+" with changes"  
  target = open(pomlocation, 'w')  
  target.truncate()  
  target.write(elementtree_to_str(pom_et))  
  target.close()  
else:  
  print pomlocation+" does not require changes"  

The script can deal with an optional classifier. When not specified it updates dependencies without looking at the classifier so be careful with this. Also the script does some pretty printing when updating. This makes it easy to compare the POM file after a version control commit to for example compare different releases.

Seeing it work

Example pom.xml file.

 <?xml version="1.0" encoding="utf8"?>  
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0               http://maven.apache.org/maven-v4_0_0.xsd">  
      <modelVersion>4.0.0</modelVersion>  
      <groupId>nl.amis.smeetsm.release</groupId>  
      <artifactId>Release</artifactId>  
      <packaging>pom</packaging>  
      <version>1.0</version>  
      <dependencies>  
           <dependency>  
                <groupId>nl.amis.smeetsm.functionalunit.HelloWorld</groupId>  
                <artifactId>HelloWorld_FU</artifactId>  
                <version>1.0</version>  
                <type>pom</type>  
           </dependency>  
      </dependencies>  
      <build>  
           <plugins>  
                <plugin>  
                     <artifactId>maven-assembly-plugin</artifactId>  
                     <version>2.5.4</version>  
                     <configuration>  
                          <descriptors>  
                               <descriptor>release-assembly.xml</descriptor>  
                          </descriptors>  
                     </configuration>  
                </plugin>  
           </plugins>  
      </build>  
 </project>  

The artifact HelloWorld_FU contains dependencies to other artifacts ending in SCA or SB to indicate if it is a SOA Suite SCA composite artifact or a Service Bus artifact. The release-assembly.xml file below puts the different types in different directories and zips the result. This way a release zip file is created.

 <assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3"  
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
   xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd">  
  <id>release</id>  
  <formats>  
   <format>zip</format>  
  </formats>  
 <dependencySets>  
   <dependencySet>  
    <outputDirectory>/composite</outputDirectory>  
    <includes>  
     <include>nl.amis.smeetsm.*:*_SCA</include>  
    </includes>  
   </dependencySet>  
   <dependencySet>  
    <outputDirectory>/servicebus</outputDirectory>  
    <includes>  
     <include>nl.amis.smeetsm.*:*_SB</include>  
    </includes>  
   </dependencySet>  
  </dependencySets>  
  </assembly>  

Updating a dependency version: releasescript.py pom.xml HelloWorld_FU nl.amis.smeetsm.functionalunit.HelloWorld 2.0 pom

 <?xml version="1.0" encoding="utf8"?>  
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0               http://maven.apache.org/maven-v4_0_0.xsd">  
      <modelVersion>4.0.0</modelVersion>  
      <groupId>nl.amis.smeetsm.release</groupId>  
      <artifactId>Release</artifactId>  
      <packaging>pom</packaging>  
      <version>1.0</version>  
      <dependencies>  
           <dependency>  
                <groupId>nl.amis.smeetsm.functionalunit.HelloWorld</groupId>  
                <artifactId>HelloWorld_FU</artifactId>  
                <version>2.0</version>  
                <type>pom</type>  
           </dependency>  
      </dependencies>  
      <build>  
           <plugins>  
                <plugin>  
                     <artifactId>maven-assembly-plugin</artifactId>  
                     <version>2.5.4</version>  
                     <configuration>  
                          <descriptors>  
                               <descriptor>release-assembly.xml</descriptor>  
                          </descriptors>  
                     </configuration>  
                </plugin>  
           </plugins>  
      </build>  
 </project>  

Adding a dependency version: releasescript.py pom.xml ByeWorld_FU nl.amis.smeetsm.functionalunit.ByeWorld 2.0 pom

 <?xml version="1.0" encoding="utf8"?>  
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0               http://maven.apache.org/maven-v4_0_0.xsd">  
      <modelVersion>4.0.0</modelVersion>  
      <groupId>nl.amis.smeetsm.release</groupId>  
      <artifactId>Release</artifactId>  
      <packaging>pom</packaging>  
      <version>1.0</version>  
      <dependencies>  
           <dependency>  
                <groupId>nl.amis.smeetsm.functionalunit.HelloWorld</groupId>  
                <artifactId>HelloWorld_FU</artifactId>  
                <version>2.0</version>  
                <type>pom</type>  
           </dependency>  
           <dependency>  
                <groupId>nl.amis.smeetsm.functionalunit.ByeWorld</groupId>  
                <artifactId>ByeWorld_FU</artifactId>  
                <version>2.0</version>  
                <type>pom</type>  
           </dependency>  
      </dependencies>  
      <build>  
           <plugins>  
                <plugin>  
                     <artifactId>maven-assembly-plugin</artifactId>  
                     <version>2.5.4</version>  
                     <configuration>  
                          <descriptors>  
                               <descriptor>release-assembly.xml</descriptor>  
                          </descriptors>  
                     </configuration>  
                </plugin>  
           </plugins>  
      </build>  
 </project>  

Finally

When you want to start with the next release, you should create a branch in your version control system of the current release. This way you can separate releases in version control and can also easily create fixes on existing releases.