TL;DR: a software release introduces changes in three areas: functional, non-functional and quality. Each change should contribute to identified objectives – and the contribution should be measurable. Product teams should strive for release statements that identify the changes, the objective they contribute to and the metric used for measuring their effect – to provide insight and the ability to influence to stakeholders and also to keep them rationally on track themselves.
The evolution of software applications happens in releases. Every new generation of a system is the result of the previous generation combined with the latest selected set of changes. These changes add value- why else would this new generation be released?
As an industry, we went from a few releases per application per year to an ever faster release cycle with in many cases multiple releases per quarter, per month, per week with the ultimate objective even multiple releases per day: continuous delivery. Agile, Automation, DevOps are some of the key factors in this acceleration.
It is useful to realize what a release consists of. Whether it is a large, quarterly release or a much smaller sprint release or quick fix, we should understand and be able to communicate to stakeholders what the relevance of a release is. If we cannot – we should not release.
Upon reflection, it would seem that any release introduces changes at three levels: R = ΔF + ΔNF + ΔQ
- ΔF = change in functionality; new features are introduced, existing features are modified and sometimes removed (see: kill your darlings); these changes are visible to users of the software as changed (for the better) behavior of the system. Note: the ΔF may not be available to all users: in canary releases to limited user audiences or through feature toggles, functionality may be released yet unavailable [to some | most | all]
- ΔNF = change in non-functionality aspects; non-functional behavior such as performance, stability and availability, recoverability, compliance, security, observability and operability, [cost of] resource usage is changed
- ΔQ = change in the quality of the software and the process that produces the software; this change is not observable by users or the operational staff. It is a design time change that supports the longer term viability of the software, by addressing technical debt, increasing readability and maintainability, applying technical upgrades, improve the build process, on board a new team member, move to a better office location and increase the test coverage of the software
The release roadmap and history for any software product should specify for each release what the ΔF, ΔNF and ΔQ for that release are or were. What are the changes that the release introduces? How can we explain or even justify to stakeholders – including the core product team – why we do the release?
Closely related to this question of why the release is the objective of the release. It is all good and well that the release brings ΔF + ΔNF + ΔQ – but what is the business relevance of that? Why should I care – and give my approval?
The higher level goals of these changes – that we should also [be able to] explicitly state for a release are expressed in terms of:
- customer happiness (including efficiency & productivity) – the customer can be the internal end user, some other department or external users of the product
- reduction of operational cost
- mitigation of risk and increase of compliancy
- future cost reduction (for example by removing a feature that we subsequently do not have to put effort into)
- insight [in usage and runtime behavior of the product] to base decisions on regarding future release of the product – fact based improvement
Every change included in a release is [at least] in one of three categories F, NF and Q. And every change should also contribute to [at least] one of these goals. A product owner may want to explicitly create and publish the matrix
Each release we create and deliver carries a cost. Part of the cost can be calculated – from the hours of effort spent on all elements in the release. Quite a bit harder is how to incorporate the cost of leveraging work that has done in preparation for a release. The ΔQ from previous releases helps to reduce the cost for the new functionality in the current release. How to account for that?
The business case for a release can be made when we compare the cost of the release with the claimed business value. The claimed (predicted) value of a release should obviously be (much) larger than the cost for a release. If that is not the case, why do it?
In addition to claiming or predicting business value, it should be measured. Ideally, for each change – or more specifically: for each benefit attributed to a change – we can designate metrics to be measured in order to assess the realized value of that change. In fact: if a measure cannot be defined – can we believe the value claim for a change? Should there not always be at least some way – even a subjective – of measuring the effect?
Not only can such [semi] objective measures help prevent spurious claims regarding value of release elements, they can also help us identify incomplete roll out (lacking user instruction for example) or imperfect implementation and thereby finetune in a next release.
Some examples of metrics:
- feature usage – who is using a new feature (when,how, conditions)
- response time and throughput
- (cost of) resource usage
- speed of recovery
- time spent on manual Ops activities
- build time
- Code QA findings
- Test Results
Concrete deliverables in a release
So far we have discussed mainly the meta-data associated with a release – in plans, discussions, descriptions. The release when it materializes consists of actual stuff. Artifacts that are real and contain or produce the effects of the release. It is useful – especially for the less technically inclined – to understand the diversity of this collection of ‘things’. A release can contain (an non-exhaustive list – a lot different types of products)
- Custom (functional) Application Code & Configuration
- 3rd party dependencies (libraries, frameworks)
- Custom Infrastructure as Code
- Documentation
- Tests (scenario’s, cases, data, automated tests)
- Build (pipelines, scripts, configuration – CI/CD, Release, …)
- Operations (reports, scripts, instructions, configuration, tooling – monitoring, cost reporting, incident management, backup & recovery)
- Development (tools, process, on boarding instructions, agile )
