In the past years I have been involved in implementation/introduction of a continuous delivery process in a lot of software projects. The activities concerned with this are also labeled as build automation, continuous integration or DevOps depending on the fashion and trends. Despite the jargon and popular buzzwords I am convinced continuous delivery practices are helpful for creating high quality software in an efficient way. The following practices have helped me a lot with implementing these practices. Here are 11 of my best practices for implementing continuous delivery. Please share your experiences as comments on this post.
1. Step-by-step implementation
Implement the continuous delivery process in small gradual changes. Do not use a big bang or waterfall approach. First work on several quick-wins / the low hanging fruit, then automate the steps in the process that will save you a lot of work and improve software quality by reducing the number of errors. Next follow a step by step approach for automating the complete delivery process. Each time focusing on a gradual improvement of the highest priority issues. This approach works best with an agile project approach. You can plan a small portion of continuous delivery work in every sprint. The team members and other stakeholders who are less involved with continuous integration and deployment automation have the chance to get used to this new way of working. This way you will get more buy-in during the implementation of the process.
2. Focus on testing
Test and review everything and do it often. You are automating the entire process of software delivery till production so every error or misinterpretation could quickly have massive consequences. The frequency of the release and delivery cycle will increase drastically. In order to safeguard your quality, a lot of automated testing is required since you won’t be able to perform thorough manual testing anymore at such a high frequence. When manual regression testing is taking 3 days you cannot keep up when you are releasing every 4 hours.
3. Version everything and commit frequently
Store everything in your version control system. The choice of version control software is less important than its usage. Just choose one and use it properly. Using a file share folder is not acceptable anymore! Your version control system needs to contain every asset of development and deployment: design documents, code, database design, database scripts, environment configuration, test scripts, deployment tools and deployment scripts. The only thing you do not store in your source control are sensitive data or environment dependent settings (endpoints, usernames, passwords). Use a release management tool for storing these settings. This means you need to store development settings, scripts and sources in your source control. In the same place as where the rest of the software is stored. Commit frequently and integrate early. At least twice a day, preferably more often. Do not save your changes till the last day of the sprint, since your brilliant changes might break the rest of the code.
4. Peer review everything
Use your peers to review everything you make. From designs, configuration plans, code, build scripts and delivery architecture. A peer review increases quality and contributes to knowledge transfer. Your team member will ask questions about your approach and methods, make sure the structure is compliant with the coding and architecture guidelines and they might find things you have overlooked while working on the item that will certainly lead to errors later on in the deployment pipeline. Along with this review there is also added value in the knowledge transfer of the things you’ve created. By reviewing your code, another team member will automatically get to know your deliverable and get familiar with the changes. This will save time later when they have to work on the same code. A review checklist helps improve the quality of the peer review.
5. Accept feedback.
A very important follow-up on the previous item; accept any feedback. When someone is providing you with feedback about anything it is only intended for improving the product or the process. Consider feedback as a gift. Approach it with an open mind. Our natural tendency is to view (negative) feedback as a personal attack and not to accept positive feedback. Discuss this in your team and create a culture of accepting feedback as a positive step towards improving the product and/or the process. And feedback can also be the opportunity to learn from your experiences and improve yourself and your team.
6. Speed up your frequency
When you are doing continuous delivery, speed up your frequency. See every run as a practice run for deploying to production. One of my favorite quotes by Martin Fowler: if it hurts, do it more often. By speeding up your frequency you know you are sure it is working since you did so many trial runs. Also the size of the changes is smaller when you speed up the frequency. When something fails you can easily pinpoint the origin and trace it back to the change which caused this error. This makes debugging, finding the error and solving it much easier.
7. Ask someone else to try your implementation
Always ask someone else to give your implementation a test run. Even when it has passed the peer review stage. By handing over your solution to another person and let it run without your direct supervision will unveil differences in interpretations, your implicit preconditions and differences in your environmental variables. And letting someone else try your software will also validate its documentation, functioning, structure and efficiency. A great test of your deliverable.
8. Always keep the delivery pipeline functioning
The top priority of any continuous delivery team is to be able to deliver software continuously. This is only possible when the complete pipeline is functioning till production. When something causes an interruption or bottleneck in the functioning of the pipeline, the team has to fix this with the highest priority. Not because we like green lights on our dashboard… that’s only superficial. When the pipeline is not functioning the safeguards and controls on your codebase are not operational. Adding more code to this situation will only increase this problem. So a functioning pipeline is the highest priority.
9. Communicate your ideas
Continuous delivery is not something you do by your own. It is affecting the core of most development processes. You cannot change a process impacting many people without communicating this with the different stakeholders. Not only for understanding the process and technology solutions. This is also important for developing a supportive environment for your changes. So share your ideas, explain what you are planning to do, explain your designs and solutions and ask for (and accept) feedback. You are probably working with a team consisting of very smart people. Make use of this and let them contribute to your solution.
10. Virtualize
The iterative development, intensive automated testing and frequent deployment requires an environment with a “reset” option. You are going to us the test environment many times and often leaving it in a ruined state. It can be very helpful when you have procedure to recreate the environment in its original state. This will allow testing of your release / deployment more often increasing its quality. Virtualization will help implementing this requirement by using snapshots or scripting tools to re-create the environment from scratch. This is more of a challenge when using physical hardware.
11. Keep improving with your whole team
Continuous delivery is not an one time effort. It is an ongoing project, evolving and improving step by step. It is not the effort of one team member assigned with the “automation task”. It is the responsibility of the whole team to make delivery of software as efficient as possible and to constantly search for ways to improve quality.
These tips have been proven to be very useful to me and I am sure they can help with improving your process. Feel free to add your opinions and experiences below. Love to hear them.
