Software development should be a fast, repeatable process. The practices of continuous integration and delivery are starting to revolutionize application delivery, by instilling changes through the stages of build, deploy, test and release processes. Facebook deploys new software into production every 24 minutes. Companies across industries now are realizing the many benefits of frequent builds and deployments, and automated testing for improved software quality.
By gradually reworking and streamlining your software delivery pipeline, over time you will:
- Increase predictability of software releases
- Significantly reduce project risk
- Eliminate waste, thereby reduce cost
- Refocus on value creation, rather than managing processes
The term ‘Continuous Integration’ originated with the Extreme Programming development process, as one of its original twelve practices.
The essence of it lies in the simple practice of everyone on the team integrating frequently, usually daily, against a controlled source code repository. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
One of the features of version control systems is that they allow you to create multiple branches. Keep your use of branches to a minimum. In particular have a mainline: a single branch of the project currently under development. Pretty much everyone should work off this mainline most of the time. (Reasonable branches are bug fixes of prior production releases and temporary experiments.)
Integration is primarily about communication. Integration allows developers to tell other developers about the changes they have made. Frequent communication allows people to know quickly as changes develop.
The one prerequisite for a developer committing to the mainline is that they can correctly build their code. This, of course, includes passing the build tests. As with any commit cycle the developer first updates their working copy to match the mainline, resolves any conflicts with the mainline, then builds on their local machine. If the build passes, then they are free to commit to the mainline.
By doing this frequently, developers quickly find out if there’s a conflict between two developers. The key to fixing problems quickly is finding them quickly. With developers committing every few hours a conflict can be detected within a few hours of it occurring, at that point not much has happened and it’s easy to resolve. Conflicts that stay undetected for weeks can be very hard to resolve.
The fact that you build when you update your working copy means that you detect compilation conflicts as well as textual conflicts. Since the build is self-testing, you also detect conflicts in the running of the code. The latter conflicts are particularly awkward bugs to find if they sit for a long time undetected in the code.
Using daily commits, a team gets frequent tested builds. This ought to mean that the mainline stays in a healthy state. In practice, however, things still do go wrong. One reason is discipline, people not doing an update and build before they commit. Another is environmental differences between developers’ machines.
As a result you should ensure that regular builds happen on an integration machine and only if this integration build succeeds should the commit be considered to be done. Since the developer who commits is responsible for this, that developer needs to monitor the mainline build so they can fix it if it breaks. A corollary of this is that you shouldn’t go home until the mainline build has passed with any commits you’ve added late in the day.
Many organizations do regular builds on a timed schedule, such as every night. This is not the same thing as a continuous build and isn’t enough for continuous integration. The whole point of continuous integration is to find problems as soon as you can. Nightly builds mean that bugs lie undetected for a whole day before anyone discovers them. Once they are in the system that long, it takes a long time to find and remove them.
A key part of doing a continuous build is that if the mainline build fails, it needs to be fixed right away. The whole point of working with CI is that you’re always developing on a known stable base. It’s not a bad thing for the mainline build to break, although if it’s happening all the time it suggests people aren’t being careful enough about updating and building locally before a commit.