Continuous integration, continuous delivery, and continuous deployment implementations cover a wide range of automation possibilities for your software. This article will provide an overview of these three principles, the benefits they can bring to your engineering efficiency, and potential challenges. A continuous integration pipeline is the first step to embracing a CI/CD process.
What is Continuous Integration?
Continuous Integration (CI) is a software development practice where builds, tests, and integrations are automatically executed when the engineering team merges code changes into a shared source-controlled repository (such as GitHub, GitLab, and Bitbucket). Utilizing a CI process helps identify and fix bugs early, improves the application’s overall stability, verifies integrations, and ultimately increases confidence for engineering and product stakeholders.
A robust CI pipeline supports scaling engineering teams and improves feature and bug development velocity. CI promotes a culture within development teams for small, iterative changes being merged into the application codebase in parallel across multiple teams, reducing the risk of code conflicts and risky merges in a scalable, efficient, and repeatable process.
A Continuous Integration Environment
CI pipelines can also include an application deployment to a shared CI environment. Development teams can use the CI environment to test and demo upcoming features for business stakeholders. Without this ability for stakeholders outside development to engage with upcoming features in their product, there can be a feeling of isolation between engineering and the wider organization. A shared CI environment can help mitigate these feelings and lead to a more productive relationship with an improved feedback loop.
These benefits alone often justify the investment in a CI deployment; however, in many modern software projects, the application’s infrastructure is managed in the code and often referred to as infrastructure as code (IaC). Therefore, when deploying a CI environment, the pipeline can also apply changes to the infrastructure during the deployment process alongside the application.
Deploying these infrastructure changes as part of the CI pipeline ensures that the automated tests verify the updated infrastructure alongside the features they support, providing maximum confidence for each update.
Leveling Up with “Pull Request Builds”
A pull request build is usually the same as the process that runs when the engineering team accepts code changes into the central repository. However, instead of waiting until the feature is available in CI, the pipeline is executed against the proposed changes. A pull request build has all the benefits of CI (running the build, tests, and deployment). It also allows engineers, testers, and product owners to review and test changes before they reach the central repository. Pull request builds have the advantage of executing earlier in the development lifecycle, resulting in a reduced need for rework and remedial bug fixes that otherwise would be discovered after CI, ultimately driving more confidence in your feature development.
One Step Further with Continuous Delivery
Continuous delivery encompasses all CI processes and extends them, resulting in every single build being prepared and ready to be deployed to the production environment. The builds are not deployed to production automatically and require a manual approval process. Because the build deployments are dependent on team decisions, not all builds will make it to production.
Continuous delivery requires a mechanism to support deploying incomplete features to production without impacting customers; this is where feature toggles (sometimes referred to as feature flags) come to the forefront.
What are feature toggles?
There are multiple types of feature toggles (or feature flags) available to control application features and operational concerns. However, for this topic, we will focus on those used to support the release of incomplete feature code.
These feature toggles allow developers to control whether features are enabled in an application. For example, engineering teams add feature toggles during the development phase, which can be enabled in the development environment for testing and left disabled for production. Upon completion and verification, the feature toggle can be enabled in production without requiring an updated deployment and disabled if the feature behaves unexpectedly or is broken, reducing impacts on the customer.
Continuous Deployment, the Final Piece of the Puzzle
Continuous deployment (CD) is the ultimate goal for development operations, building on continuous delivery. Every change merged into the application code base is automatically deployed to production without approval or manual steps.
CD requires strong engineering discipline to ensure each build is tested and verified through manual checks (at the code review stage) and automated testing during the CI process. These disciplines, coupled with feature toggles, result in application changes being delivered immediately, reducing the feedback loop from customers and enabling the development team to quickly and efficiently address business needs.
Enterprise challenges faced when embracing CI/CD
While the benefits of CI/CD may seem obvious, enterprise development organizations can face significant challenges along the journey to establishing good processes. As a result, stakeholders should never begin implementing CI/CD pipelines without a sound solution that addresses both the technological and business requirements and has buy-in from all stakeholders involved.
Security and Compliance
Incorporating security and compliance in CI/CD pipelines are often a primary blocker. While a range of security tests such as static or dynamic analysis can be integrated into stages within the CI/CD pipeline, this is not always the case. Manual checks and approvals must be used.
To help solve these challenges, external manual checks that cannot be incorporated automatically into the CI/CD pipeline could be added as explicit approval steps within the pipeline itself. Instead of an email chain or other external approval process, the approval steps halt the pipeline process until the appropriate business unit has completed the manual check and approved it. This approval and supporting documentation would be associated implicitly with the tested application build.
In the long term, DevOps can invest in tools or processes to replace the manual steps where possible to achieve a continuous deployment process.
Managing incomplete or untested features
As previously mentioned, CD requires the ability to ship incomplete or untested features to production without impacting the end user. Without this capability, all new features must be completed and tested before a build is considered production ready. This can become a severe bottleneck, especially on products with multiple development teams. Feature toggles are a standard development practice used to disable these in-flight or untested features, ensuring they do not impact the end user.
The product engineering team can develop feature toggle functionality within the application codebase or integrate one of many off-the-shelf feature toggle services.
Incomplete or inconsistent tests
One of the fundamental principles of CD is every build is considered deployable to production. A production-ready build is only possible when an application has enough robust and reliable tests (unit, integration, security, performance) to provide significant application coverage.
When an application does not have the testing capabilities to provide enough confidence as part of the move toward CD, the tests need to be improved and the development process updated so the new development includes all testing requirements.
The cultural shift from velocity to release quality
CD brings additional requirements and discipline to the engineering teams and means every build is ready to be deployed to production. The extra tasks and review time required to uphold this quality bar can initially affect perceived development velocity leading to pushback from product stakeholders.
In the long term having a robust CD pipeline will enable products to release features and fix bugs faster despite this perceived drop in velocity. Experiencing the long-term benefits will help ease productivity concerns across the business.
Managing multiple pipelines and complex builds
Large enterprise products can have many existing complex workflows and processes across multiple teams. The effort and resources required to migrate to CI/CD for an enterprise-scale application like this are significant and require a collaborative approach across all teams involved.
Businesses must ensure the benefits of migrating a legacy product to CI/CD outweigh the costs (time and money). Even when this is the case, starting small and gradually migrating the product (or products) can provide the confidence and appetite to continue.
Conclusion
Embracing a continuous deployment process and culture will increase application stability and quality, as well as the development team’s productivity, leading to more satisfied customers and stakeholders.
There are many solutions available that provide some or all of the CI/CD processes discussed here, such as Vercel, Netlify, CloudBees CI, TravisCI, CircleCI, Jenkins, Bamboo, Teamcity, and AWS.
For help deciding on the best CI tools for your business, SitePen has the experience and expertise to assist in setting up fully featured CD pipelines across any of these platforms.