In anticipation of the start of the course "CI / CD on AWS, Azure and Gitlab" we have prepared a translation of useful material for you.
Continuous Integration (CI) and Continuous Delivery (CD) are a culture, a set of principles and practices that enable developers to more frequently and reliably deploy software changes.
CI / CD is one of the DevOps practices . It also applies to agile practices : Deployment automation allows developers to focus on meeting business requirements, code quality, and security.
Definition of CI / CD
Continuous Integration is a development methodology and set of practices in which small changes are made to the code with frequent commits. And since most modern applications are developed using various platforms and tools, there is a need for an integration mechanism and testing of the changes introduced.
Technically speaking, the goal of CI is to provide a consistent and automated way to build, package, and test applications. With a streamlined continuous integration process, developers are more likely to make frequent commits, which in turn will help improve communication and improve software quality.
Continuous delivery starts where continuous integration ends. It automates the deployment of applications to different environments: most developers work with both production environments and development and testing environments.
CI / CD tools help customize specific environment settings that are configured during deployment. And also CI / CD-automation performs the necessary requests to web servers, databases and other services that may need to be restarted or perform some additional actions when deploying the application.
Continuous integration and continuous delivery need continuous testingbecause the ultimate goal is to develop quality applications. Continuous testing is often implemented as a set of various automated tests (regression, performance, and others) that are executed in the CI / CD pipeline.
The mature practice of CI / CD allows for continuous deployment: when the code successfully passes through the CI / CD pipeline, the assemblies are automatically deployed to the production environment. Teams practicing continuous delivery can afford daily or even hourly deployment. It's worth noting here, though, that continuous delivery isn't suitable for all business applications .
Continuous integration improves communication and quality
Continuous Integration is a development methodology based on regulated processes and automation. With continuous integration implemented, developers often commit their code to the source code repository. And most teams follow the rule of committing at least once a day. Small changes are easier to spot defects and miscellaneous problems than large changes that have been worked on over a long period of time. In addition, working with short commit cycles reduces the chances of multiple developers changing the same piece of code, which can lead to merge conflicts.
Teams implementing continuous integration often start by setting up a version control system and defining a workflow. Despite the fact that commits are made frequently, implementing features and fixing bugs can take a long time. There are several approaches to control which features and code are ready.
Many people use feature flags - a mechanism to turn functionality on and off at runtime. The functionality that is still under development is wrapped in feature flags and deployed from the master branch to production, but is disabled until it is fully ready for use. According to a recent study63 percent of teams using the flag feature report improved testability and improved software quality. There are special tools for working with feature flags, such as CloudBees Rollout , Optimizely Rollouts and LaunchDarkly , which integrate into CI / CD and allow you to perform configuration at the feature level.
Another way to work with features is by using branches in the version control system. In this case, you need to define a branching model (for example, such as Gitflow) and describe how the code gets into the development, testing and production branches. For features with a long development cycle, separate feature branches are created. After completing work on a feature, developers merge changes from the feature branch into the main development branch. This approach works well, but it can be inconvenient if many features are being developed at the same time.
The build phase is to automate the packaging of the required software, database and other components. For example, if you are developing a Java application, then the CI will package all static files such as HTML, CSS, and JavaScript along with the Java application and database scripts.
CI will not only package all software components and databases, but it will also automatically perform unit tests and other types of testing. This testing allows developers to get feedback that the changes they made did not break anything.
Most CI / CD tools allow you to start a build manually, on a commit, or on a schedule. Teams need to discuss a build schedule that suits them based on team size, expected daily commits, and other criteria. It is important that commits and builds are fast, otherwise long builds can become an obstacle for developers trying to commit quickly and often.
Continuous testing is more than test automation
Automated testing frameworks help QA engineers design, run, and automate various types of tests that help developers track build success. Testing includes functional tests that are developed at the end of each sprint and combined into regression tests for the entire application. Regression tests inform the team if their changes have broken something elsewhere in the application.
Best practice is to require developers to run all or part of the regression tests in their local environment. This will ensure that developers commit already checked code.
Regression tests are just the beginning. Performance testing, API testing, static code analysis, security testing - these and other types of testing can also be automated. The key point is the ability to run these tests from the command line, through a webhook, or through a web service and return the execution result: was the test successful or not.
Continuous testing involves not only automation, but also the integration of automated testing into the CI / CD pipeline. Unit and functional tests can be part of the CI and identify problems before or during the launch of the CI pipeline. Tests that require full environment deployment, such as performance and security testing, are often part of the CD and run after the builds have been deployed to the target environments.
CD pipeline automates delivery of changes to different environments
Continuous delivery is the automatic deployment of an application to its target environment. Typically, developers work with one or more development and test environments where the application is deployed for testing and review. For this, CI / CD tools such as Jenkins , CircleCI, AWS CodeBuild, Azure DevOps, Atlassian Bamboo, Travis CI are used.
A typical CD pipeline consists of build, test, and deployment steps. More complex pipelines include the following steps:
- Retrieving code from source control and executing a build.
- Configuring infrastructure automated through an infrastructure-as-code approach.
- Copying the code to the target environment.
- Setting environment variables for the target environment.
- (-, API-, ).
- , , .
- .
- .
For example, in the Jenkins conveyor determined file Jenkinsfile, which describes the various steps such as assembly (build), testing (test) and deployment (deploy). It also describes environment variables, secret keys, certificates, and other parameters that can be used in pipeline stages. The post section configures error handling and notifications.
In a more complex CD pipeline, there may be additional steps, such as data synchronization, archiving information resources, installing updates and patches. CI / CD tools usually support plugins. For example, Jenkins has over 1500 plugins for integration with third-party platforms, for extending the user interface, administration, source code management, and building.
When using a CI / CD tool, developers must ensure that all parameters are configured outside of the application via environment variables. CI / CD tools allow you to set the values of these variables, mask passwords and account keys, and customize them during deployment for a specific environment.
Also in the CD-tools there are dashboards and reporting. In the event of a build or delivery failure, they notify about it. By integrating the CD with version control and agile tools, it is easier to find code changes and user stories included in the build.
Implementing CI / CD Pipelines with Kubernetes and Serverless Architectures
Many teams using CI / CD pipelines in the clouds use containers like Docker and orchestration systems like Kubernetes . Containers help standardize packaging, delivery, and simplify scaling and destroying volatile environments.
There are many options for sharing containers, infrastructure as code, and CI / CD pipelines. You can learn more about this in the articles Kubernetes with Jenkins and Kubernetes with Azure DevOps .
Serverless computing is another way to deploy and scale applications. In a serverless environment, the infrastructure is fully managed by the cloud provider, and the application consumes resources as needed according to its settings. For example, in AWS, serverless applications are run through AWS Lambda functions , the deployment of which can be integrated into the Jenkins CI / CD pipeline using a plugin.
CI / CD enables more frequent code deployment
So, let's summarize. CI packages, tests builds, and notifies developers if something goes wrong. The CD automatically deploys applications and runs additional tests.
CI / CD pipelines are designed for organizations that need to make frequent changes to applications with a reliable delivery process. In addition to standardizing builds, developing tests, and automating deployments, we get a holistic workflow for deploying code changes. Implementing CI / CD allows developers to focus on improving their applications and not waste time deploying them.
CI / CD is one of the DevOps practicesbecause it is aimed at fighting the contradictions between developers who want to make frequent changes, and exploitation, which requires stability. With automation, developers can make changes more often, and operations teams, in turn, gain more stability, since the configuration of environments is standardized and continuous testing is carried out during the delivery process. Also, the setting of environment variables is separate from the application and there are automated rollback procedures.
The impact of implementing CI / CD pipelines can be measured in terms of DevOps Key Performance Indicators (KPIs)... KPIs such as deployment frequency, change lead time, and mean time to recovery often improve with CI / CD deployments with continuous testing. However, CI / CD is just one process that can contribute to these improvements. There are other conditions for increasing the delivery frequency.
To get started with CI / CD, the development and operations team needs to collaborate on technologies, practices, and priorities. Teams need to build consensus on the right approaches to their business and technology so that after CI / CD implementation, the team consistently adheres to the chosen practices.
CICD Tools Brief: Gitlab CI, Docker, Ansible