Dmitry Kopytov, the chief developer and project architect of OTR Group, spoke about the practical use of the CI / CD approach using the example of delivering scripts for migration of the Oracle 19 database. To solve the problem, he used GitLab Community Edition, GitLab Runner and Liquibase. The expert described in detail the technical aspects of setting up tool communication for projects.
The CI / CD development methodology is already applied on most projects. It is the automation of testing and delivery of new projects of the developed project to developers, analysts, quality engineers, end users and other interested parties. The method ensures the promptness of the output of new product functionality and the improvement of the quality of the developed solution.
In one of the projects, Dmitry Kopytov faced a CI / CD and the task of setting up from scratch the delivery of Oracle Database 19 migration scripts in one of the projects. To do this, he decided to use standard tools - GitLab and Liquibase. The first is a well-known environment for managing project repositories with the ability to work with the CI / CD pipeline. And Liquibase is an open source platform for rolling forward database migrations.
If we talk about the specific capabilities that were used to solve the problem, then we used:
GitLab Commmunity Edition 13.x - as a Git repository;
GitLab Runner - the main tool for working with CI / CD;
Liquibase is a tool for describing database roll-forward and failure scripts using chageset files consisting of SQL commands or database-independent structures.
Action plan
Before starting development, you need to make a plan for how the future migration script will work. In theory, when a developer makes a commit or a merge request, the CI / CD pipeline is launched - the very pipeline. The pipeline includes stages consisting of jobs.
— deploy. , . . , . — deploy-dev deploy-prod. . , deploy-dev , deploy-prod — . , . , , . , , .
GitLab Runner. , . Runner , , , , , .
, GitLab Runner http-. Runner GitLab. . deploy-dev deploy-prod , Runner. . Runner deploy-prod.
Runner Liquibase. . Liquibase .
, . :
, :
, . , .
«» Centos 7. , . .
Java
Liquibase Java 11+. OpenJRE 11 Centos 7 :
sudo yum install java-11-openjdk
java --version
Liquibase
Liquibase , Java. . — /usr/share/liquibase/4.3.4. driver . ojdc10.jar.
:
cd /usr/share/liquibase/4.3.4
liquibase --version
Git
: « Git, GitLab Runner ?». , , GitLab Runner Git 1.8, CI/CD.
Git , 1.8, :
#
git --version
# , 1.8
sudo yum remove git*
# (2.30)
sudo yum -y install https://packages.endpoint.com/rhel/7/os/x86_64/endpoint-repo-1.7-1.x86_64.rpm
sudo yum install git
GitLab Runner
GitLab Runner :
#
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash
#
export GITLAB_RUNNER_DISABLE_SKEL=true; sudo -E yum install gitlab-runner
. .
, GitLab Runner Liquibase . .
GitLab Runner
GitLab Runner . :
#
which gitlab-runner # /usr/bin/gitlab-runner
#
sudo chmod +x /usr/bin/gitlab-runner
GitLab Runner , :
#
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
#
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
GitLab Runner systemctl. , :
sudo gitlab-runner status
#
sudo gitlab-runner start
#
sudo gitlab-runner stop
#
sudo gitlab-runner list
GitLab. Setting, CI/CD Runners. GitLab Runners, . :
— . GitLab Runners , .
:
sudo gitlab-runner register
:
Enter the GitLab instance URL
# GitLab
Enter the registration token
#
Enter a description for the runner
# , , dev-runner
Enter tags for the runner
# . : liquibase,dev
Enter an executor
# shell
, :
sudo gitlab-runner list
, Git, Settings ⇨ CI/CD ⇨ Runners.
, GitLab Runner .
CI/CD
, CI/CD . CI/CD GitLab .gitlab-ci.yml. bash, /ci.
.gitlab-ci.yml , https://gitlab.example.com/gitlab-org/my-project/-/ci/lint , .
:
variables:
LIQUIBASE_VERSION: "4.3.4"
stages:
- deploy
deploy-dev:
stage: deploy
tags:
- liquibase
- dev
script:
- 'bash ./ci/deploy-db.sh $DEV_DB $DEV_DB_USER $DEV_DB_PASS'
environment:
name: dev
only:
- dev
deploy-prod:
stage: deploy
tags:
- liquibase
- prod
script:
- 'bash ./ci/deploy-db.sh $DEV_DB $DEV_DB_USER $DEV_DB_PASS'
environment:
name: prod
when: manual
only:
- prod
, .
variables
. LIQUIBASE_VERSION: "4.3.4" Liquibase, . Liquibase , .
variables:
LIQUIBASE_VERSION: "4.3.4"
— . Setting, CI/CD Variables. :
stage
. , .
stages:
- deploy
jobs
. deploy-dev deploy-prod. , .
deploy-dev:
stage: deploy
tags:
- liquibase
- dev
script:
- 'bash ./ci/deploy-db.sh $DEV_DB $DEV_DB_USER $DEV_DB_PASS'
environment:
name: dev
only:
- dev
deploy-prod:
stage: deploy
tags:
- liquibase
- prod
script:
- 'bash ./ci/deploy-db.sh $DEV_DB $DEV_DB_USER $DEV_DB_PASS'
environment:
name: prod
when: manual
only:
- prod
stage
, jobs.
stage: deploy
tags
. . , ? , GitLab Runners , .
, . dev, — prod.
tags:
- liquibase
- dev
script
, .
script:
- 'bash ./ci/deploy-db.sh $DEV_DB $DEV_DB_USER $DEV_DB_PASS'
environment
.
environment:
name: dev
Operations Environments. , , , , , . Environments :
when
, . , deploy-dev , , . deploy-prod , .
when: manual
only/except
only . , except.
only:
- dev
Liquibase
Liquibase bash. . . — .
./ci/deploy-db.sh:
#!/bin/bash
echo "Environment: $CI_ENVIRONMENT_NAME"
cd db/changelog
/usr/share/liquibase/$LIQUIBASE_VERSION/liquibase \
--classpath=/usr/share/liquibase/$LIQUIBASE_VERSION/drivers/ojdbc10.jar \
--driver=oracle.jdbc.OracleDriver \
--changeLogFile=master.xml \
--contexts="$CI_ENVIRONMENT_NAME" \
--defaultSchemaName=STROY \
--url=jdbc:oracle:thin:@$1 \
--username=$2 \
--password=$3 \
--logLevel=info \
update
:
classpath — ;
driver — ;
changeLogFile — - ;
contexts — ;
defaultSchemaName — ;
url — . DEV_DB $1;
username — . DEV_DB_USER $2;
password — . DEV_DB_PASS $3.
Liquibase :
Let's improve the CI / CD process by disabling merges if the deployment process goes wrong. To prohibit such merges, you need to find the Merge checks item in the General section of the repository and check the corresponding checkbox. But you need to do this after setting up the CI / CD.
You need to look for a checkmark here:
What happened?
After all the settings, our process works as follows:
1. The developer makes a merge request or commit to a branch.
2. The pipeline starts.
3. The job associated with the branch is launched.
4. With the help of a job, the runner is started.
5. The runner downloads the sources and calls it using the Liquibase script.
6. Liquibase generates and executes roll forward or rollback scripts.
Exactly what we needed initially.