Clever Geek Handbook

Canary deploy with Jenkins-X, Istio and Flagger

Good day, reader!



Here we come to the final part of a series of articles about Canary releases in Kubernetes and methods of their implementation. Enjoy your reading and hope you have found this cycle helpful.





Using Jenkins X Solution to Perform Canary Deployment on a Kubernetes Cluster







In this loop:



  1. Canary Deployment GitlabCI + GitOps/Manual Approach
  2. Canary Deployment Argo Rollouts
  3. Canary Deployment Istio
  4. ( )


?



Jenkins X k8s Python . , , JenkinsX+Flagger+Istio anary deployment , .



Canary Deployment



, , anary deployment Kubernetes.



Jenkins X



, , , Jenkins X. , .



Istio



, , Istio. , . 3, Canary deployment Istio, .



Flagger



Flagger Kubernetes, anary deployments Istio, Linkerd, App Mesh, NGINX, Contour Gloo routing Prometheus canary-. Canary- - , . ()




Jenkins X jx Kubernetes, Github , .



Jenkins X k8s



Jenkins X , Gcloud:



jx create cluster gke -n jenkinsx --machine-type n1-standard-4


“Serverless Jenkins X Pipelines with Tekton” Github . Github stage production .



(addons):



jx create addon istio
jx create addon flagger
jx create addon prometheus


Istio ( istio_requests_total Prometheus), Istio :



istioctl manifest generate --set values.kiali.enabled=true --set values.tracing.enabled=true --set values.grafana.enabled=true --set values.prometheus.enabled=true > istio.yaml
kubectl -f istio.yaml install


Flagger jx Istio sidecar namespace jx-production. :



kubectl get ns -L istio-injection


label istio-injection: enabled namespace, Istio sidecar .



Test App / Quickstart



jxing-nginx-ingress-controller namespace kube-system External IP. python-:



jx create quickstart --project-name python-test


python-http. , canary deployment . - HTTP endpoints .



Jenkins X Github .



do_GET app.py, , :



def do_GET(self):
 self.send_response(200)
 self.send_header('Content-type','text/html')
 self.end_headers()
 # Send the html message
 output = 'v1'
 self.wfile.write(output.encode('utf-8'))
 return




Jenkins X (jx get env):







Git-, GitOps. Helm Charts + , , require.yaml.



Deployment



quickstart , (jx import), 0.0.1 stage . , PROMOTE Auto stage . Pull Request stage , Jenkins X . Pipeline :



 jx get activities -w


pipelines , :



jx get applications




                        Jenkins X     Semver


, 0.0.1 stage, prod . :



kubectl -n jx-staging get all       # shows the app
kubectl -n jx-production get all    # shows nothing


stage , URL:







production



( production , 0.0.2)



(promote) production “”. jx get env , production , :



jx promote --version 0.0.2 --env production


Pull Request Jenkins X production Github . jx get applications 0.0.2 stage production:







production requirements.yaml prod .



Canary prod



prod stage canary, rolling-rollouts Kubernetes.



Python, Jenkins X -. charts/APP_NAME/values.yaml, Flagger Canary:



# Canary deployments
# If enabled, Istio and Flagger need to be installed in the cluster
canary:
 enabled: false
 progressDeadlineSeconds: 60
 canaryAnalysis:
   interval: "1m"
   threshold: 5
   maxWeight: 60
   stepWeight: 20
   # WARNING: Canary deployments will fail and rollback if there is no traffic that will generate the below specified metrics.
   metrics:
     requestSuccessRate:
       threshold: 99
       interval: "1m"
     requestDuration:
       threshold: 1000
       interval: "1m"
 # The host is using Istio Gateway and is currently not auto-generated
 # Please overwrite the `canary.host` in `values.yaml` in each environment repository (e.g., staging, production)
 host: acme.com


canary deployment 3 , 1 : 20%, 40%, 60%.



anary canary: enabled: false.



Jenkins X , production environment-jenkinsx-production. Python, Helm charts requirements.yaml. env/values.yaml environment-jenkinsx-production:



...
jenkinsx-istio-canary-python-test:
 canary:
   enabled: true
   host: jenkinsx-istio-canary-python-test.35.204.67.7.nip.io


  • xxx.IP_ADDRESS.nip.io IP_ADDRESS. .
  • host: Istio, . IP- Istio : kubectl -n istio-system get svc. , , host:, External IP Istio.


production Jenkins X, :



 jx get activity


activities successful. Istio VirtualService Gateway:



kubectl -n jx-production get virtualservices.networking.istio.io,gateways.networking.istio.io


, , , anary, , . ()

production Istio :



jenkinsx-istio-canary-python-test.35.204.67.7.nip.io:






Flagger Grafana Dashboard



Flagger Grafana namespace istio-system, , :



kubectl port-forward -n istio-system service/flagger-grafana 3000:80
# admin:admin






Grafana Istio Dashboard :



  • namespace: jx-production
  • primary: jx-jenkinsx-istio-canary-python-test-primary
  • canary: jx-jenkinsx-istio-canary-python-test-canary


Primary canary services. , Canary .



istio_requests_total, , Istio , , , jx.



Flagger Canary Workflow



Helm Chart Python , ( ):



canary:
 enabled: false # false, but we set to true in prod env git repo
 progressDeadlineSeconds: 60
 canaryAnalysis:
   interval: "1m"
   threshold: 5
   maxWeight: 60
   stepWeight: 20
   # WARNING: Canary deployments will fail and rollback if there is no traffic that will generate the below specified metrics.
   metrics:
     requestSuccessRate:
       threshold: 99
       interval: "1m"
     requestDuration:
       threshold: 1000
       interval: "1m"


. Flagger .

Canary, Flagger:



kubectl -n jx-production get canaries.flagger.app


Canary Deployment



Deploy



app.py , GET . Pull Request master- merge. Jenkins X deploy stage.



prod, , stage. 0.0.6 , :



jx promote --version 0.0.6 --env production


Canary:



kubectl -n jx-production get events --field-selector involvedObject.kind=Canary --sort-by='{.lastTimestamp}'


Canary 20%



2m39s       Normal    Synced   Canary   Advance jx-jenkinsx-istio-canary-python-test.jx-production canary weight 20


curl production Istio endpoint, , 20% :







Canary ( )



, endpoint, Canary Flagger :







Canary , .



Canary 40%



, , Flagger 40%:









Canary 60%



, , Flagger 60%:



60s         Normal    Synced   Canary   Advance jx-jenkinsx-istio-canary-python-test.jx-production canary weight 60






Canary 



25s         Normal    Synced   Canary   Routing all traffic to primary


:









Jenkins X Istio Flagger Canary Deployments. , Istio jx create addon istio, .



Flagger, , , . , Jenkins X , .



Jenkins X, , Flagger Istio Canary Deployments , , .



/



https://jenkins-x.io/docs/managing-jx/tutorials/progressive-delivery/



https://docs.flagger.app/usage/progressive-delivery





(. . — CKAD — https://killer.sh/ )



More articles:


All Articles