As part of the DevOps Practices and Tools course , we have prepared a translation of a useful article for you.
We also invite you to an open webinar on "Prometheus: Quick Start" . At the webinar, participants, together with an expert, will review the Prometheus architecture and how it works with metrics; will figure out how to generate alerts and events in the system.
Wait ... what, what? Yes, I've heard similar reactions to my suggestion to use Kubernetes to build Kubernetes clusters.
But for the automation of cloud infrastructure, nothing better comes to my mind than Kubernetes itself . Using one central K8s cluster, we create and manage hundreds of other K8s clusters. In this article, I'll show you how to do it.
: SAP Concur AWS EKS, , , Google GKE, Azure AKS , Kubernetes.
Kubernetes . AWS EKS :
$ eksctl create cluster
Kubernetes, (production ready), . « » , SAP Concur Kubernetes.
. AWS, . : IP- , AWS exports, SSM .
EKS control plane nodegroup. AWS EKS .
. , :-) Istio, logging integration, autoscaler .. .
. ( EKS ) . , . ( !)
, . , , , - , , .
Argo. , Argo Events Argo Workflows. Kubernetes CRD YAML, Kubernetes.
: (Imperative Orchestration), (Declarative Automation).
Argo Workflows
Argo Workflows — container-native workflow engine Kubernetes. Argo Workflows Kubernetes CRD.
: K8s YAML, , .
, Argo Workflows.
1.
BATS. BATS :
#!/usr/bin/env bats
@test “More than 100 available IP addresses in subnet MySubnet” {
AvailableIpAddressCount=$(aws ec2 describe-subnets --subnet-ids MySubnet | jq -r ‘.Subnets[0].AvailableIpAddressCount’)
[ “${AvailableIpAddressCount}” -gt 100 ]
}
BATS- ( avail-ip-addresses.bats
) Argo Workflow :
— name: preflight-tests templateRef: name: argo-templates template: generic-template arguments: parameters: — name: command value: “{{item}}” withItems: — bats /tests/preflight/accnt-name-export.bats” — bats /tests/preflight/avail-ip-addresses.bats” — bats /tests/preflight/dhcp.bats” — bats /tests/preflight/subnet-export.bats”
2. EKS control plane nodegroup
EKS . eksctl
, CloudFormation Terraform. EKS , CloudFormation (eks-controlplane.yaml
eks-nodegroup.yaml
), Argo Workflow .
— name: eks-controlplane
dependencies: [“preflight-tests”]
templateRef:
name: argo-templates
template: generic-template
arguments:
parameters:
— name: command
value: |
aws cloudformation deploy \
--stack-name {{workflow.parameters.CLUSTER_NAME}} \
--template-file /eks-core/eks-controlplane.yaml \
--capabilities CAPABILITY_IAM
- name: eks-nodegroup
dependencies: [“eks-controlplane”]
templateRef:
name: argo-templates
template: generic-template
arguments:
parameters:
— name: command
value: |
aws cloudformation deploy \
--stack-name {{workflow.parameters.CLUSTER_NAME}}-nodegroup \
--template-file /eks-core/eks-nodegroup.yaml \
--capabilities CAPABILITY_IAM
3.
, kubectl
, helm, kustomize . , metrics-server
helm template
kubectl
, , metrics-server
, Argo Workflows .
— name: metrics-server
dependencies: [“eks-nodegroup”]
templateRef:
name: argo-templates
template: generic-template
when: “‘{{workflow.parameters.METRICS-SERVER}}’ != none”
arguments:
parameters:
— name: command
value: |
helm template /addons/{{workflow.parameters.METRICS-SERVER}}/ \
--name “metrics-server” \
--namespace “kube-system” \
--set global.registry={{workflow.parameters.CONTAINER_HUB}} | \
kubectl apply -f -
4.
BATS- DETIK, K8s-.
#!/usr/bin/env bats
load “lib/utils”
load “lib/detik”
DETIK_CLIENT_NAME=”kubectl”
DETIK_CLIENT_NAMESPACE="kube-system"
@test “verify the deployment metrics-server” {
run verify “there are 2 pods named ‘metrics-server’”
[ “$status” -eq 0 ]
run verify “there is 1 service named ‘metrics-server’”
[ “$status” -eq 0 ]
run try “at most 5 times every 30s to find 2 pods named ‘metrics-server’ with ‘status’ being ‘running’”
[ “$status” -eq 0 ]
run try “at most 5 times every 30s to get pods named ‘metrics-server’ and verify that ‘status’ is ‘running’”
[ “$status” -eq 0 ]
}
BATS DETIK (metrics-server.bats
), , metrics-server
, Argo Workflows :
— name: test-metrics-server
dependencies: [“metrics-server”]
templateRef:
name: worker-containers
template: addons-tests-template
when: “‘{{workflow.parameters.METRICS-SERVER}}’ != none”
arguments:
parameters:
— name: command
value: |
bats /addons/test/metrics-server.bats
, . Sonobuoy conformance tests, Popeye — A Kubernetes Cluster Sanitizer Fairwinds’ Polaris. Argo Workflows!
, , AWS EKS , , , metrics-server
. !
, .
WorkflowTemplate
Argo Workflows (WorkflowTemplate), workflow. — . , , . «» workflow, ( ), . Argo Events.
Argo Events
Argo Events — Kubernetes (workflow automation framework), K8s, Argo Workflow, . , webhook, s3, , , gcp pubsub, sns, sqs ..
API (Argo Events) JSON. , (WorkflowTemplate) API. , Kubernetes, :
? API .
EKS-? eks-core (control-plane nodegroup) API.
EKS-? addons API.
- ? test API.
Argo
Argo Events, Argo Workflows , .
, :
-
-
— . Argo .
-
S3
(WorkflowTemplate)
Events Sensor
, , . Argo Events, Argo Workflows . .