The report is devoted to practical issues of developing an operator in Kubernetes, designing its architecture and basic principles of operation.
In the first part of the report we will consider:
- what is an operator in Kubernetes and why is it needed;
- how the operator simplifies the management of complex systems;
- what the operator can and what the operator cannot.
, . . :
- Kubernetes;
- , Kubernetes.
Kubernetes.
, :
- Persistent Storage ;
- Local Storage.
clickhouse-operator Amazon Google Cloud Service. ClickHouse.
:
. , . ClickHouse-operator ClickHouse.
ClickHouse?
- ClickHouse.
- ClickHouse. ClickHouse.
- ClickHouse.
. ClickHouse-operator Kubernetes.
:
- – ClickHouse Kubernetes.
- – , . . Kubernetes.
.
, ?
- , .
- , , , , Kubernetes .
, , , Kubernetes .
ClickHouse? - . open source.
. , , , , . , ClickHouse , . – ClickHouse . , ClickHouse Kubernetes.
? ? , .
- , Kubernetes. .
- : « ?». , .
- , , . . .
? , , . , , Kubernetes, ClickHouse. .
, , Kubernetes , ClickHouse . , ClickHouse- , - .
ClickHouse , DevOps:
- - ClickHouse, , , , .
- , ClickHouse . , .
- .
- , .
- .
- .
, .
Kubernetes , .
Kubernetes , :
- .
- .
- .
, , , .
, , Kubernetes.
- , , . ClickHouse Kubernetes.
, . ClickHouse-operator Kubernetes Altinity.
– , , . . .
. .
– DevOps , (DevOps) , . . (DevOps) , .
– , DevOps.
? :
- , ClickHouse, , ClickHouse , ClickHouse , , . , .
- , , . .
, , , .
, , ? Helm. ClickHouse, helm charts, ClickHouse. , , Helm?
, Helm – , . . , , , , . . , ( , ) – . . , .
, .
? , ClickHouse .
. YAML , kubectl Kubernetes. , . . ClickHouse Kubernetes.
, , . , . , .
. – open source, GitHub, . , , Quick Start Guide .
, - .
. , , -. ClickHouse, , ? , .. k8s – .
. , , , . demo.
. Storage , storage . .
. . , .
. – Pod, Service-a, StatefulSet.
, , .
. StatefulSet, Pod, ConfigMap , ConfigMap . .
– Load Balancer Service , .
. .
, . .
, . . . . , .
, . Storage , storage , .
YAML- , .
. Pod, Service-a , , 2 StatefulSet-. 2 StatefulSet-?
– , pod.
. , .
StatefulSet ? , Kubernetes Pod’.
StatefulSet, Pod’ . – Template. StatefulSet Pod’. « Pod’».
, StatefulSet. , . . , . . ClickHouse, . , StatefulSet rolling update, , , - .
, rolling update, ClickHouse, storage. , , - , , , . - , StatefulSet , .
, . StatefulSet. , . . , , . , , 2 StatefulSet’ 2 Pod’ , - .
. , .. - ClickHouse Kubernetes. .
YAML , . YAML ClickHouse, .
. . . ClickHouse. .
ConfigMap. ConfigMap , , . .
. . . . . , , . .
?
ZooKeeper. ClickHouse ZooKeeper. ZooKeeper , ClickHouse , ClickHouse .
ZooKeeper . ZooKeeper , . , . , .
. Kubernetes . ClickHouse. ZooKeeper . ClickHouse, ZooKeeper. . . .
, ClickHouse k8s.
, , .
. – ZooKeeper, Kubernetes, . . . .. . 4 pod’. storage , . Storage – .
.
. . 4- , , . ZooKeeper. .
. Persistent Storage.
Persistent Storage .
, cloud-, , Amazon, Google, storage. , .
. local storage, . , .
, cloud storage.
. . , , , storage - , - . .
. - . , - . , , .
.. ClickHouse , , , , .
, local storage.
Kubernetes local storage Kubernetes. :
- EmptyDir
- HostPath.
- Local
, , .
-, storage – , k8s . .
, . . emptyDir. ? ( – ) .
- , . .
? , .. .
. Persistent . Persistent . Kubernetes - Pod , .
, , - .
. hostPath. , . Kubernetes . . , .
. Persistent, . - .
. . Kubernetes Pod . DevOps. , pod’ , - , . .
, , . : « , instance ClickHouse - ».
, Kubernetes , , .
local. . , pod’ , - , Kubernetes. .
. YAML template. storage. . VolumeClaim template k8s. , storage .
k8s storage. StatefulSet. ClickHouse.
. Persistent Storage , , .
. ClickHouse on k8s . , , ZooKeeper, Persistent, . .
. . , ClickHouse.
– ClickHouse . , , , - , , , , , .
?
. pod’. , . .. , , , ClickHouse, storage. , .
. , ClickHouse-operator ClickHouse.
, , , K8s.
K8s . , kubectl apply? API etcd .
Kubernetes: pod, StatefulSet, service .
. .
. – k8s, . , . , , , .
K8s.
pod-, StatefulSet-, ClickHouseInstallation, . . ClickHouse, . .
K8s . , - , pod’ StatefulSet .
? -, Custom Resource Definition. ? K8s, , pod, StatefulSet , . .
kubectl apply . Kubernetes .
, etcd ClickHouseInstallation.
. . . YAML-, , «kubectl apply», Kubernetes , etcd : «, , . ClickHouseInstallation ».
, -, Kubernetes . Kubernetes, . , .
- . Kubernetes, K8s. , , Kubernetes, , .
, , Kubernetes API. API. , , . .
? , , . , K8s.
– . . Kubernetes API . Kubernetes API , . - K8s , Kubernetes , .. API , .
, - . – .
. YAML- ClickHouseInstallation. kubectl apply etcd. event, event ClickHouse-operator. . - . ClickHouseInstallation, . – .
? -, , . , .. YAML-, . , .
, pod’, , .. , . ClickHouse Kubernetes.
. Kubernetes , .. Kubernetes, .
Kubernetes , .. , system-scope. Kubernetes , pod’, , mount volumes, ConfigMap, .. , .
. . ClickHouse.
, , , . .
, , .
– . ? , StatefulSet, - , volume claim.
K8s. , ConfigMap, StatefulSet, Volume. Kubernetes . , .
ClickHouse-operator. pod, - . ClickHouse-operator . .. ClickHouse , , -, , . , -, , . .
ClickHouse, .. . . instance, , .
.
. , .
, xml, ClickHouse , .
ClickHouse. zoned deployment – , hostPath, local storage. zoned deployment.
– .
, .
. . . , . ClickHouse Prometheus, Grafana.
? - ? . , .
, - , . – , .. . . . .
? .
.
.
. .
Grafana dashboard, .
, Grafana dashboard . . DevOps .
? :
- . – .
- ZooKeeper. ZooKeeper-operator. .. ZooKeeper , .
- .
- , Templates – DONE, . . . , .
- . Re-sharding.
.
? ? Kubernetes Alitnity- .
:
- , , .
- .
- . . .
. Kubernetes, . , , ClickHouse ?
– ! , .
, TSBS. ? . , .
? . . , . .
. . :
- TimescaleDB.
- InfluxDB.
- ClickHouse.
. RedShift. Amazon. ClickHouse .
, ?
- DB Kubernetes . , , , . ClickHouse Kubernetes .
- .
- .
- , , .
Open source – !
, – open source , , . ! !
!
! . SEMrush. , . , , . , , . , . .
, . . todo. . , . . . , , , . . . , , . – , , , , . , pull request, .
! ! , Persistent Volumes. , - , ? , , , ClickHouse , ?
, – local storage, , hostPath. , , pod - , , - . , , .
. , , provisioning volumes. local storage provision , DevOps , volumes. Kubernetes provisioning, Persistent volumes - , - . Kubernetes, pod’, - local storage, labels - . - label one per host instance. , pod’ Kubernetes’ , , labels, . labels, provisioning . .
local . , , .
, . Kubernetes , . , , , ?
, Kubernetes , pod’ – cattle, - . , . Kubernetes , , .
. , , ? . ClickHouse , , .. ClickHouse.
? , DevOps. , . ClickHouse . , . , Kubernetes. ClickHouse.
, ? , , , labels. , Kubernetes instance pod’. Kubernetes . pod’ . , . ClickHouse , , . .. .
! , , , - ?
, , ?
. .
, Kubernetes. Kubernetes . . – , , replay , . . .
! ! , . haproxy? - , , ClickHouse.
Ingress?
, Ingress haproxy. haproxy , .
. , , , , . . – , . , . use case , , issues GitHub , .
.
. . haproxy todo. todo , . , , .