Clever Geek Handbook

Easy creation of Kubernetes operators with shell-operator: project progress in a year





Kubernetes operators are a convenient mechanism for extending the capabilities of this container platform, which has rightfully won wide recognition among the environment of operations engineers and their sympathizers. We talked about how they are arranged and work in the already distant 2017. And in April last year, we presented the open source shell-operator project , which greatly simplified the process of creating Kubernetes operators.



For this, a framework was developed that allows you to run arbitrary scripts (in Bash, Python, etc.) in the event of certain events occurring in the K8s cluster.



Over the past time, the shell-operator has acquired its user base (see details at the end of the article) and, of course, new features. On the occasion of the recent releasev1.0.0-beta.11 (about the beta status, see below) we decided to talk about what the project has come to during its existence, since the announcement of the first public version.



About the device and purpose



But let's start with a brief explanation of how the shell-operator works and why, in principle, it may be needed.



Shell-operator runs in the pod of the Kubernetes cluster. It is presented there as:



  • Go binary that subscribes to events in the K8s API and launches hooks (giving them details about what happened);
  • a set of hooks, each of which is a Bash script, Python script, or any other executable file.


Hooks:



  • they themselves determine which events and for which objects they need;
  • perform the necessary actions in the event of these events occurring in K8s.


Thus, the shell-operator is a layer between events in the Kubernetes API and scripts for processing them.



image



Why was the shell-operator created at all? Operators are the standard for "doing the right thing" within Kubernetes, but fully developing them (in Go using the appropriate SDK ) isn't easy. The presence of such a simple framework as the shell-operator significantly lowers the threshold for entering this area, allowing you to quickly and efficiently solve small operational problems * within the cluster. And, just as important, do it the right way.



What tasks are we talking about? Ready examples of using the shell-operator can be found in the project repository . At us, at Flant, we use it as a library(yes, that was also possible!) . It became the basis for the addon-operator , which manages additional components in Kubernetes.



NB : The announcement of this Open Source project (addon-operator) can be found here . And in the report " Expanding and Supplementing Kubernetes ", we talked in detail about the reasons for its appearance, the relationship with the shell-operator and the principles of operation.



Now about the main changes introduced to the shell-operator over the past year.



Major innovations



In the first versions of the shell-operator, only one object was available for the hook - the one associated with the event from the cluster. The evolution of the hooks used within the addon-operator resulted in the hook subscribing to an object change, but calling kubectlto get an up-to-date list of other objects. To remove unnecessary calls kubectland thereby speed up the work of hooks, several possibilities have been implemented to access the current lists of objects:



  • Synchronization + Event mode , when the hook at start receives a list of actual objects, and then works with only one object. This mode is enabled by default - we can say that the result is an analogue of the reconcile loop from operator-sdk.
  • snapshot', . (Snapshot’ Kubernetes, .)
  • snapshot'. , , , .
  • It also became possible to monitor the resource , but not react to its changes, i.e. "Accumulate snapshot". For example, a hook can react to changes in the CustomResource and still receive the actual ConfigMap object without additional call kubectl. (See flags executeHookOnSynchronizationand for details executeHookOnEvent.)


Other significant innovations:



  • Thanks to the transition to using the dynamic Kubernetes client in the shell-operator, it became possible to subscribe to any existing kind(resource type in the Kubernetes API), including Custom Resources.
  • (. queue). endpoints.
  • .
  • « », namespace’ .
  • scraping' Prometheus'. .
  • , shell.




  • YAML- ( JSON).
  • JSON logrus (. LOG_TYPE ).
  • listen-address hostNetwork: true.
  • rate limit (qps, burst) Kubernetes API.
  • kube-server Kubernetes API.
  • .
  • jqFilter libjq-go, jq.
  • zombie reaper, SIGCHLD -, Bash-. — tini.
  • Various simplifications have been implemented for connecting the shell-operator as a library.
  • Updated version kubectl(from 1.13 to 1.17.4) and made an assembly based on alpine-3.11.


Status and plans



The shell-operator project is still officially in beta . Despite this, as noted above, we very intensively use it as the basis for the addon-operator - a tool that is constantly used in many (100+) K8s clusters.



For a stable release of the shell-operator as a public project, we plan (at least):



  • add e2e testing ( # 63 ),
  • implement multi-architectural build ( # 184 ),
  • update client-go to 0.18.0, implement contextand finally deal with object caching in client-go ( # 188 ).


Community recognition



Over the years, we have seen a clear community interest in the shell-operator:





We are also pleased to announce that at the upcoming virtual conference KubeCon + CloudNativeCon Europe 2020 , which will be held in August, there will be our report on the shell-operator. Details on it are on the event website .



Thanks for your interest in the shell-operator! If you have any questions, ask them here in the comments or in the tg-channel @kubeoperator .



PS



Read also on our blog:





More articles:


All Articles