How we built a virtual infrastructure for industrial cyber operations





This year we launched a large project to create a cyber training ground - a platform for cyber training for companies from various industries. To do this, it is necessary to create virtual infrastructures that are "identical to natural" - so that they repeat the typical internal structure of a bank, energy company, etc., and not only in terms of the corporate segment of the network. A little later, we will talk about the banking and other infrastructures of the cyber landfill, and today - about how we solved this problem in relation to the technological segment of an industrial enterprise.



Of course, the topic of cyber exercises and cyber polygons did not appear yesterday. In the West, a circle of competing proposals, various approaches to cyber-training, and also simply the best practices have long been formed. The "good form" of the information security service is to periodically practice readiness to repel cyberattacks in practice. For Russia, this is still a new topic: yes, there is a small supply, and it arose several years ago, but demand, especially in industrial sectors, began to form little by little only now. We believe there are three main reasons for this - they are problems that have already become very obvious.



The world is changing too fast



Even 10 years ago, hackers attacked mainly those organizations from which they could quickly withdraw money. For the industry, this threat was less relevant. Now we see that the infrastructures of state organizations, energy and industrial enterprises are also becoming the subject of their interest. Here, we often deal with attempts at espionage, data theft for various purposes (competitive intelligence, blackmail), as well as obtaining points of presence in the infrastructure for further sale to interested comrades. Well, even trivial ransomware like WannaCry have hooked a lot of similar objects around the world. Therefore, modern realities require information security specialists to take these risks into account and form new information security processes. In particular, to regularly improve their qualifications and practice just practical skills.Personnel at all levels of operational and dispatch management of industrial facilities must have a clear understanding of what actions to take in the event of a cyber attack. But to carry out cyber-training on your own infrastructure - thank you, the risks clearly outweigh the possible benefits.



Lack of understanding of the real capabilities of attackers to hack ICS and IIoT systems



This problem exists at all levels of organizations: not even all specialists understand what can happen to their system, what are the vectors of attack on it. What can we say about leadership.



Security officers often appeal to the "air gap", which supposedly will not allow an attacker to go beyond the corporate network, but practice shows that in 90% of organizations there is a connection between the corporate and technology segments. At the same time, the very elements of building and managing industrial networks also often have vulnerabilities, which we, in particular, saw while examining MOXA and Schneider Electric equipment .



It is difficult to build an adequate threat model



In recent years, there has been a constant process of complication of information and automated systems, as well as a transition to cyber-physical systems, which involves the integration of computing resources and physical equipment. Systems are becoming so complex that it is simply impossible to predict the full consequences of cyber attacks using analytical methods. We are talking not only about the economic damage to the organization, but also about assessing the consequences that are understandable for the technologist and for the industry - the shortage of electricity, for example, or another type of product, if we are talking about oil and gas or petrochemicals. And how to prioritize in such a situation?



Actually, all this, in our opinion, became the prerequisites for the emergence of the concept of cyber training and cyber polygons in Russia.



How the technological segment of the cyber polygon works



Cyber ​​polygon is a complex of virtual infrastructures that replicate typical infrastructures of enterprises in various industries. It allows you to "train on cats" - to work out the practical skills of specialists without the risk that something will go wrong, and cyber training will harm the activities of a real enterprise. Large information security companies are starting to develop this area, and you can look at such cyber-training in a game format, for example, at Positive Hack Days.



A typical network infrastructure scheme for a conventional large enterprise or corporation is a fairly standard set of servers, work computers and various network devices with a typical set of corporate software and information security systems. The industry cyber polygon is all the same, plus serious specifics that sharply complicate the virtual model.



How we brought the cyber polygon closer to reality



Conceptually, the appearance of the industrial part of the cyber-polygon depends on the chosen method for modeling a complex cyber-physical system. There are three main approaches to modeling:







Each of these approaches has its own advantages and disadvantages. In different cases, depending on the ultimate goal and existing constraints, all three of the above modeling methods can be applied. In order to formalize the choice of these methods, we have compiled the following algorithm: The







pros and cons of different modeling methods can be represented in the form of a diagram, where the ordinate is the coverage of research areas (i.e. the flexibility of the proposed modeling tool), and the abscissa is the accuracy modeling (degree of correspondence to the real system). It turns out almost Gartner's square:







Thus, the so-called hardware-in-the-loop (HIL) simulation is optimal in terms of the ratio of simulation accuracy and flexibility. Within the framework of this approach, the cyber-physical system is partly modeled using real equipment, and partly using mathematical models. For example, an electrical substation can be represented by real microprocessor devices (relay protection terminals), servers of automated control systems and other secondary equipment, and the physical processes taking place in the electrical network can be implemented using a computer model. Okay, we've decided on the modeling method. After that, it was necessary to develop the architecture of the cyber polygon. To make cyber training really useful,all interconnections of a real complex cyber-physical system must be recreated as accurately as possible at the test site. Therefore, in our country, as in real life, the technological part of the cyber polygon consists of several interacting levels. Let me remind you that a typical industrial network infrastructure includes the lowest level, to which the so-called "primary equipment" belongs - this is optical fiber, an electrical network or something else, depending on the industry. It exchanges data and is controlled by specialized industrial controllers, which, in turn, are SCADA systems.that a typical industrial network infrastructure includes the lowest level, to which the so-called "primary equipment" belongs - this is optical fiber, electrical network or something else - depending on the industry. It exchanges data and is controlled by specialized industrial controllers, which, in turn, are SCADA systems.that a typical industrial network infrastructure includes the lowest level, to which the so-called "primary equipment" belongs - this is optical fiber, electrical network or something else - depending on the industry. It exchanges data and is controlled by specialized industrial controllers, which, in turn, are SCADA systems.



We began the creation of the industrial part of the cyber polgon from the energy segment, which is now a priority for us (the plans include the oil and gas and chemical industries).



It is obvious that the level of primary equipment cannot be realized through full-scale modeling using real objects. Therefore, at the first stage, we developed a mathematical model of the power facility and the adjacent section of the power system. This model includes all power equipment of substations - power lines, transformers, and so on, and is performed in a special RSCAD software package. The model created in this way can be processed by a real-time computing complex - its main feature is that the process time in a real system and the process time in the model are absolutely identical - that is, if in a real network a short circuit lasts two seconds, it will be simulated exactly the same amount in RSCAD). We get a "live" section of the electric power system,functioning according to all laws of physics and even reacting to external influences (for example, actuation of relay protection and automation terminals, disconnection of switches, etc.). Interaction with external devices was achieved using specialized custom communication interfaces that allow the mathematical model to interact with the level of controllers and the level of automated systems.



But the very levels of controllers and automated control systems of a power facility can be created using real industrial equipment (although, if necessary, we can also use virtual models). At these two levels, there are, respectively, controllers and automation equipment (relay protection and automation equipment, PMU, USPD, meters) and automated control systems (SCADA, OIC, AIISKUE). Full-scale modeling can significantly increase the realism of the model and, accordingly, the cyber exercises themselves, since the teams will interact with real industrial equipment, which has its own characteristics, bugs and vulnerabilities.



At the third stage, we implemented the interaction of the mathematical and physical parts of the model using specialized hardware and software interfaces and signal amplifiers.



As a result, the infrastructure looks something like this:







All equipment of the landfill interacts with each other in the same way as in a real cyber-physical system. More specifically, when building this model, we used the following equipment and computing facilities:



  • Computing complex RTDS for carrying out calculations "in real time";
  • Automated workstation (AWS) of the operator with installed software for modeling the technological process and primary equipment of electrical substations;
  • Cabinets with communication equipment, relay protection and automation terminals, and APCS equipment;
  • Amplifier cabinets designed to amplify analog signals from the digital-to-analog converter board of the RTDS simulator. Each amplifier cabinet contains a different set of amplification blocks used to generate current and voltage input signals for the relayed protection and automation terminals under study. The input signals are amplified to the level required for the normal operation of the relay terminals.






This is not the only possible solution, but, in our opinion, it is optimal for carrying out cyber exercises, since it reflects the real architecture of the vast majority of modern substations, and at the same time it can be customized so as to accurately recreate some features of a particular object.



Finally



Cyber ​​landfill is a huge project and there is still a lot of work ahead. On the one hand, we study the experience of our Western colleagues, on the other, we have to do a lot, relying on our experience of working with Russian industrial enterprises, since the specificity is not only in different industries, but also in different countries. This is both a complex and interesting topic.

Nevertheless, we are convinced that in Russia we have reached, as they say, “the level of maturity,” when the industry also understands the need for cyber-training. This means that the industry will soon have its own best practices, and we hope to strengthen the level of security.



Authors



Oleg Arkhangelsky, Lead Analyst and Methodologist of the Industrial Cyber ​​Polygon project.

Dmitry Syutov, Chief Engineer of the Industrial Cyber ​​Polygon project;

Andrey Kuznetsov, Head of the Industrial Cyber ​​Field Project, Deputy Head of the ICS Cybersecurity Laboratory for Production



All Articles