In the previous post , an algorithm for choosing a control system for various tasks of the "Internet of Things" was proposed. For medium-sized solutions, the use of cloud IoT platforms turned out to be optimal.



This article is about how we implemented a monitoring and management system based on the Amazon WEB Services (AWS) platform. A communication cabinet resembling a small “Smart House” is considered as a control object. Inside such a cabinet can be installed cellular or fixed communication equipment, city WiFi, video surveillance, lighting control, etc., as well as power supply and climate control devices. Using a smart home controller cabinet, a set of sensors and a top-level system, the operator can monitor the condition of the equipment, the microclimate and security and fire safety. 

Task

Let's consider the creation of a two-tier system, the structural diagram of which is shown below.





At the lower level, controllers with sensors, actuators and digital devices should be used.

Lower-level system functions and controller requirements

• IoT ,
• ,
• B2B , B2C « »,
• ,
• ;
• WEB , 
• ,
• .

Today, more than 600 IoT platforms have been developed in the world and this number is constantly growing . At the same time , the Amazon WEB Services (AWS) platform was the most popular among developers in 2019. The dominant position of AWS determined our choice of this platform as the basis for our ST-Eye system. The decision was also influenced by the prompt technical support and the availability of a large amount of reference information. The guys from AWS shared best practices and helped choose the most effective technologies.



Below is the architecture and description of the ST-Eye system.

ST-Eye IoT system architecture on AWS platform

Communication level with controllers

Object controllers connect to the AWS IoT-Core service using TLS. The server address used for the connection contains a unique identifier for the account. Authentication is carried out using the x509 protocol. Each device uses its own unique certificate. In addition to its own certificate, the controller has a CA certificate for server authentication. The creation and distribution of certificates is done through the AWS platform. After successful authentication, the MQTT protocol is used, which has become the de facto standard in IoT... The MQTT protocol is based on the interaction of clients with each other on a pub / sub basis. Clients publish messages to channels (topics) and subscribe to existing topics of other clients. This is provided by an intermediate service - a broker, which stores a list of topics and lists of subscribers for each of the available topics. The IoT-Core service provides several standard topics for subscribing and publishing, you can create other topics. They will be available within the AWS Region and account. For example, controllers subscribe to topics of direct control, and the WEB application publishes commands to them. Due to the low resource requirements of MQTT, the response time of the controller to the operator's command is minimal (in our system - no more than 0.5 seconds when using an office Ethernet connection). IoT-Core is a managed service, respectively,redundancy and scaling are performed without user intervention.



One of the key functions of the IoT-Core service is the shadow virtual image of the device. It allows you to get the last known state of the device and display it even when the device is not available over the network. This feature is indispensable for working with GSM devices or battery-powered devices, since they may be offline for a long time. Shadow is a JSON object containing two main fields: reported and desired. Each field contains key-value pairs corresponding to the output and / or device parameters. When the desired device parameters are changed, the delta field is generated containing the difference between the desired and reported fields. The device subscribes to the topic $ aws / things / thingName / shadow / update / deltaand updates its internal state if there are parameters in the delta field. Shadow contains the date and time of the last update. Thus, the upper-level application receives information about how long ago the controller contacted the last time.



The IoT-Core service also provides a flexible system of rules for interacting with other services. In our application, IoT-Core rules are used, for example, to send notifications and to activate devices. Notifications are configured in the administrator's workstation. Available SMS notifications, email, push.



There are strict requirements for the software update process in IoT devices. The update source and the container with the new firmware must be verified for authenticity and compatibility with the controller. Otherwise, the controller may turn into a "brick" after the update. The Job mechanism is used to perform Firmware over the air (FOTA) or device configuration. The controller receives a set of commands and additional data through a special topic, then sequentially executes the commands. For example, it downloads the firmware from the AWS S3 cloud storage using the link and verifies its signature.

Data processing layer

To process data coming from the IoT-Core service to AWS, the IoT-Analytics subsystem is used. The channel block , using sql-like rules, allows you to select a data source, for example, individual message fields from devices. The raw data arriving through the channel can be stored in the service's internal storage or in S3. The pipeline component allows you to pre-process, filter data, or supplement it. For example, based on some data, you can make a request to a database (DB) or an external service and add information from the response. The preprocessed data is saved to the datastore database . This database is of the time series database type ( TSDB) and is designed to work with data coming from time-referenced sensors. Using this data, it is possible to track the change in each parameter of the system during the entire observation period. Unlike other types of databases, TSDB provides tools such as data retention / deletion policies, a query scheduler, and flexible aggregation functions.



Data samples are subsequently applied to records in the datastore ( datasets). They can be implemented in the form of SQL queries and executed by the scheduler inside the IoT-Analytics subsystem, and, if more complex logic is required, in the Docker container. Post-processing results are stored in the service storage and are available for 90 days from the WEB application and the data visualization service (Quicksight). After the expiration date, the data is transferred to cold storage on S3 using a Lambda function.

Visualization level (WEB app)

At the top level of the system, a WEB portal is implemented, represented by several automated workstations (AWS), in particular, an administrator, analyst / manager and dispatcher.



Consider the workstation of the dispatcher. Its main window looks like a table, which summarizes data on groups of objects.





Each object can be in one of five states: normal, trouble / alarm, violation, service mode and no communication. Each cell in the table allows you to navigate to a list of relevant objects with more detailed information: 





By clicking on the corresponding device in the list, you can open the page for its monitoring and control.



The WEB application uses the TLS protocol as a transport to connect to AWS. Authentication works on the Signature V4 algorithm and uses the Cognito microservice. Application code is hosted in S3 storage. The portal uses the Continuous Integration mechanism. New versions are automatically tested and, if successful, are applied to the platform. The Beanstalk service is used for version control and automatic deployment.

Import substitution

Russian legislation for government applications (B2G) has restrictions on the use of servers outside of Russia, for example, 187-. The most advanced Russian cloud service is Yandex.Cloud, of which the IoT platform is a part. At the start of work on our system, some of the components available in AWS were not yet available from Yandex. In particular, to create a complete solution, you needed your own backend to interact with the WEB application and the MQTT broker. On the other hand, Yandex has already offered rich analytics and database tools, application deployment tools and cloud storage (Object Storage). The Datalens service for data visualization (analogous to AWS Quicksight) is already available, as well as the ClickHouse database, which is perfect for storing time series data.



Yandex makes a great contribution to the development of Open Source in the world, and also tries to make its products compatible with existing solutions. So, for example, you can work with Object Storage Yandex using the aws-cli utility using commands for AWS S3.

Selecting object controllers

The market for object controllers is quite large. The number of options and manufacturers from which to choose goes to dozens, and models to hundreds. As a rule, when creating control systems, they use controllers that they already know how to work with, or that the supplier of the top-level system recommends. This state of affairs is the only possible when using top-level systems developed for certain equipment ( hardware-specific software platforms ) and working with controllers using closed exchange protocols.



When using top-level systems operating on open protocols (cloud IoT platforms are just such), you can save money by choosing the optimal equipment for the requirements of the project. An example of such requirements is given in the paragraph “Low-level system functions and controller requirements” above.



For B2G solutions in accordance with PP-878, it is better to focus on Russian controllers. For some areas, it is becoming increasingly important to use equipment based on Russian element base.



Not everyone is familiar with the option to design custom controllers. It can pay off through the use of an inexpensive element base and the implementation of the required number of functional blocks for a specific project. In addition, the complex of equipment at the facility will consist of a minimum number of devices (sometimes only one), which means there will be less assembly, installation and commissioning work.



Customized development of the controller can pay off due to the lower cost of equipment, less installation and commissioning work.



For the described task, we used the GiC controller(Generic Internet Controller) proprietary. It is equipped with the required number of network interfaces, input-output ports, built-in power supply, supports MQTT, WEB, SNMP, ModBUS protocols and can interrogate various digital devices: metering devices, power supply systems, air conditioners (the list is constantly expanding).

Conclusion

Above, we described how our management system is made on AWS. Cloud technologies make this kind of development easier, but it still cannot be said that this is a matter of a couple of clicks. If your task is similar to ours, please contact us.



In IoT platforms, it is worth waiting for the appearance of pre-configured templates for Smart Home systems, ASKUE, NMS, ACS, etc. This will further simplify the creation of complete solutions, lower the threshold for entering this business and attract an additional audience to cloud technologies.



IoT platforms use open exchange protocols with controllers. This gives users the freedom to choose their site equipment. There is a possibility of custom development of controllers (our specialization ). Using custom controllers can reduce project budget.



For government projects and projects related to critical information infrastructure, it is better to focus on Russian systems and facility equipment, since the market for such systems has already been formed and is developing.