We develop a website for a microcontroller

With the advent of various kinds of smart sockets, light bulbs and other similar devices into our lives, the need for websites on microcontrollers has become undeniable. And thanks to the lwIP project (and its younger brother uIP), you won't surprise anyone with such functionality. But since lwIP is aimed at minimizing resources, in terms of design, functionality, as well as usability and development, such sites lag far behind those to which we are used. Even for embedded systems, compare, for example, with an administration site on the cheapest routers. In this article we will try to develop a site on Linux for some smart device and run it on a microcontroller.



To run on a microcontroller, we will use Embox . This RTOS includes a CGI-enabled HTTP server. We will use the HTTP server built into python as an HTTP server on Linux.

python3 -m http.server -d <site folder>
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Static site

Let's start with a simple static site consisting of one or more pages.

Everything is simple here, let's create a folder and index.html in it. This file will be downloaded by default if only the site address is specified in the browser.

$ ls website/
em_big.png  index.html

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

The site will also contain the Embox logo, the “em_big.png” file, which we will embed in the html.



Let's start the http server:

python3 -m http.server -d website/
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Let's go to localhost: 8000 in the browser.







Now let's add our static site to the Embox file system. This can be done by copying our folder to the rootfs / template folder (the current template is in the conf / rootfs folder). Or create a module specifying files for rootfs in it.

$ ls website/
em_big.png  index.html  Mybuild

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Content of Mybuild.

package embox.demo

module website {
    @InitFS
    source "index.html",
        "em_big.png",
}
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

For the sake of simplicity, we'll put our site directly in the root folder (@InitFs annotation with no parameters).



We also need to include our site in the mods.conf configuration file and add the httd server itself there:

    include embox.cmd.net.httpd    
    include embox.demo.website
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Also, let's start the server with our website during system startup. To do this, add a line to the conf / system_start.inc file:

"service httpd /",
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Naturally, all these manipulations need to be done with the config for the board. After that, we collect and run. We go in the browser to the address of your board. In my case, it is 192.168.2.128



And we have the same picture as for the local site.







We are not specialists in web development, but have heard that various frameworks are used to create beautiful websites. For example, AngularJS is often used . Therefore, we will give further examples using it. But at the same time, we will not go into details and apologize in advance if somewhere we have strongly adjusted with web design.



Whatever static content we put in the site folder, for example, js or css files, we can use it without any additional effort.



Let's add app.js (an angular site) to our site and in it a couple of tabs. We will put the pages for these tabs in the partials folder, images in the images / folder, and css files in css /.

$ ls website/
app.js  css  images  index.html  Mybuild  partials
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Let's launch our website.







Agree, the site looks much more familiar and pleasant. And all this is done on the browser side. As we said, the entire context is still static. And we can develop it on the host like a regular website.



Naturally, you can use all the development tools of common web developers. So, opening the console in the browser, we found an error message that the favicon.ico was missing: We







found out that this is the icon that is displayed in the browser tab. You can, of course, put a file with this name, but sometimes you don't want to spend on this place. Let me remind you that we want to run also on microcontrollers where there is little memory.



A search on the Internet immediately showed that you can do without a file, you just need to add a line to the head html section. Although the error did not interfere, it is always pleasant to make the site a little better. And most importantly, we made sure that the usual developer tools are quite applicable with the proposed approach.

Dynamic content

CGI

Let's move on to dynamic content. The Common Gateway Interface (CGI) is an interface for interaction between a web server and command line utilities that allows you to create dynamic content. In other words, CGI allows you to use the output of utilities to generate dynamic content.



Let's take a look at some CGI script:

#!/bin/bash

echo -ne "HTTP/1.1 200 OK\r\n"
echo -ne "Content-Type: application/json\r\n"
echo -ne "Connection: Connection: close\r\n"
echo -ne "\r\n"

tm=`LC_ALL=C date +%c`
echo -ne "\"$tm\"\n\n"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

First, the http header is printed to standard output, and then the data of the page itself is printed. output can be redirected anywhere. You can simply run this script from the console. We will see the following:

./cgi-bin/gettime
HTTP/1.1 200 OK
Content-Type: application/json
Connection: Connection: close

"Fri Feb  5 20:58:19 2021"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

And if instead of the standard output it is socket, then the browser will receive this data.



CGI is often implemented with scripts, even cgi scripts are said. But this is not necessary, it is just that in scripting languages ​​such things are faster and more convenient. A utility providing CGI can be implemented in any language. And since we focus on microcontrollers, therefore, we try to take care of saving resources. Let's do the same in C.

#include <stdio.h>
#include <unistd.h>
#include <string.h>

int main(int argc, char *argv[]) {
    char buf[128];
    char *pbuf;
    struct timeval tv;
    time_t time;

    printf(
        "HTTP/1.1 200 OK\r\n"
        "Content-Type: application/json\r\n"
        "Connection: Connection: close\r\n"
        "\r\n"
    );


    pbuf = buf;

    pbuf += sprintf(pbuf, "\"");

    gettimeofday(&tv, NULL);
    time = tv.tv_sec;
    ctime_r(&time, pbuf);

    strcat(pbuf, "\"\n\n");

    printf("%s", buf);

    return 0;
}
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

If we compile this code and run, we will see exactly the same output as in the case of the script.



In our app.js, let's add a handler to call a CGI script for one of our tabs:

app.controller("SystemCtrl", ['$scope', '$http', function($scope, $http) {
    $scope.time = null;

    $scope.update = function() {
        $http.get('cgi-bin/gettime').then(function (r) {
            $scope.time = r.data;
        });
    };

    $scope.update();
}]);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

A small nuance for running on Linux using the built-in python server. We need to add the --cgi argument to our launch line to support CGI:

python3 -m http.server --cgi -d .
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Automatic updating of dynamic content

Now let's take a look at another very important property of a dynamic site - automatic content updates. There are several mechanisms for its implementation:

• Server Side Includes (SSI)
• Server-sent Events (SSE)
• WebSockets
• Etc

Server Side Includes (SSI)

Server Side Includes (SSI) . It is an uncomplicated language for dynamically creating web pages. Usually files using SSI are in .shtml format.



SSI itself even has control directives, if else, and so on. But in most of the microcontroller examples we found, it is used in the following way. A directive is inserted into the .shtml page that periodically reloads the entire page. This could be, for example:

<meta http-equiv="refresh" content="1">
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Or:

<BODY onLoad="window.setTimeout("location.href='runtime.shtml'",2000)">
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

And in one way or another, content is generated, for example, by setting a special handler.



The advantage of this method is its simplicity and minimal resource requirements. But on the other hand, here's an example of how it looks.







The page refresh (see tab) is very noticeable. And reloading the entire page looks like an overly redundant action.



The standard example is from FreeRTOS - https://www.freertos.org/FreeRTOS-For-STM32-Connectivity-Line-With-WEB-Server-Example.html

Server-sent Events

Server-sent Events (SSE) is a mechanism that allows a half-duplex (one-way) connection between a client and a server. The client in this case opens a connection and the server uses it to transfer data to the client. At the same time, unlike classic CGI scripts, the purpose of which is to generate and send a response to the client, and then complete, SSE offers a “continuous” mode. That is, the server can send as much data as necessary until it either completes itself, or the client closes the connection.



There are some minor differences from regular CGI scripts. First, the http header will be slightly different:

        "Content-Type: text/event-stream\r\n"
        "Cache-Control: no-cache\r\n"
        "Connection: keep-alive\r\n"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Connection, as you can see, is not close, but keep-alive, that is, an ongoing connection. To prevent the browser from caching data, you need to specify Cache-Control no-cache. Finally, you need to specify that the special data type Content-Type text / event-stream is used.



This data type is a special format for SSE :

: this is a test stream

data: some text

data: another message
data: with two lines

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

In our case, the data needs to be packed into the following line:

data: { “time”: “<real date>”}
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Our CGI script will look like:

#!/bin/bash

echo -ne "HTTP/1.1 200 OK\r\n"
echo -ne "Content-Type: text/event-stream\r\n"
echo -ne "Cache-Control: no-cache\r\n"
echo -ne "Connection: keep-alive\r\n"
echo -ne "\r\n"

while true; do
    tm=`LC_ALL=C date +%c`
    echo -ne "data: {\"time\" : \"$tm\"}\n\n" 2>/dev/null || exit 0
    sleep 1
done
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Output if you run the script:

$ ./cgi-bin/gettime
HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive

data: {"time" : "Fri Feb  5 21:48:11 2021"}

data: {"time" : "Fri Feb  5 21:48:12 2021"}

data: {"time" : "Fri Feb  5 21:48:13 2021"}
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

And so on, once a second.



Same thing in C:

#include <stdio.h>
#include <unistd.h>
#include <string.h>

int main(int argc, char *argv[]) {
    char buf[128];
    char *pbuf;
    struct timeval tv;
    time_t time;

    printf(
        "HTTP/1.1 200 OK\r\n"
        "Content-Type: text/event-stream\r\n"
        "Cache-Control: no-cache\r\n"
        "Connection: keep-alive\r\n"
        "\r\n"
    );

    while (1) {
        pbuf = buf;

        pbuf += sprintf(pbuf, "data: {\"time\" : \"");

        gettimeofday(&tv, NULL);
        time = tv.tv_sec;
        ctime_r(&time, pbuf);

        strcat(pbuf, "\"}\n\n");

        if (0 > printf("%s", buf)) {
            break;
        }

        sleep(1);
    }

    return 0;
}

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

And finally, we also need to tell angular that we have SSE, that is, modify the code for our controller:

app.controller("SystemCtrl", ['$scope', '$http', function($scope, $http) {
    $scope.time = null;

    var eventCallbackTime = function (msg) {
        $scope.$apply(function () {
            $scope.time = JSON.parse(msg.data).time
        });
    }

    var source_time = new EventSource('/cgi-bin/gettime');
    source_time.addEventListener('message', eventCallbackTime);

    $scope.$on('$destroy', function () {
        source_time.close();
    });

    $scope.update = function() {
    };

    $scope.update();
}]);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

We launch the site, we see the following:







It is noticeable that, unlike using SSI, the page does not overload, and the data is updated smoothly and pleasantly for the eye.

Demo

Of course, the examples given are not real because they are very simple. Their goal is to show the difference between the approaches used on microcontrollers and in other systems.



We made a small demo with real tasks. Controlling LEDs, receiving real-time data from an angular velocity sensor (gyroscope) and a tab with system information.



The site was developed on the host. It was only necessary to make small plugs to emulate the LEDs and data from the sensor. Sensor data are just random values ​​received through the standard RANDOM

#!/bin/bash

echo -ne "HTTP/1.1 200 OK\r\n"
echo -ne "Content-Type: text/event-stream\r\n"
echo -ne "Cache-Control: no-cache\r\n"
echo -ne "Connection: keep-alive\r\n"
echo -ne "\r\n"

while true; do
    x=$((1 + $RANDOM % 15000))
    y=$((1 + $RANDOM % 15000))
    z=$((1 + $RANDOM % 15000))
    echo -ne "data: {\"rate\" : \"x:$x y:$y z:$z\"}\n\n" 2>/dev/null || exit 0
    sleep 1
done
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

We simply store the state of the LEDs in a file.

#!/bin/python3

import cgi
import sys

print("HTTP/1.1 200 OK")
print("Content-Type: text/plain")
print("Connection: close")
print()

form = cgi.FieldStorage()
cmd = form['cmd'].value

if cmd == 'serialize_states':
    with open('cgi-bin/leds.txt', 'r') as f:
        print('[' + f.read() + ']')

elif cmd == 'clr' or cmd == 'set':
    led_nr = int(form['led'].value)

    with open('cgi-bin/leds.txt', 'r+') as f:
        leds = f.read().split(',')
        leds[led_nr] = str(1 if cmd == 'set' else 0)
        f.seek(0)
        f.write(','.join(leds))
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

The same is trivially implemented in the C variant. If you wish, you can see the code in the repository folder (project / website).



On the microcontroller, of course, implementations are used that interact with real peripherals. But since these are just commands and drivers, they were debugged separately. Therefore, the very transfer of the site to the microcontroller did not take time.



The screenshot running on the host looks like this.







In a short video you can see the work on a real microcontroller. Note that there is not only communication over http, but also, for example, setting the date using ntp from the command line in Embox, and of course handling peripherals.





Independently, everything given in the article can be reproduced according to the instructions on our wiki

Conclusion

In the article, we showed that it is possible to develop beautiful interactive sites and run them on microcontrollers. Moreover, it can be done easily and quickly using all the development tools for the host and then run from on microcontrollers. Naturally, the development of the site can be done by a professional web designer, while the embedded developer will implement the logic of the device. Which is very convenient and saves time to market.



Naturally, you will have to pay for this. Yes SSE will require slightly more resources than SSI. But with the help of Embox, we easily fit into the STM32F4 without optimization and used only 128 KB of RAM. They didn't just check anything less. So the overhead is not that big. And the convenience of development and the quality of the site itself is much higher. And at the same time, of course, do not forget that modern microcontrollers have grown noticeably and continue to do so. After all, devices are required to be more and more intelligent.