Clever Geek Handbook

SIP phone with GUI on STM32F7

There was one of the coronavirus evenings I spent in self-isolation. The STM32F769I-Discovery board lay on the table . I looked at it and thought, it's a smartphone. There is a screen with a 800x480 touchscreen, there is an audio interface, there is a network interface, even if not wireless. All this is based on a microcontroller, therefore it is more reliable in terms of temperature conditions. And it has less consumption. Only the software is missing. Of course, no Android will even come close to this board. And I decided to try how quickly the functionality required for the phone can be developed for this board on Embox .





My project can be divided into two parts. The first is a phone, and I want to fit it into as few resources as possible. The second is the development of a minimal user interface that allows you to receive a call and communicate.



Phone on board STM32F769I-Discovery



Embox is a configurable OS for embedded systems. A distinctive feature is that it allows you to use Linux software without changing the source code on systems with limited resources.



One of the most popular VOIP phone projects is PJSIP . We will use it for our purposes.



Building PJSIP on Linux



First you need to download, build and run the main part - PJSIP, an open source SIP stack. Download the latest version . At the moment this is version 2.10.



Next, you need to build the project. It's easy to do for your host OS. In my case, it's Linux. 



$ ./configure --prefix=~/pj_build


Here I did not specify any options except prefix, the paths where the compiled libraries and header files will be installed. This is necessary in order to analyze those things that potentially end up in our microcontroller.



Then we execute 



$ make dep
$ make


Running PJSIP on Linux



If everything completed successfully, then we have compiled a PJSIP, as well as a demo application.



Let's start something simple yet functional. We need a call in both directions, take pjsip-apps / src / samples / simple_pjsua.c. This is a simple application with automatic call answering. Let's edit the selected example simple_pjsua.c in order to specify the SIP account. The following lines are responsible for this: 



 #define SIP_DOMAIN   "example.com"
 #define SIP_USER     "alice"
 #define SIP_PASSWD   "secret"


We rebuild and run: 



$ ./pjsip-apps/bin/samples/x86_64-unknown-linux-gnu/simple_pjsua


Something similar should appear: 



15:21:22.181        	pjsua_acc.c  ....SIP outbound status for acc 0 is not active
15:21:22.181        	pjsua_acc.c  ....sip:bob@sip.linphone.org: registration success, status=200 (Registration successful), will re-register in 300 seconds
15:21:22.181        	pjsua_acc.c  ....Keep-alive timer started for acc 0, destination:91.121.209.194:5060, interval:15s


Now you can receive incoming calls.



Building PJSIP on Embox



Let's put the same together on Embox. First, in order not to worry about the amount of memory, we will make an assembly for the Qemu emulator.



Embox has a mechanism for connecting external projects. It allows you to set a link for downloading projects, apply patches if required, and set rules for three stages: configure, build, install.



In order to use this mechanism, it is enough to indicate that in the annotation '@Build' you need to use 'script = $ (EXTERNAL_MAKE)'. 



@Build(stage=2,script="$(EXTERNAL_MAKE) PJSIP_ENABLE_CXX=false")
@BuildArtifactPath(cppflags="-I$(abspath $(EXTERNAL_BUILD_DIR))/third_party/pjproject/core/install/include/")
module core_c extends core {
	depends pjsip_dependencies
}


This is the Makefile used to port the assembly to Embox: 



PKG_NAME := pjproject
PKG_VER  := 2.10

PKG_SOURCES := https://github.com/pjsip/pjproject/archive/$(PKG_VER).tar.gz
PKG_MD5 	:= 13e5c418008ae46c4ce0c1e27cdfe9b5

include $(EXTBLD_LIB)

PKG_PATCHES := pjproject-$(PKG_VER).patch \
	sha256_error_fix-$(PKG_VER).patch \
	addr_resolv_sock-$(PKG_VER).patch
…

DISABLE_FEATURES := \
	l16-codec   \
	ilbc-codec  \
	speex-codec \
	speex-aec   \
	gsm-codec   \
	g722-codec  \
	g7221-codec \
	libyuv \
	libwebrtc

$(CONFIGURE) :
	export EMBOX_GCC_LINK=full; \
	cd $(BUILD_ROOT) && ( \
    	./configure \
        	CC=$(EMBOX_GCC) \
        	CXX=$(EMBOX_GXX) \
        	--host=$(AUTOCONF_TARGET_TRIPLET) \
        	--target=$(AUTOCONF_TARGET_TRIPLET) \
        	--prefix=$(PJSIP_INSTALL_DIR) \
        	$(DISABLE_FEATURES:%=--disable-%) \
        	--with-external-pa; \
	)
	touch $@

$(BUILD) :
	cd $(BUILD_ROOT) && ( \
    	     $(MAKE) dep; \
    	     $(MAKE) MAKEFLAGS='$(EMBOX_IMPORTED_MAKEFLAGS)'; \
	)
	touch $@

$(INSTALL) :
...


As you can see, these are the same configure, make dep, make as for Linux. Of course, when configuring, we indicate that you need to use cross-compilation (--host, --target, CC, CXX) for the target platform.



In addition, one more difference can be noticed. We specify --with-external-pa, that is, we say that for audio you need to use drivers from Embox. The audio drivers in Embox provide a portaudio interface, which is also available on Linux.



As you can see, we have disabled the building of the libyuv and libwebrtc libraries. We will also disable all unnecessary audio codecs in advance except PCMA / PCMU. We check the correctness of the configuration on Linux: 



$ ./configure \
	--prefix=$PREFIX \
	--disable-l16-codec   \
	--disable-ilbc-codec  \
	--disable-speex-codec \
	--disable-speex-aec   \
	--disable-gsm-codec   \
	--disable-g722-codec  \
	--disable-g7221-codec \
	--disable-libyuv \
	--disable-libwebrtc
$ make dep && make


For easier work with the simple_pjsua application, let's move the code to Embox. From the modifications, we will simply transfer the setting of the SIP account parameters from the C-shny code to the file 'simple_pjsua_sip_account.inc', which we will put in the configuration files. That is, to build an application with a different account, you only need to change this file. The content remains the same: 



#define SIP_DOMAIN 	<sip_domain>
#define SIP_USER   	<sip_user>
#define SIP_PASSWD 	<sip_passwd>


Launch the simple_pjsua application as before on Linux. If it works, then PJSIP is configured correctly. These configure options are then easily ported back to the Makefile on Embox.



Final Makefile under the spoiler 

PKG_NAME := pjproject
PKG_VER  := 2.10

PKG_SOURCES := https://github.com/pjsip/pjproject/archive/$(PKG_VER).tar.gz
PKG_MD5 	:= 13e5c418008ae46c4ce0c1e27cdfe9b5

include $(EXTBLD_LIB)

PKG_PATCHES := pjproject-$(PKG_VER).patch \
    sha256_error_fix-$(PKG_VER).patch \
    addr_resolv_sock-$(PKG_VER).patch

ifeq ($(PJSIP_ENABLE_CXX),false)
PKG_PATCHES    += pjsua2_disable-$(PKG_VER).patch
endif

DISABLE_FEATURES := \
    l16-codec   \
    ilbc-codec  \
    speex-codec \
    speex-aec   \
    gsm-codec   \
    g722-codec  \
    g7221-codec \
    libyuv \
    libwebrtc \
    #g711-codec

BUILD_ROOT  := $(BUILD_DIR)/$(PKG_NAME)-$(PKG_VER)
PJSIP_INSTALL_DIR := $(EXTERNAL_BUILD_DIR)/third_party/pjproject/core/install

$(CONFIGURE) :
    export EMBOX_GCC_LINK=full; \
    cd $(BUILD_ROOT) && ( \
   	 ./configure \
   		 CC=$(EMBOX_GCC) \
   		 CXX=$(EMBOX_GXX) \
   		 --host=$(AUTOCONF_TARGET_TRIPLET) \
   		 --target=$(AUTOCONF_TARGET_TRIPLET) \
   		 --prefix=$(PJSIP_INSTALL_DIR) \
   		 $(DISABLE_FEATURES:%=--disable-%) \
   		 --with-external-pa; \
    )
    cp ./config_site.h $(BUILD_ROOT)/pjlib/include/pj/config_site.h
    touch $@

$(BUILD) :
    cd $(BUILD_ROOT) && ( \
   	 $(MAKE) -j1 dep; \
   	 $(MAKE) -j1 MAKEFLAGS='$(EMBOX_IMPORTED_MAKEFLAGS)'; \
    )
    touch $@

$(INSTALL) :
    cd $(BUILD_ROOT) && $(MAKE) install
    # Remove AUTOCONF_TARGET_TRIPLET from file names to use them in Mybuild
    for f in $(PJSIP_INSTALL_DIR)/lib/*-$(AUTOCONF_TARGET_TRIPLET).a; do \
   	 fn=$$(basename $$f); \
   	 cp $$f $(PJSIP_INSTALL_DIR)/lib/$${fn%-$(AUTOCONF_TARGET_TRIPLET).a}.a; \
    done
    # Copy binaries and
    # remove AUTOCONF_TARGET_TRIPLET from file names to use them in Mybuild
    for f in $(BUILD_ROOT)/pjsip-apps/bin/samples/$(AUTOCONF_TARGET_TRIPLET)/*; do \
   	 cp $$f $(PJSIP_INSTALL_DIR)/$$(basename $$f).o; \
    done
    for f in $(BUILD_ROOT)/pjsip-apps/bin/*-$(AUTOCONF_TARGET_TRIPLET); do \
   	 fn=$$(basename $$f); \
   	 cp $$f $(PJSIP_INSTALL_DIR)/$${fn%-$(AUTOCONF_TARGET_TRIPLET)}.o; \
    done
    touch $@




Final Mybuild under the spoiler 
package third_party.pjproject

module pjsip_dependencies {
    depends embox.net.lib.getifaddrs

    depends embox.compat.posix.pthreads
    depends embox.compat.posix.pthread_key
    depends embox.compat.posix.pthread_rwlock
    depends embox.compat.posix.semaphore
    depends embox.compat.posix.fs.fsop
    depends embox.compat.posix.idx.select
    depends embox.compat.posix.net.getaddrinfo
    depends embox.compat.posix.net.gethostbyname
    depends embox.compat.posix.util.gethostname

    depends embox.compat.posix.proc.pid
    depends embox.compat.posix.proc.exit
    depends embox.compat.libc.stdio.fseek
    depends embox.compat.posix.time.time

    depends embox.kernel.thread.thread_local_heap

    depends embox.driver.audio.portaudio_api
}

@DefaultImpl(core_c)
abstract module core { }

@Build(stage=2,script="$(EXTERNAL_MAKE) PJSIP_ENABLE_CXX=false")
@BuildArtifactPath(cppflags="-I$(abspath $(EXTERNAL_BUILD_DIR))/third_party/pjproject/core/install/include/")
module core_c extends core {
    depends pjsip_dependencies
}

/* Currently not used. It will be used for PJSUA2 if required. */
@Build(stage=2,script="$(EXTERNAL_MAKE) PJSIP_ENABLE_CXX=true")
@BuildArtifactPath(cppflags="-I$(abspath $(EXTERNAL_BUILD_DIR))/third_party/pjproject/core/install/include/")
@BuildDepends(third_party.STLport.libstlportg)
module core_cxx extends core {
    depends pjsip_dependencies
    depends third_party.STLport.libstlportg
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpjsip {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libpjsip.a",
   		 "libpjsip-simple.a",
   		 "libpjsip-ua.a"

    @NoRuntime depends core
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpjsua {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libpjsua.a"

    @NoRuntime depends core
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpjlib_util {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libpjlib-util.a"

    @NoRuntime depends core
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpj {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libpj.a"

    @NoRuntime depends core
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpjmedia {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libpjmedia.a",
   		 "libpjmedia-codec.a",
   		 "libpjmedia-audiodev.a"

    @NoRuntime depends core
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpjnath {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libpjnath.a"

    @NoRuntime depends core
}

@BuildDepends(core)
@Build(stage=2,script="true")
static module libpj_third_party {
    @AddPrefix("^BUILD/extbld/third_party/pjproject/core/install/lib")
    source "libresample.a",
   		 "libsrtp.a"

    @NoRuntime depends core
}

@BuildDepends(libpjsua)
@BuildDepends(libpjsip)
@BuildDepends(libpjmedia)
@BuildDepends(libpj)
@BuildDepends(libpjlib_util)
@BuildDepends(libpjnath)
@BuildDepends(libpj_third_party)
@Build(stage=2,script="true")
static module libpj_all {
    @NoRuntime depends libpjsua,
   		 libpjsip,
   		 libpjmedia,
   		 libpj,
   		 libpjlib_util,
   		 libpjnath,
   		 libpj_third_party
}

@AutoCmd
@Cmd(name="streamutil", help="", man="")
@BuildDepends(core)
@Build(stage=2,script="true")
module streamutil {
    source "^BUILD/extbld/third_party/pjproject/core/install/streamutil.o"
    depends core
}

@AutoCmd
@Cmd(name="pjsua", help="", man="")
@BuildDepends(core)
@Build(stage=2,script="true")
module pjsua {
    source "^BUILD/extbld/third_party/pjproject/core/install/pjsua.o"
}

@AutoCmd
@Cmd(name="pjsip_simpleua", help="", man="")
@BuildDepends(core)
@Build(stage=2,script="true")
module simpleua {
    source "^BUILD/extbld/third_party/pjproject/core/install/simpleua.o"
    depends core
}




Now you can receive incoming calls, but on Embox.



Launching PJSIP on STM32F769I-Discovery



It remains to change the Embox configuration from PJSIP for QEMU to the configuration for a specific board - STM32F769I-Discovery. To configure Embox you need several components:



  • Compilation flags file (build.conf).
  • Linker file describing what memory is available and how the final image (lds.conf) will be located in them.
  • Embox modules configuration file (mods.conf).
  • PJSIP configuration.


The first two points are usually easy to figure out. These are compiler and linker options, and they rarely change from project to project for the same board. Except perhaps for the compilation flags. The main work on setting the characteristics of the final system will be done in the third and fourth paragraphs.



Let's start by looking at the Embox configuration. What is the difference here from running on Linux? On Linux we had an almost infinite amount of memory, we didn't care about the number of tasks, the amount of memory allocated, etc. Now we have only 2MB of ROM and 512MB of RAM, excluding external memory. Accordingly, it is necessary to set how many resources we need for specific needs.



For example, PJSIP runs on its own thread. For each new connection there is another stream. And one more thread for working with audio. Thus, even with one connection, we need at least 3 threads. Next, we want to add DHCP - we select one more stream. In total, already 4. All this is naturally transferred to the configuration: 



include embox.kernel.thread.core(thread_pool_size=5,thread_stack_size=12000)


We have set the stacks to a fixed size. You can ask different things. It all depends on the task.



Next, we select the number of required packages: 



	include embox.net.skbuff(amount_skb=28)
	include embox.net.skbuff_data(amount_skb_data=28)


Set the heap size (where malloc () works from): 



	include embox.mem.heap_bm
	include embox.mem.static_heap(heap_size=0x3C000)


Otherwise, the configuration remains the same as on QEMU.



Finding out the heap size



The main question that arises when drawing up a configuration is how to choose the necessary parameters? For example, why the heap is 0x3C000, the number of network packets is 28, and the stack is 12Kb? I often take the following approach. The first step is to deal with the stacks and the heap. A bunch of things can be explored for a start on Linux using Valgrind. You can use the Valgrind-Massif profiler for this. It works on “snapshots” at certain points in time and shows which function requested how much memory.



Launch valgrind with our application: 



$ valgrind --tool=massif --time-unit=B --massif-out-file=pjsip.massif ./pjsip-apps/bin/samples/x86_64-unknown-linux-gnu/simple_pjsua


After running the application, we visualize the data using the massif-visualizer: 



$ massif-visualizer pjsip.massif






Here you can see that memory is spent not only on PJSIP, but also on the standard library, as well as libasound (this is the host sound - ALSA). PJSIP itself asks for dimensions from the bottom red subplot. This is at the peak of 600 Kb, and during the connection about 320 Kb. Our board has 512kB RAM. Therefore, we are trying to configure PJSIP, reducing memory consumption ...



I made the following configuration: 



#define PJ_LOG_USE_STACK_BUFFER    	0

#define PJ_LOG_MAX_LEVEL 6

#define PJ_POOL_DEBUG    	0
#define PJ_HAS_POOL_ALT_API  0

/* make PJSUA slim */
#define PJSUA_MAX_ACC 3
#define PJSUA_MAX_CALLS 1
#define PJSUA_MAX_VID_WINS 0
#define PJSUA_MAX_BUDDIES 1
#define PJSUA_MAX_CONF_PORTS 4
#define PJSUA_MAX_PLAYERS 1
#define PJSUA_MAX_RECORDERS 1

/* Changing to #if 0 will increase memory consumption
 * but insreases communication speed. */
#if 1
	/* This sample derived from pjlib/include/pj/config_site_sample.h: */
	#define PJ_OS_HAS_CHECK_STACK	0
	#define PJ_ENABLE_EXTRA_CHECK	0
	#define PJ_HAS_ERROR_STRING  	0
	#undef PJ_IOQUEUE_MAX_HANDLES
	#define PJ_IOQUEUE_MAX_HANDLES   8
	#define PJ_CRC32_HAS_TABLES  	0
	#define PJSIP_MAX_TSX_COUNT  	15
	#define PJSIP_MAX_DIALOG_COUNT   15
	#define PJSIP_UDP_SO_SNDBUF_SIZE 4000
	#define PJSIP_UDP_SO_RCVBUF_SIZE 4000
	#define PJMEDIA_HAS_ALAW_ULAW_TABLE  0
#endif


Copy it to PJSIP into pjlib / include / pj / config_site.h file. We rebuild and run. Analyzing the result:







At the peak it is already about 300 KB, which can fit on the board.



Next, I put a heap of about 300 KB in Embox and set the debug pools to see if something overflows (note that as a result, the heap size was reduced to 240 KB). Debugging of pools is enabled with the option: 



#define PJ_POOL_DEBUG    	1


in the same pjlib / include / pj / config_site.h.



Okay, all that remains is to configure the thread stacks and the number of network packets. Here you need to correctly distribute the remaining resources. For example, if there are too few network packets, then the sound will simply "choke". If you allocate too many packages, then there will be nothing left for the stacks. The priority is of course the stacks. If the stack goes bad, everything is gone.



Thus, we start with the maximum possible stack size and then reduce it until the software is running under load. If we catch damage to the stack, we stop. We put interrupts on a separate stack to minimize unpredictability. That is, your stack is only for your program. 



@Runlevel(0) include embox.arch.arm.armmlib.exception_entry(irq_stack_size=1024)


After that, we give the remaining resources to network packets. As I mentioned above, we got 28 of them.



Everything, the first part was successfully completed. Simple_pjsia application runs successfully on STM32F769I-Discovery in 512KB internal memory.



We are finalizing the SIP phone. We add the user interface.



After successfully launching the console version, you need to somehow add the user interface. For simplicity, we will assume that it includes the following. When the application starts, there should be some kind of explanatory inscription on the screen. For example, “PJSIP DEMO”. If there is an incoming call, the screen displays where the call came from, and two buttons with icons appear - “Accept”, “Decline”. The call can be either accepted or rejected. If the call is accepted, the conversation starts, the contact information about the subscriber is displayed, and one button remains on the screen - "Hang". If the call was initially rejected - everything is trivial here - we return to the initial picture with “PJSIP DEMO”.



Here's an example of how this should look.







Development of a prototype on Linux



Since Embox already had support for Nuklear, I decided to use this project. Although we already have a working console version of the phone on the microcontroller, it is important here that it is much easier to modify the UI on Linux, as it was already with the PJSIP setting above.



To do this, let's take two examples. The first example is simple_pjsua from PJSIP. The second example is demo / x11_rawfb / from Nuklear. Now our task is to make them work together under Linux.



The first thing I did was to replace the automatic PJSIP call answer with an external event (such as a button press). Next, I wrote the logic on Nuklear.



In the process, it turned out that for some reason the icons were not drawn inside the buttons. In the picture below, you can see the phone icons inside the green and red buttons. These are ordinary pictures on which everything is 100% transparent except for the telephone receiver. At the same time, initially only white squares were drawn instead of them. It was about the rawfb plugin implementation. Apparently, it is not very popular, so only the cursor is drawn in it. I have added some code that simply copies the contents of the image to the correct Nuklear memory region.



As a result, after a day of work on the project, I got the following:







Knowing that the STM32F76I-Discovery has a screen size of 800x480, and in QEMU 800x600, I immediately set the required dimensions in Nuklear, so that it would be easier to navigate in creating dynamic labels and buttons. The resulting code is as follows: 



	if (nk_begin(ctx, "Demo", nk_rect(0, 0, WIN_WIDTH, WIN_HEIGHT),
        	NK_WINDOW_NO_SCROLLBAR)) {
    	int answer_pressed = 0, decline_pressed = 0;

    	if (!draw_mouse) {
        	nk_style_hide_cursor(ctx);
    	}

    	nk_layout_row_static(ctx,
        	(WIN_HEIGHT - CALL_BTN_HEIGHT - 2 * CALL_INFO_TEXTBOX_HEIGHT - WIN_HEIGHT / 4), 15, 1);

    	nk_layout_row_dynamic(ctx, CALL_INFO_TEXTBOX_HEIGHT, 1);

    	nk_style_set_font(ctx, &rawfb_fonts[RAWFB_FONT_DEFAULT]->handle);

        switch (call_info->state) {
        case CALL_INACTIVE:
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 56;
            nk_label(ctx, "PJSIP demo", NK_TEXT_CENTERED);
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 32;
            break;
        case CALL_INCOMING:
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 32;
            nk_label(ctx, "Incoming call from:", NK_TEXT_CENTERED);
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 38;
            nk_label(ctx, call_info->incoming, NK_TEXT_CENTERED);
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 32;
            break;
        case CALL_ACTIVE:
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 32;
            nk_label(ctx, "Active call:", NK_TEXT_CENTERED);
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 38;
            nk_label(ctx, call_info->remote_uri, NK_TEXT_CENTERED);
            rawfb_fonts[RAWFB_FONT_DEFAULT]->handle.height = 32;
            break;
        }

        if (call_info->state != CALL_INACTIVE) {
            nk_layout_row_static(ctx, (WIN_WIDTH - 9 * 4) / 9, (WIN_WIDTH - 9 * 4) / 9, 9);

            switch (call_info->state) {
            case CALL_INCOMING:
                nk_spacing(ctx, 2);
                demo_nk_accept_btn(ctx, im_accept, &answer_pressed);
                nk_spacing(ctx, 3);
                demo_nk_decline_btn(ctx, im_decline, &decline_pressed);
                nk_spacing(ctx, 2);
                break;
            case CALL_ACTIVE:
                nk_spacing(ctx, 4);
                demo_nk_decline_btn(ctx, im_decline, &decline_pressed);
                nk_spacing(ctx, 4);
                break;
            default:
                break;
            }
        }

    	if (answer_pressed && call_info->state == CALL_INCOMING) {
        	demo_pj_answer();
    	}
    	if (decline_pressed) {
        	demo_pj_hang();
    	}
	}
	nk_end(ctx);


Launch on board



It remains to transfer the project first to QEMU, and then to the board. We already have everything ready for the console version, so we'll just transfer the new application from Linux. To do this, just create a Mybuild file in the Embox build system: 



@AutoCmd
@Cmd(name="sip_nuklear", help="", man="")
@BuildDepends(third_party.pjproject.libpj_all)
@BuildDepends(third_party.lib.nuklear)
@Build(stage=2)
module sip_nuklear {
	@InitFS
	source "icons/phone-accept-80.png",
       	"icons/phone-decline-80.png",
       	"fonts/Roboto-Regular.ttf"

	source "main.c"

	source "nuklear_main.c"

	@IncludePath("$(CONF_DIR)")
	@DefineMacro("PJ_AUTOCONF=1")
	source "pjsua.c"

	@NoRuntime depends third_party.pjproject.libpj_all
	@NoRuntime depends third_party.lib.nuklear
	depends embox.driver.input.core
	depends rawfb_api
}


As you can see, the sources are listed. Icons and fonts are located in the internal file system and will be available as regular read-only files. Also added dependencies on pjsip and nuklear libraries.



After running the application on the board, I noticed that the default font from Nuclear looks terrible on the STM32F769I screen. Some of the letters were simply lost. For example "1" looked like "|" and "m" looked like "n". I had to connect fonts from ttf files - Roboto-Regular.ttf. This font takes up about 150 KB of flash memory, but the text is legible.



After checking on Linux, I decided that this is a small fee. And I tried to use different font sizes 32 and 38. But I got a segfault. In the end, I gave up the idea of ​​loading multiple font sizes from a file, and only loaded the 32nd and scaled it.



Features of launching on hardware



Let's go back to the board launch. It is important to understand here that in the case of the UI, a framebuffer is needed. Since we want full-screen mode, and the screen is 800x480, even with a 1-byte RGB palette, we needed 800 * 480 * 1 = 384000 bytes, that is, 375 KB. Considering that we have already occupied almost the entire 512 KB internal memory for the needs of PJSIP, it will not work to find a place for the framebuffer. For this reason, we will use SDRAM. 16 MB are available on STM32F76I-Discovery. Since we are already using external memory, we will not save much and put RGBA 32 bits. Thus, the framebuffer will be 800 * 480 * 4 = 1536000 bytes or 1.5 MB.



In our configuration, SDRAM is located at address 0x60000000. We specify it as the address of the framebuffer. 



	@Runlevel(1) include embox.driver.video.stm32f7_lcd(
    	fb_base=0x60000000, width=800, height=480, ltdc_irq=88, bpp=32
	)
	include embox.driver.video.fb


I have already covered the effects of flickering when using one buffer in another article. Therefore, we will take into account that the system uses double buffering, and, therefore, we need additional memory for another 1.5 MB buffer. In addition, the fonts will require another 256 KB. In total, you need to increase the heap by 2 MB. We also place it in external memory: 



	@Runlevel(2) include embox.driver.input.touchscreen.stm32f7cube_ts
	@Runlevel(2) include embox.driver.input.input_dev_devfs


Now the touchscreen will be the / dev / stm32-ts device in the devfs file system in Embox, and you can work with it through the usual open () / read ().



This completes the configuration. Why almost? In fact, we took into account all the nuances from memory, but did not take into account performance. If in the case of PJSIP the sound was transmitted well, then when you try to launch it with graphics, it will choke. This effect is, of course, very difficult to debug on Linux. But it turned out to be enough just to enable the caches on our board. 



	@Runlevel(0) include embox.arch.arm.armmlib.armv7m_cpu_cache(
    	log_level=4,
    	sram_nocache_section_size=0x10000
	)


In Embox, network packet descriptors and data, as well as audio buffers that DMA works with, are located in a special section of memory, marked as non-cacheable memory in the MPU. This is required so that the state of objects in this memory was always correct for both the CPU and DMA.



As a result, we get a very simple working SIP phone with a UI with buttons that works quite well.



Below I tried to depict the final memory allocation:







Development on a host system



My development process boiled down to what is shown in the figure.







And it took very little time. One day for an application on Linux and one more day for improvements on the chosen platform. Yes, Embox already had display, network card and audio drivers for this card. But the development of these parts also takes a little time, no more than a week for each driver. It takes much more time to develop such functionality directly on the board. In our case, most of the functionality is developed under a convenient host system environment. This allowed us to significantly reduce the development time.



There is a video of the results at the beginning of the article. If you wish, you can reproduce everything yourself according to the instructions on the wiki .


More articles:


All Articles