Experience in writing IDL for embedded

Foreword

When working with microcontrollers, I often came across binary protocols. Especially when there are multiple controllers. Or bluetooth low energy is used and it is necessary to write a code to process binary data in the characteristic. In addition to the code, clear documentation is always required.





The question always arises - is it possible to describe the protocol somehow and generate code and documentation for all platforms? IDL can help with this.





1. What is IDL

The definition of IDL is pretty simple and already presented on wikipedia





The IDL , or  Interface Definition Language  ( Engl.  Interface the Description the Language  and  Interface Definition the Language ) -  a specification language  to describe  interfaces , syntactically similar to the description of the classes in the language of  the C ++ .





The most important thing in IDL is that it should describe well the interaction interface, API, protocol. It should be clear enough to serve other engineers as documentation.





A bonus is also - generation of documentation, structures, code.





2. Motivation

IDL. , - QFace (https://github.com/Pelagicore/qface), swagger ( IDL, API development tool). : https://www.protlr.com/.





Swagger REST API. . cbor ( json ).





QFace , "" embedded, . , enum-.





, .





IDL "" , . QFace. - .





2.1 QFace

IDL, , :





module <module> <version>
import <module> <version>

interface <Identifier> {
    <type> <identifier>
    <type> <operation>(<parameter>*)
    signal <signal>(<parameter>*)
}

struct <Identifier> {
    <type> <identifier>;
}

enum <Identifier> {
    <name> = <value>,
}

flag <Identifier> {
    <name> = <value>,
}
      
      



jinja2. :





{% for module in system.modules %}
    {%- for interface in module.interfaces -%}
    INTERFACE, {{module}}.{{interface}}
    {% endfor -%}
    {%- for struct in module.structs -%}
    STRUCT , {{module}}.{{struct}}
    {% endfor -%}
    {%- for enum in module.enums -%}
    ENUM   , {{module}}.{{enum}}
    {% endfor -%}
{% endfor %}
      
      



. "" "", . sly IDL .





3. sly

sly - .





. . :





class CalcLexer(Lexer):
    # Set of token names.   This is always required
    tokens = { ID, NUMBER, PLUS, MINUS, TIMES,
               DIVIDE, ASSIGN, LPAREN, RPAREN }

    # String containing ignored characters between tokens
    ignore = ' \t'

    # Regular expression rules for tokens
    ID      = r'[a-zA-Z_][a-zA-Z0-9_]*'
    NUMBER  = r'\d+'
    PLUS    = r'\+'
    MINUS   = r'-'
    TIMES   = r'\*'
    DIVIDE  = r'/'
    ASSIGN  = r'='
    LPAREN  = r'\('
    RPAREN  = r'\)'
      
      



Lexer, tokens



- . - , .





- . . - -/ . - . - .





( ):





class CalcParser(Parser):
    # Get the token list from the lexer (required)
    tokens = CalcLexer.tokens

    # Grammar rules and actions
    @_('expr PLUS term')
    def expr(self, p):
        return p.expr + p.term

    @_('expr MINUS term')
    def expr(self, p):
        return p.expr - p.term

    @_('term')
    def expr(self, p):
        return p.term

    @_('term TIMES factor')
    def term(self, p):
        return p.term * p.factor

    @_('term DIVIDE factor')
    def term(self, p):
        return p.term / p.factor

    @_('factor')
    def term(self, p):
        return p.factor

    @_('NUMBER')
    def factor(self, p):
        return p.NUMBER

    @_('LPAREN expr RPAREN')
    def factor(self, p):
        return p.expr

      
      



. @_



, . sly .





.





: https://sly.readthedocs.io/en/latest/sly.html





4.

yml . sly . . - jinja2 .





, .





, :





    @_('term term')
    def term(self, p):
        t0 = p.term0
        t1 = p.term1
        t0.extend(t1)
        return t0
      
      



, , ";"(SEPARATOR



):





   @_('enum_def SEPARATOR')
    def term(self, p):
        return [p.enum_def]

    @_('statement SEPARATOR')
    def term(self, p):
        return [p.statement]

    @_('interface SEPARATOR')
    def term(self, p):
        return [p.interface]

    @_('struct SEPARATOR')
    def term(self, p):
        return [p.struct]
      
      



. (term



term



) .





:





    @_('STRUCT NAME LBRACE struct_items RBRACE')
    def struct(self, p):
        return Struct(p.NAME, p.struct_items, lineno=p.lineno)

    @_('decorator_item STRUCT NAME LBRACE struct_items RBRACE')
    def struct(self, p):
        return Struct(p.NAME, p.struct_items, lineno=p.lineno, tags=p.decorator_item)

    @_('struct_items struct_items')
    def struct_items(self, p):
        si0 = p.struct_items0
        si0.extend(p.struct_items1)
        return si0

    @_('type_def NAME SEPARATOR')
    def struct_items(self, p):
        return [StructField(p.type_def, p.NAME, lineno=p.lineno)]

    @_('type_def NAME COLON NUMBER SEPARATOR')
    def struct_items(self, p):
        return [StructField(p.type_def, p.NAME, bitsize=p.NUMBER, lineno=p.lineno)]

    @_('decorator_item type_def NAME SEPARATOR')
    def struct_items(self, p):
        return [StructField(p.type_def, p.NAME, lineno=p.lineno, tags=p.decorator_item)]

    @_('decorator_item type_def NAME COLON NUMBER SEPARATOR')
    def struct_items(self, p):
        return [StructField(p.type_def, p.NAME, bitsize=p.NUMBER, lineno=p.lineno, tags=p.decorator_item)]
      
      



- (struct



) (struct_items



). :





  • (type_def



    ), (NAME



    ), (SEPARATOR



    )





  • (type_def



    ), , (COLON



    ), (NUMBER



    - , ),





  • (decorator_item



    ), , ,





  • , , , (COLON



    ), (NUMBER



    - ),





QFace ( protlr) - . - alias.





    @_('DECORATOR ALIAS NAME COLON expr struct SEPARATOR')
    def term(self, p):
        return [Alias(p.NAME, p.expr, p.struct), p.struct]
      
      



:






enum Opcode {
    Start =  0x00,
    Stop = 0x01
};

@alias Payload: Opcode.Start
struct StartPayload {
		...
};

@alias Payload: Opcode.Stop
struct StopPayload {
		...
};

struct Message {
    Opcode opcode: 8;
    Payload<opcode> payload;
};
      
      



, opcode



= Opcode.Start



(0x00) - payload



StartPayload



. opcode



= Opcode.Stop



(0x01) - payload



StopPayload



. .





- . - , git. . flag enum, . , , .





python- . . .





- Solvable



. , . , SymbolType



( ). , reference. jinja enum . Solvable



solve



. .. .





solve :





    def solve(self, scopes: list):
        scopes = scopes + [self]
        for i in self.items:
            if isinstance(i, Solvable):
                i.solve(scopes=scopes)
      
      



, solve



- scopes



. . :





struct SomeStruct {
		i32	someNumber;

		@setter: someNumber;
		void setInteger(i32 integer);
};
      
      



- someNumber



, SomeStruct.someNumber



.





QFace - , . .





examples/uart - , html . uart . , put_u32 - MCU.





: https://gitlab.com/volodyaleo/volk-idl





P.S.

This is my first article on Habr. I would be glad to receive feedback - whether this topic is interesting or not. If anyone has good examples of kodo + doko binary protocol generators for Embedded it would be interesting to read in the comments. Or some successful practice of implementing similar systems for describing binary protocols.





In this project, I did not pay much attention to the speed of work. I did some things to "solve the problem faster". It was more important to get working code that you can already try to apply to different projects.








All Articles