OpenWrt batch system device

The OpenWrt operating system is commonly used as firmware for routers. The typical application is to set and forget. But if suddenly something is not enough for you, then you will have to understand the distribution kit.

OpenWrt uses opkg as its package manager, or rather its own fork . Debian engineers will find it familiar in many ways: similar commands, similar repository and package format.



I wanted to patch LUCI (this will not be in the article), but I did not find an adequate quick introduction, I had to independently collect fragments of information from scattered documentation, articles and examples, looking at the code and the results of the work. As a bonus, I collected a primitive (but useless in practice) package that is not yet in the repository. I share the collected educational program below.

Repository device

There is a file in the OpenWrt file system /etc/opkg/distfeeds.conf



that specifies the system (provided by the OpenWrt and opkg developers) list of repositories. Own and third-party repositories can be specified in /etc/opkg/customfeeds.conf



. The format is one-line, consists of three words:

1. src
   
   
   
   or src/gz
   
   
   
   , it depends on Packages
   
   
   
   whether the file will be downloaded or Packages.gz
   
   
   
   . Judging by the code, there are other options for the first word, but I have not found any repositories for which this would be relevant. Despite the src
   
   
   
   name, this is a repository for binary packages. Opkg does not have a special repository format for source packages similar to that used in Debian / APT.
2. The name of the repository or "feed" in opkg / OpenWrt terminology.
3. The URL that contains the Packages
   
   
   
   / Packages.gz
   
   
   
   .

When executed , or is added opkg update



to the URL , the list of packages and signatures are saved in , the file is named according to the second word in the list of repositories. There are two important conclusions from this: /



Packages



Packages.gz



/tmp/opkg-lists

1. On reboot, the cache will be cleared. On embedded systems like routers, this makes perfect sense.
2. You /etc/opkg/customfeeds.conf
   
   
   
   can override system feeds with your own, giving them the same name. opkg will swear, but swallow the override, adding the desired file instead of the one loaded earlier.

At the same time, it will be loaded Packages.sig



, which should contain the hash of the unpacked list of packages. The list itself simply lists the packages, there are several values ​​for each package, the values ​​for different packages are separated by an empty line. Here are the most important fields for each package:

• Package
  
  
  
  , package name;
• Version
  
  
  
  , version, if there are several packages with the same name, you can select the version, the latest will be installed by default;
• Depends
  
  
  
  , depending on other packages, the package manager will install the listed packages if they are not present in the system;
• Filename
  
  
  
  , the path to the file is relative to the base URL of the repository, usually the repository is flat and everything is in the same place as `Packages.gz`;
• SHA256sum
  
  
  
  the package hash declared by the repository.

If you want more details, you can simply read one of these lists right in your browser .

Binary packages

Binary packages are almost identical to Debian packages. The difference is as follows:

1. Extension .ipk
   
   
   
   instead of .deb
   
   
   
   .
2. Everything is packed with tar and compressed with gzip
   
   
   
   , the same is true for nested archives. In Debian, the top-level archive is packaged in a more primitive way ar
   
   
   
   , and nested archives often have an extension .tar.xz
   
   
   
   , tools are used appropriately.

If you change the package extension for OpenWrt to .tar.gz



and unpack it, you will find inside two archives and one text file. The file is named debian-version



, it contains the version of the binary file format and is not very interesting to us, in modern systems this version is always equal to 2.0



.



The archive data.tar.gz



contains executable files, configuration files and everything for which the package is installed. If you unpack it to the root of the FS, you will get all the expected files in the right places, in /usr/bin/



, /etc/



and so on.



And in control.tar.gz



contains auxiliary files for the package manager. Are scripts that must be performed before or after the installation and removal ( preinst



, postinst



, prerm



, postrm



), information about the files is the configuration and meta-information about the package, was very similar to the one that is contained in Packages



.

Package Build System

The assembly system (aka SDK) is made in the form of a Make-framework. The framework does not imply that you will build packages separately, its main task is to build entire repositories.



The SDK x86_64



is in git . There is an archive (the link will be outdated soon, but it is not difficult to find a fresh one) that will save you time on compiling the toolchain for assembly. Inside, the file is of particular interest feeds.conf.default



. The format is simple, separated by a space:

1. Keyword src-git
   
   
   
   . Not only git is supported , but now there are no repositories in other VCS.
2. Feed name.
3. A git repository URL where you can specify a commit or tag. If you know the name of such a specification, please tell me.

The repository with packages itself is as simple as possible: at the root of the repository there is a package category, at the second level there is a directory with the package name, and inside it is Makefile



, optionally, `Config.in` for additional options during execution make menuconfig



and a subdirectory patches



with the corresponding content. For the simplest package, only Makefile



. For example, you can look in the mirror of the main repository .

Test build

I tried building GNU Hello to test how the SDK works. This is a relatively monstrous Hello World, written in strict accordance with the GNU project guidelines, its sole purpose is to illustrate those guidelines. I did not create a separate repository for it, but instead "slipped" it into the basic SDK packages, from where I compiled it.



For operation of the SDK Surrounded by the Debian packages needed libncurses-dev



(for the assembly of the menu), build-essential



(of GCC and other standard programs, depending on the C), gawk



, unzip



, file



, rsync



and python3



. Also, to create a repository from collected packages, you will need a utility for generating keys usign



... It is not in the repository, so you will additionally need `cmake` for building. This tool can be replaced with both GPG and GPG, signify-openbsd



but it is recommended and developed by the OpenWrt project and is much more pleasant to use.



We compile and install usign



:

git clone https://git.openwrt.org/project/usign.git
cd usign
cmake .
make
sudo make install 
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Instead of setting ( sudo make install



), you can simply remember where the binar is in order to further pull it with your hands.



Now the basic SDK setup:

git clone https://git.openwrt.org/openwrt/openwrt.git
cd openwrt
./scripts/feeds update -a
./scripts/feeds install -a
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Let me remind you that instead of cloning from git, you can download and unpack the archive with the precompiled toolchain. Don't forget to just download the latest version!



When executing, ./scripts/feeds update -a



we clone / update all repositories from feeds.conf (.default), check dependencies and prepare a directory staging_dir/host/bin



with executable files (these are mainly symlinks to system utilities). The following command ./scripts/feeds install -a



,, shoves symlinks into package/feeds



, from where they will be taken for compilation. These two commands are not required to build my custom package.



Next is executed make menuconfig



... You can skip it, but when compiling the package, it will still display the appropriate window. In it, it is enough to change the target and subtarget so that everything is compiled under x86_64 and exit, agreeing to save the config. You will also need to collect auxiliary assembly tools ( make tools/install



) and toolchain ( make toolchain/install



). If you downloaded the SDK from the archive, then make menuconfig



you will not be shown options for choosing a target, and assembling the toolkit and toolchain is not required - everything is already in place.



Now I create a directory package/devel/hello



in which I place the Makefile



following content:





Basically, everything should be clear without explanation. The framework files are connected, the main parameters of the package are described, @GNU



replaced with the mirrors of the GNU project (defined in the framework), and the path consists of two parts: PKG_SOURCE_URL



which specifies the base URL for all versions and is expanded by concatenating the file name from PKG_SOURCE



with a slash. It Package/hello/install



contains instructions for assembling binaries into an archive data.tar.gz



. Additional build options, if required, are available in the documentation . By the way, do not forget that make is very picky about indentation, I had single tabs instead of leading spaces.



Call again make menuconfig



, check that the hello package is marked in the indicated section (Development in my case) and exit saving the config. Finally, we assemble the package in three steps; downloading, unpacking and actually compiling:

make package/hello/download
make package/hello/prepare
make package/hello/compile
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

As a result, I received a package bin/packages/x86_64/base/hello_2.9-1_x86_64.ipk



. You can build a repository. We generate a key pair ( usign -G -c 'openwrt test repo' -s key-build -p key-build.pub



, the private key must be called `key-build`), and collect the repository: make package/index



. At this stage, the assembly can swear to the absence usign



in the directory with auxiliary tools, I decided to symlink problem: ln -s `which usign` staging_dir/host/bin/usign



. Now next to the package is the complete set required for the repository.

Checking the repository along with the package

You can check everything on a real router (just remember to choose the correct target), but I used Docker. In Dokerhabe have OpenWrt image for x86_84, which can be run, icing inside the container directory with the SDK: sudo docker run -it --name openwrt_test -v $PWD:/opt openwrtorg/rootfs



. Poke the enter button until the Bash prompt appears.



I copy the key from the forwarded directory (the cp /opt/key-build.pub /etc/opkg/keys/usign -F -p /opt/key-build.pub



name of the key must necessarily match the identifier), add my local repository ( echo src/gz local file:///opt/bin/packages/x86_64/base >> /etc/opkg/customfeeds.conf



), update the repository ( opkg update



). The conclusion begins with encouraging text, everything is signed correctly:

# opkg update
Downloading file:///opt/bin/packages/x86_64/base/Packages.gz
Updated list of available packages in /var/opkg-lists/local
Downloading file:///opt/bin/packages/x86_64/base/Packages.sig
Signature check passed.
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

It remains only to install and check:

root@34af2f6e849b:/# opkg install hello
Installing hello (2.9-1) to root...
Downloading file:///opt/bin/packages/x86_64/base/hello_2.9-1_x86_64.ipk
Configuring hello.
root@34af2f6e849b:/# hello
Hello, world!
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Hurray, it's done! Despite the documentation scattered throughout the articles, the process of building packages is pretty straightforward.