Building Custom Embedded Linux Distributions

Thursday, September 29, 2016

This blog post will give an overview on the following subjects:

  • Why use a custom embedded Linux image
  • The build environment
  • Qt application development
  • The GitHub tool
  • Creating and editing layers and recipes
  • Building the image
Introduction

LinuxPrecompiled Linux images and distributions for embedded systems are very common among the maker movement. These distributions already have the components (sometimes more than necessary), so that the students, hobbyists, and enthusiasts can easily start to develop. Ubuntu, Debian, and Arch are a few among such distributions.

Unfortunately, a precompiled distribution for embedded Linux systems may not be the best option when we need a customized and application-specific Linux image for a Computer on Module or industrial product. There is a whole process of building images which can be done to discard any undesirable item that could increase boot time, compromise processing speed, and waste memory from our image. Many times we waste the system resources because we have a lot of unnecessary packages installed and services running. For example, in headless applications, where we don't need a desktop environment, we could build a console based distribution and thus, have a faster and lighter Linux distribution. Another downside of using a precompiled distribution is licensing. Canonical (the company behind Ubuntu), for example, does not allow someone to customize and sell Ubuntu without the proper certifications and partnership. On the other hand, with a customized Linux distribution we can have the total control of installed packages or used licenses. Therefore we can have a Linux image optimized according to the project's software or hardware requirements.

But what if we need to include in our distribution an application developed by us in Qt, or even a binary of an application developed in C? Is it necessary to first compile the application and then copy it to the board? Is it necessary to create packages like .ipk or .deb and copy it to the system? How can one include an application in "IMAGE_INSTALL_append" which is found inside the "local.conf" file? How to make an application to automatically startup after the system boot, just as seen on embedded devices?

In this blog post, we will show how to make all those things in an automated manner using the tools of OpenEmbedded/Yocto build system. We will have a quick look behind the steps used by bitbake like compiling, package installation, and directory creation as well as the addition of a service that automatically starts an application after the operational system boots. At the end we will have a custom embedded Linux distribution ready for our product or Computer on Module!

Although the details shown may vary from one development platform or SBC to another, the principles described here can be applied in general.

To follow the steps found below, it is necessary to have a build environment configured for building embedded Linux images. A tutorial for this configuration can be found at Toradex’s Developer Portal on this link. Toradex uses the OpenEmbedded-core build system for building images. Basically, the tutorial covers:

  • Prerequisites of installation
  • Repo installation
  • Download Toradex BSP version 2.5
We have a directory structure like the following:
oe-core/
+-- build
¦   +-- conf
¦   +-- downloads
¦   +-- out-glibc
¦   +-- sstate-cache
+-- stuff
    +-- meta-angstrom
    +-- meta-browser
    (... other layers)
    +-- openembedded-core
Since our application is based on Qt, we need to add meta-qt5 layer inside the stuff folder. To do that, just execute the following command inside the stuff folder:
~/oe-core/stuff/ 
$ git clone https://github.com/meta-qt5/meta-qt5.git -b fido

With all that, we can build our own custom embedded Linux distribution!

Creating applications with Qt Creator

For the purpose of our post, we developed an application following the dual-display (or dual-screen) style, which is actually two applications running on different screens. This kind of application is widely found, for example, in airport check-in kiosks or inside modern cars, where we have a screen with an instrumentation cluster behind the steering wheel and another screen on the panel with media functionalities, GPS maps, etc.

This post does not cover details about configuring Qt for cross-compiling applications. Information about that is well documented at Toradex’s Developer Portal on this link.

The source code of both the applications developed can be found at GitHub. Remember that when we build our image, both applications will be downloaded and compiled automatically following the instructions of the recipe we will write later on.

IMPORTANT: To determine where the applications will be installed when the image is built, the following code in Red needs to be added to the ".pro" project file in Qt, just like the following:
#-------------------------------------------------
#
# Project created by QtCreator 2015-10-13T16:25:09
#
#-------------------------------------------------

QT       += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

TARGET = screen1
TEMPLATE = app

SOURCES += main.cpp\
       screen1.cpp

HEADERS  += screen1.h

FORMS    += screen1.ui

RESOURCES += \
   resources.qrc

# deployment on Linux unix { target.path = /usr/bin/ INSTALLS += target \ }
Syncing our applications with GitHub

We choose to use GitHub for the fact that it is has version control tools and also because it is a cloud platform, so anyone can have access to the projects and applications stored on it. There is a "private repository" option as well. Later on, we will see the recipe to download our applications from GitHub, so they can be automatically installed on our custom Linux image. For that to be done, it is necessary to sync our local directory, where the applications are found, with a GitHub repository. We need to create one repository for each application.

From the point where we already have an account created, we need to add a repository. Click "+" on the upper right corner Account Creation and then click on "New Repository". A new page will load. Choose a name, add a description, and click "Create repository".

Page where a new repo is created
Image 1: Page where a new repo is created

On the next page, GitHub gives us a few options. For our convenience, we will choose the following:

Commands for synchronization
Image 2: Commands for synchronization

The above commands should be executed on the host computer inside each of our Qt application's directory: screen1 and screen2. Remember to edit the URL with your GitHub username and repository name.

~/myQtApplications/app-artigo-screen1/ 
$ git remote add origin https://github.com/giobauermeister/app-artigo-screen1.git

After the push command, type in your GitHub username and password. Then the projects will be uploaded. The same process should be done for the other application. Go ahead and enter your GitHub profile and you should see your new repositories.

GitHub repositories tab
Image 3: GitHub repositories tab
Creating our layer and recipes

What is a recipe? According to Yocto Reference Manual, recipes are the files terminated with .bb suffix. Basically the recipe contains information about given software. This information includes from where to fetch sources, patches to be applied, how to compile the source code, and how to package everything at the end of the process.

A good habit while adding a new recipe to a build environment is to put it inside a new layer. Layers are often organized by sets of meta-data, according to machine types, functionalities, or similar items. We could take meta-toradex layer as an example. Toradex uses layers to give its customers access to Board Support Packages (BSP's), customized kernel, U-boot, graphical features, and a lot more. Other known layers are, meta-beagleboard, meta-fsl-arm, and meta-intel-galileo. We have also found some interesting layers like meta-games, meta-maker, and meta-uav for drones. A vast list of layers can be found here. As an example we will create a layer called "meta-projects".

Inside the oe-core directory, we find the stuff folder. Inside the stuff folder, we find the layers including the layer meta-toradex. Create a new folder called meta-projects inside stuff.
~/oe-core/stuff/ 
$ mkdir meta-projects
Create a new folder called "conf" inside meta-projects
~/oe-core/stuff/meta-projects/ 
$ mkdir conf
Create a new file inside conf, called layer.conf, with the following content inside the file:
~/oe-core/stuff/meta-projects/conf/
$ vi layer.conf
# We have a conf and classes directory, add to BBPATH
BBPATH .= ":${LAYERDIR}"

# We have recipes-* directories, add to BBFILES
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
     ${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "meta-projects"
BBFILE_PATTERN_meta-projects = "^${LAYERDIR}/"
BBFILE_PRIORITY_meta-projects = "6"
The above content is minimalist and default for a layer configuration file. Note our layer name in Red.
Recipes are found inside directories organized according to type of application, category of software, etc. below the layer meta-toradex:
~/oe-core/stuff/meta-toradex/ 
$ ls
total 84
buildconf
classes
conf
recipes
recipes-benchmark
recipes-bsp
recipes-connectivity
recipes-core
recipes-devtools
recipes-fsl
recipes-gnome
recipes-graphics
recipes-kernel
recipes-lxde
recipes-mozilla
recipes-multimedia
recipes-qt
recipes-sato
recipes-support
release-notes
tasks
We can see that all recipes and applications related to Qt are inside "recipes-qt", which is related to kernel. It is inside "recipes-kernel" and so forth. Since our applications are related to Qt, we create a recipes-qt folder inside meta-projects.
~/oe-core/stuff/meta-projects/ 
$ mkdir recipes-qt
Inside recipes-qt, we create a folder with the name of our application. One folder is created for each application.
~/oe-core/stuff/meta-projects/recipes-qt/ 
$ mkdir qt-artigo-embarcados-screen1
$ mkdir qt-artigo-embarcados-screen2
Editing recipes, its functions and items (GitHub download, auto startup, etc)
We wrote our recipe based on the following simple recipe which compiles and installs a simple "Hello World" C program.
#
# This file was derived from the 'Hello World!' example recipe in the
# Yocto Project Development Manual.
#
SUMMARY = "Simple helloworld application"
SECTION = "examples"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
 
SRC_URI = "file://helloworld.c"
 
S = "${WORKDIR}"
 
do_compile() {
    ${CC} helloworld.c -o helloworld
}
do_install() {
    install -d ${D}${bindir}
    install -m 0755 helloworld ${D}${bindir}
}
C Program
#include 
int main(int argc, char **argv)
{
   printf("Hello World!\n");
   return 0;
}
Inside each recipe folder, we create a .bb file, which is the recipe itself. This file should contain some basic variables such as:
  • DESCRIPTION – A brief description of the recipe and software it contains
  • SECTION – A section or category in which the recipe fits
  • LICENSE – The license file which applies to the recipe or software
  • LIC_FILES_CHKSUM – Checksum number of the license file
  • SRC_URI – Application or its source code location
  • SRCREV – Desired commit tag from GitHub
The recipe used for the first application can be seen below. The second recipe follows the same concept.
~/oe-core/stuff/meta-projects/recipes-qt/qt-artigo-embarcados-screen1/
$ vi qt-artigo-embarcados-screen1_0.1.bb
DESCRIPTION = "Essa aplicação faz parte do artigo para o embarcados"
SECTION = "examples"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
PR = "r0"
 
SRC_URI = "git://github.com/giobauermeister/app-artigo-screen1.git;protocol=git;branch=master"
 
SRCREV = "e1200176d801393cda662ca60552c94a2b023b33"
 
S = "${WORKDIR}/git"
 
inherit systemd
 
DEPENDS = "qtdeclarative qtgraphicaleffects"
RDEPENDS_${PN} = "qtdeclarative-qmlplugins qtgraphicaleffects-qmlplugins"
 
require recipes-qt/qt5/qt5.inc
 
do_install() {
     oe_runmake INSTALL_ROOT=${D} install
     install -m 0755 ${WORKDIR}/git/qt-artigo-embarcados-screen1.sh ${D}${bindir}
     install -d ${D}${systemd_unitdir}/system/ 
     install -m 0644 ${WORKDIR}/git/qt-artigo-embarcados-screen1.service \             ${D}${systemd_unitdir}/system
}
 
NATIVE_SYSTEMD_SUPPORT = "1"
SYSTEMD_PACKAGES = "${PN}"
SYSTEMD_SERVICE_${PN} = "qt-artigo-embarcados-screen1.service"
Breaking down the recipe we can see some important items.

In the following lines we indicate de license to be used.
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
The MIT license is widely used for open source projects and can be found inside common licenses directory.
~/oe-core/stuff/openembedded-core/meta/files/common-licenses/MIT
The checksum number can be obtained using a Linux application called md5sum. To do so, execute the following:
$ md5sum MIT 
0835ade698e0bcf8506ecda2f7b4f302  MIT

Other licenses can be used or created in case the project is not focused on open source. Remember always to indicate the correct license path and checksum.

The SRC_URI indicates the path for an application, in our case it's a GitHub path. The application will be downloaded, compiled, and installed in the root filesystem of our final image.
SRC_URI = "git://github.com/giobauermeister/app-artigo-screen1.git;protocol=git;branch=master"
SRCREV = "e1200176d801393cda662ca60552c94a2b023b33"
S = "${WORKDIR}/git"

The SRCREV variable indicates the repository commit tag to be used. To check which tag to be used, go inside your repository on GitHub on the commits section, https://github.com/giobauermeister/app-artigo-screen1/commits/master, then click on the indicated button below to copy the tag. It's recommended to use the most recent commit.

GitHub commit section
In the following lines we indicate some Qt dependencies.
DEPENDS = "qtdeclarative qtgraphicaleffects"
RDEPENDS_${PN} = "qtdeclarative-qmlplugins qtgraphicaleffects-qmlplugins"
require recipes-qt/qt5/qt5.inc

Next, we have the do_install function, which is responsible for installing our application’s initialization script, as well as a unit configuration file(.service) which is responsible for automatically starting our application. Below we see the files in Red.

Initialization scripts and unit files for both applications should be inside GitHub repository.

A unit configuration file, the name of which ends in .service, encodes information about a process controlled and supervised by systemd.
Service files can be found in /etc/systemd/system/ and for distribution provided ones in /lib/systemd/system/. Services can be started or permanently enabled using the systemctl command.

inherit systemd

do_install() {
oe_runmake INSTALL_ROOT=${D} install

install -m 0755 ${WORKDIR}/git/qt-artigo-embarcados-screen1.sh ${D}${bindir}

install -d ${D}${systemd_unitdir}/system/ 
      install -m 0644 ${WORKDIR}/git/qt-artigo-embarcados-screen1.service \ ${D}${systemd_unitdir}/system
}

NATIVE_SYSTEMD_SUPPORT = "1"
SYSTEMD_PACKAGES = "${PN}"
SYSTEMD_SERVICE_${PN} = "qt-artigo-embarcados-screen1.service"
More information about systemd can be found at Toradex’s Developer website here.

Remember that the installation directory is configured in the project file(.pro) in Qt Creator, as seen in the section "Creating applications with Qt Creator" of this article.

The content of our unit file qt-artigo-embarcados-screen1.service can be found below:
[Unit]
Description=Starts Embarcados Qt demo application screen1
After=multi-user.target
 
[Service]
Type=simple
ExecStart=/usr/bin/qt-artigo-embarcados-screen1.sh
 
[Install]
WantedBy=multi-user.target
The unit file for the other screen application is the same, just change the ExecStart path.
We can notice the unit file calls our initialization script. The content of the screen1 script follows:
#!/bin/bash
ts_calibrate
while : ; do
export QT_QPA_EGLFS_HIDECURSOR=1
export QT_QPA_EGLFS_FB=/dev/fb0
screen1 -platform eglfs
done
The second script differs in just a few items:
#!/bin/bash
while : ; do
export QT_QPA_EGLFS_HIDECURSOR=1
export QT_QPA_EGLFS_FB=/dev/fb2
screen2 -platform eglfs
done

Note the command export QT_QPA_EGLFS_FB=/dev/fb0. This command indicates the framebuffer on which the application will run. This new variable was introduced in new versions of Qt5.

Both scripts should be uploaded to GitHub's repository of each application and will be downloaded and properly installed when the image is built.

Building the image
At this point we should have:
  • A configured build environment
  • Qt applications pushed to GitHub
  • Created our layer meta-projects and recipes
Inside oe-core directory Source the file 'export' to setup the environment. On first invocation this also copies a sample configuration to build/conf/*.conf.
~/oe-core/ 
$ . export
The command takes us to the build directory where we "bitbake" our images.
Inside build/conf we find the files bblayers.conf and local.conf. The layers that have all the resources for our images are listed in the bblayers.conf file. Previously we cloned meta-qt5 layer inside stuff folder. Now we should list both meta-qt5 and meta-projects in bblayers.conf.
~/oe-core/build/conf/
$ vi bblayers.conf
...  
 ${TOPDIR}/../stuff/meta-lxde \
 ${TOPDIR}/../stuff/meta-browser \
 ${TOPDIR}/../stuff/meta-qt5 \

In local.conf we find some build options and settings like, which machine to build for, how many cores to use for building the image, download directories, etc. We create a new variable called IMAGE_INSTALL_append, and indicated some items to be installed as well as our Qt applications.

Go inside local.conf file and add/edit the following content:
~/oe-core/build/conf
$ vi local.conf
MACHINE ?= "colibri-imx6"
ACCEPT_FSL_EULA = "1"
DISTRO_FEATURES_remove = "x11 wayland"
IMAGE_INSTALL_remove = "eglinfo-x11"
IMAGE_INSTALL_append= " qtbase qtbase-fonts qtbase-plugins libxkbcommon \
qt-artigo-embarcados-screen1 \
qt-artigo-embarcados-screen2 \
tslib tslib-calibrate tslib-conf tslib-tests"

The variable ACCEPT_FSL_EULA needs to be set, confirming that we agree with the license from former Freescale. That should be done for all iMX6 modules.
We also removed some Desktop environment items because our image will be a console image.

After editing both configuration files we go inside the build directory to run the bitbake command and finally start the build process.
~/oe-core/build 
$ bitbake console-trdx-image
Updating image on our computer on module

The image update process is very well documented on our Developer's Website on Flashing Embedded Linux to iMX6 modules.

Testing our final image
After image update, Linux should boot. Right after boot, we should also see our applications start automatically. Some boot messages indicate that our systemd files started:
[  OK  ] Started Serial Getty on ttymxc0.
        Starting Serial Getty on ttymxc0...
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User System.
        Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Embarcados Qt demo application screen1.
        Starting Embarcados Qt demo application screen1...
[  OK  ] Started Embarcados Qt demo application screen2.
        Starting Embarcados Qt demo application screen2...
[  OK  ] Started Hostname Service.
[  OK  ] Started WPA supplicant.
[  OK  ] Started Update UTMP about System Runlevel Changes.
[    8.986085] mxc_sdc_fb fb.19: 1920x1080 h_sync,r,l: 44,88,148  v_sync,l,u: 5,4,36 pixclock=148500000 Hz
Check out this video demonstrating the booting process and application auto-startup:
Conclusion

This article was meant to be a foundation for building custom Linux for embedded systems. We saw that an image can be customized and with a few improvements, the image can be used in a product. We saw concepts of git, layers, and recipes. Many of the concepts in this article are used by companies like Toradex, a SBC developer. Toradex provides many development resources to its customers through the layers meta-toradex and meta-toradex-extra. These development resources include Board Support Package, examples, demos, etc. Who knows if you dear reader, will be the next creator of images, layers, or applications that will revolutionize the world of embedded systems!

References

http://developer.toradex.com/how-to/how-to-set-up-qt-creator-to-cross-compile-for-embedded-linux
http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#new-recipe-writing-a-new-recipe
http://playerstage.sourceforge.net/wiki/Cross_Compile_Player_with_Openembedded_and_BitBake#Player_Recipe
http://bec-systems.com/site/501/best-practices-for-building-qt-applications-with-openembedded
https://wiki.yoctoproject.org/wiki/Creating_a_recipe_for_a_Qt_application
https://github.com/meta-qt5
https://wiki.yoctoproject.org/wiki/Building_your_own_recipes_from_first_principles
https://community.freescale.com/docs/DOC-94849
https://wiki.yoctoproject.org/wiki/How_do_I
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html
https://opensource.org/licenses/MIT
http://choosealicense.com/licenses/mit/
http://www.freedesktop.org/software/systemd/man/systemd.service.html
https://wiki.archlinux.org/index.php/Systemd
https://coreos.com/os/docs/latest/getting-started-with-systemd.html
http://developer.toradex.com/knowledge-base/how-to-autorun-application-at-the-start-up-in-linux
http://www.embeddedlinux.org.cn/OEManual/recipes_examples.html
http://wiki.openmoko.org/wiki/BitBake_recipe
https://www.wolfssl.com/wolfSSL/Docs-beginners-guide-yocto-openembedded-recipe.html

This blog post was originally featured on Embarcados.com in Portuguese. See here.

AuthorGiovanni Bauermeister, Toradex Brasil

Subscribe to our future Blog posts:


Leave a comment

Your email ID will be kept confidential. Required fields are marked *

Click to change the Code

Please enter the letters as they are shown in the image above. Letters are not case-sensitive.



* Your comment will be reviewed and then added. Thank you.