Chapter 3. The Software Layer

In this chapter, we will cover the following recipes:

• Exploring an image's contents

• Adding a new software layer

• Selecting a specific package versions and providers

• Adding supported packages

• Adding new packages

• Adding data, scripts, or configuration files

• Managing users and groups

• Using the sysvinit initialization system

• Using the systemd initialization system

• Installing package-installation scripts

• Reducing the Linux kernel image size

• Reducing the root filesystem image size

• Releasing software

• Analyzing your system for compliance

• Working with open source and proprietary code

Introduction

With hardware-specific changes on their way, the next step is customizing the target root filesystem; that is, the software that runs under the Linux kernel, also called the Linux user space.

The usual approach to this is to start with one of the available core images and both optimize and customize it as per the needs of your embedded project. Usually, the images chosen as a starting point are either core-image-minimal or core-image-sato, but any of them will do.

This chapter will show you how to add a software layer to contain those changes, and will explain some of the common customizations made, such as size optimization. It will also show you how to add new packages to your root filesystem, including licensing considerations.

Exploring an image's contents

We have already seen how to use the build history feature to obtain a list of packages and files included in our image. In this recipe, we will explain how the root filesystem is built so that we are able to track its components.

Getting ready

When packages are built, they are classified inside the working directory of your project (tmp/work) according to their architecture. For example, on a wandboard-quad build, we find the following directories:

• all-poky-linux: This is used for architecture-independent packages

• cortexa9hf-vfp-neon-poky-linux-gnueabi: This is used for cortexa9, hard floating point packages

• wandboard_quad-poky-linux-gnueabi: This is used for machine-specific packages; in this case, wandboard-quad

• x86_64-linux: This is used for the packages that form the host sysroot

BitBake will build all the packages included in its dependency list inside its own directory.

How to do it...

To find the build directory for a given package, we can execute the following...

Adding a new software layer

Root filesystem customization involves adding or modifying content to the base image. Metadata for this content goes into one or more software layers, depending on the amount of customization needed.

A typical embedded project will have just one software layer containing all non-hardware-specific customizations. But it is also possible to have extra layers for graphical frameworks or system-wide elements.

Getting ready

Before starting work on a new layer, it is good practice to check whether someone else provides a similar layer. Also, if you are trying to integrate an open source project, check whether a layer for it already exists. There is an index of available layers at http://layers.openembedded.org/.

How to do it...

We can then create a new meta-custom layer using the yocto-layer command as we learned in the Creating a custom BSP layer recipe in Chapter 2, The BSP Layer. From the sources directory, execute the following command:

$ yocto-layer create custom

Don...

Selecting a specific package version and providers

Our layers can provide recipes for different versions of the same package. For example, the meta-fsl-arm layer contains several different types of Linux sources:

• linux-imx: This corresponds to the Freescale BSP kernel image fetched from http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/

• linux-fslc: This is the mainline Linux kernel and fetched from https://github.com/Freescale/linux-fslc

• linux-timesys: This is a kernel with Vybrid platform support fetched from https://github.com/Timesys/linux-timesys

As we mentioned before, all recipes provide the package name (for example, linux-imx or linux-fslc) by default, but all Linux recipes must also provide the virtual/kernel virtual package. The build system will resolve virtual/kernel to the most appropriate Linux recipe name, taking into account the requirements of the build, such as the machine it is building for.

And within those recipes, linux-imx, for example, has both 2.6.35.3 and 3...

Adding supported packages

It is common to want to add new packages to an image that already has an available recipe in one of the included Yocto layers.

When the target image desired is very different from the supplied core images, it is recommended to define a new image rather than to customize an existing one.

This recipe will show how to customize an existing image by adding supported packages to it, but also to create a completely new image recipe if needed.

Getting ready

To discover whether a package we require is included in our configured layers, and what specific versions are supported, we can use bitbake-layers from our build directory as we saw previously:

$ bitbake-layers show-recipes | grep -A 1 htop
htop:
  meta-oe              1.0.3

Alternatively, we can also use BitBake as follows:

$ bitbake -s | grep htop
htop                                                :1.0.3-r0

Or we can use the find Linux command in our sources directory:

$ find . -type f -name "htop*.bb"
./meta-openembedded...

Adding new packages

We have seen how to customize our image so that we can add supported packages to it. When we can't find an existing recipe or we need to integrate some new software we have developed, we will need to create a new Yocto recipe.

Getting ready

There are some questions we need to ask ourselves before starting to write a new recipe:

• Where is the source code stored?

• Is it source-controlled or released as a tarball?

• What is the source code license?

• What build system is it using?

• Does it need configuration?

• Can we cross-compile it as is or does it need to be patched?

• What are the files that need to be deployed to the root filesystem, and where do they go?

• Are there any system changes that need to happen, such as new users or init scripts?

• Are there any dependencies that need to be installed into sysroot beforehand?

Once we know the answers to these questions, we are ready to start writing our recipe.

How to do it...

It is best to start from a blank template like the one that follows than to...

Adding data, scripts, or configuration files

All recipes inherit the base class with the default set of tasks to run. After inheriting the base class, a recipe knows how to do things like fetching and compiling.

As most recipes are meant to install some sort of executable, the base class knows how to build it. But sometimes all we want is to install data, scripts, or configuration files into the filesystem.

If the data or configuration is related to an application, the most logical thing to do is to package it together with the application's recipe itself, and if we think it is better to be installed separately, we could even split it into its own package.

But some other times, the data or configuration is unrelated to an application, maybe it applies to the whole system or we just want to provide a separate recipe for it. Optionally, we could even want to install some Perl or Python scripts that don't need to be compiled.

How to do it...

In those cases, our recipe should inherit the allarch...

Managing users and groups

It is also common to need to add or modify users and groups to our filesystem. This recipe explains how it is done.

Getting ready

The user information is stored in the /etc/passwd file, a text file that is used as a database for the system user's information. The passwd file is human-readable.

Each line on it corresponds to one user in the system, and it has the following format:

<username>:<password>:<uid>:<gid>:<comment>:<home directory>:<login shell>

Let's see each of the parameters of this format:

• username: A unique string that identifies the user at login

• uid: User ID, a number that Linux uses to identify the user

• gid: Group ID, a number that Linux uses to identify the user's primary group

• comment: Comma-separated values that describe the account, typically the user's contact details

• home directory: Path to the user's home directory

• login shell: Shell that is started for interactive logins

The default passwd file is stored with...

Using the sysvinit initialization manager

The initialization manager is an important part of the root filesystem. It is the first thing the kernel executes, and it has the responsibility to start the rest of the system.

This recipe will introduce the sysvinit initialization manager.

Getting ready

This is the default initialization manager in Yocto and it has been used in Linux since the operating system's origin. The kernel is passed an init command-line argument, typically /sbin/init, which is then launched. This init process has PID 1 and is the parent of all processes. The init process can either be implemented by BusyBox or be an independent program installed with the sysvinit package. Both of them work in the same way, based on the concept of runlevel, a machine state that defines which processes to run.

The init process will read an inittab file and look for a default runlevel. The default inittab file is installed with the sysvinit-inittab package and is as follows:

# /etc/inittab: init...

Using the systemd initialization manager

As an alternative to sysvinit, you can configure your project to use systemd as an initialization manager, although systemd packs many more features.

Getting ready

The systemd initialization manager is replacing sysvinit and other initialization managers in most Linux distributions. It is based on the concepts of units, an abstraction of all elements that are relevant for system startup and maintenance, and targets, which group units and can be viewed as a runlevel equivalent. Some of the units systemd defines are:

• Services

• Sockets

• Devices

• Mount points

• Snapshots

• Timers

• Paths

The default targets and their runlevel equivalents are defined in the following table:

Installing package-installation scripts

The supported package formats, RPM, ipk, and deb, support the addition of installation scripts that can be run at different times during a package installation process. In this recipe, we will see how to install them.

Getting ready

There are different types of installation scripts:

• Preinstallation scripts (pkg_preinst): These are called before the package is unpacked

• Postinstallation scripts (pkg_postinst): These are called after the package is unpacked, and dependencies will be configured

• Preremoval scripts (pkg_prerm): These are called with installed or at least partially installed packages

• Postremoval scripts (pkg_postrm): These are called after the package's files have been removed or replaced

How to do it...

An example snippet of the installation of a preinstallation script in a recipe is as follows:

     pkg_preinst_${PN} () {
         # Shell commands
     }

All installation scripts work in the same way, with the exception that the postinstallation scripts...

Reducing the Linux kernel image size

Before or in parallel with the root filesystem customization, embedded projects usually require an image size optimization that will reduce the boot time and memory usage.

Smaller images mean less storage space, less transmission time, and less programming time, which saves money both in manufacturing and field updates.

By default, the compressed Linux kernel image (zImage) for the wandboard-quad is around 5.2 MB. This recipe will show how we can reduce that.

How to do it...

An example of a minimal kernel configuration for a Wandboard that is able to boot from a microSD card root filesystem is the arch/arm/configs/wandboard-quad_minimal_defconfig file that follows:

CONFIG_KERNEL_XZ=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_EMBEDDED=y
CONFIG_SLOB=y
CONFIG_ARCH_MXC=y
CONFIG_SOC_IMX6Q=y
CONFIG_SOC_IMX6SL=y
CONFIG_SMP=y
CONFIG_VMSPLIT_2G=y
CONFIG_AEABI=y
CONFIG_CPU_FREQ=y
CONFIG_ARM_IMX6_CPUFREQ=y
CONFIG_CPU_IDLE...

Reducing the root filesystem image size

By default, the core-image-minimal size for the wandboard-quad unpacked tarball is around 45 MB, and core-image-sato is around 150 MB. This recipe will explore methods to reduce their size.

How to do it...

An example of a small image, core-image-small, that does not include the packagegroup-core-boot recipe and can be used as the base for a root filesystem image with reduced size, recipes-core/images/core-image-small.bb, is shown next:

DESCRIPTION = "Minimal console image."

IMAGE_INSTALL= "\
        base-files \
        base-passwd \
        busybox \
        sysvinit \
        initscripts \
        ${ROOTFS_PKGMANAGE_BOOTSTRAP} \
        ${CORE_IMAGE_EXTRA_INSTALL} \
"

IMAGE_LINGUAS = " "

LICENSE = "MIT"

inherit core-image

IMAGE_ROOTFS_SIZE ?= "8192"

This recipe produces an image of about 6.4 MB. You can go even smaller if you use the poky-tiny distribution by adding the following to your conf/local.conf file:

DISTRO = "poky-tiny"

The poky-tiny distribution...

Releasing software

When releasing a product based on the Yocto project, we have to consider that we are building on top of a multitude of different open source projects, each with different licensing requirements.

At the minimum, your embedded product will contain a bootloader (probably U-Boot), the Linux kernel, and a root filesystem with one or more applications. Both U-Boot and the Linux kernel are licensed under the General Public License version 2 (GPLv2). And the root filesystem could contain a variety of programs with different licenses.

All open source licenses allow you to sell a commercial product with a mixture of proprietary and open licenses as long as they are independent and the product complies with all the open source licenses. We will discuss open source and proprietary cohabiting in the Working with open source and proprietary code recipe later on.

It is important to understand all the licensing implications before releasing your product to the public. The Yocto project provides...

Analyzing your system for compliance

The Yocto build system makes it easy to provide auditing information to our legal advisers. This recipe will explain how.

How to do it...

Under tmp/deploy/licenses, we find a directory list of packages (including their corresponding licenses) and an image folder with a package and license manifest.

For the example image provided before, core-image-small, we have the following:

tmp/deploy/licenses/core-image-small-wandboard-quad-<timestamp>/package.manifest
base-files
base-passwd
busybox
busybox-syslog
busybox-udhcpc
initscripts
initscripts-functions
libc6
run-postinsts
sysvinit
sysvinit-inittab
sysvinit-pidof
update-alternatives-opkg
update-rc.d

And the corresponding tmp/deploy/licenses/core-image-small-wandboard-quad-<timestamp>/license.manifest file excerpt is as follows:

PACKAGE NAME: base-files
PACKAGE VERSION: 3.0.14
RECIPE NAME: base-files
LICENSE: GPLv2

PACKAGE NAME: base-passwd
PACKAGE VERSION: 3.5.29
RECIPE NAME: base-passwd
LICENSE:...

Working with open source and proprietary code

It is common for an embedded product to be built upon an open source system like the one built by Yocto, and to include proprietary software that adds value and specializes the product. This proprietary part usually is intellectual property and needs to be protected, and it's important to understand how it can coexist with open source.

This recipe will discuss some examples of open source packages commonly found on embedded products and will briefly explain how to use proprietary software with them.

How to do it...

Open source licenses can be broadly divided into two categories based on whether they are:

• Permissive: These are similar to Internet Software Consortium (ISC), MIT, and BSD licenses. They have few requirements attached to them and just require us to preserve copyright notices.

• Restrictive: These are similar to the GPL, which bind us to not only distribute the source code and modifications, either with the binary itself or at a later date...

The rest of the chapter is locked

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Renews at $15.99/month. Cancel anytime

Start free trial

Previous Chapter

Chapter 10 of 13

Next Chapter

You have been reading a chapter from

Embedded Linux Projects Using Yocto Project Cookbook

Published in: Mar 2015Publisher: ISBN-13: 9781784395186

© 2015 Packt Publishing Limited All Rights Reserved

Register for a free Packt account to unlock a world of extra content!

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Sign up now

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Author (1)

Alex Gonzalez

Alex González is a software engineering supervisor at Digi International and product owner of the Digi Embedded Yocto distribution. He started working professionally with embedded systems in 1999 and the Linux kernel in 2004, designing products for voice and video over IP networks, and followed his interests into machine-to-machine (M2M) technologies and the Internet of Things. Born and raised in Bilbao, Spain, Alex has an electronic engineering degree from the University of the Basque Country and he received his MSc in communication systems from the University of Portsmouth.
Read more about Alex Gonzalez

See other products by Alex Gonzalez

Personalised recommendations for you

Based on your interests and search pattern

Architectural Patterns and Techniques for Developing IoT Solutions

This book covers all the patterns and considerations that give you both the power and flexibility to build scalable, secure, and performant IoT solutions by combining various patterns in interesting ways. It also lists the benefits of combining IoT with technologies like blockchain, 3D-printing, 5G, Generative AI, quantum computing, and LLMs.

BookSep 2023304 pages

Arduino Data Communications

Arduino Data Communication focuses on IoT’s Internet aspect, guiding you in setting up your own infrastructure for storing and managing the data collected from sensors. This book goes beyond microcontroller basics, equipping you with the knowledge essential for building real-world projects.

BookNov 2023286 pages5

Arduino IoT Cloud for Developers

From fundamental principles to advanced techniques, this comprehensive book equips you with the knowledge and skills needed to design and deploy IoT applications seamlessly. Explore cloud integration, best practices, and real-world projects to harness the full potential of IoT application development with the Arduino IoT Cloud.

BookNov 2023402 pages

The Azure IoT Handbook

Building IoT Systems with Azure IoT is a comprehensive introduction for those who are new to the Internet of Things and looking to get up to speed in no time. This book will teach you how to create and develop IoT solutions with intelligent edge-to-cloud technologies in the Azure cloud.

BookDec 2023248 pages

GO TO TOP

Machine LearningKubernetesReactPythonCybersecurityUnity Game DevelopmentPower BIAzureWeb Development

• Machine Learning
• Kubernetes
• React
• Python
• Cybersecurity
• Unity Game Development
• Power BI
• Azure
• Web Development

Legal

Terms and ConditionsPrivacy PolicyCookie PolicyShipping PolicyReturn Policy

Support

HelpContact Us

Business

Partnerships

Sponsored eBooks

Custom eBooks

CareersBecome an author

Packt+ Membership

SubscriptionDataProSecPro

Packt Subscription Packt NewsLetter

Legal

Terms and Conditions

Privacy Policy

Cookie Policy

Shipping Policy

Return Policy

Support

Help

Contact Us

Business

Packt Subscription

Packt NewsLetter

---

Company Address: Packt Publishing Ltd, Grosvenor House, 11 St Paul's Square, Birmingham, B3 1RB© 2024 Packt Publishing Limited All Rights Reserved

Sysvinit	Runlevel	Systemd target	Notes
0	runlevel0.target	poweroff.target	Halt the system.
1, s, single	runlevel1.target	rescue.target	Single user mode.
2, 4	runlevel2.target, runlevel4.target	multi-user.target	User-defined/site-specific runlevels. By default, identical to 3.
3	runlevel3...

Installing package-installation scripts

The supported package formats, RPM, ipk, and deb, support the addition of installation scripts that can be run at different times during a package installation process. In this recipe, we will see how to install them.

Getting ready

There are different types of installation scripts:

• Preinstallation scripts (pkg_preinst): These are called before the package is unpacked

• Postinstallation scripts (pkg_postinst): These are called after the package is unpacked, and dependencies will be configured

• Preremoval scripts (pkg_prerm): These are called with installed or at least partially installed packages

• Postremoval scripts (pkg_postrm): These are called after the package's files have been removed or replaced

How to do it...

An example snippet of the installation of a preinstallation script in a recipe is as follows:

     pkg_preinst_${PN} () {
         # Shell commands
     }

All installation scripts work in the same way, with the exception that the postinstallation scripts...

Reducing the Linux kernel image size

Before or in parallel with the root filesystem customization, embedded projects usually require an image size optimization that will reduce the boot time and memory usage.

Smaller images mean less storage space, less transmission time, and less programming time, which saves money both in manufacturing and field updates.

By default, the compressed Linux kernel image (zImage) for the wandboard-quad is around 5.2 MB. This recipe will show how we can reduce that.

How to do it...

An example of a minimal kernel configuration for a Wandboard that is able to boot from a microSD card root filesystem is the arch/arm/configs/wandboard-quad_minimal_defconfig file that follows:

CONFIG_KERNEL_XZ=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_EMBEDDED=y
CONFIG_SLOB=y
CONFIG_ARCH_MXC=y
CONFIG_SOC_IMX6Q=y
CONFIG_SOC_IMX6SL=y
CONFIG_SMP=y
CONFIG_VMSPLIT_2G=y
CONFIG_AEABI=y
CONFIG_CPU_FREQ=y
CONFIG_ARM_IMX6_CPUFREQ=y
CONFIG_CPU_IDLE...

Reducing the root filesystem image size

By default, the core-image-minimal size for the wandboard-quad unpacked tarball is around 45 MB, and core-image-sato is around 150 MB. This recipe will explore methods to reduce their size.

How to do it...

An example of a small image, core-image-small, that does not include the packagegroup-core-boot recipe and can be used as the base for a root filesystem image with reduced size, recipes-core/images/core-image-small.bb, is shown next:

DESCRIPTION = "Minimal console image."

IMAGE_INSTALL= "\
        base-files \
        base-passwd \
        busybox \
        sysvinit \
        initscripts \
        ${ROOTFS_PKGMANAGE_BOOTSTRAP} \
        ${CORE_IMAGE_EXTRA_INSTALL} \
"

IMAGE_LINGUAS = " "

LICENSE = "MIT"

inherit core-image

IMAGE_ROOTFS_SIZE ?= "8192"

This recipe produces an image of about 6.4 MB. You can go even smaller if you use the poky-tiny distribution by adding the following to your conf/local.conf file:

DISTRO = "poky-tiny"

The poky-tiny distribution...

Releasing software

When releasing a product based on the Yocto project, we have to consider that we are building on top of a multitude of different open source projects, each with different licensing requirements.

At the minimum, your embedded product will contain a bootloader (probably U-Boot), the Linux kernel, and a root filesystem with one or more applications. Both U-Boot and the Linux kernel are licensed under the General Public License version 2 (GPLv2). And the root filesystem could contain a variety of programs with different licenses.

All open source licenses allow you to sell a commercial product with a mixture of proprietary and open licenses as long as they are independent and the product complies with all the open source licenses. We will discuss open source and proprietary cohabiting in the Working with open source and proprietary code recipe later on.

It is important to understand all the licensing implications before releasing your product to the public. The Yocto project provides...

Analyzing your system for compliance

The Yocto build system makes it easy to provide auditing information to our legal advisers. This recipe will explain how.

How to do it...

Under tmp/deploy/licenses, we find a directory list of packages (including their corresponding licenses) and an image folder with a package and license manifest.

For the example image provided before, core-image-small, we have the following:

tmp/deploy/licenses/core-image-small-wandboard-quad-<timestamp>/package.manifest
base-files
base-passwd
busybox
busybox-syslog
busybox-udhcpc
initscripts
initscripts-functions
libc6
run-postinsts
sysvinit
sysvinit-inittab
sysvinit-pidof
update-alternatives-opkg
update-rc.d

And the corresponding tmp/deploy/licenses/core-image-small-wandboard-quad-<timestamp>/license.manifest file excerpt is as follows:

PACKAGE NAME: base-files
PACKAGE VERSION: 3.0.14
RECIPE NAME: base-files
LICENSE: GPLv2

PACKAGE NAME: base-passwd
PACKAGE VERSION: 3.5.29
RECIPE NAME: base-passwd
LICENSE:...

Working with open source and proprietary code

It is common for an embedded product to be built upon an open source system like the one built by Yocto, and to include proprietary software that adds value and specializes the product. This proprietary part usually is intellectual property and needs to be protected, and it's important to understand how it can coexist with open source.

This recipe will discuss some examples of open source packages commonly found on embedded products and will briefly explain how to use proprietary software with them.

How to do it...

Open source licenses can be broadly divided into two categories based on whether they are:

• Permissive: These are similar to Internet Software Consortium (ISC), MIT, and BSD licenses. They have few requirements attached to them and just require us to preserve copyright notices.

• Restrictive: These are similar to the GPL, which bind us to not only distribute the source code and modifications, either with the binary itself or at a later date...