Packt+ | Advance your knowledge in tech

You're reading from Hands-On Embedded Programming with C++17

Product typeBook

Published inJan 2019

Reading LevelIntermediate

PublisherPackt

ISBN-139781788629300

Edition1st Edition

Languages

C++

Tools

Concepts

Embedded Systems

Author (1)

Maya Posch

Chapter 8. Example - Linux-Based Infotainment System

This chapter provides an example of how to implement an infotainment system using a Linux-based single-board computer (SBC). It also shows how to connect to remote devices using Bluetooth, and how to use online streaming services. The resulting device will be able to play back audio from a variety of sources without a complex UI. In particular, we will be covering the following topics:

Developing for a Linux-based SBC
Using Bluetooth under Linux
Playing back audio from a variety of sources and recording audio
Using GPIO for both simple input and voice recognition
Connecting to online streaming audio services

One box that does everything

Infotainment systems have become a common feature in our daily lives, starting with in-car entertainment (ICE) systems (also known as In-Vehicle Infotainment or IVI), which evolved from the basic radios and cassette players to include features such as navigation and connecting to smartphones over Bluetooth for access to one's music library, and much more. Another big feature is to provide the driver with hands-free functionality so that they can start a phone call and control the radio without having to take their eyes off the road or their hands off the steering wheel.

As smartphones became more popular, providing their users with constant access to news, weather, and entertainment, the arrival of onboard assistants that use a voice-driven interface, both on smartphones and ICEs, ultimately led to the arrival of speech-driven infotainment systems aimed at in-home use. These usually consist of a speaker and microphone, along with the required hardware for the...

Hardware needed

For this project, any SBC that's capable of running Linux should work. It also needs to have the following features for a full implementation:

An internet connection (wireless or wired) to access online content.
Bluetooth functionality (built-in or as an add-on module) to allow the system to act as a Bluetooth speaker.
Free GPIO input to allow for buttons to be hooked up.
A functioning microphone input and audio output for the voice input and audio playback, respectively.
SATA connectivity or similar for connecting storage devices like hard-drives.
I2C bus peripheral for an I2C display.

For the example code in this chapter we only require the microphone input and audio output, along with some storage for local media files.

To the GPIO pins, we can connect a number of buttons that can be used to control the infotainment system without having to use the voice-activated system. This is convenient for situations where using the voice-activated system would be awkward, such as when...

Software requirements

For this project, we are assuming that Linux has been installed on the target SBC, and that the drivers for the hardware functionality, such as the microphone and audio output, have been installed and configured.

Since we use the Qt framework for this project, all dependencies there should be met as well. This means that the shared libraries should be present on the system on which the resulting binary for the project will be run. The Qt framework can be obtained via the package manager of the OS, or via the Qt website at http://qt.io/.

Note

In Chapter 10, Developing Embedded Systems with Qt, we will look at developing on embedded platforms with Qt in more detail. This chapter will briefly touch upon the use of Qt APIs.

Depending on whether we want to compile the application directly on the SBC or on our development PC, we might have to install the compiler toolchain and further dependencies on the SBC, or the cross-compiling toolchain for Linux on the target SBC (ARM,...

Bluetooth audio sources and sinks

Bluetooth is unfortunately a technology that, despite being ubiquitous, suffers from its proprietary nature. As a result, support for the full range of Bluetooth functionality (in the form of profiles) is lacking. The profile that we are interested in for this project is called Advanced Audio Distribution Profile (A2DP). This is the profile used by everything from Bluetooth headphones to Bluetooth speakers in order to stream audio.

Any device that implements A2DP can stream audio to an A2DP receiver or can themselves act as a receiver (depending on the BT stack implementation). Theoretically, this would allow someone to connect with a smartphone or similar device to our infotainment system and play back music on it, as they would with a standalone Bluetooth speaker.

A receiver in the A2DP profile is an A2DP sink, whereas the other side is the A2DP source. A Bluetooth headphone or speaker device would always be a sink device as they can only consume an audio...

Online streaming

There are a number of online streaming services which one could integrate into an infotainment system like the type which are looking at in this chapter. All of them use a similar streaming API (usually an HTTP-based REST API), which requires one to create an account with the service, using which one can obtain an application-specific token that gives one access to that API, allowing one to query it for specific artists, music tracks, albums, and so on.

Using an HTTP client, such as the one found in the Qt framework, it would be fairly easy to implement the necessary control flow. Due to the requirement of having a registered application ID for those streaming services, it was left out of the example code.

The basic sequence to stream from a REST API usually looks like this, with a simple wrapper class around the HTTP calls:

#include "soundFoo"
// Create a client object with your app credentials.
client = soundFoo.new('YOUR_CLIENT_ID');
// Fetch track to stream.
track = client...

Voice-driven user interface

This project employs a user interface that is fully controllable by voice commands. For this, it implements a voice-to-text interface powered by the PocketSphinx library (see https://cmusphinx.github.io/) that uses both keyword-spotting and a grammar search in order to recognize and interpret commands given to it.

We use the default US-English language model that comes with the PocketSphinx distribution. This means that any commands spoken should be pronounced with a US-English accent in order to be accurately understood. To change this, one can load a different language model aimed at different languages and accents. Various models are available via the PocketSphinx website, and it is possible to make one's own language model with some effort.

Usage scenarios

We don't want the infotainment system to be activated every single time that the voice user interface recognizes command words when they are not intended as such. The common way to prevent this from happening is by having a keyword that activates the command interface. If no command is recognized after the keyword within a certain amount of time, the system reverts to the keyword-spotting mode.

For this example project, we use the keyword computer. After the system spots this keyword, we can use the following commands:

Source code

This application has been implemented using the Qt framework, as a GUI application, so that we also get a graphical interface for ease of debugging. This debugging UI was designed using the Qt Designer of the Qt Creator IDE as a single UI file.

We start by creating an instance of the GUI application:

#include "mainwindow.h" 
#include <QApplication> 
 
int main(int argc, char *argv[]) { 
    QApplication a(argc, argv); 
    MainWindow w; 
    w.show(); 
     
    return a.exec(); 
}

This creates an instance of the MainWindow class in which we have implemented the application, along with an instance of QApplication, which is a wrapper class used by the Qt framework.

Next, this is the MainWindow header:

#include <QMainWindow> 
 
#include <QAudioRecorder> 
#include <QAudioProbe> 
#include <QMediaPlayer> 
 
 
namespace Ui { 
    class MainWindow; 
} 
 
class MainWindow : public QMainWindow { 
    Q_OBJECT 
     
public: 
    explicit MainWindow(QWidget *parent...

Building the project

To build the project, the PocketSphinx project has to be built first. In the example project's source code that comes with this chapter, there are two Makefiles underneath the sphinx folder, one in the pocketsphinx folder and one in the sphinxbase folder. With these, the two libraries that form PocketSphinx will be built.

After this, one can build the Qt project, either from Qt Creator or from the command line, by executing the following command:

mkdir build
cd build
qmake ..
make

Extending the system

In addition to audio formats, one could also add the ability to play back videos and integrate the ability to make and respond to phone calls (using the Bluetooth API). One may want to extend the application to make it more flexible and modular, so that, for example, one could add a module that would add the voice commands and resulting actions.

Having voice output would be convenient as well, making it more aligned with the current commercial offerings. For this, one could use the text-to-speech API that's available in the Qt framework.

It would also be useful to add more information to the infotainment system by querying remote APIs for things such as the current weather, news updates, and maybe even running updates on a current football game. The voice-based UI could be used to set up timers and task reminders, integrate a calendar, and much more.

Finally, as can be seen in this chapter's example code, one cannot specify the name of the track that one wants to play,...

Summary

In this chapter, we looked at how one can fairly easily construct an SBC-based infotainment system, using voice-to-text to construct a voice-driven user interface. We also looked at ways that we could extend it to add even more functionality.

The reader is expected to be able to implement a similar system at this point, and to extend it to connect it to online and network-based services. The reader should also read up on the implementation of more advanced voice-driven user interfaces, the addition of text-to-speech, and the use of A2DP-based Bluetooth devices.

In the next chapter, we'll be taking a look at how to implement a building-wide monitoring and control system using microcontrollers and the local network.

The rest of the chapter is locked

You have been reading a chapter from

Hands-On Embedded Programming with C++17

Published in: Jan 2019Publisher: PacktISBN-13: 9781788629300

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Author (1)

Maya Posch

Maya Posch is a senior C++ developer with more than 15 years of experience. Discovering the joys of programming early on, and later the joys of electronics, she has always expressed a profound interest in technology, a passion that she gladly shares with others. Describing herself as a C developer who happens to like C++ and Ada, she likes to seek the limits of what can be done with the minimum of code and hardware to accomplish everything that is cool, new, and exciting. She also enjoys FPGA development, AI, and robotics research, in addition to creative writing, music, and drawing.
Read more about Maya Posch

Other recommended products

Related to this chapter

ESP8266 Home Automation Projects

The ESP8266 is a low-cost yet powerful Wi-Fi chip that is becoming more popular at an alarming rate, and people have adopted it to create interesting projects. This book shows readers how to build autonomous home automation projects, create entire home automation systems, and monitor/access their homes from the cloud. You will learn how to read values from sensors, how to send them to your cloud platform over Wi-Fi, and how to receive them on your mobile application.

BookNov 2017196 pages

Hands-On RTOS with Microcontrollers

This book covers important real-time operating systems concepts used in microcontroller-based embedded systems. You will use an STM32 board, SEGGER debugging tools, and STM32Cube IDE to build and analyze real-world embedded projects. After completing this book, you will have gained advanced techniques to implement robust real-time embedded systems.

BookMay 2020496 pages

Developing IoT Projects with ESP32

ESP32 is a low-cost, low-power MCU series by Espressif systems that can be used in wireless IoT applications. With Wi-Fi and Bluetooth technologies as well as security features, it's a full-fledged SoC for developing IoT products for consumer and industrial markets. This book will help you get up to speed with ESP32 using interesting projects.

BookSep 2021474 pages

Mastering C++ Multithreading

Multithreaded applications execute multiple threads in a single processor environment, allowing developers achieve concurrency. Moden day programming requires data to be fetched at incredible speed. Multithreading allows developers to deal with asynchronous data so their programming needs are met. This book will teach you the finer points of multithreading and concurrency concepts and how to apply them efficiently in C++.

BookJul 2017244 pages

Architecting High-Performance Embedded Systems

This book takes embedded system developers through the complete process of developing systems based on field-programmable gate arrays (FPGAs). You’ll not only understand the concepts underlying embedded systems and FPGAs but also explore the powerful free FPGA tools to design electronic circuits and construct and debug prototype embedded devices.

BookFeb 2021376 pages4

Embedded Programming with Modern C++ Cookbook

This book is a collection of practical examples for understanding how embedded development is different from other desktop application development. You’ll learn to build an embedded application and use specialized memory and custom allocators. By the end of the book, you’ll be able to build robust and secure embedded applications with C++20.

BookApr 2020412 pages

Embedded Systems Architecture

Embedded systems are self-contained units that have a dedicated purpose within a device. We come across a variety of applications of embedded systems in navigation tools, telecom applications, and networking equipment, to name just a few. This book will help you create your own parallel and distributed embedded systems.

BookMay 2018324 pages

Practical Hardware Pentesting

Hardware security has become a major concern in recent times making it crucial for companies and end users to focus on protecting hardware. If you’re a security practitioner who is more familiar with software and binary hacking or someone looking to repurpose devices, this book will demonstrate how to approach hardware hacking of embedded devices.

BookApr 2021382 pages

Personalised recommendations for you

Based on your interests and search pattern

Architectural Patterns and Techniques for Developing IoT Solutions

This book covers all the patterns and considerations that give you both the power and flexibility to build scalable, secure, and performant IoT solutions by combining various patterns in interesting ways. It also lists the benefits of combining IoT with technologies like blockchain, 3D-printing, 5G, Generative AI, quantum computing, and LLMs.

BookSep 2023304 pages

Arduino Data Communications

Arduino Data Communication focuses on IoT’s Internet aspect, guiding you in setting up your own infrastructure for storing and managing the data collected from sensors. This book goes beyond microcontroller basics, equipping you with the knowledge essential for building real-world projects.

BookNov 2023286 pages5

Arduino IoT Cloud for Developers

From fundamental principles to advanced techniques, this comprehensive book equips you with the knowledge and skills needed to design and deploy IoT applications seamlessly. Explore cloud integration, best practices, and real-world projects to harness the full potential of IoT application development with the Arduino IoT Cloud.

BookNov 2023402 pages

The Azure IoT Handbook

Building IoT Systems with Azure IoT is a comprehensive introduction for those who are new to the Internet of Things and looking to get up to speed in no time. This book will teach you how to create and develop IoT solutions with intelligent edge-to-cloud technologies in the Azure cloud.

BookDec 2023248 pages

Command	Result
Play Bluetooth	Starts playing from any connected A2DP source device (unimplemented).
Stop Bluetooth	Stops playing from any Bluetooth device.
Play local	Plays the (hardcoded) local music file.
Stop local	Stops playing the local music file, if currently playing.
Play remote	Plays from an online streaming service or server (unimplemented).
Stop remote	Stops playing, if active.
Record message	Records a message. Records until a number of seconds of silence occurs...