Developing on Edge Impulse

As we talked about in the previous chapter, developing a TinyML application requires many steps, such as collecting requirements and data, pre-processing data, model development and optimization, deployment, performance tracking, and maintenance. In a traditional software project, we have DevOps to manage the software life cycle. When it comes to machine learning, we define Machine Learning Operations or MLOps. An MLOps platform provides us with the tools and resources to design, develop, and maintain our machine learning applications. What makes an MLOps platform different from a DevOps platform is that it also manages data and the resulting models in its versioning subsystem.

Edge Impulse is the leading MLOps platform for TinyML. It helps at every step of the ML application development process. Some important features of Edge Impulse are the following:

• A web-based development environment, Edge Impulse Studio, to manage the entire ML life...

Technical requirements

The hardware requirements of the chapter are:

• ESP32-S3 Box Lite
• An RGB LED
• A 440 resistor

An account on the Edge Impulse platform is needed. You can create your account at this link: https://studio.edgeimpulse.com/login.

You can find the example of the chapter here: https://github.com/PacktPublishing/Developing-IoT-Projects-with-ESP32-2nd-edition/tree/main/ch11.

Let’s have a quick overview of the Edge Impulse platform first.

An overview of Edge Impulse

The Edge Impulse platform has distinct features to support TinyML. First of all, we have an option to collect data directly from a sensor device. Data Forwarder is a tool that comes with the platform. With suitable firmware on the device, Data Forwarder retrieves data from the sensor over a serial connection and sends it to the platform. It is important because the quality of data can change between different brands of sensors, which might eventually affect the accuracy of the model. If we collect data from the device that we are going to use in the product, this can improve the final ML model.

After retrieving training data, we can design the model as an Impulse on the platform. An Impulse consists of different blocks: an input block, a processing block, and a learning block. The input block defines the nature of data. The processing block extracts the features that we need to train the model on. Finally, the learning block applies the machine learning...

Cloning an Edge Impulse project

The Edge Impulse platform provides community projects available online to everybody. We can clone them into our accounts and develop new projects on top of them. We will use one of the public projects in this chapter as an example, which is Edge Impulse Inc. / TinyML Summit 2021 Keywords. Its goal is to detect the tinyml utterance in a model. Let’s clone this example and see what Edge Impulse provides us with as an MLOps platform in the following steps:

1. Navigate to https://www.edgeimpulse.com/projects and use the search bar to find the TinyML Summit 2021 Keywords project by name. Select it from the list.

   

   Figure 11.1: Searching the keyword-spotting example

2. Clone the project into your account by clicking on the Clone this project button at the top right. You can use the same name or give a new name to the project in the pop-up dialog. Then, the platform will create a duplicate of the project in your account with...

Using the ML model on ESP32

The goal of the example is to understand how to use the downloaded C++ model to develop an ESP-IDF project and run it on ESP32-S3 Box Lite to detect the keyword tinyml. We will connect an RGB LED to the devkit so that the application can visually indicate the inference state: green for keyword detection, blue for no sound/background noise, and red for any sound other than the keyword.

Before moving on to the code, let’s attach an RGB LED to the GP13 (red), GP12 (green), and GP11 (blue) pins of the devkit. The following Fritzing sketch shows the connections:

Figure 11.8: The connections between the devkit and the RGB LED

Unfortunately, our devkit is not officially supported by the Edge Impulse platform, so we should do the integration manually. However, we will skip all the integration work by using the application code from the book repository as it is in order to keep the focus on the main subject. We will only discuss the code...

Next steps for TinyML development

In the scope of this book, we only discussed how to run inference on ESP32 by using different TinyML frameworks. However, in real-world scenarios, we need to do more. Let’s review the ML development stages once more and have a short discussion of them in terms of the engineering work needed:

• Project requirements: A project starts with a need and requirements that list what to do in response to that need. A machine learning project is no exception for that. The requirements of an ML project usually reveal a lot about the nature of data in the project. With a requirement analysis, we can understand what data we need to collect, the sources of data, how we can collect it, any option to import external data, data versioning requirements, etc. In addition, a requirements document can have information about the performance of the output model, such as the accuracy, response time, and memory limitations. Project requirements have a direct...

Summary

MLOps is a group of activities and tools to manage an ML product life cycle. In this chapter, we discussed Edge Impulse as an MLOps platform for TinyML. After cloning an open project from the Edge Impulse project repository, we built it in Edge Impulse Studio, the online tool to manage projects. We downloaded the Edge Impulse C++ library that contains the ML model and integrated that library with the devkit in an ESP-IDF project. We discussed the application in detail to see how to use the model to make inferences on the audio data coming from the mic of the devkit.

In the final chapter of the book, we will design and develop another project to cement what we have learned about TinyML so far.

Questions

Try to answer the following questions to test your understanding of the chapter:

1. Which one of the following is NOT true for an MLOps platform?
   1. It supports data versioning for traceability.
   2. It provides tools and utilities for data import and pre processing.
   3. It always comes with an AutoML tool.
   4. Model training is a part of the solution.
2. Which of the following data collection methods is NOT supported by Edge Impulse?
   1. Connect to a mobile device.
   2. Connect to a remote database.
   3. Import a data file.
   4. Read from a sensor device.
3. Which of the following statements about transfer learning is false?
   1. It saves time.
   2. It saves resources.
   3. It refers to a specific version of its source dataset.
   4. It doesn’t allow model architecture changes.
4. Which of the following ESP frameworks can be used to develop...

Further reading

ML is a huge subject to learn about. The following books can be helpful in this journey:

• TinyML Cookbook – Second Edition, Gian Marco Iodice, Packt Publishing (https://www.packtpub.com/product/tinyml-cookbook-second-edition/9781837637362): Discusses TinyML on Arduino and Raspberry Pi with examples. The example in Chapter 4 teaches how to develop on Edge Impulse.
• Deep Learning with TensorFlow and Keras, Amita Kapoor, Antonio Gulli, and Sujit Pal (https://www.packtpub.com/product/deep-learning-with-tensorflow-and-keras-third-edition/9781803232911): Teaches about deep learning. A must-read book to design effective NN models.

Learn more on Discord

To join the Discord community for this book – where you can share feedback, ask questions to the author, and learn about new releases – follow the QR code below:

https://discord.gg/3Q9egBjWVZ