Chapter 10: Designing TinyML Applications

In the previous chapters, our projects ranged from building a weather station to a line-following robot. In this chapter, we will discuss developing TinyML applications using the Pico. We will start by introducing TinyML and its potential uses. Then, we will discuss an example that involves image classification.

In this chapter, we will cover the following topics:

• Introducing TinyML
• Keyword recognition in audio samples
• Classifying images
• Developing edge devices

Technical requirements

The following hardware is required for this chapter:

• Raspberry Pi Pico (https://bit.ly/3AJtoAf): USD 4
• Arducam Camera Shield (https://amzn.to/32KPqVN): USD 25.99
• Adafruit PDM microphone breakout (https://bit.ly/3tGCzRt): USD 4.95

Optional hardware

We also recommend the following optional hardware:

• Arduino RP2040 Connect (https://bit.ly/2YZClrY): USD 24.50
• Arducam Pico4ML TinyML Dev Kit (https://bit.ly/3DfKJTW): USD 25.99
• Arduino Tiny Machine Learning Kit (https://bit.ly/3M7m4nK): USD49.99

The code samples in this chapter can be found in this book's GitHub repository at https://github.com/PacktPublishing/Raspberry-Pi-Pico-DIY-Workshop/tree/main/chapter_10.

Code in Action videos for this chapter can be viewed at https://bit.ly/3yklorx.

Now, let's introduce TinyML.

Introducing TinyML

What is TinyML? TinyML refers to Tiny Machine Learning and it is a nascent but growing field where machine learning (ML) tools are used on resource-constrained hardware, such as an RP2040 microcontroller, to interpret sensor data. The resource constraints refer to the limited memory and processing power available on a microcontroller compared to a server with enormous processing power and GPU. TinyML allows you to interpret data on a microcontroller powered by a coin cell. A device that can interpret sensor data using TinyML tools locally instead of having to upload the data to the cloud is called an edge device.

Let's illustrate this concept with an example. The following diagram shows the flow of data in a typical IoT application, where we have a device that is collecting data from various sensors and forwarding it to the cloud. The inference happens in the cloud and the server running in the cloud instructs the gateway to turn devices on/off:

...

Keyword recognition in audio samples

In this section, we will discuss keyword recognition in an audio sample using the Pico. This is similar to the voice assistant devices that are designed and marketed by tech giants such as Amazon Echo, Google Home, and others that respond to an activation keyword such as "Hey Alexa." We will be making use of Edge Impulse (https://edgeimpulse.com) to work on this example. Sign up for a developer account on Edge Impulse to work along with this example.

Edge Impulse

Edge Impulse is a platform that allows you to develop TinyML applications using their platform and deploy them on a microcontroller. Edge Impulse is an interactive tool that allows you to develop applications without prior knowledge of machine learning.

If you are interested in becoming a certified developer, we recommend these courses from Edge Impulse on Coursera: https://www.coursera.org/edgeimpulse.

We worked on this example while using the following tutorial...

Classifying images

In this section, we will discuss an image classification example using the Pico. We will use a binary that has been already compiled and can be downloaded to determine whether a person is present in the frame that's been captured by a camera. Let's get started:

1. First, download the compiled binary from https://github.com/ArduCAM/RPI-Pico-Cam/blob/master/tflmicro/bin/person_detection_screen_int8.uf2.
2. The next step is to put the Pico in bootloader mode to load the binary. We recommend checking out Chapter 1, Getting Started with the Raspberry Pi Pico, if you are not familiar with the process.
3. Copy over the downloaded binary to the Pico.
4. Now, we need to interface the camera module to the Pico as follows, where the left-hand side of the arrow refers to a pin on the camera module and the right-hand side of the arrow refers to a pin on the Pico:
   • CS → GP5
   • MOSI → GP3
   • MISO → GP4
   • SCK → GP2
   • GND pins tied together
   • VCC &...

Developing edge devices

There are some factors to consider before you start developing a product that makes use of TinyML, as follows:

• The datasets that are available for developing your product. The best place to start is by making use of existing datasets or making use of data from an existing application.
• Product development cycles involving TinyML require a lot of trial and error to refine the parameters.
• TinyML applications are suitable for tapping new revenue streams and improving productivity.
• You also need to account for retraining your model from time to time to account for problems identified in the system.

Now that we have discussed some of the factors to consider before developing a product, let's summarize this chapter.

Summary

In this chapter, we introduced the topic of TinyML, its applications, and getting started with TinyML development. We also discussed a keyword recognition example using Edge Impulse and discussed testing it with a Pico. We also discussed an image classification example where we used a compiled binary to detect whether a person is present in the frame. We pointed a camera at a bobblehead to demonstrate this.

Finally, we wrapped up this chapter by discussing some factors to consider while developing an edge device.

In the next chapter, we will discuss developing a product around the Pico.