Mastering BeagleBone Robotics

The first thing you need to do is install a useful operating system into an 8 GB, 16 GB, or 32 GB micro SD card—just a little background for this. Initially, the default operating system on the internal memory was a version of Linux called Ångström. Version C is now shipped with a version of Debian. You're going to move to a different operating system called Ubuntu, a close relative of Debian, but with a larger feature set and community of support. It'll make the projects you are about to tackle much easier.

Now the BeagleBone Black is getting popular enough, so you can buy a micro SD card that already has Ubuntu installed. Nevertheless, you can easily download it and then install it on the card. If you're going to download a distribution, you'll need to make sure you're getting the latest version of Ubuntu.

Both of these sites come with instructions for building a card with the image on it. You'll need to use the username and password of the image you downloaded. Unfortunately, they are not the same for all images, but you should be able to easily find them in the same place where you found your image.

In this book, I'm going to assume that you'll be developing your code on an external computer while you create your projects. I'll primarily be using a Windows PC for development, but everything I'll be doing will also be available if you are using a Linux machine as the host computer. Whenever there are any significant differences between the two, I'll try to include details for both.

To do this development remotely, you'll need access to the BeagleBone Black via some type of LAN connection. To establish this connection, simply connect a LAN cable from a router or switch to the BeagleBone board, plug it into the LAN connector, and restart the BeagleBone.

You'll need to know the IP address of your BeagleBone Black. You can certainly get this by adding a display and keyboard to the device, logging in, and typing ifconfig in the prompt. However, if you want to do this without rounding up the additional hardware, then you can use an IP scan tool to scan for the address of the BeagleBone Black. I used a tool called Advanced IP Scanner, but there are others available too. If you're using Linux as a host machine, you can try the nmap command in Linux. For example, you can type sudo nmap –PR –sP 192.168.1.0/24.

These will explore your network and then print the address where there are devices, the BeagleBone Black being one of those devices. Generally, there are two types of IP addresses that your board can be assigned: one is called static and the other is called dynamic.

In the static case, you will always be assigned the same address. In the dynamic case, the address might change each time the system boots, as it asks the system for an address, which it then uses for that session. Most systems are configured for the dynamic case. However, if your system isn't changing the address, you will most likely get the same address each time you power on and log in to the system.

Once you have the address, you should create an SSH connection to the BeagleBone Black. An SSH terminal is a Secure Shell Hyperterminal connection, which simply means that you'll be able to access your board and type commands in the prompt, just like you have done without the Windows system. In order to do this, you need to have a terminal emulator on your remote computer. For Microsoft Windows, I would suggest you to use an application called PuTTY. It is free and does a very good job by allowing you to save your configuration, so you don't have to type it every time. You can go to www.putty.org for details on how to download and use PuTTY.

If you want to do this from a Linux machine, the process is even simpler. Open a terminal window and then type ssh ubuntu@157.201.194.187 –p 22. This will then bring you to the login screen of your BeagleBone Black.

SSH is an excellent way to communicate with your BeagleBone Black. However, you'll sometimes need a graphical look at your system. You can get this by installing a graphical interface on the BeagleBone Black and then using an application called VNC server.

First, let's install the Windows manager on your BeagleBone Black. Ubuntu generally comes with a very full-featured Windows system. However, it uses a good deal of memory and can interfere with the performance you might need later. So you should install a light Windows system at the top of your Ubuntu distribution. There are several choices, of which I like using Xfce. It is stable, seems to work well, and offers a fairly complete set of capabilities while not overwhelming your system resources. Before getting started, first type sudo apt-get update. This will update all the latest information about installation. To install the Windows manager, type sudo apt-get install xfce4. Again, the system will prompt you for your password and then start the installation. For large installations, the Linux system will ask you if you want to proceed. If you don't always want to type in y to this, you can use apt-get –y to automatically answer with a yes. This installation will take a significant amount of time as it has to install not only the Windows system, but also a number of packages the windowing system depends on.

Now that you have a graphical interface, you'll need to install a version of VNC server on your BeagleBone Black by typing sudo apt-get install tightvncserver in a terminal window on your BeagleBone Black. The TightVNC server is an application that will allow you to remotely view your complete Windows system. Once you have it installed, you'll need to start the server by typing vncserver in a terminal window on the BeagleBone Black. You will then be prompted for a password. This can, and preferably should, be a different password than the password using which you access your BeagleBone Black. This will be the password your remote system will send to access the VNC server running on the board. Select a password and your VNC server will be running. You only need to set this password once. Besides, you don't need to set the password for the view-only capability.

You'll need a VNC viewer application for your remote computer. On my Windows system, I used an application called Real VNC. Go to www.realvnc.com for information on how to set up and use this application. You can now access all the capabilities of your system. However they might be slower if you are doing graphics-intensive data transfers. You'll see this as you work through your projects. VNC viewer is also available via Linux.

You now have a basic BeagleBone Black configuration, so you can add some additional core packages you'll be using in the projects.

Now you'll install OpenCV, an open source library of image processing and a web camera that provides access capabilities you'll use on these projects.

First, you'll need to download a set of libraries and the OpenCV itself. There are several possible steps; I'm going to suggest one that I followed to install it on my system. Once you have booted the system and opened a terminal window, type the following commands in the same order as they have been explained:

Now you're ready to try out the OpenCV library. I prefer to use Python when programming simple tasks, so I used the Python examples. If you prefer the C examples, feel free to explore. In order to use the Python examples, you'll need one more library. So type sudo apt-get install python-numpy as you will need this to manipulate the matrices that OpenCV uses to hold the images you will get from your webcam.

Now that you have those, you can try one of the Python examples. Change the current directory to that of the Python examples by typing cd /home/ubuntu/examples/python. In this directory, you will find a number of useful examples; you only need to look at the most basic example, called camera.py. You can try running this example, but you'll need to connect a USB web camera, reboot, bring up a VNC server connection, bring up a terminal window, and type python camera.py. You should see something like this:

In my case, the camera window eventually turned black and did not show the output from the camera. I realized that I needed to change the resolution of the image to one that is supported by the camera and OpenCV. To do this, you need to edit the camera.py file, adding two lines like this:

These two lines, cv.SetCaptureProperty(capture, 3, 360) and cv.SetCaptureProperty(capture, 4, 240), change the resolution of the captured image to 360 x 240 pixels. Now run camera.py and you should see this:

Your project can now see! You will use this capability to do a number of impressive tasks that will use this video input.

The second piece of open source infrastructure that you'll add now is the voice recognition software—pocketsphinx. To make it work, you'll need to purchase a USB sound card, as the BeagleBone Black does not have an audio in/out provision. You'll also need to purchase a microphone and speaker to attach to your USB sound card. Connect these, and reboot the system. You're now ready to install the software.

You can do this over the LAN connection via an SSH terminal window, so as soon as your board flashes that it has power (look out for the heartbeat LED), open up an SSH terminal window using PuTTY or some similar terminal emulator. Once the terminal window comes up, log in with your username and password. Now type cat /proc/asound/cards. You should see the following response:

Notice that the system thinks there are two possible audio devices. The first is the HDMI sound device and the second is your USB audio card. Now you can use the USB card to both create and record sound.

First, let's get some music going. This will let you know that your USB sound device is working. You'll need to first configure your system to look for your USB card and play and record some sounds from there as default. To do this, you'll need to add a couple of libraries to your system:

If you can't hear your recording, check alsamixer to make sure your speakers and microphone are both unmuted.

Now that your projects can speak, you will want them to listen as well. This isn't nearly as simple as the speaking part, but thankfully you have some significant help. You will download a set of capabilities called pocketsphinx, and using these capabilities, you will provide your project with the ability to listen to your commands.

The first step is to download the pocketsphinx capability. Unfortunately, this is not as user friendly as the Espeak process, so follow the steps carefully. First, go to the Sphinx website, hosted by Carnegie Mellon University at http://cmusphinx.sourceforge.net/. This is an open source project that provides you with the speech recognition software you will need. With your smaller embedded system, you will be using the pocketsphinx version of this code. You will need to download two pieces of software, sphinxbase and pocketsphinx. Download these by selecting the Download section at the top of the page, and then find the latest version of both the packages. Download the .tar.gz versions of these and move them to the /usr/ubuntu directory of your BeagleBone Black. However, before you build these, you'll need another library.

This library is called bison. It's a general purpose, open source parser that will be used by pocketsphinx. To get this package, type sudo apt-get install bison.

If everything explained so far is installed and downloaded, you can build pocketsphinx as follows:

Now you are ready to actually build the sphinxbase code base. This is a two-step process. First type make, and the system will build all the executable files. Then type sudo make install and it will install all the executables on the system.

Now make the second part of the system, the pocketsphinx code itself, as follows:

Notice the 000000001: hello line. It recognized your speech! You can try other words and phrases. The system is very sensitive, so it might also pick up background noise. You are also going to find out that it is not very accurate. There are two ways to make it more accurate. One is to train the system to understand your voice more accurately. I'm not going to detail that process here. It's a bit complex, and if you want to know more, feel free to go to the CMU pocketsphinx website at http://cmusphinx.sourceforge.net/.

The second way to improve accuracy is to limit the number of words that your system can use to determine what you are saying. The default has literally thousands of words that are possible, so if two words are close, it might choose the wrong word as opposed to the word you spoke. In order to make the system more accurate, you are going to restrict the words it has to choose from. You can do this by making your own grammar.

The first step is to create a file with the words or phrases you want the system to recognize. Then you use a web tool to create two files that the system will use to define your grammar:

It will now look up that directory as it tries to find matches to your commands.

Now that your system can both hear and speak, you would want to provide the capability to respond to your speech, and perhaps even execute some commands based on the speech input. Now you're going to configure the system to respond to your simple commands.

In order to respond, you're going to edit the continuous.c code in the /home/ubuntu/pocketsphinx-0.8/src/programs directory. You can create your own .c file, but this file is already set up in the makefile system, and will serve as an excellent starting spot. You will need to edit the continuous.c file. It's very long, and a bit complicated, but you should be specifically looking out for the following section in the code:

In this section of the code, the word has already been decoded, and is held in the hyp variable. You can add some code here to make your system do things based on the value associated with the word you have decoded. First, let's try adding the capability to respond to hello and goodbye, and see if you can get the program to stop. Make the following changes to the code:

Now you need to rebuild your code. Since the make system already knows how to build the pocketsphinx_continuous program, any time you make a change to the continuous.c file, it will rebuild the application. Simply type make. The file will compile and create a new version of pocketsphinx_continuous. To run your new version, type sudo ./pocketsphinx_continuous. Make sure you type ./ at the start of pocketsphinx_continuous. If you don't, the system has another version of pocketsphinx_continuous in the library and it will run that.

If everything is set correctly, saying hello should result in a response of hello from your BeagleBone Black. Saying goodbye should elicit a response of goodbye, as well as shutting down the program. Note that the system command can be used to actually run any program that you might run with a command line. You can now use this to have your program started and run other programs based on the voice commands.