This chapter is a quick guide for setting up a development environment for Android and OpenCV. We will also look at the OpenCV sample applications, documentation, and community.
By the end of this chapter, our development environment will include the following components:
Java Development Kit (JDK) 6: It includes tools for Java programming.
Cygwin 1.7 or greater (Windows only): It is a compatibility layer that provides Unix-like programming tools on Windows.
Android Software Development Kit (Android SDK) r21.0.1 or greater: It includes tools for programming Android apps in Java.
Android Native Development Kit (Android NDK) r8d or greater: It includes tools for programming Android apps in C++. Although this book deals with Java programming, OpenCV also includes Android-compatible C++ samples that you may want to compile and run.
Eclipse 4.2.1 (Juno) or greater: It is an integrated development environment (IDE).
Java Development Tools (JDT): It is an Eclipse plugin for Java programming (already included in most Eclipse distributions).
C/C++ Development Tooling (CDT) 8.1.1 or greater: It is an Eclipse plugin for C/C++ programming.
Android Development Tools (ADT) 21.0.1 or greater: It is an Eclipse plugin for Android programming.
OpenCV4Android 2.4.3.2 or greater: It includes OpenCV's Android version, including Java and C++ libraries.
There are many possible ways to install and configure these components. We will cover several common setup scenarios, but if you are interested in even more options, see OpenCV's official documentation at http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/O4A_SDK.html.
All of the development tools for Android and OpenCV are cross platform. The following operating systems are supported with almost identical setup procedures:
Windows XP, Windows Vista, Windows 7, or Windows 8
Mac OS 10.6 (Snow Leopard) or greater
Ubuntu 10.10 (Maverick) or greater
Many other Unix-like systems (though not specifically covered in this book)
To run the OpenCV samples and, later, our own application, we should have an Android device with the following specifications:
Android 2.2 (Froyo) or greater (required)
Camera (required); front and rear cameras (recommended)
Autofocus (recommended)
Google Play Store (recommended)
Android Virtual Devices (AVDs) are not recommended. Some parts of OpenCV rely on low-level camera access and may fail with virtualized cameras.
Basically, we have a choice among the following approaches:
Install a prepackaged, preconfigured development environment that contains all the components we need.
Install various components separately and configure them to work together. Within this approach, we may do either of the following:
Use a prepackaged, preconfigured version of OpenCV
Configure and build OpenCV from source
Let's look at each of these alternatives in detail.
Tegra Android Development Pack (TADP) contains a complete, preconfigured development environment for Android, OpenCV, and some other libraries. TADP builds apps that are optimized for NVIDIA's Tegra processors. Despite being optimized for Tegra, the apps are compatible with other hardware too.
Tip
If you are setting up an Android development environment from scratch, I recommend TADP. It contains recent versions of all our required software and its setup process is simple.
TADP also contains some extras that we do not require for this book. For a complete list of TADP's contents, see the official description at https://developer.nvidia.com/tegra-android-development-pack.
To set up TADP, we just need to download and install it from a secure section of NVIDIA's website. Here are the required steps:
Join the NVIDIA Registered Developer Program at https://developer.nvidia.com/user/register. (It is free.)
Log in at https://developer.nvidia.com/user/login.
Complete your user profile at https://developer.nvidia.com/user/me/profile/rdp_profile.
Apply to join the Tegra Registered Developer Program at https://developer.nvidia.com/rdp/applications/tegra-registered-developer-program. (It is free, too!) Wait for NVIDIA to send you an acceptance email. Normally, you might receive it a few minutes after applying.
Go to https://developer.nvidia.com/tegra-resources and find the download link for TADP's latest version. At the time of writing, the latest version is 2.0r2. There are installers for Windows (32-bit or 64-bit), Mac, and Ubuntu (32-bit or 64-bit). Download and run the appropriate installer.
When the installer presents the Installation Directory step, we can enter any destination, which we will refer to as
<tadp>. By default,<tadp>isC:\NVPACK(Windows) or~/NVPACK(Mac and Ubuntu).When the installer presents the Installation Options step, we may select any option: Complete, Express, or Custom. Compared to an Express installation, a Complete or Custom installation may include additional versions of Android SDK and binary images of Tegra Android OS, which is NVIDIA's customization of Android. If in doubt, choose Express.
When the installer presents the Proxy Configuration step, we may leave all fields blank unless we are using a proxy server.
After finishing all of the installer's configuration steps, wait for TADP's content to be downloaded and installed.
That's all! Before proceeding, let's just take a note of the locations where TADP has installed certain components. For TADP 2.0r2 (the latest version at the time of writing), the locations are as follows:
Android SDK is located at
<tadp>/android-sdk-macosx. We will refer to this location as<android_sdk>.Android NDK is located at
<tadp>/android-ndk-r8d. We will refer to this location as<android_ndk>.OpenCV4Android is located at
<tadp>/OpenCV-2.4.3.2-android-sdk-tadp. We will refer to this location as<opencv>.Eclipse is located at
<tadp>/eclipse. We will refer to this location as<eclipse>.
Note
The TADP installer automatically edits the system's PATH to include <android_sdk>/platform-tools and <android_sdk>/tools. Also, it creates an environment variable called NDKROOT, whose value is <android_ndk>.
Now, we can proceed to Building the OpenCV Samples with Eclipse, later in this chapter.
Tip
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. The example code for this book is also available from the author's website at http://nummist.com/opencv/.
Instead of using TADP as a ready-made solution, we may assemble our own development environment. Broadly, this task has two stages:
Set up a general-purpose Android development environment
Set up OpenCV for use in this environment
Let's start by looking at the setup steps for a general-purpose Android development environment. We will not delve into very much detail here, because good instructions are available at the given links and, because you, as an Android developer, have probably been through similar steps before.
Tip
If you already have an Android development environment and you just want to add components to it, some of the following steps will not apply to you.
Here are the steps:
If we are using Windows or Linux, we may need to obtain JDK 6 manually. (On Mac, if JDK 6 is not present, the operating system will automatically offer to install it when needed.) The JDK 6 installers or packages are available for Windows and many Linux distributions at http://www.oracle.com/technetwork/java/javase/downloads/jdk6downloads-1902814.html. Alternatively, on Linux, check your repository for the JDK packages. Install JDK 6.
Download Eclipse and unzip it to any destination, which we will refer to as
<eclipse>. There are many versions from which we may choose. Google provides an Eclipse distribution called Android Developer Tools (ADT) Bundle, which comes with Android SDK and the ADT plugin prepackaged and preconfigured. ADT Bundle is available at http://developer.android.com/sdk/index.html. Many other up-to-date Eclipse distributions are available at http://www.eclipse.org/downloads/. Of these, Eclipse for Mobile Developers is a good and minimalist choice as a foundation for an Android development environment.If we did not get the ADT Bundle, we now need to set up Android SDK and the ADT plugin for Eclipse. Go to http://developer.android.com/sdk/index.html and get the download named Android SDK Tools. Install or unzip it to any destination, which we will refer to as
<android_sdk>. Open Eclipse and install the ADT plugin according to the official instructions at http://developer.android.com/sdk/installing/installing-adt.html. Restart Eclipse. A window, Welcome to Android Development, should appear. Click on Use Existing SDKs, browse to<android_sdk>, and click on Next. Close Eclipse.If we are using Windows, download and install Cygwin from http://cygwin.com/install.html.
Download Android NDK from http://developer.android.com/tools/sdk/ndk/index.html. Unzip it to any destination, which we will refer to as
<android_ndk>.Edit your system's
PATHto include<android_sdk>/platform-toolsand<android_sdk>/tools. Also, create an environment variable namedNDKROOTwith the value as<android_ndk>. (If you are unsure how to editPATHand other environment variables, see Appendix A: Editing environment variables.)Tip
Editing environment variables on Windows
The system's
Pathvariable and other environment variables can be edited in the Environment Variables window of Control Panel.On Windows Vista/7/8, open the Start menu and launch Control Panel. Now, go to System and Security | System | Advanced system settings. Click on the Environment Variables button.
On Windows XP, open the Start menu and go to Control Panel | System. Click on the Advanced tab. Click on the Environment Variables button.
Now, under System variables, select an existing environment variable, such as
Path, and click on the Edit button. Alternatively, make a new environment variable by clicking on the New button. Edit the variable's name and value as needed. For example, if we want to addC:\android-sdk\platform-toolsandC:\android-sdk\toolstoPath, we should append;C:\android-sdk\platform-tools;C:\android-sdk\toolsto the existing value ofPath. Note the use of semicolons as separators.To apply the changes, click on all the OK buttons until we are back in the main window of Control Panel. Now, log out and again log in.
Editing environment variables on Mac
Edit
~/.profile.To append to an existing environment variable, add a line such as
export PATH=$PATH:~/android-sdk/platform-tools:~/android-sdk/tools. This example appends~/android-sdk/platform-toolsand~/android-sdk/toolstoPATH. Note the use of colons as separators.To create a new environment variable, add a line such as
export NDKROOT=~/android-ndk.Save your changes, log out, and again log in.
Editing environment variables on Ubuntu
Edit
~/.pam_environment.To append to an existing environment variable, add a line such as
PATH DEFAULT=${PATH}:~/android-sdk/platform-tools:~/android-sdk/tools. This example appends~/android-sdk/platform-toolsand~/android-sdk/tools to PATH. Note the use of colons as separators.To create a new environment variable, add a line such as
NDKROOT DEFAULT=~/android-ndk.Save your changes, log out, and again log in.
Now, we have an Android development environment but we still need OpenCV. We may choose to download a prebuilt version of OpenCV or we may build it from source. These options are discussed in the following two subsections.
Tip
Generally, Android applications should use a prebuilt version of OpenCV. One important reason is that the prebuilt versions are available for Android users as shared libraries, which save disk space and simplify updates.
For the purpose of this book's project, there is no need to build OpenCV from source. We just mention this option for completeness, since it may be of interest to advanced users who want to modify OpenCV.
The prebuilt versions of OpenCV4Android can be downloaded from http://sourceforge.net/projects/opencvlibrary/files/opencv-android/. Look for files that have opencv-android in the name, such as OpenCV-2.4.5-android-sdk.zip (the latest version at the time of writing). Download the latest version and unzip it to any destination, which we will refer to as <opencv>.
Alternatively, the process for building OpenCV4Android from trunk (the latest, unstable source code) is documented at http://code.opencv.org/projects/opencv/wiki/Building_OpenCV4Android_from_trunk. For a summary of the process for building from trunk, continue reading this section. Otherwise, skip ahead to Building the OpenCV samples with Eclipse, later in this chapter.
Tip
Since trunk contains the latest and unstable source code, there is no guarantee that the build process will succeed. You may need to do your own troubleshooting if you want to build from trunk.
To build OpenCV from source, we need the following additional software:
Git: It is a Source Control Management (SCM) tool, which we will use to obtain OpenCV's source code. On Windows or Mac, download and install Git from http://git-scm.com/. On Linux, install it using your package manager. For example, on Ubuntu, open Terminal and run
$ sudo apt-get install git-core.CMake: It is a set of build tools. On Windows or Mac, download and install CMake from http://www.cmake.org/cmake/resources/software.html. On Linux, install it using your package manager. For example, on Ubuntu, open Terminal and run
$ sudo apt-get install cmake.Apache Ant 1.8.0 or greater: It is a set of build tools for Java. On Linux, just install Ant using your package manager. For example, on Ubuntu, open Terminal and run
$ sudo apt-get install ant. On Windows or Mac, download Ant from http://ant.apache.org/bindownload.cgi and unzip it to any destination, which we will refer to as<ant>. Make the following changes to your environment variables:Add
<ant>/bintoPATH.Create a variable,
ANT_HOME, with the value<ant>.
Python 2.6 or greater (but not 3.0 or greater): It is a scripting language that is used by some of the OpenCV build scripts. An appropriate version of Python comes preinstalled on Mac and most Linux systems, including Ubuntu. On Windows, download and install Python from http://www.python.org/getit/. If you have installed multiple versions of Python on your system, ensure that an installation of Python 2.6 or greater (but not 3.0 or greater) is the only one in
Path(Windows) orPATH(Mac, Linux, or other Unix-like systems). The OpenCV build scripts do not run properly with Python 3.0 or greater.
Once we have these prerequisites, we may download the OpenCV source code to any location, which we will refer to as <opencv_source>. Then, we may build it using an included script. The steps are platform-specific, and are described as follows:
On Windows, copy <opencv>\android\scripts\wincfg.cmd.tmpl to <opencv>\android\scripts\wincfg.cmd. Edit <opencv>\android\scripts\wincfg.cmd. The locations of several of the prerequisites are declared in this file. Modify them so that they are correct for your system. Save your changes. Then, open Git Bash (Git's command prompt) and run the following commands:
$ git clone git://code.opencv.org/opencv.git <opencv_source> $ cd <opencv_source>/android $ scripts/cmake_android.cmd $ cd build $ make -j8
On Mac, Ubuntu, or other Unix-like systems, open Terminal (or another command line shell) and run the following commands:
$ git clone git://code.opencv.org/opencv.git <opencv_source> $ cd <opencv_source>/android $ sh ./scripts/cmake_android.sh $ cd build $ make -j8
If all goes well, we should get a build of OpenCV4Android in <opencv_source>/android/build. We may move it elsewhere if we wish. We will refer to its final location as <opencv>.
Building and running a few sample applications is a good way to test that OpenCV is correctly set up. At the same time, we can practice using Eclipse.
Let's start by launching Eclipse. The Eclipse launcher should be located at <eclipse>/eclipse.exe (Windows), <eclipse>/Eclipse.app (Mac), or <eclipse>/eclipse (Linux). Run it.
We should see a window called Workspace Launcher, which asks us to select a workspace. A workspac
e is the root directory for a set of related Eclipse projects. If we are using TADP, enter <tadp>/nvsample_workspace, which is a workspace where the OpenCV4Android library, samples, and tutorials are already set up as projects. Otherwise, enter any location you choose.
If the Welcome to Eclipse screen appears, click on the Workbench button.
Now, we should see a window with several panels, including Package Explorer. If we are not using TAPD, we need to import the OpenCV sample projects into our new workspace. Right-click on Package Explorer and select Import… from the context menu.
The Import window should appear. Select General | Existing Projects into Workspace, and then click on Next>.
On the second page of the Import window, enter <opencv> in the Select root directory: field. Under the Projects: label, a list of detected projects should appear (If not, click on Refresh). The list should include OpenCV library, samples, and tutorials. Ensure that all projects are selected and click on Finish to import them.
Once the projects are imported, we may need to fix some configuration issues. Our development environment may have different paths, and different versions of the Android SDK, than the ones in the samples' default configuration.
Any resulting errors will be reported in the Problems tab.
Tip
We should start by resolving any errors in the OpenCV Library project, as the samples and tutorials depend on the library.
The following are some of the common configuration problems, and their symptoms and solutions:
The target Android version might not be properly specified. The symptoms are that imports from the java and android packages fail, and there are error messages such as
The project was not built since its build path is incomplete. The solution is to right-click on the project in Package Explorer, select Properties from the context menu, select the Android section, and checkmark one of the available Android versions. These steps should be repeated for all projects. At compile time, OpenCV and its samples must target Android 3.0 (API level 11) or greater, though at runtime they also support Android 2.2 (API level 8) or greater.
If we are working on Mac or Linux, the C++ samples might be misconfigured to use the Windows build executable. The symptom is an error message such as
Program "{ndk}/ndk-build.cmd" not found in PATH. The solution is to right-click on the project in Package Explorer, select Properties from the context menu, select the C/C++ Build section, and edit the Build command: field to remove the.cmdextension. These steps should be repeated for all the native (C++) projects, which include OpenCV Sample - face-detection and OpenCV Tutorial 2 - Mixed Processing.
Once the OpenCV projects no longer show any errors, we can prepare to test them on an Android device. Recall that the device must have Android 2.2 (Froyo) or a greater version, and a camera. To enable Eclipse to communicate with the device, we must enable the device's USB debugging option with the help of the following steps:
Open the Settings app.
On Android 4.2 or greater, go to the About phone or About tablet section and tap Build number seven times. This step enables the Developer options section.
Go to the Developer options section (on Android 4.0 or greater) or the Applications | Development section (on Android 3.2 or less). Enable the USB debugging option.
Now, open the Play Store app, and find and install the OpenCV Manager app. (The app's page in the Play Store should look similar to the previous screenshot.) OpenCV Manager takes care of checking for any OpenCV library updates when we run any OpenCV applications.
Tip
If you do not have the Play Store app on your device, then you need to install OpenCV Manager and certain OpenCV libraries via USB as per the instructions at http://docs.opencv.org/android/service/doc/UseCases.html.
Now, we must prepare our main computer for communication with the Android device. The required steps vary, depending on our operating system.
On Windows, we need to install the proper USB drivers for the Android device. Different vendors and devices have different drivers. The official Android documentation provides links to the various vendors' driver download sites at http://developer.android.com/tools/extras/oem-usb.html#Drivers.
On Linux, before connecting an Android device via USB, we must specify the device's vendor in a permissions file. Each vendor has a unique ID number, as listed in the official Android documentation at http://developer.android.com/tools/device.html#VendorIds. We will refer to this ID number as <vendor_id>. To create the permissions file, open a command prompt application (such as Terminal) and run the following commands:
$ cd /etc/udev/rules.d/ $ sudo touch 51-android.rules $ sudo chmod a+r 51-android-rules
Note that the permissions file needs to have root ownership, so we use sudo when creating or modifying it. Now, open the file in an editor such as gedit:
$ sudo gedit 51-android-rules
For each vendor, append a new line to the file. Each of these lines should have the following format:
SUBSYSTEM=="usb", ATTR{idVendor}=="<vendor_id>", MODE="0666", GROUP="plugdev"Save the permissions file and quit the editor.
On Mac, no special drivers or permissions are required.
Plug the Android device into your computer's USB port. In Eclipse, select one of the OpenCV sample projects in Package Explorer. Then, from the menu system, select Run | Run as… | Android Application.
An Android Device Chooser window should appear. Your Android device should be listed under Choose a running Android device. (If the device is not listed, try unplugging it and plugging it back in. If that does not work, also try disabling and re-enabling the device's USB debugging option, as described earlier.)
Select the device and click on OK.
If the Auto Monitor Logcat window appears, select the Yes radio button and the verbose drop-down option, and click on OK. This option ensures that all the log output from the application will be visible in Eclipse.
On the Android device you might get a message: OpenCV library package was not found! Try to install it? Make sure the device is connected to the Internet and then touch the Yes button on your device. The Play Store will open to show an OpenCV package. Install the package and then press the hardware back button to return to the sample application, which should be ready for use.
For OpenCV 2.4.3.2, the samples and tutorials have the following functionality:
Sample – 15 puzzle: It splits up a camera feed to make a sliding-block puzzle. The user may swipe blocks to move them.
Sample – color-blob-detection: It detects color regions in a camera feed. The user may touch anywhere to see the outline of a color region.
Sample – face-detection: It draws green rectangles around faces in a camera feed.
Sample – image-manipulations: It applies filters to a camera feed. The user may press the Android menu button to select from a list of filters.
Tutorial 1 – Add OpenCV: It displays a camera feed. The user may press the … menu to select a different camera feed implementation (Java or native C++).
Tutorial 2 – Use OpenCV Camera: It applies filters to a camera feed. The user may press the … menu to select from a list of filters.
Tutorial 3 – Add Native OpenCV: It draws red circles around interest points or features in a camera feed. Generally speaking, interest points or features lie along the high-contrast edges in an image. They are potentially useful in image recognition and tracking applications.
Tutorial 4 – Mix Java+Native OpenCV: It combines the functionality of Tutorial 2 and Tutorial 3. The user may press the … menu to select from a list of filters and a preview of interest points.
Tutorial 5 – Camera Control: It applies filters to a camera feed, which has a customizable resolution. The user may press the … menu to select from a list of filters and a list of resolutions.
Try these applications on your Android device! While an application is running, its log output should appear in the LogCat tab in Eclipse.
Feel free to browse the projects' source code via Package Explorer, to see how they were made. Alternatively, you might want to return to the official samples and tutorials later, after we have built our own project over the course of this book.
The OpenCV Java API and C++ API are both relevant to Android. The Java API documentation is online at http://docs.opencv.org/java/. The C++ API documentation is online at http://docs.opencv.org/. The following documents, which mostly use C++ code, are also available as downloadable PDF files:
API reference: http://docs.opencv.org/opencv2refman.pdf
Tutorials: http://docs.opencv.org/opencv_tutorials.pdf
User guide (incomplete): http://docs.opencv.org/opencv_user.pdf
If the documentation does not seem to answer your question, try talking to the OpenCV community. Here are some sites where you will find helpful people:
Official OpenCV forum: http://www.answers.opencv.org/questions/
Jay Rambhia's blog: http://jayrambhia.wordpress.com/
The support site for my OpenCV books: http://nummist.com/opencv/
Also, you can read or submit bug reports at http://code.opencv.org/projects/opencv/issues?query_id=4. Finally, if you need to take your issue to the highest authority, you can email the OpenCV4Android developers at <android@opencv.org>.
By now, we should have an Android and OpenCV development environment that can do everything we need for the project described in this book. Depending on which approach we took, we might also have a set of tools that we can use to reconfigure and rebuild OpenCV for our future needs.
We know how to build the OpenCV Android samples in Eclipse. These samples cover a different range of functionality than this book's project, but they are useful as additional learning aids. We also know where to find documentation and help.
