OpenCV 3.0 Computer Vision with Java

The first thing to notice when working with OpenCV for Java development is that OpenCV is a C++ library that should be compiled with operating system- specific compilers. The native code that would be generated is platform-dependent. So, the native Linux code won't run in Windows, neither will the Android native code run in OSX. This sounds very different from the bytecode generated for Java, which is executed by an interpreter in any platform. In order to get the native code running in a Java Virtual Machine (JVM), one needs the so called Java Native Interface (JNI). This way, the native code will be required for each platform that your application is going to be run on.

It is important to understand that JNI is a native programming interface. It allows the Java code that runs inside a JVM to interoperate with the applications and libraries written in programming languages such as C, C++, and assembly. Since it bridges the gap between Java and other languages, it needs to convert datatypes from these languages, as well as to create some boilerplate code. Curious readers should refer to the gen_java.py script, located in the modules/java/generator folder, which automates most of this work. Lucky Windows users get compiled binaries, which means source C++ OpenCV code, compiled with Windows compilers into native code that runs only on Windows, from OpenCV packages. Users from other operating systems will need to build binaries from the source code, although one can make that in Windows as well. In order to download compiled binaries, we should get version 2.4.4 or higher of the OpenCV Windows package from the OpenCV SourceForge repository, which is located at http://sourceforge.net/projects/opencvlibrary/files/.

In this section, we are mostly interested in generating all the OpenCV Java class files contained in a JAR file as well as the native dynamic library for Java OpenCV. This is a self-contained library that works with JNI and is required to run a Java OpenCV application.

In case you are working with Linux or OSX, or if you want to build from the source in Windows, then to get the latest features committed in OpenCV, you should use the source code. You can visit the OpenCV download page at http://opencv.org/downloads.html and choose the appropriate link for your distribution.

Another way to get the source code is by using the git tool. Appropriate instructions for installing it can be found at http://git-scm.com/downloads. When using git, use the following commands:

git clone git://github.com/Itseez/opencv.git
cd opencv
git checkout 3.0.0-rc1
mkdir build
cd build

These commands will access the OpenCV developers' repository and download the most updated code from branch 3.0.0-rc1, which is the release candidate for version 3.0.0.

In either method of obtaining the source code, you will need building tools in order to make binaries. The required packages are as follows:

In order to install these software in a Linux distribution such as Ubuntu or Debian, the user should issue the following command:

sudo apt-get install build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev ant

Once you have installed all these packages, you will be ready to build the library. Make sure you are in the build directory, as you should be, if you have followed the preceding Git instructions. In case you downloaded the source file from OpenCV downloads, the parent folder of your build should have CMakeLists.txt as well as the 3rdparty, apps, cmake, data, doc, include, modules, platforms, samples, and test folders.

CMake is a build tool and it will generate your compiler-specific solution files. You should then use your compiler to generate the binary files. Make sure you are in the build directory, as this should follow the last cd build command. If you are using Linux, run the following commands:

cmake -DBUILD_SHARED_LIBS=OFF

If you are using Windows, run the following command:

cmake -DBUILD_SHARED_LIBS=OFF -G "Visual Studio 10"

Notice that it is important to use the DBUILD_SHARED_LIBS=OFF flag, because it will instruct CMake to build OpenCV on a set of static libraries. This way, it will compile a single dynamic link library for Java without dependencies on other libraries. This makes it easier to deploy your Java projects.

In case you want to use MinGW makefiles, just change the CMake command to the following command:

cmake -DBUILD_SHARED_LIBS=OFF -G "MinGW Makefiles"

One of the key points to watch for when generating project files through CMake is that java is one of the modules that is going to be built. You should see a screen as shown in the following screenshot:

In case you can't see java as one of the to-be-built modules, like in the following screenshot, you should look for a couple of things, such as whether Ant is correctly configured. Also make sure that you have set the ANT_HOME environment variable and that Python is correctly configured. Check if NumPy is installed by simply typing numpy import * in a Python shell and check for any errors:

In case you are in doubt about the Python and Java installations, slide down to check their configurations. They should be similar to the next screenshot:

Once everything has been correctly configured, it is time to start compiling the sources. In order to do so in Windows, type the following:

msbuild /m OpenCV.sln /t:Build /p:Configuration=Release /v:m

Notice that you might get an error saying, 'msbuild' is not recognized as an internal or external command, operable program or batch file. This occurs when you haven't set the msbuild path. In order to set it right, open Visual Studio and in the Tools menu, click Visual Studio Command Prompt. This will yield a fully working command prompt with access to msbuild. Refer to the following screenshot for clearer directions:

In case you are using newer Visual Studio versions, press the Windows key and type VS2012 Command Prompt. This should set up your environment variables.

In order to start building in Linux, simply type the following command:

make -j8

The preceding command will compile the OpenCV library with Java support. Notice that the -j8 flag tells make to run in parallel with eight job threads, which makes the build theoretically faster.

The entire process will last for some minutes before generating a JAR file that contains the Java interfaces, which is located at bin/opencv-300.jar. The native dynamic link library containing Java bindings is generated at lib/libopencv_java300.so or bin/Release/opencv_java300.dll, depending on your operating system. These files will be used when we create our first OpenCV application.

Congratulations! You are now halfway to becoming a great developer using OpenCV!

Using OpenCV in any IDE is pretty simple. It is as simple as adding OpenCV JAR, that is, opencv-300.jar to your classpath. But, as it relies on the native code, you need to point out the dynamic link libraries—so for Linux, .dll for Windows, and dylib for MacOsX.

Notice that your project now points to the OpenCV JAR. You can also browse the main classes from the Package Explorer, as seen in the following screenshot:

After the The NetBeans configuration section, a source code to create a simple OpenCV application can be found.

In case you are more comfortable working with NetBeans, the configuration process is pretty much like Eclipse:

Make sure you change the given path to the one where your OpenCV installation is, and that it points to the folder where the native libraries are, that is, opencv_java300.dll in Windows, or libopencv_java300.so in Linux. Now, add the SimpleSample class code in your project, as pointed. Run the sample and make sure that you don't get any errors.

It's time to create a simple application that will show that we can now compile and execute Java code with OpenCV. Create a new Java class containing a Main method and paste the code given as follows. It simply creates a 5 x 10 OpenCV matrix, sets some of its rows and columns, and prints the result to the standard output.

Make sure you load the correct dynamic link libraries through a call to System.loadlibrary("opencv_java300"). Since, you might want to change the library version later, a better approach would be to use the Core.NATIVE_LIBARAY_NAME constant, which will output the correct library name. You can also find this file in the code repository for chapter1 of this book, under ant/src.

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.CvType;
import org.opencv.core.Scalar;

class SimpleSample {

  static{ System.loadLibrary(Core.NATIVE_LIBRARY_NAME); }

  public static void main(String[] args) {
    System.out.println("Welcome to OpenCV " + Core.VERSION);
    Mat m = new Mat(5, 10, CvType.CV_8UC1, new Scalar(0));
    System.out.println("OpenCV Mat: " + m);
    Mat mr1 = m.row(1);
    mr1.setTo(new Scalar(1));
    Mat mc5 = m.col(5);
    mc5.setTo(new Scalar(5));
    System.out.println("OpenCV Mat data:\n" + m.dump());
  }

}

According to Oracle's documentation, it states that, class can have any number of static initialization blocks. And they can appear anywhere in the class body. The runtime system guarantees that static initialization blocks are called in the order that they appear in the source code.

You should make sure that any calls to the OpenCV library are preceded by a single System.loadLibrary call, in order to load the dynamic libraries. Otherwise, you will receive an java.lang.UnsatisfiedLinkError: org.opencv.core.Mat.n_Mat(IIIDDDD)J error. This generally occurs in a static block.

If everything goes well, you should see the following output in the console:

Welcome to OpenCV 3.0.0-rc1
OpenCV Mat: Mat [ 5*10*CV_8UC1, isCont=true, isSubmat=false, nativeObj=0x2291b70, dataAddr=0x229bbd0 ]
OpenCV Mat data:
[ 0, 0, 0, 0, 0, 5, 0, 0, 0, 0;
  1, 1, 1, 1, 1, 5, 1, 1, 1, 1;
  0, 0, 0, 0, 0, 5, 0, 0, 0, 0;
  0, 0, 0, 0, 0, 5, 0, 0, 0, 0;
  0, 0, 0, 0, 0, 5, 0, 0, 0, 0]

If you want to rely on Apache Ant for building instead of using an IDE, a build.xml file is provided in the OpenCV samples. You can find this file in this chapter's repository as well. The following are its contents:

<project name="SimpleSample" basedir="." default="rebuild-run">
    <property name="src.dir"     value="src"/>
    <property name="lib.dir"     value="${ocvJarDir}"/>
    <path id="classpath">
        <fileset dir="${lib.dir}" includes="**/*.jar"/>
    </path>
    <property name="build.dir"   value="build"/>
    <property name="classes.dir" value="${build.dir}/classes"/>
    <property name="jar.dir"     value="${build.dir}/jar"/>
    <property name="main-class"  value="${ant.project.name}"/>

    <target name="clean">
        <delete dir="${build.dir}"/>
    </target>

    <target name="compile">
        <mkdir dir="${classes.dir}"/>
        <javac includeantruntime="false" srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath"/>
    </target>

    <target name="jar" depends="compile">
        <mkdir dir="${jar.dir}"/>
        <jar destfile="${jar.dir}/${ant.project.name}.jar" basedir="${classes.dir}">
            <manifest>
                <attribute name="Main-Class" value="${main-class}"/>
            </manifest>
        </jar>
    </target>

    <target name="run" depends="jar">
        <java fork="true" classname="${main-class}">
            <sysproperty key="java.library.path" path="${ocvLibDir}"/>
            <classpath>
                <path refid="classpath"/>
                <path location="${jar.dir}/${ant.project.name}.jar"/>
            </classpath>
        </java>
    </target>

    <target name="rebuild" depends="clean,jar"/>

    <target name="rebuild-run" depends="clean,run"/>

</project>

This is a basic build.xml Ant file that defines tasks such as cleaning, compiling, and packing a .jar file, running, rebuilding, and rebuild-running. It expects your source code to be in a sibling folder called src. Make sure that the SimpleSample.java source code provided earlier is inside this directory.

Compiling and running the project using Ant is easy. Simply type the following:

ant -DocvJarDir=path/to/dir/containing/opencv-300.jar -DocvLibDir=path/to/dir/containing/opencv_java300/native/library

In case you have downloaded and extracted pre-built binaries, use the following command instead:

ant -DocvJarDir=X:\opencv3.0.0\opencv\build\java -DocvLibDir=X:\opencv3.00\opencv\build\java\x64

A successful run of Ant build.xml will look like the following screenshot:

The provided build.xml file can be reused for building your Java OpenCV applications. In order to use it, make sure that the project name matches your main class name. If your main class is inside the package com.your.company, and it's called MainOpenCV, you should change the first line of build.xml from <project name="SimpleSample" basedir="." default="rebuild-run"> to <project name="com.your.company.MainOpenCV" basedir="." default="rebuild-run">.

You can also hardcode the ocvJarDir and ocvLibDir properties so you won't have to type them while invoking Ant. For ocvJarDir, simply change the <property name="lib.dir" value="${ocvJarDir}"/> command to <property name="lib.dir" value="X:\opencv2.47\opencv\build\java"/>.

Apache Maven is a more complex build automation tool, primarily used for Java projects. It describes not only how software is built, but also how it depends on other libraries. Its projects are configured through a Project Object Model, named pom.xml. Maven dependencies are usually located in Maven 2 Central Repository. In case they aren't found there, you will need to add other repositories. You can also create a local repository and add your own dependencies there. At the time of writing this book, there were no public dependencies for Java OpenCV. So we will cover not only the process of installing the Java OpenCV Maven dependencies in a local repository but also how to use this book's Maven repository for the Windows builds of OpenCV 3.0.0 version. In case OpenCV developers host public Maven repositories, minor changes will be required. You will only need to find out the official OpenCV JAR groupId, artifactId, and version and put them in your pom.xml.

In order to make your project dependent on any library, you only need to provide three fields in your pom.xml. They are groupId, artifactId, and version. The recommended way to make your project depend on libraries that are not hosted in the Central Maven Repository, is to install them using a simple command, like mvn install:install-file -Dfile=non-maven-proj.jar -DgroupId=some.group -DartifactId=non-maven-proj -Dversion=1 -Dpackaging=jar.

We will show you how to use the Packt repository for window builds in the next section and then we will give you the details on how to install them on your local repository, in case you need it.

mvn install:install-file -Dfile=opencvjar-3.0.0.jar -DgroupId=opencvjar -DartifactId=opencvjar -Dversion=3.0.0 -Dpackaging=jar

mvn install:install-file -Dfile=opencvjar -runtime-natives-linux-x86.jar -DgroupId=opencvjar -DartifactId=opencvjar-runtime -Dversion=3.0.0 -Dpackaging=jar -Dclassifier=natives-linux-x86

This chapter provided several different approaches for setting up OpenCV for Java, that is, by either installing compiled binaries or compiling it from the source. It also pointed to instructions for making the main configurations in Eclipse and NetBeans IDE as well as for using building tools such as Ant and Maven. The user should be ready to easily start using OpenCV in his/her Java projects.

The next chapter will go deeper into OpenCV and address basic tasks such as handling images through matrices, reading image files, retrieving frames from a webcam, and creating nice Swing GUIs for your computer vision applications.