Object Detection with TensorFlowSharp

In this chapter, we are going to introduce you to an open source package called TensorFlowSharp. More specifically, we will be using the TensorFlow[1] Object Detection API, which is an open source framework built on top of TensorFlow, which makes it easy to construct, train, and deploy various forms of object detection models.

For those not familiar with TensorFlow, the following is an excerpt from the TensorFlow website[2]:

"TensorFlow is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (such as CPUs, GPUs, and TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google's AI organization, it comes with strong...

Technical requirements

You will be required to have a basic knowledge of .NET development using Microsoft Visual Studio and C#. You will need to download the code for this chapter from this book's website: TensorFlowSharp (https://github.com/migueldeicaza/TensorFlowSharp).

Check out the following video to see Code in Action: http://bit.ly/2pqEiZ9.

Working with Tensors

Let's set the stage by talking about exactly what a Tensor is. To do so, we should also talk a little bit about vectors and matrices as well. You can skip this section if you are already familiar, but it is short and if you already know about matrices and vectors, who knows, you might remember something you've forgotten! So go ahead and read it anyway!

Now, before we talk, let me show you a graphic that may make things a tad easier to visualize:

A vector is an array of numbers, as you can see here:

A matrix is a grid of n x m numbers, a two-dimensional array. We can do all kinds of neat operations on a matrix, such as addition and subtraction, so long as the sizes are compatible:

We can multiply matrices if we so desire, like this:

And matrices can be added together, like this:

In both cases, we are working within a two-dimensional space. So...

Developing your own TensorFlow application

Now that we've shown you some preliminary code samples, let's move on to our example project—how to use TensorFlowSharp from a console application to detect objects within an image. This code is easy enough for you to be able to add into your solution if you so desire. Just tweak the input and output names, perhaps allow for user adjusted hyperparameters, and you're off!

To run this solution, you should have the source code for this chapter downloaded from the website and open in Microsoft Visual Studio. Please follow the instructions for downloading code for this book:

Before we dive into the code, let's talk about one very important variable:

private static double MIN_SCORE_FOR_OBJECT_HIGHLIGHTING = 0.5;

This variable is our threshold for identifying and highlighting objects in our base image. At 0.5, there...

Detecting images

Now it's time to move on to a real project. In this example, we are going to take our base image (seen as follows) and use it to have the computer detect objects in the image. As you can see, there are several instances of people and kites in the photograph. This is the same base image used in all TensorFlowSharp examples. You are going to see the detection and highlighting progresses changes as we change our minimum allowed threshold.

Here is our base sample image, a photograph:

Minimum score for object highlighting

We talked before about the minimum score for highlighting. Let's see exactly what that means by taking a look at what happens when we use different minimum scores for object highlighting...

Summary

TensorFlowSharp is an exciting open source project that makes working with Tensors and TensorFlow incredibly easy. In this chapter, we showed you what Tensors are and how you can work with them. We also built an incredibly powerful sample application that allows you to detect and label images within a picture.

In our next chapter, we will learn about long short-term memory networks and how you can use them to enhance your applications and processes.

References

• [1]"Speed/accuracy trade-offs for modern convolutional object detectors."Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S, Murphy K, CVPR 2017
• [2] www.tensorflow.org
• [3] JEAN, Hadrien. Deep Learning Book Series 2.1 Scalars Vectors Matrices and Tensors Web blog post. hadrienj.github.io. 26 Mar. 2018.