Chapter 2. Searching for Luxury Accommodations Worldwide

Today the bridal suite, tomorrow a prison. A secret agent's sleeping arrangements are horribly unpredictable.

Each day someone in MI6 gets the job of booking a stellar hotel room and conversely, some evil henchman has to pick a warehouse or dilapidated apartment, plus a lamp, a chair, and implements of bondage. For mini missions or brief beatings, it is tolerable to leave the choice of venue to a fallible human being. However, for long-term rentals or acquisitions, would it not be wiser to develop a specialized search engine that takes the legwork and the guesswork out of the equation?

With this motivation, we are going to develop a desktop app called Luxocator: The Luxury Locator. This is a search engine that will find images on the web by keyword search and will classify each image as a "Luxury, interior" scene, "Luxury, exterior" scene, "Stalinist, interior" scene, or "Stalinist, exterior" scene, according to certain visual cues in...

Planning the Luxocator app

This chapter uses Python. Being a high-level interpreted language with great third-party libraries for numeric and scientific computing, Python lets us focus on the functionality of the system rather than implementing subsystem details. For our first project, such a high-level perspective is precisely what we need.

Let's take an overview of Luxocator's functionality and our choice of Python libraries that support this functionality. Like many computer vision applications, Luxocator has 6 basic steps:

1. Acquire a static set of reference images: For Luxocator, we (the developers) will choose certain images that we will deem to be "Luxury, indoor" scenes, other images that we will consider as "Stalinist, indoor" scenes, and so on. We will load these images into memory.

2. Train a model based on the reference images: For Luxocator, our model will describe each image in terms of its normalized color histogram, that is, the distribution of colors across the image's pixels...

Creating, comparing, and storing histograms

	A grey-green colour that often finds itself on the walls of public institutions—e.g., hospitals, schools, government buildings—and, where appropriated, on sundry supplies and equipment.
	-- "Institutional green", Segen's Medical Dictionary (2012)

I hesitate to make sweeping statements about the ideal color of paint on a wall. It depends. I have found solace in many walls of many colors. My mother is a painter and I like paint in general.

But not all color is paint. Some color is dirt. Some color is concrete or marble; plywood or mahogany. Some color is the sky through big windows, the ocean, the golf course, or the swimming pool or jacuzzi. Some color is discarded plastics and beer bottles, baked food on the stove, or perished vermin. Some color is unknown. Maybe the paint camouflages the dirt.

A typical camera can capture at least 16.7 million (256 * 256 * 256) distinct colors. For any given image, we can count the number of pixels of each color...

Training the classifier with reference images

	Can you identify this coastline? Given time, yes.
	-- Photo caption, Dante Stella (http://www.dantestella.com/technical/hex352.html)

A small selection of reference images is included in this chapter's code bundle, which can be downloaded from my website, http://nummist.com/opencv/7376_02.zip. They are in a folder called images. Feel free to experiment with the classifier by adding more reference images, since a larger set might yield more reliable results. Bear in mind that our classifier relies on average similarity, so the more times you include a given color scheme in the reference images, the more heavily you are weighting the classifier in favor of that color scheme.

At the end of HistogramClassifier.py, let's add a main method to train and serialize a classifier using our reference images. We will also run the classifier on a couple of the images as a test. A partial implementation of the method is as follows:

def main():
  classifier...

Acquiring images from the Web

Our query images will come from a web search. Before we start implementing the search functionality, let's write some helper functions, which let us fetch images via the Requests library and convert them to an OpenCV-compatible format. Because this functionality is highly reusable, we will put it in a module of static utility functions. Let's create a file called RequestsUtils.py and import OpenCV, NumPy, and Requests, as follows:

import numpy
import cv2
import requests
import sys

As a global variable, let's store HEADERS, a dictionary of headers that we will use while making web requests. Some servers reject requests that appear to come from a bot. To improve the chance of our requests being accepted, let's set the 'User-Agent' header to a value that mimics a web browser, as follows:

# Spoof a browser's User-Agent string.
# Otherwise, some sites will reject us as a bot.
HEADERS = {
  'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; ' + \
    'rv:25...

Acquiring images from Bing image search

Microsoft's search engine, Bing, has an API that enables us to send queries and receive results in our own application. For a certain number of queries (currently, 5,000 per month), Bing Search API is free to use. However, we must register for it by taking the following steps:

1. Go to http://datamarket.azure.com/ and log in. You will need to create a Microsoft account if you do not already have one.

2. Enter Bing Search API in the Search the Marketplace field. A list of results should appear. Click on the link for Bing Search API (not any variant such as Bing Search API – Web Results Only).

3. Next to the $0.00 per month option, click on the SIGN UP button. Alternatively, if you plan to use Luxocator obsessively, to the exclusion of normal activities, sign up for one of the paid options that allows more than 5,000 queries per month.

4. Click on the My Account tab. You should see a list of Account Information, including a Primary Account Key, which is a 43-character...

Preparing images and resources for the app

Alongside RequestsUtils.py and ImageSearchSession.py, let's create another file called ResizeUtils.py with the following import statements:

import numpy
import cv2

For display in a GUI, images often must be resized. One popular mode of resizing is called aspect fill. This means that we want to preserve the image's aspect ratio while changing its larger dimension (width for a landscape image or height for a portrait image) to a certain value. OpenCV does not directly provide this resizing mode but it does provide a function, cv2.resize, which accepts an image, target dimensions, and optional arguments including an interpolation method. We can write our own function, cvResizeAspectFill, which accepts an image, maximum size, and preferred interpolation methods for upsizing and downsizing the images. It determines the appropriate arguments for cv2.resize and passes them along. Here is the implementation:

def cvResizeAspectFill(src, maxSize,
  upInterpolation...

Integrating everything into the GUI

For Luxocator's frontend, let's create a file called Luxocator.py. This module depends on OpenCV, wxPython, and some of Python's standard OS and threading functionality. It also depends on all the other modules that we have written in this chapter. Add the following import statements at the top of the file:

import numpy
import cv2
import os
import threading
import wx

from HistogramClassifier import HistogramClassifier
from ImageSearchSession import ImageSearchSession
import PyInstallerUtils
import ResizeUtils
import WxUtils

Now, let's implement the Luxocator class as a subclass of wx.Frame, which represents a GUI frame such as the contents of a window. Most of our GUI code is in the Luxocator class's __init__ method, which is therefore a big method but is not very complicated. Our GUI elements include a search control, previous and next buttons, a bitmap, and a label to show the classification result. All of these GUI elements are stored in the member variables...

Building Luxocator for distribution

To instruct PyInstaller how to build Luxocator, we must create a specification file, which we will call Luxocator.spec. Actually, the specification file is a Python script that uses a PyInstaller class called Analysis and PyInstaller functions called PYZ, EXE, and BUNDLE. The Analysis class is responsible for analyzing one or more Python scripts (in our case, just Luxocator.py) and tracing all the dependencies that must be bundled with these scripts in order to make a redistributable application. Sometimes, Analysis makes mistakes or omissions, so we will modify the list of dependencies after it is initialized. Then, we will zip the scripts, make an executable, and make an app bundle using PYZ, EXE, and BUNDLE, respectively. Here is the implementation:

import os

a = Analysis(['Luxocator.py'],
  pathex=['.'],
  hiddenimports=[],
  hookspath=None,
  runtime_hooks=None)

# Determine the platform.
platform = os.name

if platform == 'nt':
  # We are on Windows...

Summary

So much can happen in a single mission! We trained an OpenCV/NumPy/SciPy histogram classifier, performed Bing image searches, built a wxPython app, and used PyInstaller to bundle it all for redistribution to Russia with love (or, indeed, to any destination with any sentiment). At this point, you are well primed to create other Python applications that combine computer vision, web requests, and a GUI.

For our next mission, we will dig our claws deeper into OpenCV and computer vision by building a fully functional cat recognizer!