Chapter 15. Gesture Recognition

Since the beginning of time, humans have communicated with each other using gestures, even before there wasn't any formal language. Hand gestures were the primary way of communication, and it is also evident in the ancient sculptures found all across the world that the signs have been a successful way of transferring a huge amount of data in a very efficient way, sometimes, even more efficient than language itself. 

Gestures are natural, and they can occur as a reflex to a certain situation. It also happens subconsciously even without our knowing. So, it becomes an ideal way of communication with various devices. However, the question remains, how? 

We have used vision processing in our last few chapters, and we can be sure that if we are talking about gestures, then we would surely have to do a lot of programming to identify the gestures in the videos; furthermore, it would require a huge amount of processing power to make it happen as well. Hence, it is out...

Electric field sensing

Near-field sensing is a very interesting field of sensing. Be prepared for some interesting stuff. If you are feeling a little sleepy, or if you are lacking attention, then get some coffee because the working principle of this system is going to be a little new. 

Whenever there is a charge, there is an associated electrical field that comes along with it. These charges propagate through the space and go around an object. When that happens, the electric field associated with it has a specific characteristic. This characteristic will be the same till the time the environment around it is empty. 

For the gesture-recognition board that we are using, the field that would be sensed around it is only for about a few centimeters, so anything beyond that point can be disregarded. If there is nothing in that vicinity, then we can safely assume that the pattern of electric field being sensed would be unchanged. However, whenever an object such as our hand comes in the vicinity...

Using the Flick HAT

Flick HAT comes in the form of a shield, which you can simply plug into your Raspberry Pi and start using. However, once you do that, you will not be left with any GPIO pins. Hence, to save ourselves from that problem, we will be connecting it using male-to-female wires. This will give us access to the other GPIO pins and then we can have fun.

So, go ahead and connect it as follows. The following is a pin diagram of the Flick board:

Thereafter, make the connections as follows: 

Once the connection is done, simply upload this code and see what happens: 

import signal
import flicklib
import time
def message(value):
   print value
@flicklib.move()
def move(x, y, z):
   global xyztxt
   xyztxt = '{:5.3f} {:5.3f} {:5.3f}'.format(x,y,z)
@flicklib.flick()
def flick(start,finish):
   global flicktxt
   flicktxt = 'FLICK-' + start[0].upper() + finish[0].upper()
   message(flicktxt)
def main():
   global xyztxt
   global flicktxt
   xyztxt = ''
   flicktxt = ''
   flickcount = 0
 ...

Gesture recognition-based automation

Now we have interfaced the connections as per the following diagram:

Let's go ahead and upload the following code:

import signal
import flicklib
import time
import RPi.GPIO as GPIO
GIPO.setmode(GPIO.BCM)
GPIO.setup(light, GPIO.OUT)
GPIO.setup(fan,GPIO.OUT)
pwm = GPIO.PWM(fan,100)
def message(value):
   print value
@flicklib.move()
def move(x, y, z):
   global xyztxt
   xyztxt = '{:5.3f} {:5.3f} {:5.3f}'.format(x,y,z)
@flicklib.flick()
def flick(start,finish):
   global flicktxt
   flicktxt = 'FLICK-' + start[0].upper() + finish[0].upper()
   message(flicktxt)
def main():
   global xyztxt
   global flicktxt
   xyztxt = ''
   flicktxt = ''
   flickcount = 0
   dc_inc = 0
   dc_dec = 0

while True:
  pwm.start(0)
  xyztxt = ' '
  if len(flicktxt) > 0 and flickcount < 5:
    flickcount += 1
  else:
    flicktxt = ''

flickcount = 0
if flicktxt ==”FLICK-WE”:
  GPIO.output(light,GPIO.LOW)
if flicktxt ==”FLICK-EW”:
  GPIO.output(light,GPIO.HIGH)
if flicktxt...

Summary

In this chapter, we are were able to understand the concept of how gesture recognition works via electric field detection. We also understood how easy it is to use a gesture-controlled board and control the home using gestures. We will cover the machine learning part in the next chapter.