Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Leap Motion Development Essentials
Leap Motion Development Essentials

Leap Motion Development Essentials: Leverage the power of Leap Motion to develop and deploy a fully interactive application

eBook
$20.69 $22.99
Paperback
$38.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Leap Motion Development Essentials

Chapter 2. Real Talk – Real Time

When we create an application that acts on gestural input, a major concern is responsiveness. Not only is using a laggy application no fun, but there are many situations in which we desire to process and respond to the hand motion with as little latency as possible. In this chapter we'll discuss the following:

  • Blocking

  • Describing raw input as simple gestures

  • Shared-memory concurrency with pthreads

  • Building a MIDI controller for real-time audio applications

With this in mind, we will walk through creating a controller suitable for live musical performances. Because our code will be interacting with music software and hardware, we must use our operating system interfaces for transmitting MIDI (the standard musical control protocol that all devices support), which is not guaranteed to happen without blocking. While using routines that can potentially involve sending messages to hardware, it's generally good to assume that the execution of our program may be paused...

A simple gesture recognizer


One of the handy methods we can call on a Leap::Hand is sphereRadius(), which returns the size of an imaginary ball filling up a hand. The more outstretched your fingers are the larger the radius, and a more clenched fist will produce smaller values.

What does our ball gesture recognizer class look like? Feast your eyes on this:

typedef std::shared_ptr<BallGesture> BallGesturePtr;
   
void BallGesture::recognizedControls(const Leap::Controller &controller, std::vector<ControlPtr> &controls) {
    Leap::Frame frame = controller.frame();
    if (frame.hands().empty())
        return;

If no hands are found in the current frame, there's not much else for us to do here. However, if the user is waving some appendages around, we can do our thing.

    for (int i = 0; i < frame.hands().count(); i++) {
        if (i > 1) break;

If one or two hands are detected in the frame, we might be able to do something useful. We can get extra fancy if we return...

Using the MIDI output


A MIDI client is an entity that can communicate with other MIDI clients, and may have one or more inputs and outputs. Our client will be sort of useless without any I/O, so we create a virtual source into which we can inject our freshly minted MIDI messages. The descriptive string passed in will appear in the MIDI configuration dialogs of any application that supports MIDI, such as my DAW of choice, Ableton Live.

Let's get to the meat of sending a MIDI control message now; we will deal with the packet initialization in a minute. Remember our onControlUpdated() routine? We can use the mapped control value and the controller index number to build a message to transmit.

    midi_control_index ctrlIdx = control->controlIndex();
    midi_control_value ctrlVal = control->mappedValue();

ctrlIdx and ctrlVal correspond to the number of the MIDI control and the output value for that control respectively. For our ball radius gesture, the control indices will be BALL_RADIUS_HAND_1...

Blocking and latency


So, can we play the Leap now? We absolutely can! There is a wealth of free and trial music software available for generating all sorts of noises a quick Google search away. You should be able to select the Leap MIDI control device in your music program's settings, and use its MIDI learn feature to map your ball radius gestures to anything you desire (filter cut-offs, effect parameters, and synthesizer knobs are especially fun).

This is all well and good but there is one very serious, unaddressed issue with our MIDI controller: blocking. The sad truth is that our code is not ready to be used by the general public to give performances because the gesture response time is utterly nondeterministic. In the onControlUpdated() routine described earlier, our call to MIDIReceived() is part of a poorly documented API that may or may not involve communication with blocking subsystems or hardware, and that frankly we know very little about. Depending upon the user's setup, hardware...

Multiprocessing with threads


The solution to our conundrum is to run more than one thread of execution at the same time. CPUs and modern operating systems are very good at dealing with multiple processes and multiple threads of execution within processes, and this suits our purposes nicely. What we really want is one task that is receiving callbacks from the controller and handling the input, with a separate task taking our control messages and outputting them. This would give us a robust setup in which we can easily do anything we want in the output thread, without introducing issues associated with blocking the input processing thread. We could be happily logging output to a file, sending it over a network, outputting MIDI to hardware, controlling GPIO pins, doing expensive calculations or 3D graphics, and not affect the response time of our Leap callbacks.

As with many things in software development, there is more than one way to achieve our desired result of multitasking: POSIX threading...

Refactoring for multithreading


Before continuing further on, perhaps it would be a good time to refactor our virtual MIDI device output functionality into its own class, as the MIDI output routines with the attendant threading code should be encapsulated for maintaining some semblance of clean program structure.

namespace leapmidi {
class Device {
public:
    Device();
    ~Device();
    void init();
    void queueControlMessage(midi_control_index controlIndex, midi_control_value controlValue);

We'll go ahead and create a leapmidi::Device class. Users of our class construct a new Device class and call init() to set up the MIDI virtual device, and then send MIDI control messages using queueControlMessage().

protected:
    void initPacketList();
    void createDevice();

Our earlier MIDI virtual device setup code can go in here.

    pthread_t messageQueueThread;
    void *messageSendingThreadEntry();

This function will be called when our MIDI output thread (kept track of by messageQueueThread) begins...

The producer-consumer race condition


What we have created is a classic producer-consumer pattern, where one thread is producing work and the other is processing the work on the queue. We are trying to make our application take care of the following:

  • Respond to events in as close to real-time as possible

  • Not lose any events

So, we need to be absolutely certain that we handle new items added to the queue as rapidly as possible. A naive implementation might look something like this:

            - release message queue lock
            - (possible race condition here!)
            - wait for condition variable broadcast
            - check queue again

But that would be utterly wrong. Suppose that queueControlMessage() is called after unlocking our message queue mutex, but before our thread starts waiting for the main Leap processing thread condition variable signal. The condition variable signals are ephemeral in the extreme. If no thread is waiting at the exact moment the signal is sent, we will...

Summary


Multithreaded C++ applications are inherently nontrivial and full of debugging pitfalls. Your best strategy is to abstract the communication between your threads to a simple and well-tested mechanism for safe transportation of messages between your threads. Note that the callbacks used by the Leap library are invoked in separate threads, so take care while sharing data between your callbacks and any other threads your program may be running. This makes it possible for applications to easily receive updates concurrently with the rest of the program's execution, but you are still responsible for appropriately handling any other blocking that may take place in your program.

Left arrow icon Right arrow icon

Key benefits

  • Comprehensive and thorough coverage of many SDK features
  • Intelligent usage of gesture interfaces
  • In-depth, functional examples of API usage explained in detail

Description

Leap Motion is a company developing advanced motion sensing technology for human–computer interaction. Originally inspired by the level of difficulty of using a mouse and keyboard for 3D modeling, Leap Motion believe that moulding virtual clay should be as easy as moulding clay in your hands. Leap Motion now focus on bringing this motion sensing technology closer to the real world. Leap Motion Development Essentials explains the concepts and practical applications of gesture input for developers who want to take full advantage of Leap Motion technology. This guide explores the capabilities available to developers and gives you a clear overview of topics related to gesture input along with usable code samples. Leap Motion Development Essentials shows you everything you need to know about the Leap Motion SDK, from creating a working program with gesture input to more sophisticated applications covering a range of relevant topics. Sample code is provided and explained along with details of the most important and central API concepts. This book teaches you the essential information you need to design a gesture-enabled interface for your application, from specific gesture detection to best practices for this new input. You will be given guidance on practical considerations along with copious runnable demonstrations of API usage which are explained in step-by-step, reusable recipes.

Who is this book for?

This book is for developers who are either involved in game development or who are looking to utilize Leap Motion technology in order to create brand new user interaction experiences to distinguish their products from the mass market. You should be comfortable with high-level languages and object-oriented development concepts in order to get the most out of this book.

What you will learn

  • Read finger and hand positions as well as motion information
  • Detect where a user is pointing on a screen
  • Recognize gestures ‚Äì both built-in and user-defined
  • Deal with multithreaded programming challenges to create responsive interfaces
  • Explore the theory and concepts of gestural interfaces along with best practices
  • Integrate the Leap with 3D web capabilities using WebGL and Three.js
  • Understand the detailed coverage of C++ and JavaScript APIs
  • Add Leap support to a web page with no additional software or downloads required by users
Estimated delivery fee Deliver to United States

Economy delivery 10 - 13 business days

Free $6.95

Premium delivery 6 - 9 business days

$21.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Oct 25, 2013
Length: 106 pages
Edition : 1st
Language : English
ISBN-13 : 9781849697729
Vendor :
Leap Motion

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to United States

Economy delivery 10 - 13 business days

Free $6.95

Premium delivery 6 - 9 business days

$21.95
(Includes tracking information)

Product Details

Publication date : Oct 25, 2013
Length: 106 pages
Edition : 1st
Language : English
ISBN-13 : 9781849697729
Vendor :
Leap Motion

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 71.98
Leap Motion Development Essentials
$38.99
Raspberry Pi for Secret Agents
$32.99
Total $ 71.98 Stars icon

Table of Contents

5 Chapters
Leap Motion SDK – A Quick Start Chevron down icon Chevron up icon
Real Talk – Real Time Chevron down icon Chevron up icon
Actual Gestures Chevron down icon Chevron up icon
Leap and the Web Chevron down icon Chevron up icon
HTML5 Antics in 3D Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.3
(3 Ratings)
5 star 66.7%
4 star 0%
3 star 33.3%
2 star 0%
1 star 0%
Richter, Volkmar Jan 12, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Wir forschen zur Zeit an Einsatzmöglichkeiten für die Gestensteuerung, da war das Buch für uns eine große Hilfe beim schnellen Erstellen von Prototypen.
Amazon Verified review Amazon
Zak Nov 17, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Good read, helped a lot with a gesture-based midi controller I'm trying to build. The possibilities are endless!
Amazon Verified review Amazon
Martin Oct 24, 2014
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
This is a very short book (89 pages), and is written for advanced C++ programmers. As I don't know this language, it's almost useless to me. The "examples" are just random snippets of code and so are useless, as nothing compiles.On the plus side, it is written in an accessible, good-natured style and has nudged me towards re-learning C++. The snippets do eventually combine to make a functioning app, but are generally geared towards Mac developers.Still, it's the only book available for the Leap Motion, so I am grateful for that!
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the digital copy I get with my Print order? Chevron down icon Chevron up icon

When you buy any Print edition of our Books, you can redeem (for free) the eBook edition of the Print Book you’ve purchased. This gives you instant access to your book when you make an order via PDF, EPUB or our online Reader experience.

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
Modal Close icon
Modal Close icon