Kinect for Windows SDK Programming Guide


Kinect for Windows SDK Programming Guide
eBook: $26.99
Formats: PDF, PacktLib, ePub and Mobi formats
$22.94
save 15%!
Print + free eBook + free PacktLib access to the book: $71.98    Print cover: $44.99
$44.99
save 37%!
Free Shipping!
UK, US, Europe and selected countries in Asia.
Also available on:
Overview
Table of Contents
Author
Reviews
Support
Sample Chapters
  • Building application using Kinect for Windows SDK.
  • Covers the Kinect for Windows SDK v1.6
  • A practical step-by-step tutorial to make learning easy for a beginner.
  • A detailed discussion of all the APIs involved and the explanations of their usage in detail
  • Procedures for developing motion-sensing applications and also methods used to enable speech recognition

 

Book Details

Language : English
Paperback : 392 pages [ 235mm x 191mm ]
Release Date : December 2012
ISBN : 1849692386
ISBN 13 : 9781849692380
Author(s) : Abhijit Jana
Topics and Technologies : All Books, Game Development, Games, Microsoft

Table of Contents

Preface
Chapter 1: Understanding the Kinect Device
Chapter 2: Getting Started
Chapter 3: Starting to Build Kinect Applications
Chapter 4: Getting the Most out of Kinect Camera
Chapter 5: The Depth Data – Making Things Happen
Chapter 6: Human Skeleton Tracking
Chapter 7: Using Kinect's Microphone Array
Chapter 8: Speech Recognition
Chapter 9: Building Gesture-controlled Applications
Chapter 10: Developing Applications Using Multiple Kinects
Chapter 11: Putting Things Together
Index
  • Chapter 1: Understanding the Kinect Device
    • Components of Kinect for Windows
      • Inside the Kinect sensor
        • The color camera
        • IR emitter and IR depth sensor
        • Tilt motor
        • Microphone array
        • LED
    • Kinect for Windows versus Kinect for Xbox
    • Where can you use Kinect
    • Summary
    • Chapter 2: Getting Started
      • System requirements for the Kinect for Windows SDK
        • Supported operating systems
        • System configuration
        • The Kinect sensor
          • The Kinect for Windows sensor
          • The Kinect for Xbox sensor
        • Development tools and software
      • Evaluation of the Kinect for Windows SDK
      • Downloading the SDK and the Developer Toolkit
      • Installing the Kinect for Windows SDK
        • Installing the Developer Toolkit
        • Components installed by the SDK and the Developer Toolkit
          • Kinect management service
        • Connecting the sensor with the system
          • Verifying the installed drivers
      • Testing your device
        • Testing Kinect sensors
        • Testing the Kinect microphone array
      • Looking inside the Kinect SDK
      • Features of the Kinect for Windows SDK
        • Capturing the color image data stream
        • Processing the depth image data stream
          • Near Mode
        • Capturing the infrared stream
        • Tracking human skeleton and joint movements
        • Capturing the audio stream
        • Speech recognition
        • Human gesture recognition
        • Tilting the Kinect sensor
        • Getting data from the accelerometer of the sensor
        • Controlling the infrared emitter
      • The Kinect for Windows Developer Toolkit
        • The Face Tracking SDK
        • Kinect Studio
      • Making your development setup ready
      • The Coding4Fun Kinect Toolkit
      • Summary
      • Chapter 3: Starting to Build Kinect Applications
        • How applications interact with the Kinect sensor
          • Understanding the classification of SDK APIs
        • Kinect Info Box – your first Kinect application
          • Creating a new Visual Studio project
            • Adding the Kinect libraries
          • Getting the Kinect sensor
            • The Kinect sensor
            • Defining the Kinect sensor
            • The collection of sensors
          • Starting up Kinect
            • Inside the sensor.Start() method
            • Enabling the data streams
          • Identifying the Kinect sensor
            • Initializing the sensor using device connection ID
          • Stopping the Kinect sensor
            • The Stop() method does the clean-up operation
          • Displaying information in the Kinect Info Box
            • Designing the Info Box UI
            • Binding the data
          • That's all!
        • Dealing with the Kinect status
          • Monitoring the change in sensor status
            • Properties of the StatusChangedEventArgs class
          • Resuming your application automatically
          • Building KinectStatusNotifier
            • Setting up an application
            • How it works
          • Using KinectStatusNotifier
            • Test it out
        • Summary
        • Chapter 4: Getting the Most out of Kinect Camera
          • Understanding the Kinect image stream
            • Types of color images
          • Different ways of retrieving the color stream from Kinect
            • Event model
            • Polling model
          • KinectCam – a Kinect camera application
            • Setting up the project
            • Designing the application – XAML and data binding
            • Capturing color image from the Kinect camera
          • Enabling the color stream channel
            • Enabling a channel with the image format
            • Choosing the image format
            • Disabling the color stream channel
            • Attaching the event handler
            • Processing the incoming image frames
            • Rendering image frames on the UI
            • Running the KinectCam
          • Looking inside color image stream helpers
            • The ColorImageStream class
            • The ColorImageFrame class
          • Capturing frames on demand
          • Extending the KinectCam
            • Getting the frame number
            • Changing image format on the fly
              • Bind available image formats
              • Changing the color image format
            • Calculating frame rate
              • How to calculate frame rate
            • Capturing and saving images
              • Saving images periodically
              • Trying to save image frames directly
            • Changing sensor elevation angles
              • Maximum and minimum elevation angle
              • Adjusting the Kinect sensor angle
            • Playing around with color pixels
              • Applying RGB effects
              • Making grayscale effects
              • Inverting the color
          • Applying more effects to the camera
            • Applying the backlight compensation mode
            • Applying slow motion effects
            • Kinect Camera Effects – application
          • Seeing in low light
          • Making your application perform better
          • Using the Coding4Fun toolkit
            • Installing the Coding4Fun Kinect toolkit
              • Using assembly
              • Using the NuGet package
            • Using Coding4Fun Kinect libraries in your application
          • Summary
          • Chapter 5: The Depth Data – Making Things Happen
            • Understanding the depth data stream
              • Depth data – behind the scenes
              • Stereo triangulation
            • Capturing and processing depth data
              • Enabling the depth stream channel
              • Attaching the event handler
              • Processing the depth frames
              • Depth data at first look
            • Looking inside depth image stream helpers
            • Depth data and distance
              • How the distance is calculated
              • Getting the distance from a particular pixel
              • Accessing the range of distance
              • Colorize depth data processing
            • Working with depth range
              • Special depth range values
            • Depth data distribution
            • Player index with depth data
              • How player index works
              • Identifying players
            • Getting the depth and player index automatically
            • A 3D view of depth data
              • The basics of the coordinate system
              • Basic elements of 3D graphics
              • Setting up the project
              • Give it a 3D effect
                • Creating the ViewPort
                • Using the camera
                • Creating the 3D Model
                • Setting up the initial data points
              • Getting the depth data from Kinect
              • Have a look at 3D depth
            • Summary
            • Chapter 6: Human Skeleton Tracking
              • How skeleton tracking works
                • Steps to remember
              • Skeleton tracking with the Kinect SDK
              • Start tracking skeleton joints
                • Tracking the right hand
                  • Setting up the project
                  • Creating a joint placeholder
                  • Get Kinect running and instantiate skeleton tracking
                  • Processing the skeleton frames
                  • Mapping the skeleton joints with UI elements
                  • Running the application
                  • Adding more fun
              • Flow – capturing skeleton data
              • An intrusion detector camera application
                • Adding night vision
              • Looking inside skeleton stream helpers
                • The skeleton frame
                • The skeleton stream
              • Skeleton tracking mode
                • Default skeleton tracking
                • Seated skeleton tracking
                  • Using seated-skeleton tracking
                  • Points to be considered with seated-skeleton tracking
              • Skeleton-tracking in near mode
              • The Skeleton
                • Skeleton-tracking state
                • Choosing which skeleton to track
                  • Skeleton tracking ID
                  • Monitoring changes in the skeleton
                  • Limiting tracking for the intrusion-detector camera
                • The building blocks – Joints and JointCollection
                  • Joint-tracking state
                • Steps to be followed for joint tracking
                • Create your own joints data point
              • Bones – connecting joints
                • Bone sequence
                  • Bone sequence for a default skeleton
                  • Bone sequence for a seated skeleton
                • Drawing bones between joints
              • Adjusting the Kinect sensor automatically and giving live feedback to users
              • Skeleton smoothing – soften the skeleton's movement
                • What causes skeleton jitters
                • Making skeleton movement softer
                  • Smoothing parameters
                • How to check if skeleton smoothing is enabled
                • Exponential smoothing
              • Skeleton space transformation
              • The Advanced Skeleton Viewer application
              • Debugging the applications
                • Using conditional breakpoints
                • Using Kinect Studio
              • Getting data frames together
              • Summary
              • Chapter 7: Using Kinect's Microphone Array
                • Verifying the Kinect audio configuration
                  • Using the Kinect microphone array with your computer
                • The Kinect SDK architecture for Audio
                • Kinect microphone array
                  • The major focus area of Kinect audio
                  • Why microphone array
                • Audio signal processing in Kinect
                • Taking control over the microphone array
                  • Kinect audio stream
                  • Starting and stopping the Kinect audio stream
                    • Starting audio streaming after a time interval
                • Kinect sound recorder – capturing Kinect audio data
                  • Setting up the project
                  • Designing the application – XAML and data binding
                  • Recording the Kinect audio
                    • Starting the recording
                    • Playing the recorded audio
                  • Running the Kinect Sound Recorder
                • Processing the audio data
                  • Echo cancellation
                  • Noise suppression
                  • Automatic gain control
                  • Audio data processing with the Kinect sound recorder
                • Sound source localization
                  • Sound source angle
                    • Confidence level
                  • Beamforming
                    • Beam angle mode
                  • Extending the Kinect Sound Recorder with sound source localization
                • Summary
                • Chapter 8: Speech Recognition
                  • How speech recognition works
                  • Using Kinect with your Windows PC speech recognition
                  • Beginning with Microsoft Speech API (SAPI)
                    • Steps for building speech-enabled applications
                    • Basic speech-recognition approach
                    • Building grammar
                      • Using Choice and GrammarBuilder
                      • Building grammar using XML
                      • Creating grammar from GrammarBuilder
                      • Loading grammar into a recognizer
                      • Unloading grammars
                  • Draw What I Want – a speech-enabled application
                    • Setting up the project
                    • Designing the application – XAML and data binding
                      • Data binding
                    • Instantiating speech recognizer
                    • Working with the speech recognition engine
                      • Configuring Kinect audio
                      • Creating grammar
                      • Start the speech recognizer
                    • Drawing an object when speech is recognized
                    • Testing your application
                  • Summary
                  • Chapter 9: Building Gesture-controlled Applications
                    • What is a gesture
                    • Approaches for gesture recognition
                    • Basic gesture recognition
                      • Gesture-detection technique
                        • Representing skeleton joints
                        • Calculating the distance between two joints
                      • Building a clapping-hands application
                        • Setting up the project
                        • Implementing the gesture recognizer
                        • Plugging gestures into the application
                        • Testing your application
                      • A virtual rope workout application
                      • Hands-raised-above-head gesture recognition
                      • Steps to recognize basic gestures
                    • Algorithmic gesture recognition
                      • Which gestures can be considered as algorithmic
                      • Understanding the algorithmic gesture detection approach
                      • Implementing an algorithmic gesture
                        • Adding gesture types
                        • Extending the Event argument
                        • Adding a GestureHelper class
                        • Defining the GestureBase class
                        • Implementing the SwipeToLeftGestures class
                        • Adding the ZoomIn, ZoomOut, and SwipeToRight gesture classes
                        • Implementing the GestureRecognitionEngine class
                        • Using the GestureRecognitionEngine class
                        • A demo application
                      • Making it more flexible
                    • Weighted network gesture recognition
                      • What is a neural network
                      • Gesture recognition with neural networks
                      • Jump tracking with a neural network – an example
                    • Template-based gesture recognition
                    • Building gesture-enabled controls
                      • Making a hand cursor
                        • Getting the hand-cursor point
                      • Identifying the objects
                      • Enabling action for the objects
                    • The Basic Interaction – a WPF application
                    • Key things to remember
                    • Summary
                    • Chapter 10: Developing Applications Using Multiple Kinects
                      • Setting up the environment for multiple Kinects
                        • Plugging the first Kinect sensor
                        • Plugging the second Kinect sensor
                        • Kinect sensors require an individual USB Controller
                      • Multiple Kinects – how to reduce interference
                      • Detecting multiple Kinects
                        • Getting access to the individual sensor
                        • Different ways to get a Kinect sensor's reference
                      • Developing an application with multiple Kinects
                        • Setting up the project
                        • Designing the UI
                        • Creating the KinectInfoCollection
                        • Getting information from Kinects
                        • Running the application
                      • Controlling multiple sensor status changes
                        • Extending Multiple Kinect Viewer with status change
                          • Registering and handling the status change
                          • Running the application
                        • Identifying the devices automatically
                        • Integrating with KinectStatusNotifier
                        • Capturing data using multiple Kinects
                      • Handling a failover scenario using Kinects
                      • Challenges faced in developing applications using multiple Kinects
                      • Applications where multiple Kinects can be used
                      • Summary
                      • Chapter 11: Putting Things Together
                        • Taking Kinect to the Cloud
                          • Required components
                            • Windows Azure
                            • The Windows Azure SDK
                            • The Kinect for Windows SDK
                          • Designing the solution
                          • Real-time implementations
                        • Remotely using the Kinect with Windows Phone
                          • Required components
                            • The Windows Azure Service Bus
                            • The Windows Phone SDK
                          • Designing the solution
                          • Real-time implementations
                        • Using Kinect with a Netduino microcontroller
                          • Required components
                            • Microsoft .NET Micro Framework
                            • Netduino
                            • The Netduino SDK
                          • Blinking of the on-board LED
                            • Changing the Deployment Transport
                            • Running the application
                          • Connecting Kinect to a Netduino
                            • Using an Internet connection
                            • Listening to the request
                            • Sending a request from a Kinect application
                          • Taking it further
                        • Augmented reality applications
                        • Working with face tracking
                        • Working with XNA and a 3D avatar
                        • Summary

                        Abhijit Jana

                        Abhijit works with Microsoft as a .NET Consultant as part of Microsoft Services Global Delivery, India. As a Consultant, his job is to help customers design, develop and deploy enterprise level secure solutions using Microsoft Technologies. Apart from being a Former Microsoft MVP (Most Valuable Professional), he is a speaker, and author, as well as an avid Technology Evangelist. He has delivered sessions at prestigious Microsoft events like TechED, Web Camps, Azure Camps, Community Tech Days, Virtual Tech Days, Devdays, and developer conference. He loves to work with different .NET Communities and help them with different opportunities. He is a well-known author and has published many articles on various .NET Community sites. He shares his thoughts on his personal blog at http://abhijitjana.net. You can follow him on Twitter at @abhijitjana. Abhijit lives in Hyderabad, India, with his wife, Ananya and a beautiful little angel Nilova.

                        Code Downloads

                        Download the code and support files for this book.


                        Submit Errata

                        Please let us know if you have found any errors not listed on this list by completing our errata submission form. Our editors will check them and add them to this list. Thank you.

                        Sample chapters

                        You can view our sample chapters and prefaces of this title on PacktLib or download sample chapters in PDF format.

                        Frequently bought together

                        Kinect for Windows SDK Programming Guide +    jMonkeyEngine 3.0 Beginner's Guide =
                        50% Off
                        the second eBook
                        Price for both: £24.65

                        Buy both these recommended eBooks together and get 50% off the cheapest eBook.

                        What you will learn from this book

                        • Understand the Kinect sensor in terms of a device, different components, and how they work.
                        • Step by step installation guide of SDK, troubleshooting tips and development environment setup.
                        • Reading the Kinect device information, monitoring and notifying the sensor status change.
                        • Automatically adjust the sensor elevation angle
                        • Understand different approaches of capturing data stream from Kinect sensor.
                        • Learn the uses of color, depth and IR data and how to leverage them in your applications.
                        • Use skeleton-tracking for interactive applications and leveraging the applications to gesture-enabled application
                        • Understand usages of Microphone array and build speech-enabled applications with Speech API
                        • Develop applications using Multiple Kinects
                        • Understand how to design cross device application with Kinect

                        In Detail

                        Kinect has been a game-changer in the world of motion games and applications since its first release. It has been touted as a controller for Microsoft Xbox but is much more than that. The developer version of Kinect, Kinect for Windows SDK, provides developers with the tools to develop applications that run on Windows. You can use this to develop applications that make interaction with your computer hands-free.

                        This book focuses on developing applications using the Kinect for Windows SDK. It is a complete end to end solution using different features of Kinect for Windows SDK with step by step guidance. The book will also help you develop motion sensitive and speech recognition enabled applications. You will also learn about building application using multiple Kinects.

                        The book begins with explaining the different components of Kinect and then moves into to the setting up the device and getting thedevelopment environment ready. You will be surprised at how quickly the book takes you through the details of Kinect APIs. You will use NUI to use the Kinect for Natural Inputs like skeleton tracking, sensing, speech recognizing.

                        You will capture different types of stream, and images, handle stream event, and capture frame. Kinect device contains a motorized tilt to control sensor angles, you will learn how to adjust it automatically. The last part of the book teaches you how to build application using multiple Kinects and discuss how Kinect can be used to integrate with other devices such as Windows Phone and microcontroller.

                        Approach

                        This book is a practical tutorial that explains different features of Kinect for Windows SDK by creating sample applications throughout the book

                        Who this book is for

                        The purpose of this book is to explain how to develop applications using the Kinect for Windows SDK. If you are a beginner and looking to start developing applications using the Kinect for Windows SDK, and if you want to build motion-sensing,speech-recognizing applications with Kinect, this book is for you.

                        This book uses C# and WPF (Windows Presentation Foundation) in the examples, so you need to know the basics of C# and WPF. You should be familiar with the Visual Studio IDE as well. You don't have to know anything about the Kinect for Windows SDK.

                        Code Download and Errata
                        Packt Anytime, Anywhere
                        Register Books
                        Print Upgrades
                        eBook Downloads
                        Video Support
                        Contact Us
                        Awards Voting Nominations Previous Winners
                        Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
                        Resources
                        Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software