Log In
Or create an account ->
Imperial Library
Home
About
News
Upload
Forum
Help
Login/SignUp
Index
Start Here!™ Learn Microsoft® Kinect API
Dedication
A Note Regarding Supplemental Files
Introduction
Who Should Read This Book
Assumptions
Who Should Not Read This Book
Organization of This Book
Conventions and Features in This Book
System Requirements
Code Samples
Installing the Code Samples
Using the Code Samples
Acknowledgments
Errata and Book Support
We Want to Hear from You
Stay in Touch
I. Getting Started
1. An Introduction to Kinect
The Kinect Sensor
Getting Inside a Kinect Sensor
The Depth Sensor
The Kinect Microphones
Recognizing People with Kinect
Programming the Kinect
Kinect for Xbox and Kinect for Windows
Summary
2. Getting Started with Kinect
Kinect for Windows SDK Prerequisites
Kinect Device
Visual Studio
DirectX Studio
Installing the Kinect for Windows SDK
Installing the Kinect SDK
Installing the Kinect SDK
Installing the Kinect SDK
Connecting the Kinect Sensor Bar
Powering the Kinect Sensor
Installing the Kinect Sensor USB Drivers
Testing the Kinect Sensor Bar
The Kinect SDK Sample Browser
Troubleshooting Your Kinect Installation
Remove Old SDK Installations
Ensure That Visual Studio 2010 Is Installed but Not Running During Installation
Ensure That There Are No Windows Updates in Progress
Ensure That the Kinect Is Powered Correctly
Remove Any Old USB Drivers
Summary
3. Writing Software for Kinect
Making a Kinect Video Camera
Creating a New Visual Studio Project for Kinect
Creating a new Kinect Program
Creating a new Kinect Program
Getting the Kinect Sensor Working
Using the KinectSensor Class to Control the Kinect Sensor
Using the KinectSensor Class to Control the Kinect Sensor
Displaying a Video Frame
Creating a WPF Image Display Element for the Kinect Camera
Adding an Image Display Element to the WPF Window
Displaying the Kinect Camera Picture
Obtaining the Kinect Color Camera Video Data
Testing the Kinect Camera
Adding Error Handling
Detecting the Presence of a Kinect
Handling Setup Errors
Summary
II. Using the Kinect Sensor
4. Your First Kinect Application—Video Snapshots
Image Storage in Computers
Getting the Kinect Image Data onto the Screen
Controlling the Color of the Pixels
Creating a Color Adjustment Program
Improving the Speed by Writing Unsafe Code
Managed Code and Unsafe Code
References and Pointers
Pointers and Fixed Memory Locations
Using 32-Bit Integer Pointers
Saving the Image to a File
Improving Video Quality
Improving Performance by Waiting for Each Kinect Frame
Creating a Video Display Thread
Updating the Image from a Different Thread
Stopping the Background Thread
Summary
5. Moving Pictures
Detecting Movement in Video Images
Storing a Video Image in Program Memory
Detecting Changes in Video Images
Dealing with Noise
Using Change Thresholds to Filter Out Noise
Sounding the Alarm
Adding Sound Playback to a Project
Adding Sound Playback to a Project
A Complete Alarm Program
Switching to Black and White
Triggering Pictures with Motion Capture
Capturing Multiple Frames
Summary
6. Fun with the Depth Sensor
Visualizing Kinect Depth Information
The Kinect Depth Sensor
Obtaining Depth Information from the Sensor
Depth Data Values
Visualizing Depth Information
Using the Depth Information to Detect Intruders
Using the Depth and Video Sensors at the Same Time
Drawing in the Air
Detecting Objects
Counting Depth Values
Making You into the Controller
Using the Kinect Sensor with an XNA Game
Getting the Control Value from the Kinect
Drawing the Kinect Depth Image in XNA
Summary
7. Fun with the Sound Sensor
Capturing Sound Using Kinect
Sound and Computers
Receiving Sound Signals from Kinect
Playing Sound Using XNA
Stopping the Program
Sound Signals and Latency
Visualizing a Sound Signal in XNA
Storing Sound Data in a File and Replaying It
Creating a WAV File
Playing a Recorded Sound
Triggering the Sound Playback
Sound Playback Management
Summary
III. Creating Advanced User Interfaces
8. Body Tracking with Kinect
Kinect Body Tracking
Kinect Skeleton Information
Kinect Joint Positions
A Head Tracking Program
The Joints Collection and C# Dictionaries
Using Format Strings to Build a Message
Skeleton Information Quality
Joint Tracking State
Drawing a Skeleton
Drawing Lines in WPF
Converting Joint Positions to Image Coordinates
Clearing the Canvas
Turning Yourself into a Paintbrush
Drawing a Complete Skeleton
Detecting Gestures
Calculating the Distance Between Two Points in Space
Using a Gesture to Trigger an Action
Biometric Recognition with Kinect
Creating a “Kiss-Detecting” Program
Finding Two Skeletons That Are Being Tracked
Summary
9. Voice Control with Kinect
Using the Microsoft Speech Platform
Testing Voice Recognition
Creating a Program That Recognizes Color Names
Adding the Speech Platform SDK Assemblies to a Project
Creating a Speech Recognition Engine
Building the Commands
Creating a Grammar
Getting Audio into the Speech Recognizer
Responding to Recognized Words
Creating a Voice-Controlled Painting Program
Speech Commands
Drawing a Skeleton Cursor
Drawing Using the Artist’s Hand
Saving the Drawing Canvas to a File
Rendering a Canvas to a Bitmap
Saving a Bitmap into a Named File
Tidying Up When the Program Ends
Improving the Drawing Program
Adding Speech Output to Programs
Feedback Problems
Summary
10. Augmented Reality with Kinect
An Augmented-Reality Game
Creating Sprites
Creating Sprite Graphics
Adding Images to a Project
Drawing Sprite Images
Setting the Sprite Position
Making a Sprite Move
Putting the Bug Back on the Top
Creating Augmented Reality
Displaying Computer Graphics on Top of the Video Image
Adding a Mallet for the Player
Matching Together Screen and Depth Coordinates
Drawing the Mallet
Splatting Bugs
Isolating the Player Image from the Background
Using a Display Mask
Finding the Player Pixels
Using Depth Information to Make a Mask
Drawing the Mask Image
Setting up the Game Image
Putting the Whole Game Together
The Kinect Manager Class
Creating a KinectManager Instance
Using Kinect Data
Displaying the Kinect Status
Polling the Sensor
Starting and Stopping the KinectManager Sensor
Using the KinectManager Class in Your Programs
Improving the Game
Summary
VI. Kinect in the Real World
11. Real-World Control with Kinect
Controlling MIDI Devices with Kinect
The MIDI Protocol
Creating a Class to Manage a MIDI Connection
Using MIDI in Windows PC Programs
Constructing a MIDI Connection Class
Creating a MIDIControl Instance
Creating MIDI Messages
The MIDI Command Byte
The Note Value
The Velocity Value
Sending MIDI Messages
Playing MIDI Notes
Releasing MIDI Notes
Creating the MIDI Connection
Sending Note Messages
Making a Multi-Note Piano
Creating the Keyboard Display Elements
Responding to Keyboard Events
Playing a Proper Scale
Creating a Human MIDI Keyboard
Creating a List of MIDI Note Keys
Controlling Note Playback
Using the Player Position to Control Notes
Developing the MIDI Program
Using the Kinect with a Serial Port
Serial Ports and Devices
Serial Ports and Devices
The Robot Slave
Linking a Kinect Program to a Serial Port
Creating a Serial Port Connection
Sending Messages to the Robot Using the Serial Port
Creating Command Touch Areas
Receiving Messages from the Robot Using the Serial Port
Summary
12. Taking Kinect Further
Adjusting the Sensor Angle
Using Kinect to Track Multiple People
Identifying Particular People in a Scene
Combining Skeleton and Person Depth Information
Sound Location with the Kinect Microphone Array
Using Kinect with the Microsoft Robotics Development Studio
Mobile Autonomous Reference Using Kinect
Emulating a Robot Environment
Robots and Kinect in the Future
Taking Kinect Further
Mount the Sensor in Different Orientations
Use Multiple Sensors
Move the Sensor Around
Use Skeleton Tracking to Measure Things
Investigate TransformSmoothParameters
Use Voice Response to Do Anything
Have Fun Playing with Video
Make More of MIDI
Good Luck and Have Fun!
Summary
Index
About the Author
Copyright
← Prev
Back
Next →
← Prev
Back
Next →