Don't touch: The Microsoft Kinect sensor lets people use gestures and voice commands to control video games. It reveals the problems of a gestural interface as well as its benefits.
Credit: Microsoft
Computing
Gesturing at Your TV Isn't Ready for Prime Time
The Microsoft Kinect is fun, but it shows that so-called natural user interfaces still have a long way to go.
- Monday, November 8, 2010
- By Erica Naone
The Microsoft Kinect, a sensor that works with the Xbox 360 game console, offers the first experience most people will have with a "natural" user interface. A player controls the $150 device with voice and gestures; there's no need to hold any sort of controller or wear any special gloves or clothing. In a recent talk at MIT, Microsoft's chief research and strategy officer, Craig Mundie, described the Kinect as a preview of what's to come for user interfaces, suggesting that what works in gaming now will soon be used for shopping, design, and many other common computing tasks. Instead of thinking about controllers, keyboards, and other "application-specific prosthetics," Mundie said, people could focus on the task at hand, making software much more appealing and easy to use.
But while using the Kinect for gaming is a fun and interesting experience, the device also illustrates that natural user interfaces have a long way to go before they could be suited to most everyday applications.
The Kinect uses both software and hardware to pick up a person's position, gestures, and voice. To measure position, it emits an infrared beam and measures how long that light takes to bounce back from objects it encounters. Four microphones can receive voice commands, and software filters out background noise and even conversation from other people in the room.
Since all these systems need to be calibrated, setting up the Kinect takes some time. After you connect the sensor to an Xbox 360 and position it near the center line of a television, the Kinect's motors automatically adjust its angle so that it can get a complete picture of the user.
The Kinect also needs a lot of space. It needs to be able to see the floor as a reference point for objects in the room, and the user has to stand at least six feet from the device (eight feet if two people plan to use it).
It also tests the sound levels in the room and adjusts for noise coming from the television's speakers. If anything changes in the room—if furniture is moved, for instance, or the sound environment changes significantly—the device is thrown off and needs to be recalibrated.
All this means that as an everyday interface, the Kinect would make little practical sense. Its space requirements strain the capacity of a typical urban apartment. If Microsoft wants to make natural user interfaces accessible to everyone, it will have to consider the needs of the dorm room and the cubicle. The calibration process is also too finicky to make the Kinect useful for any critical application. Users would never tolerate needing to recalibrate in order to check e-mail.
- 1
- 2
Related Articles
Computers Get in Touch with Your Emotions
Machines that respond to emotional state could help you focus better on the task at hand.
Gestures that Your TV Will Understand
The company behind Microsoft's Kinect controller wants to kill the remote.
tonym
3 Comments
- 926 Days Ago
- 11/08/2010
Erica I agree that Kinect type interfaces would not make sense for most business users, for all the reasons you stated and more.
However, I don't think that is where people would try and take it. I think the next real place for a Kinect style interface would be for consumers. Imagine (similar to Minority Report) that you walk into a store, and a digital clerk greets you and asks you how you liked your previous purchase. If you liked it, the clerk then gives you a list of similar but new options they have. After selecting which one you are interested in, it lets you know where in the store to head to.
This would actually be pretty simple, as it would only take a second to link your face to your purchase history on checkout. If they used algorithms similar to Netflix (which can predict user satisfaction with over 90% accuracy) to drive sales of new products to the users that will love them, it could be a huge revenue generator.
Erica Naone
70 Comments
- 926 Days Ago
- 11/08/2010
Your consumer scenario is interesting, but I'd still worry about speed. I find it amazing how impatient I can be with an interface. In the shopping scenario you describe, holding a gesture to confirm interest in a potential purchase, etc, would get old to me very quickly.
I am much more patient with people than with interfaces because I can see what people are doing (folding something, walking toward me, etc). Since the work the interface does is invisible, part of me believes it should be instant. I don't think I'm alone in this--look at how many people will continue to click a button on a web page when it doesn't respond immediately.
I think speed will be a major issue for gestural interfaces in any context outside of a few very specialized ones.
timmaay
1 Comment
- 926 Days Ago
- 11/08/2010
This article is criticizing the Kinect for something is doesn't claim to be, an all inclusive device used for gaming, shopping, business, etc.
I'm not a fan of Microsoft by any measure but this device works quite well for what it's used for, namely gaming. I bought this the day it came out and I can honestly say I was blown away with the accuracy of it.
Thanks to MS for spending the money (approx. 1 billion from what I heard) and bringing us the first truly hands free interactive controller.
I see this expanding to other areas of people's lives but will need to be perfectly specifically for those applications before it becomes a reality.
Erica Naone
70 Comments
- 926 Days Ago
- 11/08/2010
Good for gaming, not sure about other applications
To be clear, I think the Kinect is a very impressive piece of technology. It's incredible to use a gestural interface without needing any extras (even in "Minority Report," Tom Cruise's character needed gloves).
However, the idea for this story came because I recently saw Craig Mundie demonstrate the Kinect and talk about it as a first step toward natural user interfaces that he thought would become pervasive.
I think that the Kinect, as incredible as it is, also illustrates why the grander vision of natural user interfaces everywhere is impractical. That's what I'm trying to get at here.
rick_merrill@alum.mit.edu
16 Comments
- 925 Days Ago
- 11/09/2010
I am 69 yrs old and bought the first one in Massachusetts at 12:02AM at Walmart. This helps me stay competitive as a grandfather.
The system recognizes new players who just jump into the game! This requires enough space for two players at a time and an 8x8 foot space is
adequate for that and I'm sure would accommodate most living spaces, even in an apartment. You will eventually find you did not need that coffee table there!
True, it does not recognize one finger or two, but it spots your "command" guesture in a second.
Criticism is good and will lead to the next product, and journalist can find that negative comments get peoples attention. But I am very positive about a device that lets adults play equally well with little people.
ohene
1 Comment
- 925 Days Ago
- 11/09/2010
It's very hard to find a product Microsoft came up with and led in it. Unfortunately, the same guy - Craig Mundie - who failed at delivery the Tablet is taking a central role as chief strategist in this. I remember very well when he led the Tablet effort, shout out the younger voices and kept as a design criteria "ability to haul a Tablet in a backpack"......of course we all know what became of Windows Tablet. If this technology (nothing original here...) is going to be successful as a compelling gaming device, folks like Craig need to retire and let young minds take over.
profquatermass
57 Comments
- 925 Days Ago
- 11/09/2010
The average British living room or bedroom doesn't have a 6x6 of open free space let alone 8x8!
So a lot of UK people are going to be very disappointed with this gadget....
rick_merrill@alum.mit.edu
16 Comments
- 925 Days Ago
- 11/09/2010
For a single player Kinect will work
within a 3foot by 6 foot area.
This may vary with placement. I put it at belly-button height.
Also, it turns out that different games allow more and more advanced Moves!
aunderdown
78 Comments
- 925 Days Ago
- 11/09/2010
Glass Half Full Tone of Article - indicative of bias?
I have to agree with the commenters who think that the article is too critical of Microsoft's new gaming interface. I got the sense that the author was intent on focusing on Kinect's limitations, even those that are not relevant to the specific application. There wasn't too much about how this new technology actually works and where developments may be heading. Perhaps, I'm jumping to conclusions in thinking the author is not a big fan of Microsoft and may have let that influence the tone and emphasis of the article.
Iamthep
1 Comment
- 925 Days Ago
- 11/09/2010
There is no need to recalibrate if you move things around in the background. If I just want a quick romp through a game I don't bother moving couches. If I want to use it for two player I move couches. No recalibration is necessary. I don't have to worry about turning lights on or off or people walking around behind me. It just works.
And the UIs that are being used today are simply a limitation of the UI designers. Not the system. Try Dance Revolution menus. They respond nearly instantly. You choose an item with the angle of a single arm and then select by bending the same arm at the elbow.
You could also easily do flicking motions to mimick scrolling type effects. The depth is there. Simply put a ripple effect on your UI whenever you "touch" the surface of the UI.
Sure it would be easier if the skeleton tracked fingers, and I expect you will find that enabled in version 2.0.
Kinect is amazing and is worth every penny. It is every bit ready for primetime.
jayfray
1 Comment
- 925 Days Ago
- 11/09/2010
The Right Part: Kinect, while just the start, is huge leap forward does portend the future of natural user interface (NUI).
The Fail Part: It is the first in market and already playing on primetime in thousdands and thousands of livingn rooms as we type
Finally: It's not minority report but puts us on a clear path to that end. By Noane's measure all evolution and innovation in technology must apparently be born as a fully grown adult - rediculous miss from MIT.
jpgerri
1 Comment
- 916 Days Ago
- 11/18/2010
e-mail, a critical application/? You need to get out of the cubicle, Honey.
On Twitter
Become a Fan on Facebook
Subscribe to the Feed
neiladam
1 Comment
misplaced criticism
Come on, this is hardly fair. Criticising a specialist games interface for not being good for a wide range of other applications is daft.
If Microsoft (of whom I'm not a particular fan) wished a gestural control for an office user surfing the web, I doubt it would look much like the Kinect.
From the little I've read (i've not got one), it's a superb device for what it claims to do. It's just not designed for doing the washing up or remotely swatting flies.
Reply
hendersonmj
33 Comments
Re: misplaced criticism
I've got one. I knew this was the real deal the moment I saw my technology-phobic girlfriend rolling on the floor howling with laughter while we were "boxing". She had completely forgotten that this was a "video game". Okay, I don't see how this will interface with my office applications or database tools. But those applications' user interfaces were built for the keyboard-and-mouse regime. There is nothing to prevent the Kinect device from being scaled to fit my desk instead of my living room.
Reply