For all the buzz about the new Kinect that will ship with the Xbox One, there are remarkably few facts to go around. Sources trumpet its infrared-enabled ability to detect motion in a dark room, for example, but so could the original Kinect. Taking a look at what we know about the Kinect 2.0, it isn’t at all clear whether it is an exciting breakthrough or just a group of incremental updates.
Improved camera and personalised voice recognition
One clear step up in the new Kinect is the 1080p camera, with better resolution and a 60 per cent wider field of view. This will allow the camera to cover a wider range of simultaneous player positions. The new Kinect also helps the Xbox One quickly and (at least in the demo) accurately respond to voice commands. If it works as well in real living rooms, it’ll be a huge improvement over existing user interfaces. A big part of making that possible is the recognition of individual voices, which should make an unprecedented degree of personalisation possible.
Time of flight replaces structured light
Rather than the coded-light patterns used by the original Kinect, the new version is reported to use direct time of flight (TOF) measurement. TOF sensors are essentially small infrared “radars” that instantly create a depth map. However, TOF sensor sites normally take up room that could otherwise be used for traditional visible light sensing – also crucial for gaming.
Samsung’s integrated Visible+TOF sensor, for example, has a fill factor under 50 per cent, and that’s with a relatively low VGA resolution for its depth map. If Microsoft is truly offering high accuracy depth sensing in its new Kinect, it is either using a completely separate sensor, sacrificing quite a bit of the camera’s potential, or it has truly made a breakthrough in fundamental CMOS sensor technology.
Microsoft hasn’t revealed much about the specifics of its proprietary TOF solution, but does say that the sensor (possibly including the visible light camera) gathers about two gigabits of data – not the same as the 2GB that some other tech sites have reported.
Assuming the TOF sensor captures data at 30 frames per second (matching the camera), that’s about 60 million bits per frame. If it records depth in the same 11-bits per sample precision as the original Kinect, that would more than match the throughput needed for a 1080p resolution TOF sensor. If so, it is indeed much higher resolution than the 640 x 480 depth map of the current Kinect. I suspect the depth precision has also been improved, perhaps to 12-bits or even 16-bits per sample – to help allow the detection of subtle motions in the Xbox One.
Has Microsoft caught up on motion accuracy?
When Kinect was first introduced, it was the benchmark by which other motion sensitive peripherals were measured. Then tiny LEAP Motion made headlines last year by claiming 200 times the accuracy of the Kinect. A year later, Leap is finally about to ship its product, so it would be natural for Microsoft to leapfrog, or at least match, Leap’s claimed 0.01mm accuracy. However, Microsoft has been completely silent on the technical specs achieved through its switch to TOF technology – other than saying it can now track small movements like those made with the wrist.
Some tech sites are reporting the new Kinect’s ability to measure player heartbeats as some type of breakthrough, but numerous research projects have shown it isn’t hard to do that with a standard 1080p video camera.
Similarly, Microsoft’s comment that TOF measures the return time of individual photons has been taken out of context and reported as if the Kinect can measure single photons. In reality, physics makes it impossible to reliably predict the course of a single photon, so any system that tried to measure at that level would be dominated by noise.
In fairness, the gestures demonstrated by Microsoft on stage to control the Xbox One were pretty impressive. However, it wasn’t clear how much of that was made possible by the new Kinect and how much was simply down to improvements in the gesture recognition software in the new Xbox.