Microsoft Research recently took the wraps off IllumiRoom, an augmented reality, peripheral projection technology that could be one of the killer features of the Xbox 720 or Kinect 2.0.
By combining a Kinect camera and a projector, IllumiRoom augments the area around your television to increase immersion in the game you’re playing or the movie you’re watching. The best way to get your head around IllumiRoom is to watch the mind-blowing video below. Microsoft says the video is “captured live,” and hasn’t had any special effects added in post-production.
IllumiRoom is surprisingly simple in its operation. Kinect captures the appearance and geometry of the room, and then this data is used to adapt the extra visuals that are projected against the wall and furniture around your TV. Microsoft Research doesn’t go into details on how the extra, augmented reality visuals are created, but presumably they’re just a secondary output from whatever software is running on the television. In the case of games, it’s fairly safe to assume that the developer would want full control over the projected visuals.
The use cases of IllumiRoom, as you can probably imagine after watching the video, are extensive. In first-person shooters, the projection might simply be bullet trails and explosions, but in RPGs and adventure games cityscapes and landscapes could be projected. Imagine a horror/suspense game where beasties skitter over some shelves in your peripheral vision. IllumiRoom might also work with movies and other forms of media (action and horror movies, music visualisations, and so on).
How does IllumiRoom dovetail into Xbox 720 and Kinect 2.0? Well, for a start, IllumiRoom was shown off at CES 2013 – odd, considering this is a Microsoft Research project, and MS didn’t have an official CES presence, instead piggybacking on keynotes presented by Qualcomm and Samsung. Secondly, the video almost looks like a professional commercial, whereas Microsoft Research videos are generally ad hoc and amateur-shot. In short, IllumiRoom seems like much more than a pet project to Microsoft – and remember, it is Microsoft Research that developed the original Kinect, too.
It is very clear from the video, however, that the IllumiRoom prototype currently uses a strong, standalone projector – probably hanging from the ceiling. While enthusiasts might not have a problem with such a setup, it would be a hard sell to the average, console-buying consumer.
This is where an interesting patent comes in, though: Back in 2011, Microsoft filed a patent on an Immersive Display Experience that includes a spherical “environmental display” projector (pictured above, number 116, on top of the TV). The patent goes on to say that the environmental display could consist of two wide-angle RGB projectors (one facing left, one right), and that the projector could be placed anywhere in the room. There is even mention of 3D projection – which makes sense, if the TV is displaying 3D content.
Is it feasible to produce a small, spherical projector that’s capable of providing the experience you see in the video? That’s a good question. IllumiRoom was demonstrated during Samsung’s CES presentation – and Samsung does make the world’s brightest pico projector (30 ANSI lumens), and has previously squeezed a pico projector into the Galaxy Beam smartphone. By the time the Xbox 720 and Kinect 2.0 roll around at the end of 2013, maybe Samsung will have upped that to 40 or 50 lumens – not a lot, but enough if the room is darkened. It’s quite reasonable to infer that Samsung and Microsoft are working together on future hardware.
As to whether IllumiRoom is actually a key feature of the Xbox 720 or Kinect 2.0, we’ll have to wait until E3 in June, where Microsoft and Sony should finally unveil their next-gen consoles. Given Microsoft and Microsoft Research’s recent focus on augmented reality and hands-free, gestural interfaces, IllumiRoom would certainly make sense.