MIT scientists develop UI that fuses smartphone and desktop screens (video)

MIT scientists have developed a new user interface that allows smartphones and laptops to overlap and push the boundaries of what the two devices can achieve together.

Related: MIT develops soft robot ‘tentacle’

THAW, as the innovation has been named, allows the two to work together in such a way that they seem as if they are sharing the same brain and wherever the smartphone is moved it appears as part of the landscape of the laptop.

“We own multiple devices—we have a phone, we have a laptop, now everyone is going to have a watch,” MIT Media Lab student Philipp Schoessler told Wired. “We think they need to be ways to more seamlessly integrate everything.

The concept is the result of a collaboration between San-win Leigh and Philipp Schoessler, students from MIT’s Media Lab. It works by using a piece of software on the laptop that overlays a rainbow coloured grid on the computer screen and using the iPhone rear camera to detect which area of the monitor it is hovering over. Both devices are connected to each other wirelessly and use this to communicate positions to produce the resulting synchronisation.

“We’re displaying the pattern only behind where the phone’s camera is looking; it’s like a peep hole you can see through,” explains Leigh. “When you move the phone around, the peep hole follows your position and the phone can see the color pattern through the peep hole.”

The pair don’t plan to bring the product to the commercial market for the moment, if at all, and are more likely to release it as open source to see what other apps developers can come up with.

Related: MIT researchers develop 3D pixel screen that changes with your movements

“We don’t really think of it as a product,” says Schoessler. “We’ve really just touched the surface of the applications—I think it would be nice to see what other people might come up with.”Porthole Ad