If Microsoft Research gets its way, you might soon be controlling computers using the tiniest of muscle twitches, and no actual controller in sight. With a hand still in your pocket, or behind your back, you will be able to control a computer with a variety of pinching and finger-waggling gestures. (Not that we don’t try to do that already, to no avail, when the darn thing crashes).
This new technology relies on electromyography (EMG), which is basically the muscular equivalent of an EEG – electroencephalography, the basis for brain-computer interfaces. An EMG detects the electrical charges created by muscle cells after they receive a signal from the brain. By placing an armband around the muscles in your forearm – the muscles that control your fingers – an EMG can accurately detect the movement of your fingers. With training, software can then convert specific EMG readings into gestures, which can be fed into the computer as normal keyboard, mouse, or touchscreen inputs.
The video below, of one of Microsoft’s early prototypes, gives a good introduction to EMG-based human-computer interaction (or HCI for short).
In a recent patent application, Microsoft notes that the EMG device doesn’t necessarily have to be an armband: It could be a wristwatch, an article of clothing, or even a pair of spectacles (though do you really want to control a computer with a waggle of your ears?)
In all cases, these EMG controllers would be battery powered and connect to a wireless hub. Presumably you could wear multiple EMGs at the same time, too – maybe you will waltz and jive your way through the Xbox 720′s menu system, and Modern Warfare 4.
On the feedback side of things – which is always a problem with these new-fangled, non-keyboard input methods – Microsoft says that the EMG armband would also include actuators that buzz in response to your gestures. If you perform a pinch gesture, you might be treated to a light vibration that travels all the way around the armband; if you perform a gesture that isn’t recognised, you might be treated to a short, sharp buzz on the top of your arm. (Let’s hope tomorrow’s malware writers don’t manage to hijack these things).
There is also the issue of calibration: Irrespective of its location, the EMG device needs to be placed accurately – and if you’re particularly fat, bony, or muscular, it might be hard to get a good EMG reading. There is also one big omission from Microsoft’s patent filings, though: Accuracy. The video shows Guitar Hero being played, but that’s still just a series of on/off buttons – there’s no indication that EMG could be used to finely control a pointer on the screen.
Ultimately what we’re looking at is a smaller, localised version of Kinect – or a version of Leap that doesn’t require a static, wired-up external input device. Even if an EMG HCI’s resolution isn’t up to scratch, we’re still talking about a technology that could help us interact with smartphones, MP3 players, cars, and other objects while on the move.