Vid North Carolina boffins have been watching text entered into iPhones from 60 meters (197ft) behind the shoulders of users – or from the front, by reading the reflections in the users' glasses.
The process uses a standard video camera. It is even possible using an iPhone's camera, though the range decreases and relies on the iPhone's habit of popping up big versions of characters typed. Once the video has been fed through the researcher's image stabilisation software, and run through some optical character recognition software and natural-language analysis, the meaning emerges, as this (silent) video demonstrates:
Apple's iPhone isn't the only smartphone to provide visual feedback by popping up an enlarged version of the character pressed, but the technique won't work with those that don't. The researchers also admit that alternative text-entry techniques, such as Swype, will confound the recognition, but those are only used by a minority.
There are some other videos showing how reflections can be read, and the accuracy possible, on the boffin's own site. Their full paper (PDF, interesting, but very mathematical in places) demonstrates that with a decent video camera they were able to collect very accurate renditions of what was typed from a considerable distance.
It seems that the biggest limitation was motion blur. Stabilisation can only work so well and as the characters pop up on the screen only for a moment, a single blur make a character impossible to read. That's easily addressed with better video equipment, and better analysis, but this research was deliberately based on standard kit.
One can imagine Jason Bourne using such a technique, and it's interesting to hear that it is possible. It might pay to think about one's surroundings when entering a password, but in reality there are already plenty of other threats to be concerned about without worrying about what people might be able to pick up reflected in your sunglasses. ®
