Phone designers to improve reality

New meaning for "deep pockets"

Multicore Expo Future phones will recognize buildings and people by sight and replace reality with something better. They'll also have roll-out HD displays. Or projectors. Or they'll dock with your PC's display.

At least, that's the vision of some visual-computing visionaries at this week's Multicore Expo, inspired by the graphic and computing power of high-performance multicore embedded processors that will power tomorrow's smartphones.

Kari Pulli, who heads the Visual Computing and User Interfaces research team at the Nokia Research Center in Palo Alto, California, described a prototype phone that his company has developed that visually recognizes buildings and people. The object, as he puts it, is to "make the device aware of its surroundings and react to it; to connect the digital and real world."

Using what he calls "a 'magic lens' metaphor," a smartphone's camera captures an image - say, of a building or a person - then compares it with a database of images either in the smartphone's memory or downloaded over the phone's wireless connectivity. Once the image is recognized, the phone provides information about the object in the image, displayed over the image.

Note that this capability doesn't depend upon GPS coordinates, so the object being imaged can be at any reasonable distance from the phone's camera.

The challenge, according to Pulli, is to make the systems good enough "so that they work at least 98 per cent of the time - otherwise people just don't want to use them."

Pulli's team at NRC created a prototype device he calls an "automated tourist guide" of Palo Alto's Stanford University. "You could go to the Stanford campus, point your camera at any of the buildings, and it would recognize which building you were looking at and then provide information about it."

Nokia isn't the only company working on such a system. Startup Mobilizy was recently a Top 50 developer in the Google's Android Developer Challenge with a similar app called Wikitude AR Travel Guide.

The "AR" stands for augmented reality, a catch-all concept that describes layering digital content onto live video while matching perspective. Mobilizy's layer contains straightforward descriptive 2D text (you can watch a demo of it here), but 3D augmented-reality demos from Toyota, Total Immersion, GE, and even a simple game for the Nokia N95 put it to shame.

Pulli believes we'll start to see all this stuff in five years or fewer.

Of course, for full-scale enjoyment of full-scale augmented reality, a smartphone will need a better display. Tony King-Smith, Imagination Technologies' VP of marketing, has a couple of ideas, all driven by the current trends toward improved resolution, more-efficient manufacturing and lower power consumption and prices.

King-Smith cites microprojectors as one possibility. He may have a point. At this year's Macworld Expo, The Reg took a look at the Show WX from Microvision and was pleased indeed.

He also suggested that roll-out displays were another possibility. Perhaps, but the roll-up full-color OLED has been "just over the horizon" for a number of years. Forgive us if we'd rather not hold our breath.

King-Smith's final suggestion was the most practical, if the least exciting: simply being able to easily dock your iPhone into your PC display, HDTV, or other larger screen whenever you need to slake your desire for some augmented reality - or video, games, full-screen social networking, or whatever.

But a display dock would negate the biggest lure of a mobile device: its very mobility. Nokia's Pulli did provide one inventive solution for the big-screen, small-device challenge, however: "bigger pockets." ®