A computer without sensors is a pitiful, useless thing. Keyboards are sensors, as are mechanical-optical paper-tape readers, magnetic heads on storage discs, and the logic scanning for ones and zeroes on an ethernet interface. Everything a computer does - outside of calculations - involves a sensor.
Despite this, we tend to judge of our computers by their CPUs, rather than their complement of sensors - or we did, until the smartphone came along. Although CPU is important on a smartphone - my iPhone 6S Plus is faster than any desktop I’d purchased before this year - the raw grunt at the core only matters if the mobile has the right complement of sensors. The history of the smartphone is an arms race, fought with sensors.
The most important sensor on a smartphone - it’s raison d'être - is its microphone. (Or was, back when we used our smartphones to make voice calls.) A video camera came along quickly. Apple cleverly added a proximity sensor - infrared reflected off skin - so the smartphone screen dims (saving battery) when cradled against your head.
Apple’s competitors, looking for a bigger slice of the smartphone market, upped the ante with sensors of their own. Before it imploded, Nokia built high-quality, ultra-high-resolution cameras into its smartphones. And, from the first iPhone, smartphones have used gyroscopes and accelerometers, so they can detect and adapt to portrait and landscape orientation.
While that last may seem a trifling addition, it opened the smartphone to something completely unexpected.
Flash back a quarter of a century: I’m sourcing components for a consumer virtual reality system. An accelerometer is an absolute necessity in a head-mounted display, because it senses the motion of the head. Accelerometers exist in silicon, but priced at US$25 apiece, their only customer is the automotive industry - sensors used to trigger deployment of the airbags in a crash.
In the end, I invented my own sensor, because silicon accelerometers cost too much.
A few hundred million smartphones later, accelerometers and gyroscopes have become cheap as chips. Literally. From twenty-five dollars to less than twenty-five cents, the conjunction of Moore’s Law and Steve Jobs made these sensors cheap and abundant.
With many smartphones using high-quality accelerometer/gyroscope sensors, the groundwork had been laid for Google’s Cardboard - really no more than a cheap set of plastic lenses set at the right distance from a smartphone screen. Everything else about the Cardboard experience happened inside the smartphone - because the smartphone suddenly had the right suite of sensors to generate a head-tracking display.
Theoretically, Google’s Cardboard should give you the same smooth virtual reality experience as Samsung’s Gear VR. But it’s like chalk and cheese: Cardboard does the job, but it always feels as though you’re fighting the hardware, where Gear VR feels as comfortable as an old shoe.
The reason for that lies with the sensors built into Gear VR. Oculus CTO John Carmack worked with Samsung to specify an accelerometer/gyroscope sensor suite that could feed Samsung's flagship Galaxy S6 smartmobe with a thousand updates a second. The average sensors, on a typical smartphone - even the very powerful Galaxy S6 - won’t come anywhere near that.
Head tracking can only be as good as the sensors used to track the head. The proof of this is the difference between Galaxy S6 in Cardboard, and Galaxy S6 in Gear VR - try both and see for yourself.
This is one bleeding edge in the smartphone sensor arms race. Within the next eighteen months, every high-end smartphone will specify incredibly sensitive and fast accelerometers and gyroscopes. Smartphones work well both in the palm of your hand and when mounted over your eyes. Every major manufacturer will have their own Gear VR-like plastic case for wearing their latest top-of-the-line handset. Except at the very high end - the province of serious gamers and information designers - smartphones and VR will become entirely interchangeable.
Studded with sensors, our smartphones become increasingly sensitive. My iPhone tracks every step I take and every flight of stairs I climb, courtesy of a dedicated motion sensor. I can track my energy output against my weight in Apple’s Health app, and I don’t need a Fitbit for that - my smartphone is enough.
But sensitivity also has its downside: A 2014 demonstration by security researchers at Stanford demonstrated that the accelerometer of a smartphone can be used to ‘listen’ to conversations happening around the smartphone, because the accelerometer senses the pressure of the voice.
Back during the Cold War, the Soviets were caught out shining laser beams onto the windows at the White House, reading voices out of the reflections. The White House responded by pointing speakers at their windows, playing music just loud enough to drown out any other signal. We may need a new app for our smartphones, one that keeps just enough music piping out its speaker to confound anyone using our newly sensitive accelerometers against us. ®