Visions through a narrow Lens
Using HoloLens works something like that. With the current iteration of the hardware, the experience is anything but immersive. Your view of the "holograms" is limited to a small rectangle in the center of your vision. As long as you're looking directly at them, the illusion is pretty much what the promo videos describe. Move your head even slightly from side to side, however, and you cut off pieces of the hologram, like cropping a photo.
The "window" through which you see the holograms isn't at a fixed point in space. It moves with you, always directly in front of your face. Once you've located a hologram – not always an easy task, since they're invisible unless you're looking directly at them – you can walk all the way around it and view it on all sides. But you're always looking at it through a window, or a tunnel. Unless it's very small (or you're very far away from it), it's almost impossible to keep the whole thing visible at once. You have to pan your head around to get the full picture.
Could Microsoft one day come up with a version of HoloLens that extends the hologram window to reach the wearer's peripheral vision, so that there isn't such a stark separation between the hologram world and the real one? Maybe. But it hasn't yet, so don't be fooled.
Interacting with thin air
One thing the HoloLens designers got right is the audio, which is provided by a set of speakers, one above each ear. Nothing goes into or covers your ears. The speakers produce a remarkably lifelike directional 3D sound effect, which can even fool you into thinking sounds are coming from directly behind you. (This proves extremely useful for figuring out where holograms are in the room when you can't actually see them.)
If the visual element of HoloLens was a disappointment, however, interacting with the device is also more awkward than the promo videos make it seem. The difficulty in keeping holograms visible is just one part of it. The controls available are also limited – by necessity, since it's a hands-free device.
The three forms of user input HoloLens accepts are gaze, gestures, and voice. Gaze input involves a cursor that follows the center of your vision and automatically zooms forward until it hits an obstruction, like a wall or an object, whether real or holographic. Whatever you let the cursor rest on is "selected."
If you want to activate whatever the cursor is resting on, you can "click" it by making an up-and-down motion with your index finger, like clicking a mouse button. If HoloLens sees you do this, it interprets it as a button press. (This seems to be the only gesture that's supported so far.)
Another option is to use voice commands. You might say "activate" out loud to produce the mouse click event, for example, rather than using a gesture. What specific controls and commands are available in any given app, and their effects, are strictly up to the developer.
Visual apps? Fire up Visual Studio
That's where we came in. The purpose of the deep-dive demo at Build was to walk participants through what it's like to develop an app for HoloLens. It achieved that, albeit only to a simplistic degree.
As announced during the Build keynote, everything that runs on HoloLens is a Windows Universal App. You can create ones designed specifically for HoloLens using a toolkit that Microsoft is calling Windows Holographic, or you can adapt existing ones. A HoloLens user could hang a chat app on the wall like a picture frame, for example, and interact with it that way.
Because HoloLens apps are Windows apps, you build them using Visual Studio – specifically, a preview release of Visual Studio 2015. For purposes of the demo, however, we did all of our work using the Unity 3D games development toolkit. While we were shown some code – in this case, written in C# – we didn't touch it.
Unity was obviously a co-sponsor of the demo with Microsoft, and while we have no reason to suspect it will be the only toolkit for building HoloLens apps, when we asked what other tool vendors Redmond was partnering with, our demo chaperone dodged the question.
Redmond's demo-wranglers walked us through the process of implementing our HoloLens app in a number of stages. The first was simply bringing a hologram into existence. We did that by linking up a few pre-built resources in Unity, compiling the project in Visual Studio, and then downloading it to the HoloLens and running it.