Yesterday during the CES day-two keynote—somewhere between the future dancers trying to communicate to us through their strange, alien movements and Bill Clinton reminiscing about how heavy cellphones were back in the day—Microsoft CTO Eric Rudder took the stage and demoed…something. I mean, this wouldn’t even be the first time Microsoft jumped in on a keynote this year at CES (what with CEO Steve Ballmer hijacking the stage from Qualcomm CEO Paul Jacobs), but it was at least interesting this time.
We were shown a video of some gaming, but the images and effects and whatnot extended well past the screen of the television in front of the players. In what we were assured was actual video of a real lab scenario, snow fell around the room and explosions expanded well into the walls and space in front of the TV. Little strobe lights from within the game would shift from the screen to the floor and splash colored light at your feet. It was a little like Philips’ Ambilight and amBX technologies but way more detailed and, I dunno, interesting. Here’s what we saw yesterday:
It’s a technology called IllumiRoom, a proof-of-concept experiment that uses a Kinect to map the physical layout of the room you’re playing in and then uses a projector to cast appropriately morphed images into your world from the game. It all happens in real time and due to the physical-to-digital mapping, totally changes your peripheral perception of your surroundings. It aims to fundamentally alter the level of visual immersion of gaming.
It goes so far beyond just making sparks and snowflakes drop in around you. You’ll see the entirety of the player’s vision extended onto the floors and walls, but at 0:36 in, you’ll see a crazy highlighted wireframe version of the world being displayed, the gameplay implications and possibilities of which are huge. And at 0:42, you’ll see gunfire actually warp the real life surroundings of the TV. The shelves and plants and stuff all pulses and distorts with each bullet fired, inducing apparent motion in the player. It’s super neat.
But that’s just about all we get. We get a name and a video and nothing else. It looks a bit like Microsoft’s Immersive Display Experience patent that was found back in September, but who knows. Right now you just get to lust of this thing for a while. Or, buy into Microsoft’s plea, and get into computer science. I did, and I can tell you that computer vision is one of the most fascinating things you’ll ever study.