360 Video Virtual Reality

This is actually quite momentous, and something I’ve been musing about (how it would work) for a while. High def *video* capture – NOT CGI, not a 3d model, but something that you can experience in virtual reality space as if you were standing there in the real world.

This is a critical shift, eliminating the need for the artificial creation of worlds/experience. Can you interact with it (touch anything)? Probably not. But it will come.

I knew there were companies working on taking moving pictures and interpolating the 3D models out of them (required to move around things, and have them shift as you move your POV), am finally seeing some of it coming to fruition. I’m not actually sure that is the case here (that it’s not just smart faking of 3d perspective), but in order to develop a world where you can interact with things they need to have physical definition – otherwise you won’t be able to touch them, pick them up etc. So you need to not only record a scene, but interpolate the depth of objects and spaces, map that to a wireframe (3d talk for…an object) AND THEN give the user (visitor?!) the ability to move around the scene. That is a *lot* of computing power.

The road is long and steep to get this to be in market, but can you imagine what will happen when this technology is merged with all those 360 camera captures out there? You can literally experience or relive what someone else is doing. Literally travel the world without leaving your comfy Barca Lounger.

Now we only need some smart company to start engineering “Smell-O-Vision” so you can really experience what it’s like to walk around Jaipur (I only pick on Jaipur because I’ve been and can say, you don’t want Smell-O-Vision).

The interesting bit about this isn’t that there are realistic created 3d VR experiences, but that creating them just got one step closer to being easier – and more real. Creating an immersive experience in VR isn’t as easy as it sounds; you know that suspension of disbelief you have to have while watching a movie? And how when one thing is off – say a science fact (umm, maybe that’s just me lol) – you are immediately pulled out of the experience? VR has that even worse. We have a lifetime of experiences in the real world to check against, so any tiny little thing that’s off when you’re there –  say, a shadow not being right – and you’ll be pulled out of the realism. Being able to use 360 footage to do it goes a long way to solving those kinds of problems.

I have a friend who is about to leave the city he’s lived in for 20 years, to move to New Zealand. It’s a long distance and he probably won’t get back to there often – but with this, he could create his own 360 video, and after he moves relive his favorite walks as if he were there, whenever he feels like it. And some day, he can be joined by his long distance daughter virtually, and they can walk together.

Isn’t that a wonderful way VR will add to people’s lives?

Reality, Virtually Hackathon

So stoked….I am going up to MIT Media Lab‘s all day workshop this Saturday, to learn about programming in Augmented and Virtual Reality as part of their Reality, Virtually Hackathon…while I freely admit that a portion of the nitty gritty programming will undoubtedly be over my head, I’m going to get a crash course and overview of the essential process, by all the companies who are the big players in the space.

I’m well chuffed, as they say in the UK.

Companies presenting include Unity, the programming language used to create both Augmented Reality, and Virtual Reality; Microsoft – who is involved because of the Hololens; Google’s Tango, which is technology that helps devices understand where they are spatially, and in the world , and others.

Here’s the full agenda. Don’t fall asleep 😉

Not quite feeling it

One of the major challenges facing virtual reality is that when you’re visually immersed in another world, your internal body mechanisms are screaming “Danger Will Robinson!” since they know you’re not *actually* moving/flying/speeding/whatever. There are plenty of companies working on simulating all that, to trick your brain into coming along with your eyes; but it it’s early days for VR, it’s even earlier days for that.

Take for example, the situation where you’re exploring Mars. As one does (or will be!). How are you “moving” through the terrain, without your legs actually moving? – and if you do decide to walk, how big is your living room?

If you’re driving. how will it feel if there’s no acceleration?

Things like that drive your brain batty. And create nausea / cyber sickness in about 2/3 of people using VR.

Another problem is that you can’t actually interact with anything in VR {yet}. So, can’t crash into walls, fall, pick something up….shakes someone’s hands…it’s like being a ghost!

So you can see all this cool stuff around you but actually interacting with it all is still a challenge. One that many companies are working on solving.

Some solutions these days include relying on game pads or sticks to interact with virtual objects. While practical and, in some cases, effective, it’s not ideal. In addition to not taking into account fine hand and finger motion…and it also doesn’t match what your hand is actually doing in the virtual environment. So yet another point of discontinuity between your hands and brain (more opportunity to get sick).

There are VR gloves, of course, which – while they do come a bit closer to how real hands and fingers work – leave out natural feedback (resistance) that is necessary when “picking up” objects in the virtual world. In order to grasp and pick up a virtual object it has to feel as if you’re holding an actual one.

One company who’s working on the “being able to touch” something in VR is Dexta Robotics, a Chinese company that has developed gloves that not only track the location and movement of fingers for input, but the gloves also apply “force” to simulate actual objects. Importantly, the resistance changes depending on the virtual material you’re interacting with; bouncy like a rubber ball, hard if it’s wood.

Genius. And paving the way for truly immersive interaction. No launch details available yet though *sigh*.

Still, I have to wonder: with all the suiting up you’re going to have to do, to properly experience immersive VR experience though – headset, exo-skeleton gloves, pressure suits, rotary treadmills – how easy will it be to actually fully engage?

Going to put some thought towards what future VR interaction will look like. Because all this gear just isn’t practical.

And it’s still not even multiplayer!

That’s a hill of beans (a hill of seed?)

These people got financing for, among other reasons, they aren’t inventing any technology, rather are closer to a traditional studio creating Virtual Reality content for clients. That’s a lot of capital for a seed round!

Penrose Studio raises seed capital for VR content studio

Lucid trips

This virtual reality “experience” (it’s not really a game) does sound like serious fun…instead of the traditional combat-themed shoot ’em up aggression-based environment, this strives to create an out 0f body experience.

Lucid dreamsThe article’s author says it feels a lot like being Ironman! You’re floating and sinking and through using your virtual arm and hand positions, interact with different worlds. There is apparently a game component as well….

Virtual and other alternative reality experiencing is going to take off this year. And just like video killed the radio star, it will kill the flat screen video (we are after hyper visual creatures who like escapism). Articles screaming about VR addiction, and treatments, starting in 3…2…1..