Not quite feeling it

One of the major challenges facing virtual reality is that when you’re visually immersed in another world, your internal body mechanisms are screaming “Danger Will Robinson!” since they know you’re not *actually* moving/flying/speeding/whatever. There are plenty of companies working on simulating all that, to trick your brain into coming along with your eyes; but it it’s early days for VR, it’s even earlier days for that.

Take for example, the situation where you’re exploring Mars. As one does (or will be!). How are you “moving” through the terrain, without your legs actually moving? – and if you do decide to walk, how big is your living room?

If you’re driving. how will it feel if there’s no acceleration?

Things like that drive your brain batty. And create nausea / cyber sickness in about 2/3 of people using VR.

Another problem is that you can’t actually interact with anything in VR {yet}. So, can’t crash into walls, fall, pick something up….shakes someone’s hands…it’s like being a ghost!

So you can see all this cool stuff around you but actually interacting with it all is still a challenge. One that many companies are working on solving.

Some solutions these days include relying on game pads or sticks to interact with virtual objects. While practical and, in some cases, effective, it’s not ideal. In addition to not taking into account fine hand and finger motion…and it also doesn’t match what your hand is actually doing in the virtual environment. So yet another point of discontinuity between your hands and brain (more opportunity to get sick).

There are VR gloves, of course, which – while they do come a bit closer to how real hands and fingers work – leave out natural feedback (resistance) that is necessary when “picking up” objects in the virtual world. In order to grasp and pick up a virtual object it has to feel as if you’re holding an actual one.

One company who’s working on the “being able to touch” something in VR is Dexta Robotics, a Chinese company that has developed gloves that not only track the location and movement of fingers for input, but the gloves also apply “force” to simulate actual objects. Importantly, the resistance changes depending on the virtual material you’re interacting with; bouncy like a rubber ball, hard if it’s wood.

Genius. And paving the way for truly immersive interaction. No launch details available yet though *sigh*.

Still, I have to wonder: with all the suiting up you’re going to have to do, to properly experience immersive VR experience though – headset, exo-skeleton gloves, pressure suits, rotary treadmills – how easy will it be to actually fully engage?

Going to put some thought towards what future VR interaction will look like. Because all this gear just isn’t practical.

And it’s still not even multiplayer!

Temporary interactive tattoos


The ultimate “wearable”; smart temporary tattoos that have functionality. Seriously I think all the gadgets we currently carry – phones, trackers, health monitoring devices will very soon just be stickers that we attach to ourselves (or be woven into our clothing) – wrote about both of those before ( and

These are fascinating. Coming out of the MIT Media Lab, these temporary tattoos have different configurations: one iteration has an NFC chip embedded in it (allowing for interaction with nearby gadgets), another lets you use your finger to interact with your computer screen…made out of real gold or silver leaf, anyone can make their own design / circuits, and “tattoo” them on their skin.

While it’s early days on how powerful the interaction is, the battery / power needs are limited – and of course, the programming that drives the interaction is a big question mark for me! – it’s a harbinger of the future and has fascinating potential applications.

We are already moving towards being a disposably minded society. From apparel to jewelry, electronics to furniture, we are in the midst of a revolution in attitude about how we acquire and consume goods. Clothing isn’t bought to wear for years, but is increasingly cheaply made and sold, intended to be worn and discarded a few months later…people aren’t buying a “serious” piece of jewelry once every few years, intended to be eventually be handed to children and posterity, instead cheap “fun” mass produced jewelry that scratches a trendy itch prevails. Once, our grandparents bought a sofa when they got married and meant for it to last decades; electronics are traded in for upgrades every six months.

Even music has gone from owning physical albums to streaming, meaning we don’t have to “own” anything, but can stream whatever we want on demand.

So is it any surprise that when the need for a piece of equipment is eliminated because we can have one to do whatever we want at the moment, the desire or need to own an object dedicated to one function will disappear?



Augmented reality: A primer

I recently had a friend who is a professional artist ask me to explain what the tech behind the PokemonGo phenomenon is, and how he can use it in his work. I’m pasting a bit of my response to him for those who are curious. Augmented Reality (AR) is a fascinating technology, one that’s actually been around for a while – but PokemonGo has brought it into the mainstream.

maxresdefaultWhat is augmented reality?

A way to superimpose something created over the “real” world. It adds to reality, rather than replaces it. Some call it a mashup – a combination of two or more functional elements + data to create something new and different.

It is usually visual, but can be aural or even touch. Currently phones are used as the interface (looking at the world through the phone screen to see the overlay), but in the near enough future eye glasses, car windshields, etc will be able to do it as well so the interaction isn’t so artificial (or hazardous).

The difference between augmented reality and virtual reality is AR continues to let you interact with the real world, while VR immerses you in a totally different one.

Here’s a good video that explains it (from 2007!)

How does it work?

There are different ways it can be triggered:

  • Using your phone’s sensors: this is Pokemon Go uses, it “places” objects at geo coordinates and people “find” them. The sensors it can use includes digital compasses, accelerometers, GPS, radar, etc to sense objects in the real world and draw the virtual data in relation to them. These are most often used in vehicles and mobile phones.
  • Using a pre determined visual trigger: This is where the phone recognizes the trigger and superimposes the artificial over it. Sometimes it uses bar codes printed on objects. To do this the phone needs to analyze the video feed from a camera. It then uses the marker as a reference point onto which to draw the virtual data over the real objects. A good example is this:
  • Markerless: This is a more complex variant of the simpler bar code system, where the computer is programmed to detect markers that are not as specific. This could potentially be any feature in the real world as long as it is visually identifiable enough (street signs are one). The complexity of detecting these features is greater, but has the capability to be more robust.

The content that you’re overlaying doesn’t necessarily have to be 3d content: it could trigger a video, for example. This was used in an ad campaign for the Stella Artois Beerfinder App (see examples below)

How to create it?

You need to create an app. The predominant programming language is one that’s used in gaming, called Unity 3D and a framework to run it on, which seems to predominantly be Qualcomm’s Vuforia (the alternative, with different strengths, is Layar, although it’s not mentioned much). The third competitor – Metaio – was recently bought by Apple and doesn’t seem to be public anymore.

Some good AR examples:

Here are six different examples of different AR experiences.

1. Fixed Experience: Desktop Computer – DeBeers

This AR application helps people shop by visualizing an item “on” at home, without having to go to a store.

This example works by printing out a sheet of paper with a graphic on it. This graphic is called a “marker”. She then holds up the paper so the computer’s camera can see the paper.

De-Beers-Forevermark-Virtual-Try-on-MillemoiHer browser accesses the image from the camera, identifies the marker, checks a database for the correct animation.  Her web browser overlays an animation on top of the graphic and the user can move it around,  spin it like a 3D object by moving the paper or using the keys of their keyboard

De Beers used this in their “forevermark” campaign (link here).

2. Mobile Experience: Stella Artois

This AR application helps people find nearby restaurants and bars that serve the product you’re looking for.

This example works by downloading the Stella Artois Beerfinder App to a smartphone. As someone holds up the phone – it accesses the phone’s camera and GPS/Maps functions, as well as mobile data. As he turns around he can see streets in front of him through the phone screen.

The app connects to a database and shares the loci (gps coordinates and direction the camera is facing) and gets back loci data in response (addresses/storefront info for stores selling the beer. As he looks through the screen, the app overlays an icon on restaurants and bars that sell Stella Artois Beer.

It’s a lot faster than googling “who serves Stella Artois near me” and hoping to find an answer.

3. Mobile Experience: Pokemon GO

PokemonGo is currently all the rage. It’s not the first AR game, but it is the most successful to date. It’s the ultimate scavenger hunt! – and it’s inadvertently getting people into shape lol.

This is how it works.

A woman downloads Pokemon Go game app and creates an account. The game accesses her camera and GPS and mobile data; as she plays the game, she “finds” Pokemon tracks superimposed on real world locations – and follows them


To catch a Pokemon she has to hold up her phone so the camera can show where the Pokemon is; now the app is capturing camera data as well as GPS and direction the camera is pointing in. It “shows” a Pokemon Character standing on the path of a park or on a bench in a museum. She then captures the Pokemon by “throwing” a ball at it on the screen and earns points, which she shares the points team (other players aligned with a certain team).

4. Mobile Experience: Zombie Run (Audio Only)

This AR uses only audio to superimpose a game onto the real world.

The Zombie Run app connects to mobile data, then uses the phone’s GPS and accelerometer data to play sounds to the person playing. When he turns on the game – which is audio-only – he starts running and are given both directions (thanks to GPS) as well as running speed data. The directions they hear push user to “find” caches of supplies.

At determined intervals the game will play sounds and give commands that the player is being chased by Zombies, which they can outrun if they speed up.

I don’t actually know what happens when they catch you, but I suspect it sounds grisly.

5. Mobile Experience: Glassholes

Terminator-Version-app-pop_10766AR that helps recognize and identify people, overlaying information about them on the ‘real” world with the help of Heads Up Display (HUD) (a la Google Glass, or tech enabled eyeglasses).

The Google Glass has a built-in camera, mobile data connectivity and a transparent display in the field of view.

As a person walks through a crowd or a party, the Google Glasses are capturing pictures of people’s faces at the party, instantly processing that info using facial recognition and comparing it to data source such as LinkedIn, then providing an overlay to the wearer of data such as their name, company they work for, and other useful stats pulled from various social media and other sources.

6: Mobile Experience: Word Lens (now integrated with Google Translate)

This is a very simple AR application, but a great one: triggered by signage, it overlays whatever language you prefer over the sign through your phone screen (although I’m waiting for it to be integrated into your car windshield, so when driving in a foreign country you will “see” the signs in your preferred language).

So, that’s a basic introduction to Augmented Reality. Any other great examples I’ve missed? Tell me in the comments!