OSKAR – A proposed heuristic for AR control

So, I answer my front door, and I see someone dressed in yellow from head to toe. Assuming I like yellow, this is a pleasant surprise. However, if yellow brings out the worst in me, then I’m not a happy camper!

But, what if I could control my view of someone else’s outfit, i.e. change the outfit’s colour, or maybe even change how I view their hair, eyes, or skin? Instead of a yellow outfit, I see purple, or military garb, or pink lingerie, or a person who is red, white and blue all over?

Well, a recent technology called Augmented Reality (AR) is opening the door to this kind of control, and the quintessential AR question we should all be talking about is: How do we determine who owns what we look like in an AR space? I have come up with an idea for a potential protocol. But first, let’s look at today’s AR technology and where it’s headed.

The most recent AR technology is brought to us by companies such as Microsoft in the form of wearable goggles they call “HoloLens.” Currently, these goggles allow the wearer to view a mix of what is actually there in the environment, along with various limited, yet cool controllable simulated items including; the ability to project a TV or computer image onto a blank wall or view a 3D immersive video game like Minecraft. So far, it all seems fairly harmless. However, there is explosive growth on the AR horizon. Exciting new, more controllable and debatable features are coming our way and it is our responsibility to consider the potential effects and figure out how we want to proceed.

Thus far, only one person wearing goggles can interact with a simulated space. Soon, two or more people will be able to wear goggles and manipulate the same space. And once we can control someone’s height, weight, clothing, skin colour, etc., we can potentially invade their privacy, or leave them feeling offended or bullied. We already have huge issues with internet privacy invasion. Just this past week we saw a very intriguing and complicated case in which a judge awarded Mr. Hulk Hogan $115 million for sex-tapes released against his will. So with AR, we must be debating the ultimate question: Who gets to control objects in a simulated space and the way people appear in them? Let’s follow the logic and see how some seemingly harmless AR scenarios can morph into something unacceptable for at least one of the parties involved.

Imagine I love yellow and want anyone who comes near me to appear to be wearing yellow. As long as this is a non-shared space (something only I can see), I should be able to 'dress' a person in any colour I like, right? How about if the person I meet is yellow-phobic and I tell them that in my eyes/view they are dressed in yellow, and then they experience a panic attack. Is that still ok? How about if they are in the same augmented space and when they approach I force the system to dress them in yellow. Since they share the space, they too can see that they are in the colour I chose (which I know they hate). Is that ok? How about if that same someone chooses to be part of a simulated paintball game and they get splattered with yellow paint. Is that ok? Lastly, how about if I made a business deal with Mountain Dew so anyone coming into my building would automatically be shown in yellow and have a Mountain Dew Emblem on their shirt. Is that ok?

If you answered “no” or even “I’m not sure” to any of the above questions, therein lies the need for discussion and decisions on how to protect each other’s personal appearance thresholds. So, how do we figure out what is acceptable to all parties, and what are the standards that should be used for managing this?

Right now I see three possible options for who gets control. The first one involves a single person chosen to control everything in the space, like in the scenarios I’ve mentioned above. Another is some negotiated form of control whereby each person gives the others rights to manipulate parts of the same scene. And the third option grants all control to the product designer, like in multi-person online games where each player controls their own character, and the game designer controls the rest of the system and sets the limitations for how people can look and interact with each other and the environment.

So, does it make sense for the designer to have full control over the way someone looks? Or, does it make sense to allow us to have some kind of negotiated control over each other’s appearance? Perhaps as a default, a person should be able to control their own appearance just as they do outside of an AR space. However, there is also some value to be gained in giving up some of that control in an acceptable manner. Considering the way these systems are evolving, I expect a negotiated system makes sense. Therefore, I am proposing a heuristic that I loosely call 'OSKAR' to help technologists and AR space designers evaluate who should own the control in the spaces they enable.

Here are the factors which define OSKAR.

  • Offense - is the alteration of appearance going to be offensive? - to the impacted person or to any observer
  • Scope - from what distance and in what area/space will the alteration occur? - private house, business, public space, school, internet, etc.
  • Knowledge - who can or will be aware that the alteration is occurring and how to let them know? - the one making the change, the one impacted or other observers
  • Allowed – What circumstances will require permissions and/or preventions? - i.e. pre-acceptances, or implied or stated acceptance like alerts upon entry that the building owners have control, etc.
  • Reason – what is the reason/intent for the alteration? - education, non-profit, for profit, personal interest, entertainment, bullying/hurting someone (this relates to the ‘allowed step’)

Applying a fair heuristic like OSKAR or something similar to the designs we create should enable us to more comfortably embrace the future of AR. I believe it is time to have an open dialog about who controls what in the AR space. What do you think? Is this a reasonable approach for AR designers to use as they build controls into their system?

Can you think of other important factors that are missing? There is much more to think about. How do you want to be treated in the newly created AR world? Chime in here and let’s talk about it!

Image source: Microsoft, HoloLens Demo