augmented reality

Microsoft Steamrolls Everyone with HoloLens

Within days of Google cancelling its Glass Explorer program, Microsoft just steamrolled the whole augmented reality and virtual reality industries with the announcements (and demos) of the new Microsoft HoloLens.

The take-away: Augmented Reality and Virtual Reality are coming. They aren't just fads. There is still a long way to go to reach what we all imagine and hope for, but Microsoft just made a giant step forward, and has the capability to drive hard and go big. 

Before I dig into this, let me make a few points for clarification:

  • No, they aren't really holograms. The HoloLens is another wearable device with transparent displays, just like more than a dozen other companies have on the market.
  • This is augmented reality, not virtual reality. If you aren't sure about the difference, read my other blog posts on the subject. 
  • And no, I have not had a hands-on experience with these yet, so keep that in mind as you read through my comments. I'm making a few assumptions about the tech and experience, and accepting what other people are saying (including the marketing) at face value (for now).
  • This post is based on my initial thoughts about what I've read so far regarding HoloLens, and some general observations about the industry.
Microsoft's HoloLens

Microsoft's HoloLens

Shock and Awe

For me at least, HoloLens is quite unexpected, especially the timing, and how well put together everything seems to be. It seems like a more complete package than anything else out there, and the demo examples were very compelling. I mean, come on...exploring mars, playing minecraft in the living room, and having a remote person guide you through a task and literally marking up objects in front of you with arrows and stuff. Pretty awesome.

The reliance here is on the demo, not some fancy concept marketing video. I feel like I could expect to have this in my hands any time, compared to some other products that feel like they are going to be in development for a few more years before anything substantial is released.

After reading a few articles on the demos from USA TodayCNet, Wired, The Verge, and others, I got the sense that those that were fortunate enough to experience HoloLens were pretty blown away and excited. I mean, this is MICROSOFT we are talking about. When was the last time you heard someone really excited about something Microsoft is doing?

All that aside, I think that the eventual impact of HoloLens is pretty understated right now. Sure, people will talk a lot about augmented reality and minecraft or whatever, but there is more at play here, and the shockwaves are still tiny right now, but they will turn into a major force as the Spring turns into Summer.

Name of the Game

Why did Microsoft go with "Holo" for HoloLens, and why are they calling everything Holograms? While technically incorrect, this is definitely a master stroke by some genius at Microsoft.

  • Everyone knows what holograms are. Most people are clueless about augmented reality, even if you give them the usual Iron Man and Minority Report examples. Virtual Reality is easier to explain, but I find that you still have to explain it to people and even then, more blank stares than not. But holograms...very accessible. "It's like real 3D holograms in your living room! Imagine Princess Leia and R2D2!". Done. People get that.
  • When you name or label something, that puts you in control or at least makes you the perceived authority. 
  • "Augmented Reality" is a mouthful and not very euphonic. I've had this discussion with many people in the industry and most have expressed some desires to find another name or description for the tech. Microsoft just did it for everyone. Watch now as everyone (especially marketers) start using variations of holograms and holo-. Heck, I bet every domain variation with the word holo in it will be scooped up in a matter of days. How will Microsoft respond? Will they try to enforce branding and trademark to some degree?

Augmented Reality's Dirty Little Secret

Well, two dirty little secrets. Dirty secret number one...augmented reality, seen through wearables with transparent displays are, well, transparent. Without some black or opaque background, any graphics on a transparent lens will be see-through. It doesn't matter how photorealistic it is, it will look a bit ghostly. Obviously this is different when you experience augmented reality on a mobile device like a smartphone or a tablet, but that is because the AR graphics are overlaid on a video on a screen that is not transparent.

This is definitely a negative, but since we are calling virtual objects and data that you see in AR holograms now, it suddenly makes sense and resets our biases and expectations a little bit.

The other dirty little secret is the actual window or field of view for wearable displays. The world doesn't magically change when you put on a pair of displays, but rather there is a small square or rectangular-ish space in each lens that you are looking through that can display graphics. This is one of the reasons why people wearing these move their heads around a lot, instead of just their eyes. We can still have some amazing experiences with the current-state-of-the-art, but this is one area where the hardware guys will be chipping away at for years to come, until we have full peripheral augmented, er "holographic" I mean, vision.

So, all those awesome concept videos videos for AR HMDs (head mounted displays) aren't quite what the actual experience is. Fortunately though, there is a growing number of researchers, developers, and startups working on this sort of thing. It won't be long.

Indoor, Outdoor

One thing that struck me about the HoloLens videos and use cases, is that they were all indoors. 

  • Transparent displays offer a dismal experience in brightly lit areas. This will ultimately be fixed when the whole opacity issue is sorted out. In the meantime, a dark shade or lens behind the transparent lenses helps, but that also diminishes the real world part of the experience and makes it harder to see real objects indoors.
  • If I'm going to wear some augmented reality glasses outside, they need to look like a pair of sunglasses. Not a Robocop dome or a bulky headband.

Having said that, all of the use cases were indoors, and that is likely where HoloLens and other AR products will excel. Microsoft played this well here. HoloLens doesn't have to be this amazing thing you use everywhere that does everything, helps you pick up dates, or make you look smarter or like a successful valley hipster.

Let's not forget about privacy

Remember all the complaints about Google Glass and privacy? HoloLens has a camera on it as well. I think the difference is you aren't likely to see someone wearing HoloLens at a bar, a stripclub, or at the movie theater, but still, the point remains. You need a camera feed for all of these awesome experiences (especially the one where your dad is giving you tips on how to fix the plumbing under the sink and needs to see what you are doing). 

Personally, I think all the privacy alarmists with Google Glass are making a big deal out of nothing. If Glass just added a tiny red "recording" light, it wouldn't be any different than snapping video with your smartphone. Everyone knows your camera is on and recording. If it is just on for computer vision tracking or image recognition of objects, no one should care and the light shouldn't need to be on. 

Anyone that complains about privacy with HoloLens is just going to be looking for attention or trolling.

Why is HoloLens such a big deal?

There are a variety of reasons, but the biggest one here is that we are talking about Microsoft. Microsoft is going to take augmented reality mainstream. Watch how many startups will suddenly appear to develop games and other apps for HoloLens. If Google doesn't wake up, Microsoft is going to eat their lunch, in more ways than one. Apple can keep filing overly broad patents all day long and hinting at i-this and i-that, but Microsoft is doing it now. Everyone is going to play catch up. I meant it when I said steamrolled. 

Something else to consider...the showcase of HoloLens at the Windows 10 premiere event is not to be ignored. I expect that there is more going on behind the scenes that is going to tie a lot of interesting things together, which is going to be awesome for consumers and developers.  We aren't talking about some fun hardware and maybe an SDK or yet another app store, or a cool AR demo based on one of the off-the-shelf AR platforms already out there...HoloLens is going to be tightly integrated into the whole Microsoft ecosystem and culture and all that entails. Half of the story is the HoloLens itself (hardware) but the real big deal in my opinion is everything else. Just think of these things will be integrated with the XBox. 

The industry is drowsy and mumbling in its sleep right now, but I expect that sometime in the very near future a lot of companies, developers, and investors are going to jolt awake as they realize what is going on, and the augmented reality war will really kick off. One happy result will be an acceleration in innovation and advancement (hardware and software) as everyone starts to try to leverage the platform for more amazing applications or compete directly with another ecosystem for market share. Anyway you look at it, Microsoft's HoloLens is going to make waves. 

Personally, I plan on surfing those waves. Speaking of which, Investors, Venture Capitalists, and the Press are always welcome to contact me and chat about the industry, what trends are evolving, where the opportunities are, and of course, what I'm working on. 

Thank you Microsoft. The future just got a bit more interesting. 

Feel free to chime in if you have a comment or a different perspective on what I'm talking about here. Seriously, leave a comment. Lots of comments means I'll publish blog posts more often (I know, I'm a slacker). Also, feel free to get me introduced to Alex Kipman at Microsoft. I want to see these things first hand. 

Augmented or Virtual Reality?

While Augmented Reality (AR) has been buzzing in the tech industry for a few years, Virtual Reality (VR) has made a huge splash and comeback with the recent announcement of Facebook acquiring Oculus VR for about $2B.

I've heard some conversation lately about whether VR just knocked AR out of the picture, or if one is better than the other. The recent Facebook acquisition at such an enormous buying price is likely going to spawn a bit of a gold rush as people suddenly try to get on the bandwagon, and this might make it harder for AR startups to find funding (already a hard thing to do).

I think first, we need to understand the differences between AR and VR as well as how these technologies are going to be used. Both have the potential to be massive agents of change, but not in the way that most people think.

Augmented Reality

Augmented Reality (AR) is best described as a digital layer of information superimposed on what you are looking at. Most people are familiar with the term "heads up display" or HUD (as you would see in an airplane cockpit) or the digital yellow first-down line in football. That line isn't actually there, but it is added to the video feed as a layer of information.

Wikitude.jpg

Many implementations of mixed reality are referred to as Augmented Reality. Unless you are really picky like me, or an academic, its probably fine to refer to both as Augmented Reality or AR. 

I wanted to make the distinction here for a couple of reasons, but if you are new to all this, just think Augmented Reality when I say Mixed Reality and you will be fine.

Mixed Reality

Mixed Reality (MR) is essentially the same as Augmented Reality (and almost always referred to as AR) except instead of an overlay with information (for example, imagine people's names floating over their heads), the experience would have digital 3D objects placed in the environment around you, which in some cases you could interact with. Examples of this would be 3D Zombies shambling around a college campus, or replacing your head with Darth Vader's helmet while you are on a video chat with your friends. 

Mixed Reality, or the full blending of the real world and the virtual, is where I think the greatest potential for amazing things is at, and certainly (in my opinion) the coolest stuff to work on.

Virtual Reality

Virtual Reality (VR) is where you are fully immersed in a virtual environment or virtual world (VW), generally through a wearable display, like those that Oculus VR makes, much larger and complicated contraptions (see image to the right, lol), or through something like a bunch of giant screens around you or a dome that you would stand inside.

VR really captured the minds of the public in the late 80s and through the mid-90s with a bunch of very popular and high profile movies. Unfortunately, these movies set the bar of expectations as to what the current day technology could do, and VR was ultimately a huge disappointment. It eventually faded away, with barely a handful of VR evangelists still proclaiming the world-changing implications of VR.

Still though, with the right interface and experience, VR is going to be pretty mind-blowing.

Double Rainbows! What does it all mean?

Augmented Reality (both types) and Virtual Reality are pretty damn awesome and compelling when you think about it. 

AR is ultimately going to have an effect on every day life much in the way the Internet and the Web has so far. You can disagree with this all you want, but it is coming. It might to envision right now while things are still so early, but at some point critical mass is going to occur, and some tremendously amazing application in an area like education or entertainment (early on in any case) will blow the industry up and we will see another information age evolution and rapid revolution occur before our eyes. Some people today are worried about competitors in this industry...honestly, I think they are missing the point. AR is going to spawn entirely new professions and industries, and ultimately change the way we think about and interact with the environment around us and other people. Pick an industry and I'll tell you how it will be reinvented with AR.

VR on the other hand, is, at first glance, much more compelling and engaging. While AR is all about who and what is around you, VR has the potential to take you to an entirely new place (real or imaginary) where you can be anyone and experience anything. Even the rules of physics can go right out the door. Imagine experiencing the world from the perspective of a fly, or exploring something while you are no bigger than half a dozen atoms. On the other end of the scale, flying through space and seeing galaxies in the palm of your hand. What about traveling through time and seeing an event unfold, first-hand, from the point of view of someone that was actually there? Anything and everything is possible with VR, and I haven't even started talking about things like new ways to learn or interact with pure data and information. Of course, I'll be first in line to see the digital frontier like TRON did. 

Remember, AR is about the world around you, blended with digital objects, whereas VR is about an entirely different world, and all immersive. Both of these have tremendous potential and great promise, provided that we don't give up on getting there, and we can get away from horribly designed and implemented gimmicks, just for the sake of a quick buck, or trying to get on the bandwagon.

Having said that, the really smart people out there understand that there are very similar underlying technologies in both AR and VR. It may make sense to differentiate the two for marketing of some product or another, or in some academic discussion, but the real power of both is going to happen when the technologies mature to some level, and we start seeing some convergence and coadunation of both, along with other elements like the internet of things and sensors, new control devices, and a reinvention of information display (are windows and pages really the right metaphor for AR and VR? I don't think so).

Not so fast Buckaroo Banzai!

The best way I can describe the state of the industry compared to what people expect from all the marketing is to use a time machine as the example.

What everyone expects because of all the marketing hype: Dr. Who's T.A.R.D.I.S

Step right up folks!

Step right up folks!

What is actually available on the market: A cardboard box with some blinking lights, a fan blowing some air, and a questionable looking guy outside shaking the box and making sounds like  "EERrwWwoooooSHHHh" "EERrwWwoooooSHHHhEERrw" as loud as he can.

Joking aside, we have seen glimpses of what AR and VR could be like in the future mostly thanks to Hollywood. This sort of stuff is coming, it may just take a while.

Having said that, the question of whether or not AR and/or VR are just technological fads still remains for many people. It is fun for a couple of minutes and then...meh? What's the point?

Virtual Reality still has a few things to work out with the head mounted displays, the big thing being the irritating sense of dizziness and motion sickness that a lot of people experience after only a few brief minutes wearing the gear. I expect that will be conquered very soon. The other issue is that of user interface...how do we interact with these amazing virtual worlds? Right now, we have to stumble blindly using a keyboard we can't see, or some other type of hardware like a joystick, motion controller, or maybe a glove studded with sensors. This too will probably be sorted out in the near future, but for all of the glorious experiences we see with VR on TV and in the movies, we will have to wait until we can jack into our brains directly, or something else that can record impulses from our spinal cord and interpret those to control a 3D avatar in the virtual world. This by the way, is quite different from the traditional approach of trying to "read" the mind by using generic EEG sensors to identify certain brainwave patterns. This is commonly referred to as reading the mind by a lot of companies and a few recent tech toys, but nothing could be further from the truth. 

Augmented Reality has its own issues. It is one thing using printed "markers" (basically black and white patterns) or simple image recognition to tell the software where to display the content associated with those markers, but it is an entirely different beast to do full-on AR like in the videos under Mixed Reality at the beginning of this post. To do that, you have to understand the environment around you (I call this spatial intelligence) and that is a bit tricky. Even trickier trying to do that for an experience that multiple people are engaging in at the same time. More on this later in a different blog post. I have a lot to say about this : )

Needs more cowbell

needs-more-cowbell.jpg

In any case, at the end of the day, both AR and VR need "more cowbell". Even if the technology is getting where it needs to go (albeit slowly, mostly due to lack of risk capital available for innovation and R&D, but also lack of enough visionaries really trying to push things forward with ambition), there still remains the quality of the experience itself, and what the application is. 

For now, VR, in all of its glory and recent industry excitement, still lacks killer experiences. Most of what I've seen out there, is essentially just replacing the monitor on your desktop with a wearable display, and then calling it a day. VR is unique in many respects, and I don't think it will take off as much as it should until there are experiences and content designed specifically for VR instead of just being tacked on.

AR is probably in a worse position right now...the industry has been flooded with a bunch of novelty gimmicks meant to attract attention, but without giving any real substance or added value to the experience. There is plenty of hype coming from people releasing awesome concept videos that look good, but have little to do with what their tech or app actually does, and in some cases, there is little thought given to the realities of what is necessary to pull off some of the capabilities shown in these videos. I'll save my comments for what AR needs and how the tech needs to evolve to get where it needs to be for the magic to happen for another post soon.

A little critical...

I might be a little bit critical with a dose of cynicism in my posts here and elsewhere, but this is more about the state of the industry and not so much about the technology itself. Most of my frustration comes from over-marketing, get-rich-quick people, and venture capital pouring into dumb ideas and companies instead of seeking out real innovators that have the potential to make a difference. So, whatever you think, don't let me discourage you about the tech or the industry. Some amazing things are going to happen, and relatively soon.

A few companies I like to keep an eye on:

Paracosm

Occipital

OculusVR

These guys are all doing it.  Really smart people, solid technology, and thoughtful roadmaps. One step at a time, but that is the way to go. 

You won't get a bunch of razzle dazzle from these three companies or concept videos that strain the sense of credulity. There are too many of those already in the industry (and they shall remain unnamed here). It is safe to say (in my opinion) that at least half the industry is either full of it, a waste of time, or just trying to ride the hype and repacking something else and making it seem great. You know who you are!

In the Movies

In case you still don't know what AR or VR is, here are a couple of movies you can watch easily enough to get a better idea. Have fun.

AR in Movies: Terminator, Minority Report, Iron Man, Avatar, Robocop

VR in Movies: The Matrix, Lawnmower Man, Johnny Mnemonic, Virtuosity, and many more.

You should also check out Dennou Coil (anime), and read stuff like Neuromancer (one of my all time favorites), Cryptonomicon , Diamond Age, Rainbow's End, and Dream Park. There is a ton of fantastic material out there that will blow your mind. Not hard to find. Speaking of which, feel free to list your favorite movies or books that have anything to do with AR or VR in the comments. Other readers will appreciate it.

See you in the future.

My first tech job was creating virtual reality

Oculus Rift Virtual Reality Wearable Display

Oculus Rift Virtual Reality Wearable Display

I am sure you have heard about Oculus VR by now. They have an awesome head mounted display for virtual reality. Great company, great team, and they just scored another $75M in funding. They even managed to get John Carmack to leave Id, which is something like getting Steve Jobs to leave Apple. While Carmack is known for his immense contributions to 3D graphics and the game industry, not many know that he had a connection to Virtual Reality a good twenty years ago. I'll explain in a minute, but I need to provide some background. 

AWT's  Reality Rocket

AWT's Reality Rocket

After working for a few years in various interesting jobs, I started my first business, a comic book store in the early 90's. This lasted for a little over a year (maybe a year and a half) when I closed the store down and took a job as the first employee for the first virtual reality game company based in the US, called Alternate Worlds Technology (or AWT). This was around 1993 or 1994 I think, I don't remember off hand. I was their first full time employee and had the lofty title of "Senior Multimedia Engineer" which meant that I knew how to use a computer, I could teach myself graphics, 3D modeling, animation, and sound effect design. Plus I could handle other design, user experience, and whatever else the CEO wanted me to do. 

Somehow or another, the CEO of AWT managed to convince Id and John Carmack to agree to some sort of license to Wolfenstein 3D (at the time, Doom was still in early development). AWT was going to do a virtual reality conversion of the game for arcades. We were developing stand-up and seated arcade game cabinets (the seated one had the unfortunate name "Reality Rocket").

Clip from the AWT Marketing Brochure

Clip from the AWT Marketing Brochure

Not only did we end up finishing and releasing Wolfenstein VR (which was a huge amount of fun), but built a whole new game on the engine called Cybertag VR...the first multi-player VR arcade game if I remember correctly. Cybetag is my first real game design credit...I did the overall design, gameplay, character design, graphics, animation, and sound effects. 

Those were fun times. I had to teach myself to do a lot of this from scratch and then figure out how to train other people. Deluxe Paint ,Animator Pro, Pre-release version of 3D Studio, Wave for Windows, the list goes on. I had to stumble through trial and experimentation with some of this software...didn't have manuals and certainly didn't have Google to go search Youtube for tutorials haha. 

Anyway, I remember John Carmack being very interested in virtual reality, even back then. I'm really excited to see what Oculus is going to do in the future. We live in interesting times!

 

Your Augmented Reality concept video is awesome, but...

 I love watching new augmented reality concept videos. It seems like all of the cool kids have one these days. While these are fantastic at communicating the potential of augmented reality (including perceptual computing, mixed reality, virtual reality, sensor fusion, and much more), these videos are a little problematic.

First, they are setting unrealistic expectations from the consumers that are watching them, and I'm willing to bet fooling potential investors and venture capitalists that are looking at funding these companies.   

lots of windows.jpg

These videos are sometimes used as marketing pieces, and for the common man who doesn't know better, it looks like this is what the company is developing, when the reality couldn't be farther.  

It seems to me, and I could be wrong, that very little thought is being given to the technical elements required for these experiences or any consideration at all for design challenges, particularly around areas like user interface, cognitive overload, information visualization, and much more. Have you noticed that there is a good amount of mind-reading in half of these videos, not to mention context driven experiences that would probably require a quantum computer or two to actually pull off?

At least make it clear that your concept video is a work of fiction and be sure to pair it with actual videos of what your product or tech actually does, or what you expect it to do within a reasonable amount of time, maybe not more than a year or so.  

Here are a few examples, good and bad. 

There is sooo much that is misleading about some of these, it is almost embarrassing.  

 

Augmented Vision and the Decade of Ubiquity

[Originally posted in March 2009]

There is one  thing stronger than all the armies in the world, and that is an idea whose  time has come. - Victor Hugo

“The best way  to predict the future is to invent it. Really smart people with  reasonable funding can do just about anything…” - Alan Kay


The Past

The concept of Augmented Reality has been around for a very  long time, and not just in fiction. I’m not going to spend much time talking about what augmented reality (“AR)” is or should be, you can do that on your own. There are plenty of resources like Ori Inbar’s Games Alfresco out there that will get you up to speed quickly. Start there if you want to know who is who, and who is doing what. I’m not aware of any other resource on the net that is as definitive as this site is.

The Present

Augmented Reality is quickly becoming one of the buzzwords of 2009 mostly due to social networking, blogs, twitter, and early exposure in mass media. Unless you have been living under a rock recently, you should have seen some marketing by GE, Toyota, Lego, and many others. While I think that it is too early for AR to have so much attention in the mass market, and it is already beginning to suffer overexposure in some circles, it is undeniably building momentum and the early experimenters/adopters are diving right in with accessible tools.

For now, AR is mostly about superimposing graphics on a video stream (from a webcam). This requires some type of marker such as a glyph or fidicial with a symbol, or some other type of image such as a picture (like the front of a baseball card). In either case, the software uses the marker for two things…first, to determine registration and tracking (where should the content and media be displayed) and second, what content to display. Some companies advertise the second method as markerless, but what they really mean is that they aren’t using the first method of a symbol or pattern. Let’s call all of this Level 1 AR.

In most of these cases, this type of AR is pretty novelty and fairly useless. Aside from some games like Sony’s Eye of Judgment, Int13’s Kweekies, and even Frank Lasorne’s AR Toys concept, which are all pretty damn cool, you won’t see very many applications worth more than a glance unless you break away from the desktop, take it mobile, and get rid of all types of printed markers. Now, we are talking about Level 2 AR.

Probably the most well known example of level 2 is Mobilizy’s Wikitude-AR for the Android platform. As we move away from the desktop AR toys and start paying attention to where you are and what is around you, things get much more interesting. The mobile device becomes a lens that gives us the sensation of looking through and seeing the world around us layered with information, data, and visualizations. As an industry, we are only beginning to explore the possibilities here. The transformation of mobile phones into mobile internet devices (MIDs) with powerful processors, 3D graphics, and GPS functionality has already changed the way we think, communicate, and interact with media. Some, like MIT’s improperly named “Sixth Sense” have this backwards by trying to project images on to objects instead of augmenting what you see. Others, like Tonchidot’s Sekai Camera has the right idea, but their approach feels incomplete. It is one thing to associate or link media to a general location, but it is much better to link to specific objects and things. SprxMobile’s ATM finder for ING is another example of how early location-based augmented reality can be very useful.

The Future

Level 3 becomes Augmented Vision. This is an important distinction. We must break away from the monitor and display to lightweight transparent wearable displays (in an eyeglasses formfactor). Once AR becomes AV, it is immersive. The whole experience immediately changes into something more relevant, contextual, and personal. This is radical and changes everything. As I have said before, this will be the next evolution in media. Print, Radio, Television, Internet, Augmented Reality (well, Vision).

L3 must also be mobile massively multi-user, persistent, shared, dynamic, and ubiquitous. This requires a full on convergence of a variety of technologies and disciplines, particularly powerful multi-core MIDs, pervasive wireless broadband, semantic search, intelligent pattern and image recognition, intelligent agents, hybrid service oriented and client-server architectures, gesture interfaces, standardized communications protocols and data formats, easy-to-use and intuitive tools for application development and content creation, and many others. Depending on a number of factors and variables, we are two to three years from this being realized commercially, and maybe five to seven from dominating the mass market. Maybe longer.

2010 to 2020 will become The Decade of Ubiquity. Not only will Level 3 become a reality, but the advent of this will spawn entirely new industries, professions, and hundreds of thousands of jobs. The impact of L3 will be equal to or greater than the effect of the Internet and the Web combined. Nearly every industry will change in some way, and L3 technologies will have a dramatic effect on our day to day lives, jobs, education, entertainment, culture, politics, society, and so on. Even newspapers will evolve and reinvent themselves. Today’s web designers and artists will become holoscape designers…developers will create intelligent agents and bots that are capable of seamlessly interacting with the real and digital worlds (think about Star Trek Voyager’s Holographic Doctor). Marketing and advertising will be completely reinvented and will be more interactive and dynamic than the targeted holographic advertising in The Minority Report. The world around you becomes your display and your interface. Any and everything will be tagged, labeled, interpreted, remembered, and filtered, in real-time. Cyberspace, combined with L3 devices, will become something like a hive-mind collective conscience and memory that we can all tap into at will. We don’t quite know how this is going to happen yet, but a lot of thought and effort is going on right now. Ideas are beginning to become reality.

Early on, entertainment, advertising, and social communication will feel the effects the strongest. Massive amounts of revenue will be generated and the technology will begin to explode, disrupting the way we do everything. Next, education will get a huge shock, as will training, medicine, and business. Industry domination will first be focused on the hardware and software that users need. Then it will be controlled by whoever masters what goes on behind the scenes in the cloud of cyberspace.

Imagine…

You only have to see the Yellowbook Ads, HP’s Roku’s Reward, Soryn’s The Future of Education, Bruce Branit’s World Builder, Nokia’s Morph concept phone, and Microsoft’s Future Vision Series to get a glimpse of what is COMING and in some cases is almost already here.

The best examples of L3 AR, at least where we are headed to and what everyone is talking about for the near-future, include Vernor Vinge’s Rainbows End and Mitsuo Iso’s Denno Coil. If you don’t bother with anything else, at least pay attention to those two.

The Decade of Ubiquity is defined as the next ten years where every aspect of our lives will be permeated by digital, mobile, media, data, information, augmented, virtual, and so forth. It will be everywhere and accessible almost instantly. Everything will be connected, labled, monitored, tracked, tagged, and interactive to some degree or another. We will break away from the desk, we will throw away our monitors, and our children will laugh at how large our IPhones are. They will struggle with how we ever managed to get work done with “windows” “webpages” and keyboards. They will be unable to fathom the concept of vinyl disks, typewriters, and landlines. But it all starts, and accelerates, during this next decade. Imagine everything that happened in the last decade, and multiply it. You haven’t seen anything yet. The next decade will make the last one pale in comparison.

The Distant Future

Level 4 is a long way off and is where we upgrade to contact lens displays and/or direct interfaces to the optic nerve and the brain. At this point, multiple realities collide, merge, and we end up with the Matrix. Without some amazing breakthroughs in a dozen fields, don’t expect this for another two or three decades. That is, assuming there is aggressive funding and R&D in the right areas. It won’t just happen on its own. There needs to be dedicated effort here. This is where Virtual Reality will finally come into its own and our dreams of pure and total immersion where we forget our bodies will finally be realized. Ok, maybe just Playstation 9.

Back to the Future

VAST Media is Virtual, Augmented and Simulations Technology Media. Virtual Worlds, Virtual Reality, Augmented Reality, MMORPGs, Simulations, and so on. In other words any media that is usually based on technology and is generally three dimensional. Print, Radio, and Television don’t count (this includes video). VAST Media today is still heavily segregated into individual industries with very little cross-pollination and sharing of theory, methodology, application, and leaders. This is slowly changing, but the fact remains that the technologies used for each are very similar. Until industry-wide convergence begins to occur, there will be little growth or advancement in any of the individual sectors. Virtual Reality went into a coma in the earl-mid 90s. Innovation in Virtual Worlds is barely measurable, with much of today’s state-of-the-art barely different from where it was a decade ago. MMORPGs have actually devolved in nearly every aspect. Some of the leading titles focus more on single-player gameplay, repetitive and static content, or aren’t even real 3D anymore. Augmented Reality, even as it is gaining momentum and excitement, is at risk of over-exposure and hype.

New leaders and thinkers are emerging and the hunger for creative innovation is beginning to gnaw at the bellies of Gen X’ers that miss the good old days of the Internet boom. Rapid advancements in mobile internet devices and tools for open development are fanning the fires. L2 will burst into the mainstream very soon, and the main thing holding L3 back are the wearable display companies that keep making promises but don’t seem to actively and aggressively be pushing the limits of technology. Too much emphasis is on miniature projectors or wearable displays so people can watch IPod/IPhone videos on the plane in privacy.

The world is nearing another dramatic paradigm shift and explosive growth in technology and economics, but we need to wake up. Demand more, better, stronger, faster, smaller. The future is ours to invent. Don’t be satisfied with mediocrity or lazy development.

We still have a long way to go, and there are plenty of obstacles and problems to be sorted out. Hardware has got to keep up this time (remember what happened to VR). This means that mobile devices have to crank it up real soon and compete with the desktop. Wearable display companies have got to quit screwing around, or they will single-handedly snuff out most efforts to push the envelop by years.

The architects of our augmented future need to think outside of the box as well. Forget everything you know about the internet, the web, web 2.0, virtual worlds, interface design, client/server, internet domains, etc. They MUST look at massively multiuser ubiquitious augmented reality with fresh eyes and vision. The paradigm is completly different. You can’t think about website design and development and ubiquitous AR at the same time. It isn’t about pages, servers, websites, or everything we have created over the last two decades. AR is about WHO you are, WHERE you are, WHAT is around you, WHAT you are doing, and WHO is nearby. Even things we take for granted like anonymity on the internet needs to be thrown out and rethought. The user’s identity is absolutely key to building the future. So are other things like privacy, interoperability, context, semantics, interface, and so on. We have to be thinking about these things NOW if we are going to build the future in the next decade.

Even the way we think about media and content is going to be important. Types of media can be categorized as Passive, Active, Interactive, Dynamic, and Meta. Passive media is text, an image, a 3D object, or something else that just is, and is static. Active media does something. It might be animated, it could turn on and off, and it can have multiple states. Interactive media requires input and interaction with a user. Games are a good example of interactive media. Dynamic media has the ability to change or evolve. It can be influenced. Meta media is beyond all other media types and is usually created and driven by other media or data sources. An example of this would be dynamic media, such as a constantly shifting and transforming 3D shape with attributes such as size, color, texture, volume, and morphability determined by live input from some other source such as the stock market or an orchestra.

Think about all of that, but with other attributes and influences that are based on the who, what, where, when, how, and why that become important with mobile multiuser ubiquitous augmented reality and vision. Now, make it intelligent. The rabbit hole is getting very deep, isn’t it? You absolutely cannot create, architect, and develop this stuff while in the mindset of 1.0 or 2.0. You have to think ahead to 9.0, or better yet, throw out the whole “point oh” system to start with. Never mind Shrödinger’s Cat, think about his Dog.

You must change your perspective, if you want to change how we see the world.

One good place to find out what some out of the box thinkers are thinking, is over at Tish Shute’s blog. Check it out, definitely worth your time. Her recent interviews with Mike Kuniavsky, Adam Greenfield, Usman Haque, and Andy Stanford-Clark are very interesting and in-depth.

What is your vision of the future?

#

Note to Venture Capitalists: Don’t even THINK about investing in anything remotely associated with Augmented Reality unless you are absolutely familiar (that means having seen or read) with these and others like Neal Stephenson’s Snow Crash and The Diamond Age, Roger Zelazny’s Donnerjack, Charles Stross’ Halting State, Larry Niven’s Dream Park, and just about anything by William Gibson and Bruce Sterling. My apologies to anyone I left out, this isn’t a definitive list by any means. Make sure you watch Masamune Shirow’s Ghost in the Shell as well. Beware of the slew of startups that will come out of nowhere in the next few years with no discernable business model or any real understanding of the tech. Everyone and their brother is going to try to jump on this bandwagon once the realization sets in that the next billion dollar world-changing corporations are going to have something to do with L3. Get rich quick is going to be redefined.