AR

AR Fiction: Forensics and Crime Scene Analysis

This was originally posted in my personal blog (Dec 17, 2009), and I thought it would be better placed here in this section. I couldn't just move it without losing the comments or the original URL, so I'm just going to copy it here. Enjoy.

- Robert

In a recent post, I mentioned that I was a contributing author to Working Through Synthetic Worlds

Anyway, the format of the book is unique. As the description says:

The editors use a distinctive format for the book, consisting of a set of chapters composed of three parts: a story or vignette that describes work conducted within a synthetic world based loosely on the question, ‘what will work be like in the year 2025?’, founded on the expert authors’ expectations of plausible future technologies; a scholarly review of the technologies described by the stories and the current theories related to those technologies; and, a prescription for future research required to bridge the current state-of-the-art with the notional worlds described in the stories.

What I wanted to do here was share part of my chapter with you, at least just the fiction part. The rest of the chapter gets into the technical analysis, scholarly review, etc. This is the unedited version (the final is much cleaner), so please ignore any glaring writing errors. I hope you enjoy the short story…I might continue it at a later date depending on reader response (I had a limited amount of space to work with in the chapter).

******

The call had arrived at a little past two-thirty in the morning and Kendra had drawn the short straw to venture out in the freezing rain that had been cascading from an inky black sky for the last several hours. She waited at the stoplight, listening to the soft hum of her electric car and the turn signal’s metronome clicking away little segments of time. She was tempted to let her mind wander while staring at the glistening raindrops snaking their way down her windows and brightly lit by the stoplight. Kendra had always felt something mysterious about inclement weather in the middle of the night, especially when there were only few people on the roads heading to some unknown destination.  The only thing that could make the atmosphere a little more surreal would be snow materializing out of eternity and blanketing the city in silence, with only the reflections of street lights free to echo through the night.

The light finally changed to green and Kendra turned her mind back to the task at hand and drove the rest of the way to the crime scene, preparing herself for her work.

Kendra arrived at the scene and shouldered her way through the overly eager press reporters that had already gathered like hungry vultures desperate for a meal. She ignored their barrage of questions and ducked under the police cordon, showing her badge to the officers attempting to control the growing crowd. This was the fourth murder in as many days and the public was beginning to demand answers. She made her way across the driveway and up the brick stairs leading to the front door of the house, dodging the growing pools of rainwater that would soak her feet if she wasn’t careful.

She entered the house and waited while most of the other officers cleared out to give her some room. The medical examiner had already been on the scene and the bodies were taken away. Kendra could sync up her databases with the examiner’s later, it was time for her to do her own work now. She set down her bag and reached up to her glasses, tapping a barely noticeable touch sensitive key at the corner of the frames. A softly glowing heads up display pulsed into existence in front of her, immediately sensing her location in the real world and on the meta-net.

“Initialize.” she commanded. “Voice authentication: Alpha-Zulu-Romeo-Bravo-Zero-Niner-Seven.”

A nondescript male voice spoke in her ear. “Authenticated, Lieutenant Kendra Jackson, Enhanced Sensory Perception, ESPer Team 5”

“Begin scanning, crime scene, domestic, murder.” She began to look at every area of the room, making a few mental notes of things she would pay special attention to when she returned for her detail and evidence collection sweep. The voice spoke in her ear again. “Location confirmed. Date and time confirmed. Scanning in process. Comparing visual point cloud to public architectural records and builder plans. Synchronizing. Process complete.”

“Display wireframe” she said.

“Wireframe active. No anomalies detected.”

Faint blue lines appeared, following the course of the walls, the floor, the ceiling, the doorframes, and every other part of the house that she could see. It was like seeing a 3D blueprint of the house overlaid on the real thing. This served multiple purposes, but for now she needed it to act as a reference for the scale, positioning, and orientation of everything in the house, particularly any evidence she would come across. She remembered hearing stories of a SWAT team that had used augmented glasses and the wireframe to navigate through a pitch black office building where terrorists had cut the power and had set off multiple smoke bombs. They never knew what hit them. After that particular incident, Fire departments all over the country began requesting the same gear. Not being able to see where you were going in a smoke filled building was no longer a problem. You simply needed to follow the blue lines or the floating translucent red arrows that pointed towards the nearest exit. Life was so much easier with augmented reality and the glasses.

“Activate WATSON” she requested, almost without thinking. She had grown accustomed to working with the artificial life intelligent agent. Originally he was not much more than a tool that helped gather information and run intelligent semantic database searches, all wrapped up in a life sized human representation. Kendra had spent a lot of time and a fair amount of her own money to customize him to her liking and upgrade his program with a more realistic personality and some law enforcement grade functionality that was not available to the public.

“WATSON is now online.” The voice stated. She wondered if she should change her settings and give it a personality or if she preferred it to be completely devoid of any emotion. Having WATSON around was more than enough she guessed.

Kendra turned her head slightly and regarded WATSON, who was now standing in the middle of the room wearing his usual Victorian garb and peering at his surroundings while muttering to himself. He was faintly translucent, which was required by law for any virtual person, human controlled avatar or AI-driven persona. There were some public safety reasons for this and people generally preferred to keep it that way, mostly for the same reasons that humanoid synthetic androids were restricted from being too lifelike.

“WATSON, access the domestic security system and copy the logs from the last 72 hours” she said to her virtual companion while she began to start walking around each of the rooms on the first floor, observing everything.

WATSON nodded and pulled a worn leather-bound journal from thin air and started to make notes in it. He stopped muttering to himself and looked quite intent on his recording task. When he was finished, he walked out of the room, waddling slightly (he was a tad overweight, and it affected his walk cycle) and caught up with her.

When Kendra completed her initial walkthrough, she returned to where she began and started collecting evidence. Every bullet and casing were recovered, lasers used to determine angle of the bullet holes in the wall, fingerprints lifted, bloodstains sampled, and so on. By the time she was done, a highly detailed 3D mirror of the entire house had been created on the servers back at the station. Every photograph she took and corresponding video clips of everything she saw were also stored, but each was tagged with appropriate time and location data. She had taken the time to attach some notes and observations to a few key items and places of interest in the house. Anyone looking at the 3D version of the crime scene would have ready access to all of the data and her notes.

As Kendra was packing up her gear and preparing to leave, she had a sudden hunch. “WATSON, display all augmented reality content and channels associated with this residence please.”

“Ah, yes yes, good idea Ma’am. Complying.” WATSON looked like he was staring off into the distance at something vaguely interesting, while a wide variety of virtual objects began popping into existence. Normally, this would only be visible to the residents and friends that have access to their channels. Some abstract art materialized on one wall, shifting patterns and undulating waves rippling across the largest piece, their smooth motion was mesmerizing. A few virtual pets popped into existence next and started scampering around playing at some learning game intended for young children. A wonderfully detailed plan for a new house slowly appeared on the dining room table complemented by a soft woman’s voice reciting a sales pitch and mentioning a number of optional additions that could be purchased.

WATSON frowned and coughed politely. “I’m sorry, but there seems to be some additional content here that is restricted. It is not listed on the normal public and private channels. It may be an illegal hack. Shall I attempt to access?”

Kendra considered for a moment. Normally, law enforcement personnel have access to all AR content, public and private, within the domain of a crime scene, but there were some levels of protection and privilege that required a court order or security clearance to access. In cases like these, discretion was usually prudent and it was safe to obtain a warrant first. Then again, there was a strong possibility it was an illegal hack and this was a murder scene, not a regular crime scene. Kendra’s curiosity and intuition got the better of her.

“Force access WATSON, on my authorization. All means necessary.”

“Yes Ma’am. I shall do my best”. WATSON’s eyebrows furrowed and little beads of sweat began to form on his virtual brow. A few long minutes passed and Kendra started wondering if WATSON had locked up…he hadn’t moved in a bit and even flickered a few times, which was extremely unusual. She was about to give up and restart his program when he lurched with a gasp and a pained look on his face.

“Ahem, that was extraordinarily difficult, but I seem to have succeeded. A spot of tea would be really nice methinks.”

“Thank you WATSON, what is the location of the hidden content?”

“In the living room I believe. Shall we investigate?” queried WATSON.

“Of course; this should be interesting.” Kendra grinned and headed towards the living room. She stopped short at the doorway and stared at the center of the room in disbelief. A massive floating black skull hovered there, blood-red fire silently blazing in its hollow eye sockets, and a nasty black ichor dripping out of its mouth on to the carpet. The pools of liquid that formed stretched out and oozed across the rug to form intricate letters spelling something out in a bizarre language she did not recognize.

“Oh, my, this cannot be good.” WATSON said, stating the obvious.

“No kidding. Begin scanning the usual databases for gang signs, military unit insignia, secret societies, tattoos, and ancient symbology. I think we may have stumbled onto something that we weren’t meant to discover” said Kendra. “Let’s head back to the office and start putting the pieces together.”

The next several days were a blur for Kendra. Most of her time was spent processing evidence, populating the database with all of the information and media, and focusing every remaining waking moment trying to research the origin and meaning of the skull. It didn’t take too long to establish a timeline of the crime and how the murders occurred, but her research efforts resulted in a lot of dead-ends and few leads.

It was Thursday afternoon and Kendra headed over to the simulations room to tweaking her presentation before the staff briefing the next day. The higher-ups were demanding some progress and wanted to look at her work so far. She went to Room Three, which she had reserved earlier, and looked around. Room Three was entirely empty and slightly larger than a racquetball court. The walls and floor were painted a light grey color with thin black registration marks at every corner. She thought it was pretty depressing and wanted to get started.

She touched the corner of her glasses and went through the usual initialization and authentication process. “Load SIM four-zero-three, set location to living room.” Kendra paused and waited while her command was processed. After a few second a photorealistic 3D mirror of the crime scene faded into existence. Every detail was reproduced exactly and to scale, all based on the visual data she had recorded during her initial walkthrough and subsequent evidence collection at the scene.

Kendra continued giving the main computer additional commands, and after half an hour or so, the victim’s bodies were also visible. Faint red beams crossed the room, originating at each of the areas where the bullets were recovered, giving her a good idea where the killer or killers had stood and fired from. All of the fingerprints that were taken were also referenced, faintly glowing various shades of green. She touched a pair that was on the coffee table and a small window materialized above the prints with a zoomed in version and information about who the prints belonged to. In this case, the prints were marked as unknown and not in any of the usual fingerprint databases.

She moved on, making notations and observations to each area of evidence. Everything was checked and rechecked. Kendra reviewed all of the collected data and media, making sure everything was properly linked to the relevant objects. If there were any questions at the meeting, she literally wanted everything at her fingertips. All it would take was a brief touch on anything (or anyone) and all of the reports, photographs, video clips, and lab data would be instantly viewable to anyone in the room.

Once she was satisfied with everything, Kendra started tweaking the reconstruction of the crime based on the evidence they had so far. She started by tracking backwards from what the Police discovered when they responded to the crime, continually adjusting the position of each of the victims as she worked through the timeline. When she finished, she directed the system to fill in the blanks based on the forensic data already in the database.

After the processing was complete, she would be able to view a realistic recreation of the crime from any angle and perspective. The combination of the 3D objects, advanced physics modeling, and actual forensics data made these simulations incredibly lifelike. She had heard stories of the occasional jury member being shocked after witnessing these simulations in court, particularly if the crime was unusually brutal and violent. It felt like being an unseen ghost right in the middle of the crime as it occurred, and was much more compelling than the old way of relying on photographs, charts, and awkward animations on a projector screen. This put you THERE and left little doubt as to what happened. Kendra wondered how long it would be before the engineers would figure out how to implement smell and tactile feedback.

She ran through the simulation a few times and checked all of the datapoints once again to make sure she hadn’t missed anything. It was good enough she supposed. Maybe she should take another hour or two and check in with some of her contacts that were helping her with the skull research and then call it a night. Her boss didn’t take kindly to oversleeping or flubbing a presentation with the brass due to lack of sleep. Tomorrow should be interesting. She hoped the press didn’t show up, that always made her nervous.

“Save all files, backup, and copy to private storage. End simulation.” Kendra looked around at the empty room once more and left for her office.

(To be continued…?)

******

-Excerpt from Working Through Synthetic Worlds, Chapter 11, Augmented Reality Tools for Enhanced Forensics Simulations and Crime Scene Analysis by Robert Rice

Facebook to Acquire Oculus Rift for $2B...

oculus-rift-slider.jpg

So, yeah, Facebook just announced it is acquiring VR headset company Oculus Rift for $2B. Wow and holy cow, what does this mean for the industry?

Take a look at Mark Zuckerberg's Status Update announcement. Mark discusses the amazing potential of virtual reality as a new platform and some pretty amazing experiences, but doesn't really mention anything about how this relates to Facebook, other than their mission to make the world more open and connected. 

This acquisition is certainly going to make the founders and investors at Oculus very happy, and if they are allowed to continue operating independently, this huge cash influx should hopefully accelerate everything that Oculus is working on, but there are still a lot of remaining questions about age-old problems with VR headsets, the main one being the eye-strain and motion-sickness that tends to occur after using the headsets for several minutes. We will probably see a bunch of VR-enabled games hitting Facebook, but honestly, creating good experiences for VR is COMPLETELY different from just making a 3D game or mobile app. We will probably see some conversions or whatever just to build up a portfolio, which will either be great and really amazing, or it will fizzle out very quickly as gimmicky and novelty as everyone tries to get on the bandwagon. 

Having said that, I am generally optimistic. There are some great people at Oculus and they are making tremendous strides in an industry that went *poof* in the mid-90s. The hardware isn't quite there yet, but it is getting pretty damn close. Once the hardware is right, all of the amazing experiences we have been promised for many years about virtual reality (VR) and its potential will start coming true. However! Who are the VR developers? Where are they? Where are the investment dollars going into VR startups? Who is pushing the edge of innovation for VR in terms of user interface, user experiences, immersion, and so forth? What happened to all of the VR pioneers and innovators from the 80s and 90s? I'd say that they are out there, but they are disillusioned or disinterested... try talking to a venture capitalist with an idea for changing the world or technology now if you are over 25. Good luck with that. I'd argue there is a lost generation of expertise and vision for virtual reality out there that needs to be recovered. 

Virtual Reality isn't going anywhere any time soon until sources of capital (angel investors, seed funds, venture capitalists, etc.) decide to embrace the vision and potential of the technology (and its inherent risks as "new" technology) and start pumping capital into new startups. These startups must be driven by the pioneers and visionaries that already understand this tech and how to apply it...young guys with an idea for a facebook VR game just aren't going to cut it. Much more is at stake, and the potential for really changing the world is huge. 

Now, I'm not knocking the young guys, I've been there myself. But with VR in particular, there is already a wealth of knowledge, expertise, and vision (pardon the pun) that can be tapped into, and I'm just not seeing it happen. I really don't want to see Farmville VR, Flappy Bird VR, or Snapchat VR. 

I would guess though that Facebook's move here might just open the floodgates for VR companies as everyone starts thinking that hell, if Facebook thinks this is the future, we better get on the bandwagon or we are going to miss out. This might be a good thing, as capital will flow, but on the other hand, in the past when this sort of thing has happened, it has created a bubble and tons of really stupid ideas and teams get crazy funding, which just goes up in smoke as they are busy burning the cash doing "cool" things instead of building something really awesome

Getting back to Facebook though, I'm still not clear on why Facebook decided to acquire Oculus...I sincerely hope we don't see a Facebook VR application any time soon. VR is about immersion and experience, not 2D windows full of status updates, "likes", or selfies. There is nothing about VR (now at least) that will contribute to Facebook's mission of making the world more open or connected. Dean Putney has an interesting post on boingboing that is worth reading. I think that it would have been better for Oculus (and the world) if Google or Apple had acquired them instead. It would have made much more sense, and would align with their efforts in other areas, like augmented reality, wearables, and so forth. 

All in all, I think that Facebook's acquisition will ultimately prove out to be a good thing, but in the short term I think there is a big risk of novelty gimmicks running on the buzz of VR for a money grab, and very little actual innovation. Time will tell I guess.

I do believe that there is insane potential for virtual reality, augmented reality, virtual worlds, wearable displays, wearable devices, and the internet of things, particularly in areas where they intersect or are complimentary. Anyone interested in changing the world and building the future is welcome to contact me. 

I have to hand it to Zuckerberg though, this is a really interesting development in the industry and I'm looking forward to seeing how it plays out and what Facebook is ultimately going to do with Oculus. 

Just remember, there is no spoon. 

Really, there is no spoon.

Update: And now, there is no Minecraft VR for Oculus Rift.

My first tech job was creating virtual reality

Oculus Rift Virtual Reality Wearable Display

Oculus Rift Virtual Reality Wearable Display

I am sure you have heard about Oculus VR by now. They have an awesome head mounted display for virtual reality. Great company, great team, and they just scored another $75M in funding. They even managed to get John Carmack to leave Id, which is something like getting Steve Jobs to leave Apple. While Carmack is known for his immense contributions to 3D graphics and the game industry, not many know that he had a connection to Virtual Reality a good twenty years ago. I'll explain in a minute, but I need to provide some background. 

AWT's  Reality Rocket

AWT's Reality Rocket

After working for a few years in various interesting jobs, I started my first business, a comic book store in the early 90's. This lasted for a little over a year (maybe a year and a half) when I closed the store down and took a job as the first employee for the first virtual reality game company based in the US, called Alternate Worlds Technology (or AWT). This was around 1993 or 1994 I think, I don't remember off hand. I was their first full time employee and had the lofty title of "Senior Multimedia Engineer" which meant that I knew how to use a computer, I could teach myself graphics, 3D modeling, animation, and sound effect design. Plus I could handle other design, user experience, and whatever else the CEO wanted me to do. 

Somehow or another, the CEO of AWT managed to convince Id and John Carmack to agree to some sort of license to Wolfenstein 3D (at the time, Doom was still in early development). AWT was going to do a virtual reality conversion of the game for arcades. We were developing stand-up and seated arcade game cabinets (the seated one had the unfortunate name "Reality Rocket").

Clip from the AWT Marketing Brochure

Clip from the AWT Marketing Brochure

Not only did we end up finishing and releasing Wolfenstein VR (which was a huge amount of fun), but built a whole new game on the engine called Cybertag VR...the first multi-player VR arcade game if I remember correctly. Cybetag is my first real game design credit...I did the overall design, gameplay, character design, graphics, animation, and sound effects. 

Those were fun times. I had to teach myself to do a lot of this from scratch and then figure out how to train other people. Deluxe Paint ,Animator Pro, Pre-release version of 3D Studio, Wave for Windows, the list goes on. I had to stumble through trial and experimentation with some of this software...didn't have manuals and certainly didn't have Google to go search Youtube for tutorials haha. 

Anyway, I remember John Carmack being very interested in virtual reality, even back then. I'm really excited to see what Oculus is going to do in the future. We live in interesting times!

 

Your Augmented Reality concept video is awesome, but...

 I love watching new augmented reality concept videos. It seems like all of the cool kids have one these days. While these are fantastic at communicating the potential of augmented reality (including perceptual computing, mixed reality, virtual reality, sensor fusion, and much more), these videos are a little problematic.

First, they are setting unrealistic expectations from the consumers that are watching them, and I'm willing to bet fooling potential investors and venture capitalists that are looking at funding these companies.   

lots of windows.jpg

These videos are sometimes used as marketing pieces, and for the common man who doesn't know better, it looks like this is what the company is developing, when the reality couldn't be farther.  

It seems to me, and I could be wrong, that very little thought is being given to the technical elements required for these experiences or any consideration at all for design challenges, particularly around areas like user interface, cognitive overload, information visualization, and much more. Have you noticed that there is a good amount of mind-reading in half of these videos, not to mention context driven experiences that would probably require a quantum computer or two to actually pull off?

At least make it clear that your concept video is a work of fiction and be sure to pair it with actual videos of what your product or tech actually does, or what you expect it to do within a reasonable amount of time, maybe not more than a year or so.  

Here are a few examples, good and bad. 

There is sooo much that is misleading about some of these, it is almost embarrassing.  

 

Augmented Vision and the Decade of Ubiquity

[Originally posted in March 2009]

There is one  thing stronger than all the armies in the world, and that is an idea whose  time has come. - Victor Hugo

“The best way  to predict the future is to invent it. Really smart people with  reasonable funding can do just about anything…” - Alan Kay


The Past

The concept of Augmented Reality has been around for a very  long time, and not just in fiction. I’m not going to spend much time talking about what augmented reality (“AR)” is or should be, you can do that on your own. There are plenty of resources like Ori Inbar’s Games Alfresco out there that will get you up to speed quickly. Start there if you want to know who is who, and who is doing what. I’m not aware of any other resource on the net that is as definitive as this site is.

The Present

Augmented Reality is quickly becoming one of the buzzwords of 2009 mostly due to social networking, blogs, twitter, and early exposure in mass media. Unless you have been living under a rock recently, you should have seen some marketing by GE, Toyota, Lego, and many others. While I think that it is too early for AR to have so much attention in the mass market, and it is already beginning to suffer overexposure in some circles, it is undeniably building momentum and the early experimenters/adopters are diving right in with accessible tools.

For now, AR is mostly about superimposing graphics on a video stream (from a webcam). This requires some type of marker such as a glyph or fidicial with a symbol, or some other type of image such as a picture (like the front of a baseball card). In either case, the software uses the marker for two things…first, to determine registration and tracking (where should the content and media be displayed) and second, what content to display. Some companies advertise the second method as markerless, but what they really mean is that they aren’t using the first method of a symbol or pattern. Let’s call all of this Level 1 AR.

In most of these cases, this type of AR is pretty novelty and fairly useless. Aside from some games like Sony’s Eye of Judgment, Int13’s Kweekies, and even Frank Lasorne’s AR Toys concept, which are all pretty damn cool, you won’t see very many applications worth more than a glance unless you break away from the desktop, take it mobile, and get rid of all types of printed markers. Now, we are talking about Level 2 AR.

Probably the most well known example of level 2 is Mobilizy’s Wikitude-AR for the Android platform. As we move away from the desktop AR toys and start paying attention to where you are and what is around you, things get much more interesting. The mobile device becomes a lens that gives us the sensation of looking through and seeing the world around us layered with information, data, and visualizations. As an industry, we are only beginning to explore the possibilities here. The transformation of mobile phones into mobile internet devices (MIDs) with powerful processors, 3D graphics, and GPS functionality has already changed the way we think, communicate, and interact with media. Some, like MIT’s improperly named “Sixth Sense” have this backwards by trying to project images on to objects instead of augmenting what you see. Others, like Tonchidot’s Sekai Camera has the right idea, but their approach feels incomplete. It is one thing to associate or link media to a general location, but it is much better to link to specific objects and things. SprxMobile’s ATM finder for ING is another example of how early location-based augmented reality can be very useful.

The Future

Level 3 becomes Augmented Vision. This is an important distinction. We must break away from the monitor and display to lightweight transparent wearable displays (in an eyeglasses formfactor). Once AR becomes AV, it is immersive. The whole experience immediately changes into something more relevant, contextual, and personal. This is radical and changes everything. As I have said before, this will be the next evolution in media. Print, Radio, Television, Internet, Augmented Reality (well, Vision).

L3 must also be mobile massively multi-user, persistent, shared, dynamic, and ubiquitous. This requires a full on convergence of a variety of technologies and disciplines, particularly powerful multi-core MIDs, pervasive wireless broadband, semantic search, intelligent pattern and image recognition, intelligent agents, hybrid service oriented and client-server architectures, gesture interfaces, standardized communications protocols and data formats, easy-to-use and intuitive tools for application development and content creation, and many others. Depending on a number of factors and variables, we are two to three years from this being realized commercially, and maybe five to seven from dominating the mass market. Maybe longer.

2010 to 2020 will become The Decade of Ubiquity. Not only will Level 3 become a reality, but the advent of this will spawn entirely new industries, professions, and hundreds of thousands of jobs. The impact of L3 will be equal to or greater than the effect of the Internet and the Web combined. Nearly every industry will change in some way, and L3 technologies will have a dramatic effect on our day to day lives, jobs, education, entertainment, culture, politics, society, and so on. Even newspapers will evolve and reinvent themselves. Today’s web designers and artists will become holoscape designers…developers will create intelligent agents and bots that are capable of seamlessly interacting with the real and digital worlds (think about Star Trek Voyager’s Holographic Doctor). Marketing and advertising will be completely reinvented and will be more interactive and dynamic than the targeted holographic advertising in The Minority Report. The world around you becomes your display and your interface. Any and everything will be tagged, labeled, interpreted, remembered, and filtered, in real-time. Cyberspace, combined with L3 devices, will become something like a hive-mind collective conscience and memory that we can all tap into at will. We don’t quite know how this is going to happen yet, but a lot of thought and effort is going on right now. Ideas are beginning to become reality.

Early on, entertainment, advertising, and social communication will feel the effects the strongest. Massive amounts of revenue will be generated and the technology will begin to explode, disrupting the way we do everything. Next, education will get a huge shock, as will training, medicine, and business. Industry domination will first be focused on the hardware and software that users need. Then it will be controlled by whoever masters what goes on behind the scenes in the cloud of cyberspace.

Imagine…

You only have to see the Yellowbook Ads, HP’s Roku’s Reward, Soryn’s The Future of Education, Bruce Branit’s World Builder, Nokia’s Morph concept phone, and Microsoft’s Future Vision Series to get a glimpse of what is COMING and in some cases is almost already here.

The best examples of L3 AR, at least where we are headed to and what everyone is talking about for the near-future, include Vernor Vinge’s Rainbows End and Mitsuo Iso’s Denno Coil. If you don’t bother with anything else, at least pay attention to those two.

The Decade of Ubiquity is defined as the next ten years where every aspect of our lives will be permeated by digital, mobile, media, data, information, augmented, virtual, and so forth. It will be everywhere and accessible almost instantly. Everything will be connected, labled, monitored, tracked, tagged, and interactive to some degree or another. We will break away from the desk, we will throw away our monitors, and our children will laugh at how large our IPhones are. They will struggle with how we ever managed to get work done with “windows” “webpages” and keyboards. They will be unable to fathom the concept of vinyl disks, typewriters, and landlines. But it all starts, and accelerates, during this next decade. Imagine everything that happened in the last decade, and multiply it. You haven’t seen anything yet. The next decade will make the last one pale in comparison.

The Distant Future

Level 4 is a long way off and is where we upgrade to contact lens displays and/or direct interfaces to the optic nerve and the brain. At this point, multiple realities collide, merge, and we end up with the Matrix. Without some amazing breakthroughs in a dozen fields, don’t expect this for another two or three decades. That is, assuming there is aggressive funding and R&D in the right areas. It won’t just happen on its own. There needs to be dedicated effort here. This is where Virtual Reality will finally come into its own and our dreams of pure and total immersion where we forget our bodies will finally be realized. Ok, maybe just Playstation 9.

Back to the Future

VAST Media is Virtual, Augmented and Simulations Technology Media. Virtual Worlds, Virtual Reality, Augmented Reality, MMORPGs, Simulations, and so on. In other words any media that is usually based on technology and is generally three dimensional. Print, Radio, and Television don’t count (this includes video). VAST Media today is still heavily segregated into individual industries with very little cross-pollination and sharing of theory, methodology, application, and leaders. This is slowly changing, but the fact remains that the technologies used for each are very similar. Until industry-wide convergence begins to occur, there will be little growth or advancement in any of the individual sectors. Virtual Reality went into a coma in the earl-mid 90s. Innovation in Virtual Worlds is barely measurable, with much of today’s state-of-the-art barely different from where it was a decade ago. MMORPGs have actually devolved in nearly every aspect. Some of the leading titles focus more on single-player gameplay, repetitive and static content, or aren’t even real 3D anymore. Augmented Reality, even as it is gaining momentum and excitement, is at risk of over-exposure and hype.

New leaders and thinkers are emerging and the hunger for creative innovation is beginning to gnaw at the bellies of Gen X’ers that miss the good old days of the Internet boom. Rapid advancements in mobile internet devices and tools for open development are fanning the fires. L2 will burst into the mainstream very soon, and the main thing holding L3 back are the wearable display companies that keep making promises but don’t seem to actively and aggressively be pushing the limits of technology. Too much emphasis is on miniature projectors or wearable displays so people can watch IPod/IPhone videos on the plane in privacy.

The world is nearing another dramatic paradigm shift and explosive growth in technology and economics, but we need to wake up. Demand more, better, stronger, faster, smaller. The future is ours to invent. Don’t be satisfied with mediocrity or lazy development.

We still have a long way to go, and there are plenty of obstacles and problems to be sorted out. Hardware has got to keep up this time (remember what happened to VR). This means that mobile devices have to crank it up real soon and compete with the desktop. Wearable display companies have got to quit screwing around, or they will single-handedly snuff out most efforts to push the envelop by years.

The architects of our augmented future need to think outside of the box as well. Forget everything you know about the internet, the web, web 2.0, virtual worlds, interface design, client/server, internet domains, etc. They MUST look at massively multiuser ubiquitious augmented reality with fresh eyes and vision. The paradigm is completly different. You can’t think about website design and development and ubiquitous AR at the same time. It isn’t about pages, servers, websites, or everything we have created over the last two decades. AR is about WHO you are, WHERE you are, WHAT is around you, WHAT you are doing, and WHO is nearby. Even things we take for granted like anonymity on the internet needs to be thrown out and rethought. The user’s identity is absolutely key to building the future. So are other things like privacy, interoperability, context, semantics, interface, and so on. We have to be thinking about these things NOW if we are going to build the future in the next decade.

Even the way we think about media and content is going to be important. Types of media can be categorized as Passive, Active, Interactive, Dynamic, and Meta. Passive media is text, an image, a 3D object, or something else that just is, and is static. Active media does something. It might be animated, it could turn on and off, and it can have multiple states. Interactive media requires input and interaction with a user. Games are a good example of interactive media. Dynamic media has the ability to change or evolve. It can be influenced. Meta media is beyond all other media types and is usually created and driven by other media or data sources. An example of this would be dynamic media, such as a constantly shifting and transforming 3D shape with attributes such as size, color, texture, volume, and morphability determined by live input from some other source such as the stock market or an orchestra.

Think about all of that, but with other attributes and influences that are based on the who, what, where, when, how, and why that become important with mobile multiuser ubiquitous augmented reality and vision. Now, make it intelligent. The rabbit hole is getting very deep, isn’t it? You absolutely cannot create, architect, and develop this stuff while in the mindset of 1.0 or 2.0. You have to think ahead to 9.0, or better yet, throw out the whole “point oh” system to start with. Never mind Shrödinger’s Cat, think about his Dog.

You must change your perspective, if you want to change how we see the world.

One good place to find out what some out of the box thinkers are thinking, is over at Tish Shute’s blog. Check it out, definitely worth your time. Her recent interviews with Mike Kuniavsky, Adam Greenfield, Usman Haque, and Andy Stanford-Clark are very interesting and in-depth.

What is your vision of the future?

#

Note to Venture Capitalists: Don’t even THINK about investing in anything remotely associated with Augmented Reality unless you are absolutely familiar (that means having seen or read) with these and others like Neal Stephenson’s Snow Crash and The Diamond Age, Roger Zelazny’s Donnerjack, Charles Stross’ Halting State, Larry Niven’s Dream Park, and just about anything by William Gibson and Bruce Sterling. My apologies to anyone I left out, this isn’t a definitive list by any means. Make sure you watch Masamune Shirow’s Ghost in the Shell as well. Beware of the slew of startups that will come out of nowhere in the next few years with no discernable business model or any real understanding of the tech. Everyone and their brother is going to try to jump on this bandwagon once the realization sets in that the next billion dollar world-changing corporations are going to have something to do with L3. Get rich quick is going to be redefined.