Intelligent Reality
*This article was published in contribution to the new frontier of Intelligent Reality (IR). Click here to learn more about the 2022 IEEE 2nd International Conference on Intelligent Reality (ICIR).
If you are unfamiliar with the term Intelligent Reality, your initial reaction may well be, “Do we really need yet another term?”. We already have virtual reality, augmented reality, mixed reality and extended reality. Surely these are enough? Despite the seeming similarity to existing terms, Intelligent Reality actually describes something rather different. VR, AR, MR and XR are all closely related. In many ways, you can think of them as existing in a continuum. In virtual reality, the real world is entirely replaced with a virtual environment. In augmented reality, a virtual environment is added to the real world. Mixed reality lies somewhere in between, and extended reality is an all-encompassing term that includes all of the others. For each of these terms, the distinction between them comes down to how much reality is present, and how the virtual environment is experienced.
In contrast, intelligent reality is not primarily concerned with how the virtual component is experienced. Instead, it is focused on making actual reality, the literal world around us, more intelligent and responsive to our needs. As such, an intelligent reality experience could be delivered as augmented reality, mixed reality or virtual reality. Intelligent reality is fundamentally about augmenting human intelligence through the seamless application of technology.
Artificial intelligence and machine learning, are commonly used components, especially natural language processing, computer vision and deep learning. The goal is to be additive to reality, but not just by overlaying graphics. Intelligent reality aims to use AI technology and other technologies to seamlessly overlay knowledge, understanding and intelligence on look the world around us.
Benefits of intelligent reality
Intelligent Reality is a relatively new term. As already discussed, the core idea is to augment the world around us with intelligence, not just graphics. What does that really mean? Imagine a virtual reality experience that combines a training simulation with artificial intelligence. The simulation models reality because it is based on a real world process. The AI system could enhance the training process by providing a natural language interface. It could also modify the training program dynamically in response to user behavior. The result is a more intelligent virtual reality application, and potentially a more effective one. If the experience was changed to use augmented reality, we could add computer vision so that the training program would recognize key real world objects used during training. Additional machine learning could be applied to anticipate user behavior and reinforce key learnings.
Intelligent automation systems, often in the form of chatbots, have become increasingly popular in the past few years. Building intelligent automation systems into virtual and augmented experiences is a logical and inevitable next step.
But all of these examples use narrow AI. Even with the latest developments in deep learning, today’s AI applications systems are quite limited in their scope. They require significant training and can easily be confused. Should artificial general intelligence, or super AI, become possible, it would radically redefine the abilities of such a system in ways that are hard to predict.
The primary goal of intelligent reality is to more seamlessly integrate technology, knowledge and understanding with the world around us. We all stand to benefit from it.
Crucial components of intelligent reality
Intelligent reality is created through the seamless integration of multiple components and systems that add a layer of digital reality to actual reality. The crucial components of intelligent reality are a mixture of hardware and software, and the underlying user experience. Intelligent reality might be delivered via virtual reality, augmented reality or mixed reality, requiring the technologies and systems usually associated with such applications. But the “intelligence” of intelligent reality requires some form of data processing and synthesis, most likely using artificial intelligence or machine learning. As already noted, augmented and mixed reality experiences may well use computer vision. Other artificial intelligence technology such as neural networks or deep learning may also be used for data integration and analysis, natural language processing, and to drive autonomous virtual avatars. Additional sensing capabilities can be provided by IoT devices, further enhancing a intelligent reality experience.
As intelligent reality gains traction, new uses for artificial intelligence will be found. For experiences that use head mounted displays, the power demands of artificial intelligence will be a critical factor. Fortunately, new low power processors optimized for machine learning algorithms are already available.
Intelligent reality relies on many different components and draws from many modern sciences. The size, weight and performance of immersive headsets will improve with new display technology, more advanced “pancake” optics and new battery technology. AI algorithms will run faster and graphics will be more immersive with greater processing power. The continued miniaturization of electronics will also play a role. Low latency high bandwidth wireless network connections will enable distributed and edge computing to share the load. As new technologies become available, many will find a use in the process of creating intelligent reality, or will become use cases for intelligent reality.
Implementing intelligent reality
As should be abundantly clear at this point, intelligent reality combines many different disciplines. As an emerging concept, many of the key components have yet to be standardized and are being built on a project by project basis.
However, very few organizations are in a position to build every element of an intelligent reality experience from scratch. Instead, implementers must make choices about which existing technologies they will use, and which areas they will choose to focus their resources on. For example, a group may decide to build on the Microsoft Hololens platform, and focus on the creation of autonomous avatars for a particular application.
Implementing intelligent reality starts like any other technology project. The core team must discuss and rank key goals, features and benefits. In particular, the team must decide which technologies they will create and which existing technologies they will use. For many intelligent reality projects, the bulk of the effort may lie in system integration, rather than the creation of new technology itself. Once goals have been agreed upon, the next step is to decide how the experience will be delivered. Which form of extended reality will be used? Virtual reality, augmented reality or mixed reality? From there, the implementer can choose a suitable platform, which in turn will heavily influence the software toolkit they choose to use.
Depending on the focus and needs of the project, a review of existing artificial intelligence and machine learning tools will either help identify an existing AI system that can be used, or reveal promising research in computer science that may lead the team to their desired goal. For example, the project may target business intelligence, or help in future pandemics. As the components become identified, the focus of the project will shift to systems integration and the user experience, both of which should not be underestimated.
The ultimate goal of intelligent reality is to augment our human capabilities. The world is an increasingly complicated place and technology is developing faster than ever before. Intelligent Reality is a potential solution to our natural limitations, and could prove critical to humanity’s future.
Best uses for intelligent reality
Just a few years ago, few would have believed that conversational AI would be a primary driver for consumer adoption and use of AI. But the birth of digital assistants like Alexa, Siri and others showed otherwise. Similarly, the mass consumer adoption of augmented reality experiences like Pokemon Go and SnapChat filters was both unexpected and massive. Consumers are increasingly aware of the AI system behind the proverbial curtain in social media and other recommendation systems. The more recent successes of consumer tools for AI-driven art creation, deepfakes and other entertainment applications has both popularized the use of the advanced AI application, while also hiding the complexities of generative adversarial networks, stable diffusion, deep learning and other AI and machine learning techniques.
It is hard to know where intelligent reality will see consumer adoption. Increasingly smart and autonomous avatars seem like a likely use case, from personal tutors to financial advisors. As our homes become increasingly automated, there may be a role for intelligent reality to play. And of course, entertainment will always be a significant driver in the consumer space. Last but not least, new ideas like biosyncing may well prove to be ideal use cases too.
In a commercial setting, intelligent reality has countless applications, from training to the monitoring and management of complex systems and processes. It is a simple fact that most organizations produce more data than any individual can reasonably process. Intelligent reality could be the tool that helps us regain control.
Applied to a global scale, intelligent reality could be game changing. From international supply chains to money markets, operating and maintaining smart cities to governing countries, Intelligent Reality could help us overcome our human limitations, and in doing so, change the world.
2022 IEEE 2nd International Conference on Intelligent Reality (ICIR)
Want to learn more about Intelligent Reality? Why not register to attend IEEE's International Conference on Intelligent Reality (ICIR). The IEEE International Conference on Intelligent Reality aims at identifying the challenges and opportunities inherent in deploying intelligent tools and interactive disruptive technologies into immersive environments. It provides a forum for leading researchers, industry professionals, and standards experts to share their research findings and ideas.