Extended reality (XR) is a technical term that refers to real, virtual, and combined environments where human-machine interactions are produced by computer technology and various mobile devices. XR technology encompasses Augmented Reality (AR), Virtual Reality (VR), and relative areas that intersect with each other. While all the above terms do not represent the exact same functionality, they are all based on common technologies such as 3D virtual imaging and artificial intelligence.
From VR to XR
Virtual Reality is a simulation created entirely by a computer system. To have a virtual reality experience, one must have a special headset to see specific images, hear sounds and even feel vibrations. On the other hand, AR is a real-time digital projection of a physical object in space. Modifications are made in real-time using a computer system. With AR technology, the digital version of an object is achieved by using a camera and special software. The most common devices currently available for the use of AR technology are smartphones, as they have high-quality cameras and relatively powerful processors.
Mixed Reality (MR) is another term that is used to combine the real and the digital world. MR creates an intermediate world, where objects can interact in real-time, just as they would in the real world.
Extended Reality (XR), however, is the newest term in this field of interacting in the physical and virtual world. XR refers to how the natural objects of our space interact with us and our movements. In XR, these movements are recognized and recorded using specific wearables or other portable devices. XR is, therefore, a superset that encompasses the entire spectrum from “completely real” to “complete virtual” and the concept of “virtuality continuum.” Furthermore, its importance lies in the extension of human experiences related to consciousness (represented by VR) and the acquisition of knowledge (represented by AR).
XR basics
Although the term “Spatial Computing” has been around for some time, it seems less well-known to the general public. Ridesharing applications, GPS, smart mobile devices, etc., are part of spatial computing technology. However, with the increasing adoption of 5G networks, IoT applications and devices, AR, VR, MR, and their combination known as XR, the field of spatial computing has expanded by offering the possibility of seamless interaction between machines in a 3D and multidimensional cyberspace.
This new technological field digitizes analog and physical processes, collects data from people and environments through sensors, and enables computers to guide users through experiences of a purely digital or hybrid world. XR is a continuous new medium that encompasses all VR, AR, and MR technologies in a single term and is currently attracting the interest of investors and entrepreneurs that seek to contribute to the next technological revolution.
XR and AI
Artificial Intelligence (AI) includes technologies such as Machine Learning, Deep Learning, and Reinforced Learning. With the enormous possibilities offered by these AI technologies, XR could help bring massive opportunities with a substantial positive impact in many areas such as human health, both mental and physical.
XR with AI should not be approached simply as a new technology. Society should be able to understand and accept them as new media that will affect every aspect of our lives, such as radio, television, and the internet have done in the past.
The user of XR technologies directly integrates his senses, mainly the visual and acoustic, with the digital system and indirectly with the designer behind it. The designer also has absolute control over the user, who delivers his senses in a completely controlled digital environment, where his experience is guided through direct intervention. The enormous power of this new technology can allow us to expand our physical reality and embrace our understanding of ourselves and each other in digital, synthetic environments, where we have a strong sense of existence.
Legal & Ethical challenges
The notion of our “existence” in synthetic, digital environments with our own bodies is crucial to understanding VR and XR. There is a large number of academic literature on the user’s sense of existence in a digital environment. In this context, existence is defined as a construction of illusions produced by the digital system translating or influencing our senses. The first component of this illusion of existence is the “sense of being there,” often called the “illusion of space,” and the second is the “illusion of logic,” which refers to the illusion that the scenario evolves through the experience has the user feel it is really happening.
VR may become a very controversial topic in the coming years. This is due not only to the revolutionary applications in the various professional and industrial fields of action that are to be reshaped but mainly because they can influence the way we perceive space-time and our existence within it. VR allows us to experience infinite dimensions outside of our physical reality. This notion may act as a catalyst on how we might shape our future societies, structures, laws, and economic and professional activity.
Using VR, XR and AI technologies to their full potential means that personal data, such as images, reactions, movements, or other information, will be used and produced through biosensors or brain signals that have to be recorded and analyzed. It is evident that to maximize these benefits, the legal and ethical challenges posed by using these technologies must also be addressed.
XR Evolution
We are currently at a point where start-ups developing XR technologies are beginning to successfully integrate their healthcare products and services. It is expected that the full development of these services will create terabytes of personal and other sensitive data. Various studies have examined how Deep Reinforcement Learning (AI) technology can be used with VR/XR technologies to develop interactive physical exercise games and visual aids to improve various skills.
Therefore, XR will meet with AI to expand the possibilities of utilizing personalized and generalized data. Furthermore, XR with the use of AI will help design and create information models, medical reports, evaluation, personalized treatment, and other automated services, covering various aspects of the future healthcare sector.
XR Implementation & Challenges
There are many criteria to consider to identify potential use cases for a back-end solution for a real-time immersive XR experience. As far as infrastructure is concerned, a considerable amount of sensor data is needed, whereas, the growing number of XR scenarios will require input data from additional external data sources. Furthermore, to guarantee a smooth user experience, the time between signal measurement and rendering must not exceed 30 milliseconds. This is challenging for existing back-end architectures.
On top of that, superfast interactions with the real and virtual world are required. Additionally, XR use cases should come with high safety and security requirements. Application areas that fulfill the above criteria are XR gaming, XR remote control/monitoring & maintenance, and XR social interaction. Furthermore, XR real-time education and entertaining could change the environment as we currently perceive it.
The future of XR
In the near future, customers will be able to experience XR services in the best possible way since low latencies can be guaranteed by the new 5G networks. However, there are also some factors that might delay XR adoption. The existing infrastructure is based mostly on proprietary protocols and APIs around XR, and EDAs. This could result in poor stability and deployment risks when it comes to existing distributed architectures. Therefore, a new architectural solution is needed that gathers and distributes signals and data fast and also predicts application-specific data or detects network anomalies on the fly.
Designing an optimized back-end architecture for XR technologies is crucial to ensure a real-time immersive experience of use cases in the XR ecosystem. This means, that the industry should further explore opportunities on how to invest wisely in XR-related use cases as well as the required infrastructure.
Currently, the market for back-end solutions to ensure a real-time XR experience is under development. Several big players such as Facebook, Microsoft, HTC, and Samsung focus on developing XR software, applications & hardware. XR applications are under extended research, with an estimated market size in 2024 reaching about $270 bn.