Chris Pearson, President, 5G Americas – November 2022
We are living in an era of connected intelligent things. Increasingly, we’re seeing mundane items and everyday objects becoming imbued with data that are generated by their sensors and connected back to the cloud. This digital data is a layer of information that sits on top of the physical object, perceivable only to people and smart objects that can view that data.
For humans, this ability to perceive this digital metadata is referred to as ‘Extended Reality’ (XR), which is often provided to us via our devices, such as a headset, smartphone, or some other electronic screen. Of course, XR itself is comprised of other types of “reality” such as Augmented Reality (AR) and Virtual Reality (VR). Imagine this ability to perceive digital metadata as being on a sliding scale. On the far left, you experience zero digital data (otherwise known as ‘reality’), while on the far right, you experience 100% digital data, which completely covers up the underlying reality underneath – this is referred to as Virtual Reality. Somewhere in the middle of that lies Augmented Reality, which blends the real with the digital.
But the worlds of the data center, where most of the world’s data resides, and the real world “out there” could not be more different. In getting this data out into the real world, you need to connect it – and most non-electronic things in real life aren’t connected by a physical wire. This, of course, is where wireless connectivity – and 5G wireless cellular plays a very important role. People and objects in the real world are often mobile, so being tethered to data cable just isn’t feasible all the time.
VR, but particularly AR, requires significant development in multiple areas, including but not limited to multi-media, artificial intelligence, computing, display systems, and communication to provide experiences that incorporate XR into our daily lives.
In our latest 5G Americas white paper, entitled “Extended Reality and 3GPP Evolution,” we explain how 5G offers the potential to transmit up to 10 gigabits per second, at less than 10 milliseconds of latency, and the ability to manage up to one million devices per square kilometer. As it turns out, all these capabilities can make a user’s Extended Reality experience significantly better, allowing higher resolution video with very little headache-inducing “lag” or jitter, and better control over energy usage and positioning, which can be critical for understanding where a mobile device like a headset or hand controllers reside in space.
Initially, many use cases for XR include consumer applications like gaming, but we are also beginning to see many applications in enterprises, institutions, service entities and manufacturing. VR use cases span the gamut of online gaming, virtual event participation, and educational experiences, while additional AR use cases may include other uses like mobile AR video gaming, mission critical services, online shopping, spatial-audio multiparty calls & conferences, and digital co-design.
Since the introduction and wide commercialization of mobile video and haptic services in 4G LTE, wireless cellular networks have been able to provide degrees of possibility both indoor and outdoor for early kinds of Extended Reality. We are at a stage now where it is possible to have entry-level XR services with full-view 8K 3D video, both indoors and outdoors along with simple haptic feedback in 5G New Radio systems. The future holds even more promise with higher video resolution, more multi-sensory options beyond haptic (touch) – and even better connectivity and coverage.
With 4G LTE serving as a guide, 5G was designed to address many Extended Reality uses beginning with Third Generation Partnership Project (3GPP) Release 15, allowing 5G networks to enable end-to-end XR systems by usingsplit computation architecture across various system components. This provides benefits for lower latency, higher reliability, higher rates, and less device computation. As you can see in the image below, an edge cloud server can address computationally intensive processing demands, communicating with a 5G radio that wirelessly communicates with an XR headset for low latency applications. The emergence of Mobile Edge Computation (MEC), alongside the development of cloud computing services, have opened the door for 5G wireless networks to be able to achieve much more for XR uses.
However, while evolution of 5G systems from Rel-15 and 16 introduced features for Ultra-Reliable Low Latency (URLLC) and power savings that provide higher reliability, lower latency, and larger power savings, these features were not specifically designed and optimized for XR. Extended Reality data traffic characteristics often include quasi-periodic traffic in large chunks, irregular intervals and variable size, high data rate including uplink (UL) for AR services, simultaneous transmission of 3D video stream, and control data over the same end-to-end connection.
VR wireless service requirements are dependent on the current VR generation. While early deployments of VR only consisted of 360 videos or 360 videos with simple haptic feedback, “Advanced VR” requires a video resolution of full-view 12K video, and transmission data in the range of 796 Mb/s-11.94 Gb/s, depending on compression and content. Of course, AR has even higher demands, and depends on the user’s input more than VR, necessitating a high-rate bidirectional (downlink and uplink) rate. The table below shows the requirements needed to deliver a quality experience at each stage of VR evolution – note the increasing demands in terms of data throughput and network latency.
In order to deliver against these stringent requirements, 3GPP Rel-17 through Rel-18 are expected to optimize XR support including XR awareness, power optimizations, and capacity enhancements. In particular, addressing mobile positioning and power efficiency will be critical, as XR applications become increasingly demanding, and more XR-capable devices share the same locality. With 5G-Advanced, Rel-19 may potentially include support for localized metaverse services, which could unlock the door for a vast array of new services.
So 5G networks will continue evolve right alongside XR. This is expected to be a long successful journey. Rome was not built in a single day – and neither will a truly immersive XR experience or metaverse that rivals human perception. Fortunately, concrete steps have already been laid out that show us the pathway to getting there. We have a good understanding of what 5G and 5G-Advanced networks will need to be able to accomplish in the intervening years. We know the kinds of challenges that the next few 3GPP Releases are intended to address.
Like many other co-evolving and technologies, wireless cellular and XR truly enable – and are enabled by – each other. The next few years will unveil some amazing new progress. I can’t wait to put on a headset or other device and see for myself all the great changes.