Mobile Metaverse services
Standardization landscapeNew applications are being supported by current and future networks. In particular, it is expected that metaverse applications will be deployed in the near future, as 5G and 6G networks provide suficient bandwidth and suficiently low latency to provide a satisfying end‑user experience. However, networks still need to evolve to better support this type of application. We present here a basic taxonomy of the metaverse, which allows us to identify some of the networking requirements for such an application; and we also provide an overview of the current state of the standardization efforts in different standardization organizations, including ITU‑T, 3GPP, IETF and MPAI.
ITU standardizationMetaverse standardization discussions had initiated in several ITU‑T Study Groups (SGs), including SG16, SG17, and SG20, and other SGs had also shown an interest in the metaverse. In particular, ITU‑T SG16 (Multimedia and related digital technologies) established a correspondence group on metaverse (CG-Metaverse) in January 2022 to discuss metaverse standardization issues. The CG‑Metaverse also discussed the need to establish an ITU‑T Focus Group on Metaverse (FG‑MV). Finally, the ITU‑T FG‑MV was established under the ITU‑T Telecommunication Standardization Advisory Group (TSAG) in December 2022, as the scope of metaverse standardization is not limited to a specifc working group.ITU‑T Focus Group is an instrument to providing an additional working environment for the quick development of standards in specific areas and it is acting as an incubator for a future emerging issue. The focus group is open to all experts with an interest and even to non‑ITU members. The documents developed in the focus groups are called deliverables and are categorized as Technical Specifications (TSs) or Technical Reports (TRs) depending on their nature. Deliverables are later submitted to the ITU‑T study groups for further review and adoption as Recommendations, Supplements or Technical Reports.FG‑MV was originally scheduled to operate for one year, but a proposal to extend it for another year will be discussed at the last FG‑MV meeting in December 2023. This will be decided at the next TSAG meeting in early 2024. Oversight and reporting of the FG‑MV’s activities will be directed to the ITU Telecommunication Standardization Advisory Group. The FG‑MV is organized into Working Groups (WGs) and Task Groups (TGs). WGs are responsible for standardization work in a major area and TGs conduct discussions and develop one or more documents focusing on more specific topics that fall within the scope of the WG’s work. There are currently 9 WGs and 19 TGs approved, but the structure may change in the future as standardization work is progressed. Since the FG‑MV has established, a total of 59 standardization work items have been adopted to develop deliverables, and the first ofiicial deliverable, ‘Exploring the metaverse: opportunities and challenges’, was approved at the second meeting of FGMV in July 2023. The rapid publication of an ofiicial report is highly unusual and demonstrates the rapid pace of FG‑MV standardization work. The deliverable explores the background of the metaverse, including its history, ecosystem and development stages, challenges and opportunities. Discussions on metaverse terminology and concept, CitiVerse, industry metaverse, domain‑specific pplications and services, infrastructure, platform interoperability, integration of real and virtual worlds, and security and privacy, etc., will be actively progressed at the ITU FG‑MV, resulting in expecting the first major deliverables by the end of 2023.
3GPP standardizationThe metaverse has been used in various ways to refer to the broader implications of AR, VR or XR and multi‑modality services. Currently in 3GPP, the metaverse‑related SIDs and WIDs have spread to different Working Groups (WGs) belonging to different stages. In stage‑1 (SA1), there are three metaverse‑related projects, i.e., the TACMM or TACtile and multimodal communication service; the high mobility for XR services; and the FS_Metaverse or the localized mobile metaverse. Based on the stage‑1 study conclusions and consolidated requirements, the stage‑2 WGs (e.g., SA2 ‑ SA6) have developed their own corresponding projects. The SA2 WG has two projects, namely the XRM or extended reality and media services, and the XR & Metaverse. The SA4 WG has multiple related projects, studying and exploring XR and AR use cases, Key Performance Indicators (KPIs), media formats, and more. While both the SA2 and SA4 have 3GPP approved projects on XR & Metaverse, the SA6 WG is still in the proposal phase for two potential projects, namely the network enabler for XR, and the FS_MetaApp study. Comparably, the stage‑3 (CT WGs) XR & Metaverse work is fairly premature, for which only the CT3 WG has started the protocol studies.
The term metaverse has been used in various ways to refer to the broader implications of AR and VR. Metaverse in diverse sectors evokes a number of possible immersive user experiences. Products and services can emerge once virtual reality and augmented reality become commonly available and find application in our work, leisure and other activities. The present document uses this term to refer to a shared, perceived set of interactive perceived spaces that can be persistent.In addition to services that offer location-independent user experiences, this feature also considers content and services that are associated or applicable only in a particular location. These metaverse services are mobile in the sense that mobile users are able to interact with services anywhere and in particular when located where specific services are offered. The present document focuses on how to make these services function well, consistently and with diverse support mechanisms over mobile telecommunications networks. Requirements for diverse service enablers are introduced to the 5G system to support these services, including avatar call functionality, coordination of mobile metaverse services, digital asset management and support for spatial anchors. The term mobile metaverse implies the combination of various technologies to fuse physical and digital worlds, widely impacting society and the economy. Users will access these mobile metaverse services with devices for interaction with XR media and sensors, enabled by mobile telecommunication standards.
Mobile metaverse servicesIntroduction. This document defines the mobile metaverse as the user experience enabled by the 5G system of interactive and/or immersive XR media, including haptic media. For clarity and unambiguity with respect to many divergent technical and commercial developments, this document avoids the use of the term metaverse without qualification with the term mobile. This annex provides background information for the present document and further explanation of the term mobile metaverse, and the services that it enables. The term mobile metaverse implies the combination of various technologies to fuse physical and digital worlds, widely impacting society and the economy. Users will access these mobile metaverse services with devices for interaction with XR media and sensors, enabled by mobile telecommunication standards. Mobile metaverse services are expected to be provided via many and diverse service providers, each catering to different customers, companies and communities. An entertainment company might provide one mobile metaverse service for consumers such as a virtual theme park, while offering a separate mobile metaverse service to its employees. Some providers may specialize in interactive tools to create content, while others present that content. In general, mobile metaverse services can be divided into three general categories: industrial, enterprise and consumer mobile metaverse services. There are commonalities among these categories. These mobile metaverse services will apply across these domains to varying degrees, sharing technologies, devices and interfaces, and functionalities described in this document. Ultimately, mobile metaverse services will be defined by the applications they enable and the business models they adopt. Besides experiencing virtual world and/or augmented real world media as a passive consumer (where the media is read-only), mobile metaverse services also can enable interaction, the user can create and even control elements of the media. Depending on the mobile metaverse service, the consequence of user interaction could be experienced by other users. As discussed below, in some cases, user interaction may result even in changes to the real world, through actuation (remote control mechanisms). Mobile metaverse services face technical challenges specific to the media and interactions they offer to users, in the context of mobile networks. Some of these technical challenges are addressed by mobile telecommunications standards, of which this document is a contribution. Another set of challenges are not technical in nature, such as regulatory and safety implications, only some of which are discussed in the present document. Other technical and non-technical challenges are expected to be addressed separately depending on the target environment/vertical. Consumer mobile metaverse services. There has been some initial development of commercial consumer-oriented mobile metaverse services, specifically in metaverse gaming and VR social media 'virtual worlds'. Mobile metaverse consumer services considered during the study of mobile metaverse services relate to the following use cases:
Enterprise mobile metaverse services. Remote working has driven demand for better collaboration and communication tools, and thanks to extended reality (XR) and virtual reality (VR) technologies, many of those tools contribute to the enterprise metaverse. The enterprise mobile metaverse will eventually envelop the core productivity applications that make business function, e.g., as a digital drafting tool for architecture and engineering, and training sessions via AR and/or VR. Eventually, the enterprise and industrial mobile metaverses will interlink, merging the IT systems in office branches with the Operational Technology (OT) systems of the shop floor. Main mobile metaverse enterprise services considered during the study of mobile metaverse services relate to the following use cases:
Industrial mobile metaverse services. Industrial mobile metaverse services are expected to provide cost, productivity, safety and flexibility gains. In particular, industrial OT systems begin to support mobile services that include XR media for monitoring and analysis, and also for control of operations through the digital orchestration of robot fleets or user-guided remote operations. Industrial users will gain the ability to visualize and reconfigure their operations, e.g., in response to changing supply and demand or disruptions. By fusing digital and physical realities, the digital twin (real-time representation of physical assets in a digital world) has begun to transform many industries. Seaports have begun using digital twins to track every container on their docks. Aerospace companies design and build engines and fuselages virtually to simulate how an aircraft will fly and perform. Many new factories exist just as much digitally, as data and virtual representation, as they do in the physical world. This kind of mobile metaverse service aims at control and awareness of operations down to the smallest detail. Some of the mobile metaverse service enabler requirements support digital twin applications. Main mobile metaverse industrial services considered during the study of mobile metaverse services relate to the following use cases:
The term metaverse has been used in various ways to refer to the broader implications of AR and VR. Metaverse in diverse sectors evokes a number of possible immersive user experiences. Products and services can emerge once virtual reality and augmented reality become commonly available and find application in our work, leisure and other activities. The present document uses this term to refer to a shared, perceived set of interactive perceived spaces that can be persistent.In addition to services that offer location-independent user experiences, this feature also considers content and services that are associated or applicable only in a particular location. These metaverse services are mobile in the sense that mobile users are able to interact with services anywhere and in particular when located where specific services are offered. The present document focuses on how to make these services function well, consistently and with diverse support mechanisms over mobile telecommunications networks. Requirements for diverse service enablers are introduced to the 5G system to support these services, including avatar call functionality, coordination of mobile metaverse services, digital asset management and support for spatial anchors. The term mobile metaverse implies the combination of various technologies to fuse physical and digital worlds, widely impacting society and the economy. Users will access these mobile metaverse services with devices for interaction with XR media and sensors, enabled by mobile telecommunication standards.
Mobile metaverse servicesIntroduction. This document defines the mobile metaverse as the user experience enabled by the 5G system of interactive and/or immersive XR media, including haptic media. For clarity and unambiguity with respect to many divergent technical and commercial developments, this document avoids the use of the term metaverse without qualification with the term mobile. This annex provides background information for the present document and further explanation of the term mobile metaverse, and the services that it enables. The term mobile metaverse implies the combination of various technologies to fuse physical and digital worlds, widely impacting society and the economy. Users will access these mobile metaverse services with devices for interaction with XR media and sensors, enabled by mobile telecommunication standards. Mobile metaverse services are expected to be provided via many and diverse service providers, each catering to different customers, companies and communities. An entertainment company might provide one mobile metaverse service for consumers such as a virtual theme park, while offering a separate mobile metaverse service to its employees. Some providers may specialize in interactive tools to create content, while others present that content. In general, mobile metaverse services can be divided into three general categories: industrial, enterprise and consumer mobile metaverse services. There are commonalities among these categories. These mobile metaverse services will apply across these domains to varying degrees, sharing technologies, devices and interfaces, and functionalities described in this document. Ultimately, mobile metaverse services will be defined by the applications they enable and the business models they adopt. Besides experiencing virtual world and/or augmented real world media as a passive consumer (where the media is read-only), mobile metaverse services also can enable interaction, the user can create and even control elements of the media. Depending on the mobile metaverse service, the consequence of user interaction could be experienced by other users. As discussed below, in some cases, user interaction may result even in changes to the real world, through actuation (remote control mechanisms). Mobile metaverse services face technical challenges specific to the media and interactions they offer to users, in the context of mobile networks. Some of these technical challenges are addressed by mobile telecommunications standards, of which this document is a contribution. Another set of challenges are not technical in nature, such as regulatory and safety implications, only some of which are discussed in the present document. Other technical and non-technical challenges are expected to be addressed separately depending on the target environment/vertical. Consumer mobile metaverse services. There has been some initial development of commercial consumer-oriented mobile metaverse services, specifically in metaverse gaming and VR social media 'virtual worlds'. Mobile metaverse consumer services considered during the study of mobile metaverse services relate to the following use cases:
- attending (live) VR events (sports, gaming, concerts, etc.) either as spectator or performer, including while moving / commuting;
- virtual shopping or visit experience (tourism, real estate, etc.);
- presentation of AR content on a virtual screen, e.g., a feature length movie;
- interaction with AR content in a location-aware manner, offering spatial localization and mapping to support applications generating AR content for enhanced localized experiences (e.g., in museums, shopping malls);
- situation awareness about the user's physical surroundings, while walking or driving;
Enterprise mobile metaverse services. Remote working has driven demand for better collaboration and communication tools, and thanks to extended reality (XR) and virtual reality (VR) technologies, many of those tools contribute to the enterprise metaverse. The enterprise mobile metaverse will eventually envelop the core productivity applications that make business function, e.g., as a digital drafting tool for architecture and engineering, and training sessions via AR and/or VR. Eventually, the enterprise and industrial mobile metaverses will interlink, merging the IT systems in office branches with the Operational Technology (OT) systems of the shop floor. Main mobile metaverse enterprise services considered during the study of mobile metaverse services relate to the following use cases:
- XR-enabled collaborative and concurrent engineering based on geospatial digital twins (and avatars), for research & prototyping, visual testing & simulation, planning & optimisation, which can further be used for training
- Virtual showrooms, products or stores;
- Interaction with AR content in a location-aware manner, e.g., creating spatial anchors and discovery of them for AR content enhanced in-store shopping experiences;
Industrial mobile metaverse services. Industrial mobile metaverse services are expected to provide cost, productivity, safety and flexibility gains. In particular, industrial OT systems begin to support mobile services that include XR media for monitoring and analysis, and also for control of operations through the digital orchestration of robot fleets or user-guided remote operations. Industrial users will gain the ability to visualize and reconfigure their operations, e.g., in response to changing supply and demand or disruptions. By fusing digital and physical realities, the digital twin (real-time representation of physical assets in a digital world) has begun to transform many industries. Seaports have begun using digital twins to track every container on their docks. Aerospace companies design and build engines and fuselages virtually to simulate how an aircraft will fly and perform. Many new factories exist just as much digitally, as data and virtual representation, as they do in the physical world. This kind of mobile metaverse service aims at control and awareness of operations down to the smallest detail. Some of the mobile metaverse service enabler requirements support digital twin applications. Main mobile metaverse industrial services considered during the study of mobile metaverse services relate to the following use cases:
- remote critical health care, including surgery and treatment;
- AR/VR based tele-operation of a remote device or vehicle (e.g., driving).
Mulsemia communication in 6G wireless systemsAlthough humans have five basic senses, sight, hearing, touch, smell, and taste, most multimedia systems today only capture two of them, namely, sight and hearing. With the development of the metaverse and related technologies, there is a growing need for a more immersive media format that leverages all human senses. Multisensory media (Mulsemedia) that can stimulate multiple senses will play a critical role in the near future. This paper provides an overview of the history, background, use cases, existing research, devices, and standards of mulsemedia. Emerging mulsemedia technologies such as extended Reality (XR) and Holographic‑Type Communication (HTC) are introduced. Additionally, the challenges in mulsemedia research from the perspective of wireless communication and networking are discussed. The potential of 6G wireless systems to address these challenges is highlighted, and several research directions that can advance mulsemedia communications are identifiied.