Network Working Group G. Fioccola Internet-Draft Huawei Technologies Intended status: Informational P. Mendes Expires: 11 January 2024 Airbus J. Burke UCLA REMAP D. Kutscher HKUST(GZ) 10 July 2023 Information-Centric Metaverse draft-fmbk-icnrg-metaverse-01 Abstract This document aims to explore the new challenges for the transport network brought by the development of Metaverse. It discusses the Metaverse as an Information-Centric Network (ICN). Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 11 January 2024. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. Fioccola, et al. Expires 11 January 2024 [Page 1] Internet-Draft Metaverse July 2023 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Requirements and Gaps . . . . . . . . . . . . . . . . . . . . 3 3. Current and Emerging Mainstream Approaches . . . . . . . . . 4 4. Opportunities and Challenges for Information-Centric Metaverse . . . . . . . . . . . . . . . . . . . . . . . . 6 4.1. Scalable multimedia communication . . . . . . . . . . . . 7 4.1.1. Issues today . . . . . . . . . . . . . . . . . . . . 7 4.1.2. ICN Support . . . . . . . . . . . . . . . . . . . . . 7 4.1.3. Research Opportunities . . . . . . . . . . . . . . . 8 4.2. Interaction with applications . . . . . . . . . . . . . . 8 4.2.1. Issues today . . . . . . . . . . . . . . . . . . . . 8 4.2.2. ICN Support . . . . . . . . . . . . . . . . . . . . . 9 4.2.3. Research Opportunities . . . . . . . . . . . . . . . 9 4.3. In-Network Computing . . . . . . . . . . . . . . . . . . 10 4.3.1. Issues today . . . . . . . . . . . . . . . . . . . . 10 4.3.2. ICN Support . . . . . . . . . . . . . . . . . . . . . 10 4.3.3. Research Opportunities . . . . . . . . . . . . . . . 10 4.4. Interoperability with existing infrastructure . . . . . . 11 5. Security Considerations . . . . . . . . . . . . . . . . . . . 11 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 11 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 11 9. Informative References . . . . . . . . . . . . . . . . . . . 11 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 1. Introduction "Metaverse" is a place-holder for a range of new technologies and experiences that is not particularly well-defined, but some working definitions include the notion of shared, interoperable, and persistent eXtended Reality (XR). Whereas initial prototypes and blueprints suggest leveraging or extending existing Internet and Web protocols, we can already identify gaps with respect to performance and scalability, for example as reported by [socialVR-measurements]. Fioccola, et al. Expires 11 January 2024 [Page 2] Internet-Draft Metaverse July 2023 Some of the observed performance problems seem to stem from fundamental gaps in today's Internet and Web technologies, for example, the lack of scalable and robust multi-destination communication and the lack of leveraging computing in the network with the required level of flexibility and trustworthiness to provide offloading services and to enable the ultra-low latencies that some Metaverse applications claim to require. Different remedies are being proposed, e.g., providing more (costly) deterministic communication services through resource reservation and scheduling on the Internet, requiring/enabling the network to understand application requirements and to provide corresponding QoS, extended overlay infrastructure for reducing latencies for CDN-like distribution etc. Alternatively, one might also take a more principled approach and do not take current design and deployment models as a given, but rather take Metaverse as a first candidate proposal for a future web infrastructure that enables fine-granular 3D content exchange, rich interaction between physical and virtual infrastructure, access to static data and dynamic computation results for individual users as well as for large user groups without the limitations of today's platform- and overlay-based system -- i.e., conceive the Metaverse and the future web as a fundamentally information-centric system. This document addresses three aspects: 1. the documentation of requirements and observed gaps with the current technology stack; 2. a discussion of the applicability of different networking and distributed computing technologies; and 3. an initial discussion of a more fundamental and comprehensive re- design of a future web that provides useful services for Metaverse systems and beyond. 2. Requirements and Gaps [I-D.han-iccrg-arvr-transport-problem] started to analyze the requirements of Virtual Reality (VR) and Augmented Reality (AR) to networking from a transport protocol perspective. Some of the requirements are: * low latency and High-Speed transport to reach services in one-hop and for real-time user interactions; Fioccola, et al. Expires 11 January 2024 [Page 3] Internet-Draft Metaverse July 2023 * intelligent control and SLA real-time monitoring to convey the traffic and manage network resources and source/route re- selection; * decentralization and Edge Services by positioning the data close to the user; and * reducing data sizes through resolution changes, compression, and more efficient encodings. [socialVR-measurements] performed measurements with five popular social VR platforms. The experimental results revealed that all these platforms face fundamental technical challenges considering the claims that are often associated with the Metaverse. One issue was poor scalability with respect to the number of users in one session: throughput, end-to-end latency, and on-device computation resource utilization increase almost linearly with the number of users. Other issues include noticeable load and reduced achievable video rendering frame rates and considerable network utilization even with smaller numbers of users. 3. Current and Emerging Mainstream Approaches Different IETF technologies have been proposed to address some of the above-mentioned issues and to provider a better service for Metaverse applications. In the following, we list a few of them and will discuss them in more detail in a future version of this document. For example, on the network and transport layer, there are elaborate solutions for dealing with bandwidth limitations, network congestion, lossy transport protocols, and the ever growing size of video data, to address the above requirements, for instance: * MPTCP[RFC8684] and MPQUIC[I-D.ietf-quic-multipath] are the expansions of TCP[RFC9293] and QUIC[RFC9000] in order to dispatch packets over multiple paths to maximize throughput. * Dynamic Adaptive Streaming over HTTP (DASH) aim to improve the viewport quality of immersive videos by refining the tiles delivery. But client-driven nature of DASH introduces less control on the server side. * Media over QUIC (MoQ) ([I-D.ietf-moq-requirements]) and extensions such as QuicR ([I-D.jennings-moq-proto]) use similar concepts and delivery mechanisms to those used by CDN and named objects. There are fundamental characteristics that QuicR provides for ultra low latency delivery, by leveraging the characteristics of QUIC protocol. Fioccola, et al. Expires 11 January 2024 [Page 4] Internet-Draft Metaverse July 2023 * The APplication-aware Networking (APN) aims to develop a framework to enable fine-granularity network service provisioning (traffic operations) within the network domain(s) that supports APN ([I-D.li-apn-framework]). APN aims to use the ability to apply policies to traffic flows entering into the infrastructure. In modern networks, where things such as deterministic networking and networking slicing are required, there is a requirement for more functionality than QoS can provide. * The Computing-Aware Traffic Steering (CATS) aims to analyze the problem on the edge node, which makes a decision based on the metrics of interest, and then steers the traffic to a node that serves a service instance. Indeed, for AR/VR services, the performance experienced by the end users depends on both network metrics such as bandwidth and latency, and compute metrics such as processing, storage capabilities, and capacity. In all of these approaches, the Metaverse is considered as an overlay application with corresponding infrastructure dependencies, but this potentially increases the current gaps (and resulting costs and technical complexity) between distributed applications and the underlying network architecture. In the 3D hypermedia space, one proposal for a new "Spatial Web" Framework is the HyperSpace Transaction Protocol (HSTP), as described by [IEEE-P2874], intended to "enable interoperable, semantically compatible connections between connected hardware (e.g. autonomous drones, sensors, smart devices, robots) and software (e.g. services, platforms, applications, artificial intelligence systems)". The specification, which is not accessible publically, is supposed to include 1. a spatial range query format and response language for requesting data about objects within a dimensional range (spatial, temperature, pressure, motion) and their content. 2. a semantic data ontology schema for describing objects, relations, and actions in a standardized way 3. a verifiable credentialing and certification method for permissioning create, retrieve, update, and delete (CRUD) access to devices, locations, users, and data; and 4. a human and machine-readable contracting language that enables the expression and automated execution of legal, financial and physical activities. Fioccola, et al. Expires 11 January 2024 [Page 5] Internet-Draft Metaverse July 2023 We cannot review the technical specification, but the feature description seems to suggest an application layer protocol that would enable more expressiveness and functionality in the "web" (i.e., application and presentation) layer, however based on the assumption of existing networking technology and overlay approaches. 4. Opportunities and Challenges for Information-Centric Metaverse Considering the gaps and perceived requirements from applications and proposed application layer protocols, we can reason about a holistic design that can address the afore-mentioned problems _and_ provide a more useful foundations for future hypermedia communication. Information-Centric Networking (ICN) introduces named information objects, e.g. media contents, as the central concept as opposed to a physical computer, or node ([RFC7927]). In ICN approaches, the principal paradigm is not host-to-host communication as in the current Internet architecture. The increasing demand for highly scalable and efficient distribution of content has motivated the development of architectures that focus on information objects, their properties, and receiver interest in the network to achieve efficient and reliable distribution of such objects. Therefore, we can conceive the Metaverse as an information-centric system where most applications participate in granular 3D content exchange, context-aware integration with the physical world, and other Metaverse-relevant services. The assumption is that the Metaverse is an information-centric concept that will become synonymous with the network itself. Many applications already work with data-oriented paradigms. Mapping them to a host-centric network model creates complexities and robustness issues, which can be addressed with an ICN oriented approach. The overlay approach to deal with real-time interactive media adds significant complexity. It is needed a fine-grained, hierarchical media exchange for low-latency interactive communication that enables scalable multi-destination distribution, and in-network replication and transformation that exposes object hierarchy for fine grained access and security. Since the Metaverse is an extension of the Web into immersive XR modalities that are often aligned with physical space, leveraging ICN concepts provides support for decentralized publishing, content interoperability and co-existence, based on general building blocks and not within separated application silos as today’s initial prototypes. Fioccola, et al. Expires 11 January 2024 [Page 6] Internet-Draft Metaverse July 2023 In the following, we discuss issues with today's technologies, ICN support that could be leveraged, and research opportunities for the ICN community for four topics: 1. Scalable multimedia communication; 2. Interaction with applications; 3. In-network computing; and 4. Interoperability with existing infrastructure. 4.1. Scalable multimedia communication 4.1.1. Issues today * Low-latency live streaming is not easy and not efficient in the Internet today. CDN-based DASH incurs high latencies. * Current trends: blending of real-time interactive (WebRTC with RTP) and streaming (DASH), for example: Amazon Twitch, Meta Rush, IETF Media-over-QUIC, QuicR * What is needed: - fine-granular media distribution that supports both interactive and streaming; - scalable multi-destination distribution, i.e., some kind of in- network replication; - ability to leverage wireless broadcast such as 5G Broadcast; and - support for hetergeneous devices and edge networks, i.e., different quality layers, possibly dynamic transcoding. 4.1.2. ICN Support ICN is generally supporting most of these requirements: * multi- destination distribution can be achieved through automatic in-network replication and interest aggregation, further supported by opportunistic caching. * ICN provides a uniform interface for unicast and "multicast" (i.e., there is no difference for consumers). * IP- Multicast issues (inter-domain, routing scalability) do not apply * Wireless broadcast could be leveraged where available, without requiring application-aware in edge routers. * Receiver-driven operation conducive to supporting different quality levels, like in Fioccola, et al. Expires 11 January 2024 [Page 7] Internet-Draft Metaverse July 2023 DASH today: receiver has all the knowledge directly (current performance) and can make timely decisions. * Consumer mobility is an efficient operation in ICN (a non-operation). 4.1.3. Research Opportunities * Based on ICN's basic capabilities, actual systems would need specific approaches for quality adaptation, e.g., - use of layered coding; and - role of in-network transcoding. * In order to achieve low-latency and QoS, more work is needed for - fine-tuning with respect to interest aggregation, caching; and for - prioritizing "flows", e.g., audio over video. * Sender mobility has seen some proposals in research that need to be validated. * More experiments are needed with large-scale interactive multimedia communication and low-latency transport. * Actual application development and deployment is needed to gradually develop best practices, software stacks, and re-usable application components. - For initial deployments, some kind of overlay topology over the current Internet might be needed. - Whereas the technology (different faces and underlay protocols) is essentially ready, there are other issues the deployment and efficient operation of such overlays (shortest path communication, routing, reliability). 4.2. Interaction with applications 4.2.1. Issues today Many applications, not only the Metaverse, work inherently with data- oriented paradigms when they are accessing named data, objects etc. Mapping this to a connection-based communication model creates complexities and robustness issues and would not result in good abstractions for systems. Fioccola, et al. Expires 11 January 2024 [Page 8] Internet-Draft Metaverse July 2023 In Metaverse applications, data can potentially be shared efficiently between nodes and within one node/process. Connection-based communication models make it hard/impossible to do so. Application layer data structures in VR (3D models, scene descriptions) are based on object hierarchies, connection-based systems may not be able to take advantage of it. 4.2.2. ICN Support ICN generally enables direct data-oriented communication: just names and objects so that location, storage contexts (files etc.) become less relevant. In addition ICN provides * named-based APIs to applications; * support for object collections through manifests; * additional "middleware" such as dataset synchronization; and * data-sharing is generally supported, which has benefits beyond networking, e.g., zero-copy sharing in processes etc. 4.2.3. Research Opportunities * Work should be started on the development of information-centric hypermedia concepts, i.e., linking from object collections to other objects/collections. * Manifest technologies such as FLIC should be extended to support dynamically created objects. * Concepts for dealing with "mutable objects" (or mutable "information") should be developed, i.e., how to deal with updates without giving up data immutability. * The relationship between application-layer data-oriented operation and network-layer needs to be explored further, e.g., - Would there be any differences? - Would it be needed to think about robust namespace mappings schemes? * Concepts and mechanisms for privacy, selective attention, content filtering, and autonomous interactions, as well as ownership and control on the publishing side are needed. Fioccola, et al. Expires 11 January 2024 [Page 9] Internet-Draft Metaverse July 2023 * In general more applications should be developed to enable more experiments. 4.3. In-Network Computing In-network computing can support Metaverse systems in different ways: * transcoding (to support heterogeneous receiver groups and networks better); * coding for better communication robustness & efficiency; * rendering for offloading clients and servers; * compression and decompression on various levels, including semantic communication; * ad insertion; and * potentially for future decomposed Metaverse systems. 4.3.1. Issues today * In-network computing today is typically limited to coarse-grained CDN-style computing, include Multi-Access Edge Computing. * Current trust and security frameworks require TLS connection termination, i.e., represent an overlay approach, which is not conducive to low latency communication. * Dynamic, just-in-time, instantiation of computing function on application-agnostic platforms is not available. 4.3.2. ICN Support * The named-data approach is generally useful for distributed computing (NFN, RICE, CFN). * ICN's security model makes it possible to do on-path computing / data transformation securely * discovery of functions and request forwarding can be punted on regular ICN mechanisms (name-based forwarding). 4.3.3. Research Opportunities * Robust distributed computing interaction models (RMI, Dataflow, REST) should be further developed and tested. Fioccola, et al. Expires 11 January 2024 [Page 10] Internet-Draft Metaverse July 2023 * Specific approaches such as in-network media transcoding should be developed. * In general, more experiments for different types of applications are needed. 4.4. Interoperability with existing infrastructure In addition, the interoperability aspects also need to be investigated, and, for example, Hybrid Information-Centric Networking (hICN), which implements information-networking functionalities into IPv6 ([I-D.muscariello-intarea-hicn], can provide a solution. It would be theoretically possible to leverage the solutions mentioned in the previous section in order to reach the above ICN oriented capabilities. But a systemic approach would be highly desirable in the longer term. 5. Security Considerations TBD 6. IANA Considerations This document makes no request of IANA. 7. Contributors TBD 8. Acknowledgements TBD 9. Informative References [I-D.han-iccrg-arvr-transport-problem] Han, L. and K. Smith, "Problem Statement: Transport Support for Augmented and Virtual Reality Applications", Work in Progress, Internet-Draft, draft-han-iccrg-arvr- transport-problem-01, 12 March 2017, . [I-D.ietf-moq-requirements] Gruessing, J. and S. Dawkins, "Media Over QUIC - Use Cases and Requirements for Media Transport Protocol Design", Work in Progress, Internet-Draft, draft-ietf-moq- Fioccola, et al. Expires 11 January 2024 [Page 11] Internet-Draft Metaverse July 2023 requirements-01, 10 July 2023, . [I-D.ietf-quic-multipath] Liu, Y., Ma, Y., De Coninck, Q., Bonaventure, O., Huitema, C., and M. Kühlewind, "Multipath Extension for QUIC", Work in Progress, Internet-Draft, draft-ietf-quic-multipath-05, 10 July 2023, . [I-D.jennings-moq-proto] Jennings, C. F. and S. Nandakumar, "QuicR - Media Delivery Protocol over QUIC", Work in Progress, Internet-Draft, draft-jennings-moq-proto-00, 13 March 2023, . [I-D.li-apn-framework] Li, Z., Peng, S., Voyer, D., Li, C., Liu, P., Cao, C., and G. S. Mishra, "Application-aware Networking (APN) Framework", Work in Progress, Internet-Draft, draft-li- apn-framework-07, 3 April 2023, . [I-D.muscariello-intarea-hicn] Muscariello, L., Carofiglio, G., Auge, J., Papalini, M., and M. Sardara, "Hybrid Information-Centric Networking", Work in Progress, Internet-Draft, draft-muscariello- intarea-hicn-04, 20 May 2020, . [IEEE-P2874] "IEEE SA P2874 Standard for Spatial Web Protocol, Architecture and Governance", n.d., . [RFC7927] Kutscher, D., Ed., Eum, S., Pentikousis, K., Psaras, I., Corujo, D., Saucez, D., Schmidt, T., and M. Waehlisch, "Information-Centric Networking (ICN) Research Challenges", RFC 7927, DOI 10.17487/RFC7927, July 2016, . Fioccola, et al. Expires 11 January 2024 [Page 12] Internet-Draft Metaverse July 2023 [RFC8684] Ford, A., Raiciu, C., Handley, M., Bonaventure, O., and C. Paasch, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 8684, DOI 10.17487/RFC8684, March 2020, . [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, May 2021, . [RFC9293] Eddy, W., Ed., "Transmission Control Protocol (TCP)", STD 7, RFC 9293, DOI 10.17487/RFC9293, August 2022, . [socialVR-measurements] Cheng, R., Wu, N., Varvello, M., Chen, S., and B. Han, "Are we ready for metaverse?: a measurement study of social virtual reality platforms", ACM, Proceedings of the 22nd ACM Internet Measurement Conference, DOI 10.1145/3517745.3561417, October 2022, . Authors' Addresses Giuseppe Fioccola Huawei Technologies Palazzo Verrocchio, Centro Direzionale Milano 2 20054 Segrate (Milan) Italy Email: giuseppe.fioccola@huawei.com Paulo Mendes Airbus 82024 Taufkirchen Germany Email: paulo.mendes@airbus.com Jeff Burke UCLA REMAP 102 East Melnitz Hall Los Angeles, CA 90095 United States of America Email: jburke@remap.ucla.edu Fioccola, et al. Expires 11 January 2024 [Page 13] Internet-Draft Metaverse July 2023 Dirk Kutscher HKUST(GZ) Guangzhou China Email: ietf@dkutscher.net Fioccola, et al. Expires 11 January 2024 [Page 14]