| Internet-Draft | Gap Analysis for Enhanced DetNet Data Pl | July 2023 |
| Xiong | Expires 7 January 2024 | [Page] |
From charter and milestones, the enhanced Deterministic Networking (DetNet) is required to provide the enhancement of flow identification and packet treatment for data plane to achieve the DetNet QoS in large-scale networks.¶
This document describes the requirements for multiple deterministic services with differentiated DetNet QoS, discusses the characteristics of scaling networks and analyzes the gaps of the existing technologies especially applying the DetNet data plane as per RFC8938.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 7 January 2024.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
As per [RFC8655], it defined the overall architecture for Deterministic Networking (DetNet) , which provides a capability for real-time applications with extremely low data loss rates and bounded latency within a network domain. It has three goals: minimum and maximum end-to-end latency from source to destination, bounded jitter (packet delay variation), packet loss ratio and upper bound on out-of-order packet delivery. To achieve the above objectives, multiple techniques need to be used in combination, including explicit routes, service protection and resource allocation defined by DetNet.¶
As defined in [RFC8938], the DetNet data plane describes how application flows, or App-flows are carried over DetNet networks and it is provided by the DetNet service and forwarding sub-layers with DetNet-related data plane functions and mechanisms. The enhanced DetNet is required to provide the enhancement of flow identification and packet treatment for data plane to achieve the DetNet QoS in large-scale networks. It is required to analyse the applicability in DetNet for large-scale networks.¶
This document describes the requirements for multiple deterministic services with differentiated DetNet QoS, discusses the characteristics of large-scale networks and analyzes the gaps of the existing technologies especially applying the DetNet data plane as per [RFC8938].¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
5G network is oriented to the internet of everything. It need to supports the Ultra-reliable Low Latency Communications (uRLLC) services. The uRLLC services demand SLA guarantees such as low latency and high reliability and other deterministic and precise properties especially in Wide Area Network (WAN) applications.The uRLLC services should be provided in large-scale networks which cover the industries such as intelligent electrical network, intelligent factory, internet of vehicles, industry automation and other industrial internet scenarios. The industrial internet is the key infrastructure that coordinate various units of work over various system components, e.g. people, machines and things in the industrial environment including big data, cloud computing, Internet of Things (IOT), Augment Reality (AR), industrial robots, Artificial Intelligence (AI) and other basic technologies. For the intelligent electrical network, there are deterministic requirements for communication delay, jitter and packet loss rate. For example, in the electrical current difference model, a delay of 3~10ms and a jitter variation is no more than 100us are required. For the automation control, it is one of the basic application and the the core is closed-loop control system. The control process cycle is as low as millisecond level, so the system communication delay needs to reach millisecond level or even lower to ensure the realization of precise control. There are three levels of real-time requirements for industrial interconnection: factory level is about 1s, and process level is 10~100ms, and the highest real-time requirement is motion control, which requires less than 1ms. So the deterministic latency requirements are different with varying services and network scenarios.¶
As defined in [RFC8655], the DetNet QoS can be expressed in terms of : Minimum and maximum end-to-end latency, bounded jitter (packet delay variation), packet loss ratio and an upper bound on out-of-order packet delivery. As described in [RFC8578], DetNet applications differ in their network topologies and specific desired behavior and different services requires differentiated DetNet QoS. In large-scale networks, multiple services with differentiated DetNet QoS can be co-existed in the same DetNet network. The classification of the deterministic flows within different levels should be taken into considerations. It is required to provide Latency, bounded jitter and packet loss dynamically and flexibly in all scenarios for each characterized flow.¶
As the Figure 1 shows, the services can be divided into 5 levels and level 2~5 is the DetNet flows and level-1 is non-DetNet flow. DetNet applications and DetNet QoS is differentiated within each level.¶
+-------------+-----------+----------+----------+----------+-----------+ | Item | Level-1 | Level-2 | Level-3 | Level-4 | Level-5 | +-------------+-----------+----------+----------+----------+-----------+ | Applications| Email | Voice | Audio and| AR/VR | Industrial| | Examples | | | Video | | | +-------------+-----------+----------+----------+----------+-----------+ | DetNet QoS | Bandwidth | Jitter | Delay | Low | Ultra-low | | | Guarantee | Guarantee| Guarantee| delay | delay and| | | | | |and jitter| jitter | +-------------+-----------+----------+----------+----------+-----------+
From the perspective of deterministic service requirements, deterministic Quality of Service (QoS) in the network can be divided into five types or levels:¶
Level-1: bandwidth guarantee. The indicator requirements include basic bandwidth guarantee and certain packet loss tolerance. There is no requirement for the upper bound of the latency, and no requirement for the jitter. Typical services include download and FTP services.¶
Level-2: jitter guarantee. The indicator requirements include: jitter 50ms, delay 300ms. Typical services include synchronous voice services, such as voice call.¶
Level-3: delay guarantee. The indicator requirements include: delay 50ms, jitter 50ms. Typical services include real-time communication services, such as video, production monitoring, and communication services.¶
Level-4: low delay and jitter guarantee. The indicator requirements include: delay 20ms, jitter 5ms. Typical services include video interaction services, such as AR/VR, holographic communication, cloud video and cloud games.¶
Level-5: ultra-low delay and jitter guarantee. The indicator requirements include: delay 10ms, jitter 100us. Typical services include production control services, such as power protection and remote control.¶
Moreover, different DetNet services is required to tolerate different percentage of packet loss ratio such as 99.9%, 99.99%, 99.999%, and so on.¶
Traditional Ethernet, IP and MPLS networks which is based on statistical multiplexing provides best-effort packet service and offers no delivery and SLA guarantee. As described in [RFC8655], the primary technique by which DetNet achieves its QoS is to allocate sufficient resources. But it can not be achieved by not sufficient resource which can be allocated due to practical and cost reason. So it is required to achieve the high-efficiency of resources utilization when provide the DetNet service.¶
As described in [RFC8557], deterministic forwarding can only apply to flows with such well-defined characteristics as periodicity and burstiness. As defined in DetNet architecture [RFC8655], the traffic characteristics of an App-flow can be CBR (constant bit rate) or VBR (variable bit rate) of L1, L2 and L3 layers (VBR takes the maximum value when reserving resources). But the current scenarios and technical solutions only consider CBR flow, without considering the coexistence of VBR and CBR, the burst and aperiodicity of flows. The operations such as shaping or scheduling have not been specified. Even TSN mechanisms are based on a constant and forecastable traffic characteristics.¶
It will be more complicated in a large-scale network where much more flows coexist and the traffic characteristics is more dynamic. A huge number of flows with different DetNet QoS requirements is dynamically concurrent and the state of each flow cannot be maintained. It is required to offer reliable delivery and SLA guarantee for dynamic flows. For example, periodic flow and aperiodic flow (including micro burst flow, etc.), CBR and VBR flow, flow with different periods or phases, etc. When the network needs to forward these deterministic flows at the same time, it must solve the problems of time micro bursts, queue processing and aggregation of multiple flows.¶
In large-scale applications, the network topology may consists of a large number of nodes and links which leads to difficulty with controlling the end-to-end delay and jitter. High speed, long-distance transmission and asymmetric links may also co-exists and affects the bounded latency such as increasing transmission latency, jitter and packet loss in large-scale networks.¶
The network topology in a large-scale network may across multiple domains within a single administrative control or a closed group of administrative control as per [RFC8655]. Moreover, DetNet domains or nodes may be interconnected with different sub-network technologies such as FlexE tunnels, TSN sub-network, IP/MPLS/SRv6 tunnels and so on. It is required to support the inter-domain deterministic metric and routes to achieve the end-to-end bounded latency.¶
As defined in [RFC8938], the DetNet data plane describes how application flows, or App-flows are carried over DetNet networks and it is provided by the DetNet service and forwarding sub-layers with DetNet-related data plane functions and mechanisms. This section analyzes the DetNet technical gaps when applying the DetNet data plane as per RFC8938 in large-scale networks. [I-D.xiong-detnet-large-scale-enhancements] has proposed the overall framework of DetNet enhancements for scaling deterministic networks based on the gaps.¶
In [RFC8938], the DetNet data plane can provide the DetNet-Specific Metadata such as Flow-ID for both the service and forwarding sub-layers. The flow-based state information is required to be maintained for per-flow processing rules. For example, the resource reservation configuration is required for each flow. DetNet as per [RFC8938] provides the capability to aggregate the individual flows to downscale the operations of flow states. However, it still requires large amount of control signaling to establish and maintain DetNet flows. It may be challenging for network operations with a large number of deterministic flows and network nodes in large-scale networks. It may provide traffic class scheduling than the flow scheduling. And the traffic class for DetNet should consider the requirements of multiple services with differentiated DetNet QoS.¶
As described in [RFC8655], the primary goals are to achieve the DetNet QoS to provide minimum and maximum end-to-end latency and bounded jitter, low packet loss ratio and an upper bound on out-of-order packet delivery. But the data plane [RFC8938] particularly focuses on the DetNet service sub-layer which provides a set of Packet Replication, Elimination, and Ordering Functions (PREOF) functions to provide end-to-end service assurance. It mainly provides the capabilities for DetNet to guarantee the reliability.¶
The DetNet forwarding sub-layer provides corresponding forwarding assurance with IETF existing functions using resource allocations and explicit routes. But these functions can not provide the deterministic latency (bounded latency, low packet loss and in-order delivery) assurance in large-scale networks. The following sections mainly discuss the gap analysis for the forwarding sub-layer functions to provide deterministic latency assurance.¶
Traditional routes only have reachability. As per [RFC8938], explicit optimized paths with allocation of resources should be provided to achieve the DetNet QoS. But the deterministic requirements such as end-to-end delay and jitter are only used as path computation constraints. Multiple network metrics which are measured and distributed by the routing system should be taken into consideration.¶
In large-scale networks, it may be challenging to compute the best path to meet all of the requirements. In multi-domain scenarios, the inter-domain deterministic routes need to be established and provisioned. Especially when interconnecting with sub-networks, the selection of intra-domain paths acrossing cooperating domains should consider the bounded latency in each domain and the stitching of the paths.¶
Moreover, the paths vary with the real-time change of the network topology. On the basic of the resources, the steering path and routes for deterministic flows should be programmed before the flows coming and able to provide SLA capability. And the routes should be considered to be established in distributed and centralized control Plane.¶
As described in [RFC8557], the packet replication and elimination service protection should be provided to achieve the low packet loss ratio. It will copy the flows and spread the data over multiple disjoint forwarding paths. The bounded latency and jitter of each path should be meet service deterministic requirement. And the difference of latency within these paths should be limited. So the replication and elimination deterministic routes with configured latency and jitter policy should be taken into consideration. It is required to generate two disjoint paths with very close delay to form 1+1 protection and perform concurrent transmission and dual reception, and make replication and elimination on the egress PE.¶
As per [RFC8938], the forwarding sub-layer uses buffer resources for packet queuing, as well as reservation and allocation of bandwidth capacity resources. The reservation of the bandwidth can not guarantee the deterministic latency. In large-scale networks, the bandwidth, buffer and scheduling resources are combined with queuing mechanisms to guarantee the deterministic latency. The deterministic resources may be include the resources that can guarantee the deterministic latency such as the nodes, links, interfaces, buffers, bandwidth, queuing and scheduling mechanisms and so on. The planning, reservation and allocation of deterministic resources should be taken into consideration in DetNet data plane.¶
As per [RFC8938], the forwarding sub-layer provides the QoS-related functions needed by the DetNet flow including the use of queuing techniques. But the queuing techniques which are defined in existing IETF technologies can not guarantee the bounded latency such as Active Queue Management(AQM). And the queuing mechanisms which are defined in IEEE802.1 TSN can not be directly applied in large-scale networks such Time Aware Shaping [IIEEE802.1Qbv] and Cyclic Queuing and Forwarding [IEEE802.1Qch] with time synchronization.¶
Enhancement of queuing mechanisms have been discussed in DetNet such as cyclic-scheduling queuing mechanism [I-D.eckert-detnet-tcqf], [I-D.dang-queuing-with-multiple-cyclic-buffers] and [I-D.chen-detnet-sr-based-bounded-latency], deadline-scheduling queuing mechanism [I-D.stein-srtsn] and [I-D.peng-detnet-deadline-based-forwarding], timeslot-scheduling queuing mechanism [I-D.peng-detnet-packet-timeslot-mechanism] and asynchronous queuing mechanism [I-D.joung-detnet-asynch-detnet-framework] and [I-D.joung-detnet-stateless-fair-queuing]. The queuing-based requirements in DetNet enhanced data plane has been described in [I-D.ietf-detnet-scaling-requirements]. The function of multiple queuing mechanisms and related DetNet-Specific metadata should be defined in DetNet data plane as per [I-D.xiong-detnet-data-fields-edp].¶
TBA¶
TBA¶