| Internet-Draft | DetNet POF | July 2023 |
| Varga, et al. | Expires 25 January 2024 | [Page] |
Replication and Elimination functions of DetNet Architecture can result in out-of-order packets, which is not acceptable for some time-sensitive applications. The Packet Ordering Function (POF) algorithm described herein enables to restore the correct packet order when replication and elimination functions are used in DetNet networks.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 25 January 2024.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The DetNet Working Group has defined packet replication (PRF) and packet elimination (PEF) functions for achieving extremely low packet loss. PRF and PEF are described in [RFC8655] and provide service protection for DetNet flows. This service protection method relies on copies of the same packet sent over multiple maximally disjoint paths and uses sequencing information to eliminate duplicates. A possible implementation of PRF and PEF functions is described in [IEEE8021CB] and the related YANG model is defined in [IEEEP8021CBcv].¶
In general, use of per packet replication and elimination functions can result in out-of-order delivery of packets, which is not acceptable for some deterministic applications. Correcting packet order is not a trivial task, therefore details of a Packet Ordering Function (POF) are specified herein. The IETF DetNet WG has defined in [RFC8655] the external observable result of a POF function, i.e., that packets are reordered, but without any implementation details.¶
So far in packet networks, out-of-order delivery situations were handled at higher OSI layers at the end-points/hosts (e.g., in the TCP stack when packets are sent to application layer) and not within a network in nodes acting at the Layer-2 or Layer-3 OSI layers.¶
Figure 1 shows a DetNet flow on which PREOF functions are applied during forwarding from source to destination.¶
+------------+
+--------------E1----+ | |
+----+ | | +---R3---+ | +----+
|src |------R1 +---+ | E3----O1---+ dst|
+----+ | | E2-------+ +----+
+----------R2 |
+-----------------+
R: replication point (PRF)
E: elimination point (PEF)
O: ordering function (POF)
Important to note that different applications can react differently to out-of-order delivery. A single out-of-order packet (E.g., packet order: #1, #3, #2, #4, #5) is interpreted by some application as a single error, but some other applications treat it as 3 errors in-a-row situation. 3 errors in-a-row is a usual error threshold and can cause the application to stop (e.g., to go to a fail-safe state).¶
This document uses the terminology established in the DetNet architecture [RFC8655], and the reader is assumed to be familiar with that document and its terminology.¶
The following abbreviations are used in this document:¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The requirements on a POF function are:¶
Some aspects are explicitly out-of-scope for a POF function:¶
The POF Algorithm discussed in this document makes some assumptions and tradeoffs regarding the characteristics of the sequence of received packets. In particular, the algorithm assumes that a Packet Elimination Function (PEF) is performed on the incoming packets before they are handed to the POF function. Hence, the sequence of incoming packets can be out of order or incomplete but cannot contain duplicate packets. However, the PREOF functions run independently without any state exchange required between the PEF and the POF or the PRF and the POF. Error cases in which the POF is presented duplicate packets can lead to out of order delivery of duplicate packets as well as to increased delays.¶
The algorithm further requires that the delay difference between two replicated packets that arrive at the PRF before the POF is bounded and known. Error cases that violate this condition (e.g., a packet that arrives later than this bound) will result in out-of order packets.¶
The algorithm also makes some tradeoffs. For simplicity, it is designed in a way that allows for some out of order packets directly after initialization. If this is not acceptable, Section 4.5 provides an alternative initialization scheme that prevents out-of-order packets in the initialization phase.¶
The method described herein provides POF for DetNet networks. The configuration parameters of POF can be derived during engineering the DetNet flow through the network.¶
The POF method is provided via:¶
Note: the conditional buffer of POF increases the burstiness of the traffic as it adds delay only for some of the packets.¶
Figure 2 shows the building blocks of a possible POF implementation.¶
+------------+ +--------------+
| Delay calc | | Conditional |
+--| for packet >--->>---| Delay Buffer >---+
| +------------+ +--------------+ |
| |
+------^--------+ |
->>--| POF selector >----------------------------------+-->>----
| (Flow ident.) |
+---------------+
->>- packet flow
The basic POF algorithm delays all out-of-order packets until all previous packet arrives or a given time (POFMaxDelay) elapses. The basic POF algorithm works as follows:¶
If (seq_num <= POFLastSent + 1)¶
Note: the difference of sequence number in consecutive packets is bounded due to the history window of the Elimination function before the POF. Therefore "<=" can be evaluated despite of the circular sequence number space.¶
The state used by the basic POF algorithm (i.e., "POFLastSent") needs initialization and maintenance. This works as follows:¶
The basic POF algorithm has two parameters to engineer:¶
Design of these parameters is out-of-scope in this document.¶
Note: multiple network failures can impact the POF function (e.g., complete outage of all redundant paths).¶
The basic POF algorithm increases the delay of packets with maximum "POFMaxDelay" time. Packets being in order are not delayed. This basic POF method can be applied in all network scenarios where the remaining delay budget of a flow at the POF point is larger than "POFMaxDelay" time.¶
Figure 3 shows the delay budget relations at the POF point.¶
Path delay
difference
/-------------/
<-- fastest path delay -> /--- remaining delay budget ---/
|-----------------------|-------------|------------------------------|
0 t1 t2 T
<--------- slowest path delay -------->
/-------------------- delay budget at POF point ---------------------/
In network scenario where the remaining delay budget of a flow at the POF point is smaller than "POFMaxDelay" time the basic method needs extensions.¶
The issue is that packets on the longest path cannot be buffered in order to keep delay budget of the flow. It must be noted that such a packet (i.e., forwarded over the longest path) needs no buffering as it is the "last chance" to deliver a packet with a given sequence number. This is because all replicas are already arrived via shorter path(s).¶
The advanced POF algorithm needs two extensions of the basic POF algorithm:¶
By identifying the path of a given frame, the POF algorithm can use this information to select what predefined time "POFMaxDelay_i" to apply for the buffered frame/packet. So, in the advanced POF algorithm "POFMaxDelay" is an array, that contains the predefined and path specific buffering time for each redundant path of a flow. Values in the "POFMaxDelay" array are engineered to fulfill the delay budget requirement.¶
The method for identification of the packet's path at the POF location depends on the network scenario. It can be implemented via various techniques, for example using ingress interface information, encoding the path in the packet itself (e.g., replication functions can set different FlowID per egress what can be used as a PathID), or in other means. Method for identification of the packet's path is out of scope in this document.¶
Note: in case of using the advanced POF algorithm it might be advantageous to combine PEF and POF locations in the DetNet network, as it can simplify the method used for identification of the packet's path at the POF location.¶
POF algorithms can be further enhanced by distinguishing the case of initialization from normal operation at the price of more states and more sophisticated implementation. Such enhancements could for example react better after some failure scenarios (e.g., complete outage of all paths of a DetNet flow) and can be dependent on the PEF implementation.¶
The challenge for POF initialization is that for example after a reset it is not known whether the first received packet is in-order or out-of-order. The original initialization (see before) considers the first packet as in-order, so out-of-order packet(s) during "POFMaxTime"/"POFMaxTime_path_i" time - after the first packet was received - cannot be corrected. Motivation behind such an initialization is POF implementation simplicity.¶
A possible enhancement of POF initialization works as follows:¶
The selection of the POF algorithm depends on the network scenario and the remaining delay budget of a flow. Using POF and calculating its parameters require proper design. Knowing the path delay difference is essential for the POF algorithms described here. Failure scenarios breaking the design assumptions can impact the result of POF (e.g., packet received out of the expected worst-case delay window - calculated based on the path delay difference - can result in unwanted out-of-order delivery).¶
In DetNet scenarios there is always an Elimination function before the POF (therefore duplicates are not considered by the POF). Implementing them together in the same node allows POF to consider PEF events/states during the re-ordering. For example, under normal circumstances the difference of sequence number in consecutive packets is bounded due to the history window of PEF. However, in some scenarios (e.g., reset of sequence number) the difference can be much larger than the history window size.¶
POF algorithms needs setting of the following parameters:¶
Note, that in a proper design "POFTakeAnyTime" is always larger than "POFMaxDelay".¶
PREOF related security considerations (including POF) are described in section 3.3 of [RFC9055]. There are no additional POF related security considerations originating from this document.¶
This document makes no IANA requests.¶
Authors extend their appreciation to Gyorgy Miklos, Mohammadpour Ehsan, Ludovic Thomas, Greg Mirsky, Jeong-dong Ryoo, Shirley Yangfan, bToerless Eckert, Norman Finn and Ethan Grossman for their insightful comments and productive discussion that helped to improve the document.¶