| RFC 9574 | EVPN Optimized IR | May 2024 |
| Rabadan, et al. | Standards Track | [Page] |
Network Virtualization Overlay (NVO) networks using Ethernet VPNs (EVPNs) as their control plane may use trees based on ingress replication or Protocol Independent Multicast (PIM) to convey the overlay Broadcast, Unknown Unicast, or Multicast (BUM) traffic. PIM provides an efficient solution that prevents sending multiple copies of the same packet over the same physical link; however, it may not always be deployed in the NVO network core. Ingress replication avoids the dependency on PIM in the NVO network core. While ingress replication provides a simple multicast transport, some NVO networks with demanding multicast applications require a more efficient solution without PIM in the core. This document describes a solution to optimize the efficiency of ingress replication trees.¶
This is an Internet Standards Track document.¶
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.¶
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc9574.¶
Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Ethernet Virtual Private Networks (EVPNs) may be used as the control plane for a Network Virtualization Overlay (NVO) network [RFC8365]. Network Virtualization Edge (NVE) and Provider Edge (PE) devices that are part of the same EVPN Broadcast Domain (BD) use Ingress Replication (IR) or PIM-based trees to transport the tenant's Broadcast, Unknown Unicast, or Multicast (BUM) traffic.¶
In the ingress replication approach, the ingress NVE receiving a BUM frame from the Tenant System (TS) will create as many copies of the frame as the number of remote NVEs/PEs that are attached to the BD. Each of those copies will be encapsulated into an IP packet where the outer IP Destination Address (IP DA) identifies the loopback of the egress NVE/PE. The IP fabric core nodes (also known as spines) will simply route the IP-encapsulated BUM frames based on the outer IP DA. If PIM-based trees are used instead of ingress replication, the NVEs/PEs attached to the same BD will join a PIM-based tree. The ingress NVE receiving a BUM frame will send a single copy of the frame, encapsulated into an IP packet where the outer IP DA is the multicast address that represents the PIM-based tree. The IP fabric core nodes are part of the PIM tree and keep multicast state for the multicast group, so that IP-encapsulated BUM frames can be routed to all the NVEs/PEs that joined the tree.¶
The two approaches are illustrated in Figure 1. On the left-hand side of the diagram, NVE1 uses ingress replication to send a BUM frame (originated from Tenant System TS1) to the remote nodes attached to the BD, i.e., NVE2, NVE3, and PE1. On the right-hand side, the same example is depicted but using a PIM-based tree, i.e., (S1,G1), instead of ingress replication. While a single copy of the tunneled BUM frame is generated in the latter approach, all the routers in the fabric need to keep multicast state, e.g., the spine keeps a PIM routing entry for (S1,G1) with an Incoming Interface (IIF) and three Outgoing Interfaces (OIFs).¶
To WAN To WAN
^ ^
| |
+-----+ +-----+
+----------| PE1 |-----------+ +----------| PE1 |-----------+
| +--^--+ | | +--^--+ |
| | IP Fabric | | | IP Fabric |
| PE | | (S1,G1) |OIF to G1 |
| +----PE->+-----+ No State | | IIF +-----+ OIF to G1 |
| | +---2->|Spine|------+ | | +------>Spine|------+ |
| | | +-3->+-----+ | | | | +-----+ | |
| | | | 2 3 | | |PIM |OIF to G1| |
| | | |IR | | | | |tree | | |
|+-----+ +--v--+ +--v--+ | |+-----+ +--v--+ +--v--+ |
+| NVE1|---| NVE2|---| NVE3|-+ +| NVE1|---| NVE2|---| NVE3|-+
+--^--+ +-----+ +-----+ +--^--+ +-----+ +-----+
| | | | | |
| v v | v v
TS1 TS2 TS3 TS1 TS2 TS3
In NVO networks where PIM-based trees cannot be used, ingress replication is the only option. Examples of these situations are NVO networks where the core nodes do not support PIM or the network operator does not want to run PIM in the core.¶
In some use cases, the amount of replication for BUM traffic is kept under control on the NVEs due to the following fairly common assumptions:¶
If the above assumptions are true for a given NVO network, then ingress replication provides a simple solution for multi-destination traffic. However, statement c. above is not always true, and multicast applications are required in many use cases.¶
When the multicast sources are attached to NVEs residing in hypervisors or low-performance-replication Top-of-Rack (ToR) switches, the ingress replication of a large amount of multicast traffic to a significant number of remote NVEs/PEs can seriously degrade the performance of the NVE and impact the application.¶
This document describes a solution that makes use of two ingress replication optimizations:¶
Assisted Replication consists of a set of procedures that allows the ingress NVE/PE to send a single copy of a broadcast or multicast frame received from a TS to the BD without the need for PIM in the underlay. Assisted Replication defines the roles of AR-REPLICATOR and AR-LEAF routers. The AR-LEAF is the ingress NVE/PE attached to the TS. The AR-LEAF sends a single copy of a broadcast or multicast packet to a selected AR-REPLICATOR that replicates the packet multiple times to remote AR-LEAF or AR-REPLICATOR routers and is therefore "assisting" the ingress AR-LEAF in delivering the broadcast or multicast traffic to the remote NVEs/PEs attached to the same BD. Assisted Replication can use a single AR-REPLICATOR or two AR-REPLICATOR routers in the path between the ingress AR-LEAF and the remote destination NVEs/PEs. The procedures that use a single AR-REPLICATOR (the non-selective Assisted Replication solution) are specified in Section 5, whereas Section 6 describes how multi-stage replication, i.e., two AR-REPLICATOR routers in the path between the ingress AR-LEAF and destination NVEs/PEs, is accomplished (the selective Assisted Replication solution). The procedures for Assisted Replication do not impact unknown unicast traffic, which follows the same forwarding procedures as known unicast traffic so that packet reordering does not occur.¶
PFLs provide a method for the ingress NVE/PE to prune or remove certain destination NVEs/PEs from a flooding list, depending on the interest of those NVEs/PEs in receiving BUM traffic. As specified in [RFC8365], an NVE/PE builds a flooding list for BUM traffic based on the next hops of the received EVPN Inclusive Multicast Ethernet Tag routes for the BD. While [RFC8365] states that the flooding list is used for all BUM traffic, this document allows pruning certain next hops from the list. As an example, suppose an ingress NVE creates a flooding list with next hops PE1, PE2, and PE3. If PE2 and PE3 did not signal any interest in receiving unknown unicast traffic in their Inclusive Multicast Ethernet Tag routes, when the ingress NVE receives an unknown unicast frame from a TS, it will replicate it only to PE1. That is, PE2 and PE3 are "pruned" from the NVE's flooding list for unknown unicast traffic. PFLs can be used with ingress replication or Assisted Replication and are described in Section 7.¶
Both optimizations -- Assisted Replication and PFLs -- may be used together or independently so that the performance and efficiency of the network to transport multicast can be improved. Both solutions require some extensions to the BGP attributes used in [RFC7432]; see Section 4 for details.¶
The Assisted Replication solution described in this document is focused on NVO networks (hence its use of IP tunnels). MPLS transport networks are out of scope for this document. The PFLs solution MAY be used in NVO and MPLS transport networks.¶
Section 3 lists the requirements of the combined optimized ingress replication solution, whereas Sections 5 and 6 describe the Assisted Replication solution for non-selective and selective procedures, respectively. Section 7 provides the PFLs solution.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The following terminology is used throughout this document:¶
The ingress replication optimization solution specified in this document meets the following requirements:¶
The solution is compatible with [RFC7432] and [RFC8365] and has no impact on the Customer Edge (CE) procedures for BM traffic. In particular, the solution supports the following EVPN functions:¶
The ingress replication optimization solution specified in this document extends the Inclusive Multicast Ethernet Tag routes and attributes described in [RFC7432] so that an NVE/PE can signal its optimized ingress replication capabilities.¶
The Network Layer Reachability Information (NLRI) of the Inclusive Multicast Ethernet Tag route [RFC7432] is shown in Figure 2 and is used in this document without any modifications to its format. The PMSI Tunnel Attribute's general format as provided in [RFC7432] (which takes it from [RFC6514]) is used in this document; only a new tunnel type and new flags are specified, as shown in Figure 3.¶
+------------------------------------+
| RD (8 octets) |
+------------------------------------+
| Ethernet Tag ID (4 octets) |
+------------------------------------+
| IP Address Length (1 octet) |
+------------------------------------+
| Originating Router's IP Address |
| (4 or 16 octets) |
+------------------------------------+
0 1 2 3 4 5 6 7 +---------------------------------+ +--+--+--+--+--+--+--+--+ | Flags (1 octet) | -> |x |E |x | T |BM|U |L | +---------------------------------+ +--+--+--+--+--+--+--+--+ | Tunnel Type (1 octet) | T = Assisted Replication Type +---------------------------------+ BM = Broadcast and Multicast | MPLS Label (3 octets) | U = Unknown (unknown unicast) +---------------------------------+ x = unassigned | Tunnel Identifier (variable) | +---------------------------------+
The Flags field in Figure 3 is 8 bits long as per [RFC7902]. The Extension (E) flag was allocated by [RFC7902], and the Leaf Information Required (L) flag was allocated by [RFC6514]. This document defines the use of 4 bits of this Flags field:¶
Bits 5 and 6 are collectively referred to as the Pruned Flooding Lists (PFLs) flags.¶
The T field and PFLs flags are defined as follows:¶
T is the Assisted Replication Type field (2 bits), which defines the AR role of the advertising router:¶
The PFLs flags define the desired behavior of the advertising router for the different types of traffic:¶
Please refer to Section 11 for the IANA considerations related to the PMSI Tunnel Attribute flags.¶
In this document, the above Inclusive Multicast Ethernet Tag route (Figure 2) and PMSI Tunnel Attribute (Figure 3) can be used in two different modes for the same BD:¶
This route is used by the AR-REPLICATOR to advertise its AR capabilities, with the fields set as follows:¶
Originating Router's IP Address MUST be set to an IP address of the advertising router that is common to all the EVIs on the PE (usually this is a loopback address of the PE).¶
An NVE/PE configured as an AR-REPLICATOR for a BD MUST advertise a Replicator-AR route for the BD and MAY advertise a Regular-IR route. The advertisement of the Replicator-AR route will indicate to the AR-LEAFs which outer IP DA, i.e., which AR-IP, they need to use for IP-encapsulated BM frames that use Assisted Replication forwarding mode. The AR-REPLICATOR will forward an IP-encapsulated BM frame in Assisted Replication forwarding mode if the outer IP DA matches its AR-IP but will forward in Ingress Replication forwarding mode if the outer IP DA matches its IR-IP.¶
In addition, this document also uses the Leaf Auto-Discovery (Leaf A-D) route defined in [RFC9572] in cases where the selective AR mode is used. An AR-LEAF MAY send a Leaf A-D route in response to reception of a Replicator-AR route whose L flag is set. The Leaf A-D route is only used for selective AR, and the fields of such a route are set as follows:¶
Each AR-enabled node understands and processes the T (Assisted Replication type) field in the PMSI Tunnel Attribute (Flags field) of the routes and MUST signal the corresponding type (AR-REPLICATOR or AR-LEAF type) according to its administrative choice. An NVE/PE following this specification is not expected to set the Assisted Replication Type field to decimal 3 (which is a RESERVED value). If a route with the Assisted Replication Type field set to decimal 3 is received by an AR-REPLICATOR or AR-LEAF, the router will process the route as a Regular-IR route advertised by an RNVE.¶
Each node attached to the BD may understand and process the BM/U flags (PFLs flags). Note that these BM/U flags may be used to optimize the delivery of multi-destination traffic; their use SHOULD be an administrative choice and independent of the AR role. When the PFL capability is enabled, the BM/U flags can be used with the Regular-IR, Replicator-AR, and Leaf A-D routes.¶
Non-optimized ingress replication NVEs/PEs will be unaware of the new PMSI Tunnel Attribute flag definition as well as the new tunnel type (AR), i.e., non-upgraded NVEs/PEs will ignore the information contained in the Flags field or an unknown tunnel type (type AR in this case) for any Inclusive Multicast Ethernet Tag route.¶
Figure 4 illustrates an example NVO network where the non-selective AR function is enabled. Three different roles are defined for a given BD: AR-REPLICATOR, AR-LEAF, and RNVE. The solution is called "non-selective" because the chosen AR-REPLICATOR for a given flow MUST replicate the BM traffic to all the NVEs/PEs in the BD except for the source NVE/PE. NVO tunnels, i.e., IP tunnels, exist among all the PEs and NVEs in the diagram. The PEs and NVEs in the diagram have TSs or VMs connected to their ACs.¶
( )
(_ WAN _)
+---(_ _)----+
| (_ _) |
PE1 | PE2 |
+------+----+ +----+------+
TS1--+ (BD-1) | | (BD-1) +--TS2
|REPLICATOR | |REPLICATOR |
+--------+--+ +--+--------+
| |
+--+----------------+--+
| |
| |
+----+ VXLAN/NVGRE/MPLSoGRE +----+
| | IP Fabric | |
| | | |
NVE1 | +-----------+----------+ | NVE3
Hypervisor| ToR | NVE2 |Hypervisor
+---------+-+ +-----+-----+ +-+---------+
| (BD-1) | | (BD-1) | | (BD-1) |
| LEAF | | RNVE | | LEAF |
+--+-----+--+ +--+-----+--+ +--+-----+--+
| | | | | |
VM11 VM12 TS3 TS4 VM31 VM32
In AR BDs, such as BD-1 in Figure 4, BM traffic between two NVEs may follow a different path than unicast traffic. This solution recommends the replication of BM traffic through the AR-REPLICATOR node, whereas unknown/known unicast traffic will be delivered directly from the source node to the destination node without being replicated by any intermediate node.¶
Note that known unicast forwarding is not impacted by this solution, i.e., unknown unicast traffic SHALL follow the same path as known unicast traffic.¶
An AR-REPLICATOR is defined as an NVE/PE capable of replicating incoming BM traffic received on an overlay tunnel to other overlay tunnels and local ACs. The AR-REPLICATOR signals its role in the control plane and understands where the other roles (AR-LEAF nodes, RNVEs, and other AR-REPLICATORs) are located. A given AR-enabled BD service may have zero, one, or more AR-REPLICATORs. In our example in Figure 4, PE1 and PE2 are defined as AR-REPLICATORs. The following considerations apply to the AR-REPLICATOR role:¶
When a node defined as an AR-REPLICATOR receives a BM packet on an overlay tunnel, it will do a tunnel destination IP address lookup and apply the following procedures:¶
An AR-REPLICATOR MUST follow a data path implementation compatible with the following rules:¶
When an AR-REPLICATOR receives a BM packet on an overlay tunnel, it will check the destination IP address of the underlay IP header and:¶
The AR-REPLICATOR/LEAF nodes will build an unknown unicast flooding list composed of ACs and overlay tunnels to the IR-IP addresses of the remote nodes in the BD. Some of those overlay tunnels MAY be flagged as non-U (unknown unicast) receivers based on the U flag received from the remote nodes in the BD.¶
An AR-LEAF is defined as an NVE/PE that, given its poor replication performance, sends all the BM traffic to an AR-REPLICATOR that can replicate the traffic further on its behalf. It MAY signal its AR-LEAF capability in the control plane and understands where the other roles are located (AR-REPLICATORs and RNVEs). A given service can have zero, one, or more AR-LEAF nodes. In Figure 4, NVE1 and NVE3 (both residing in hypervisors) act as AR-LEAF nodes. The following considerations apply to the AR-LEAF role:¶
In a BD where there are no AR-REPLICATORs due to the AR-REPLICATORs being down or reconfigured, the AR-LEAF MUST use regular ingress replication based on the remote Regular-IR Inclusive Multicast Ethernet Tag routes as described in [RFC7432]. This may happen in the following cases:¶
In a service where there are one or more AR-REPLICATORs (based on the received Replicator-AR routes for the BD), the AR-LEAF can locally select which AR-REPLICATOR it sends the BM traffic to:¶
An AR-LEAF MUST follow a data path implementation compatible with the following rules:¶
The AR-LEAF nodes will build two flooding lists:¶
When an AR-LEAF receives a BM packet on an AC, it will check the AR-REPLICATOR-set:¶
An RNVE is defined as an NVE/PE without AR-REPLICATOR or AR-LEAF capabilities that does ingress replication as described in [RFC7432]. The RNVE does not signal any AR role and is unaware of the AR-REPLICATOR/LEAF roles in the BD. The RNVE will ignore the flags in the Regular-IR routes and will ignore the Replicator-AR routes (due to an unknown tunnel type in the PMSI Tunnel Attribute) and the Leaf A-D routes (due to the IP-address-specific Route Target).¶
This role provides EVPNs with the backward compatibility required in optimized ingress replication BDs. In Figure 4, NVE2 acts as an RNVE.¶
Figure 5 is used to describe the selective AR solution.¶
( )
(_ WAN _)
+---(_ _)----+
| (_ _) |
PE1 | PE2 |
+------+----+ +----+------+
TS1--+ (BD-1) | | (BD-1) +--TS2
|REPLICATOR | |REPLICATOR |
+--------+--+ +--+--------+
| |
+--+----------------+--+
| |
| |
+----+ VXLAN/NVGRE/MPLSoGRE +----+
| | IP Fabric | |
| | | |
NVE1 | +-----------+----------+ | NVE3
Hypervisor| ToR | NVE2 |Hypervisor
+---------+-+ +-----+-----+ +-+---------+
| (BD-1) | | (BD-1) | | (BD-1) |
|LEAF-set-1 | |LEAF-set-1 | |LEAF-set-2 |
+--+-----+--+ +--+-----+--+ +--+-----+--+
| | | | | |
VM11 VM12 TS3 TS4 VM31 VM32
The solution is called "selective" because a given AR-REPLICATOR MUST replicate the BM traffic to only the AR-LEAFs that requested the replication (as opposed to all the AR-LEAF nodes) and MUST replicate the BM traffic to the RNVEs (if there are any). The same AR roles as those defined in Sections 4 and 5 are used here; however, the procedures are different.¶
The selective AR procedures create multiple AR-LEAF-sets in the EVPN BD and build single-hop trees among AR-LEAFs of the same set (AR-LEAF->AR-REPLICATOR->AR-LEAF) and two-hop trees among AR-LEAFs of different sets (AR-LEAF->AR-REPLICATOR->AR-REPLICATOR->AR-LEAF). Compared to the selective solution, the non-selective AR method assumes that all the AR-LEAFs of the BD are in the same set and always creates single-hop trees among AR-LEAFs. While the selective solution is more efficient than the non-selective solution in multi-stage IP fabrics, the trade-off is additional signaling and an additional outer source IP address lookup.¶
The following subsections describe the differences in the procedures for AR-REPLICATORs/LEAFs compared to the non-selective AR solution. There are no changes applicable to RNVEs.¶
In our example in Figure 5, PE1 and PE2 are defined as selective AR-REPLICATORs. The following considerations apply to the selective AR-REPLICATOR role:¶
The selective AR-REPLICATOR MUST follow the procedures described in Section 5.1, except for the following differences:¶
When a node defined and operating as a selective AR-REPLICATOR receives a packet on an overlay tunnel, it will do a tunnel destination IP lookup, and if the destination IP address is the AR-REPLICATOR AR-IP address, the node MUST replicate the packet to:¶
A selective AR-REPLICATOR data path implementation MUST be compatible with the following rules:¶
The selective AR-REPLICATORs will build two flooding lists:¶
Composed of ACs and overlay tunnels to the remote nodes in the BD, always using the IR-IPs in the tunnel destination IP addresses.¶
Composed of ACs, a selective AR-LEAF-set, and a selective AR-REPLICATOR-set, where:¶
When a selective AR-REPLICATOR receives a BM packet on an overlay tunnel, it will check the destination and source IPs of the underlay IP header and:¶
A selective AR-LEAF chooses a single selective AR-REPLICATOR per BD and:¶
In the example in Figure 5, we consider NVE1/NVE2/NVE3 as selective AR-LEAFs. NVE1 selects PE1 as its selective AR-REPLICATOR. If that is so, NVE1 will send all its BM traffic for BD-1 to PE1. If other AR-LEAFs/REPLICATORs send BM traffic, NVE1 will receive that traffic from PE1. A selective AR-LEAF and a non-selective AR-LEAF behave differently, as follows:¶
In a service where there is more than one selective AR-REPLICATOR, the selective AR-LEAF MUST locally select a single selective AR-REPLICATOR for the BD. Once selected:¶
All the AR-LEAFs in a BD are expected to be configured as either selective or non-selective. A mix of selective and non-selective AR-LEAFs SHOULD NOT coexist in the same BD. If a non-selective AR-LEAF is present, its BM traffic sent to a selective AR-REPLICATOR will not be replicated to other AR-LEAFs that are not in its selective AR-LEAF-set.¶
A selective AR-LEAF MUST follow a data path implementation compatible with the following rules:¶
The selective AR-LEAF nodes will build two flooding lists:¶
In addition to AR, the second optimization supported by the ingress replication optimization solution specified in this document is the ability of all the BD nodes to signal PFLs. As described in Section 4, an EVPN node can signal a given value for the BM and U PFLs flags in the Regular-IR, Replicator-AR, or Leaf A-D routes, where:¶
The ability to signal and process these PFLs flags SHOULD be an administrative choice. If a node is configured to process the PFLs flags, upon receiving a non-zero PFLs flag for a route, an NVE/PE will add the corresponding flag to the created overlay tunnel in the flooding list. When replicating a BM packet in the context of a flooding list, the NVE/PE will skip the overlay tunnels marked with the flag BM = 1, since the NVEs/PEs at the end of those tunnels are not expecting BM packets. Similarly, when replicating unknown unicast packets, the NVE/PE will skip the overlay tunnels marked with U = 1.¶
An NVE/PE not following this document or not configured for this optimization will ignore any of the received PFLs flags. An AR-LEAF or RNVE receiving BUM traffic on an overlay tunnel MUST replicate the traffic to its local ACs, regardless of the BM/U flags on the overlay tunnels.¶
This optimization MAY be used along with the Assisted Replication solution.¶
In order to illustrate the use of the PFLs solution, we will assume that BD-1 in Figure 4 is optimized ingress replication enabled and:¶
NVE1 and NVE3 are administratively configured as AR-LEAF nodes due to their low-performance software-based replication capabilities. They will advertise a Regular-IR route with type AR-LEAF. Assuming that both NVEs advertise all of the attached VMs' MAC and IP addresses in EVPNs as soon as they come up and these NVEs do not have any VMs interested in multicast applications, they will be configured to signal BM/U flags = 11 for BD-1. That is, neither NVE1 nor NVE3 is interested in receiving BM or unknown unicast traffic, since:¶
Based on the above assumptions, the following forwarding behavior will take place:¶
The procedures explained in Sections 5 and 6 assume that the AR-REPLICATOR can use two local routable IP addresses to terminate and originate NVO tunnels, i.e., IR-IP and AR-IP addresses. This is usually the case for PE-based AR-REPLICATOR nodes.¶
In some cases, the AR-REPLICATOR node does not support more than one IP address to terminate and originate NVO tunnels, i.e., the IR-IP and AR-IP are the same IP addresses. This may be the case in some software-based or low-end AR-REPLICATOR nodes. If this is the case, the procedures provided in Sections 5 and 6 MUST be modified in the following way:¶
The rest of the procedures will follow those described in Sections 5 and 6.¶
This section extends the procedures for the cases where two or more AR-LEAF nodes are attached to the same ES and two or more AR-REPLICATOR nodes are attached to the same ES in the BD. The mixed case -- where an AR-LEAF node and an AR-REPLICATOR node are attached to the same ES -- would require extended procedures that are out of scope for this document.¶
If a VXLAN or NVGRE is used and if the split-horizon is based on the tunnel source IP address and "local bias" as described in [RFC8365], the split-horizon check will not work if an ES is shared between two AR-LEAF nodes, and the AR-REPLICATOR replaces the tunnel source IP address of the packets with its own AR-IP.¶
In order to be compatible with the source IP address split-horizon check, the AR-REPLICATOR MAY keep the original received tunnel source IP address when replicating packets to a remote AR-LEAF or RNVE. This will allow AR-LEAF nodes to apply split-horizon check procedures for BM packets before sending them to the local ES. Even if the AR-LEAF's source IP address is preserved when replicating to AR-LEAFs or RNVEs, the AR-REPLICATOR MUST always use its IR-IP as the source IP address when replicating to other AR-REPLICATORs.¶
When EVPNs are used for MPLSoGRE or MPLSoUDP, the ESI-label-based split-horizon procedure provided in [RFC7432] will not work for multihomed ESs defined on AR-LEAF nodes. Local bias is recommended in this case, as it is in the case of a VXLAN or NVGRE as explained above. The local-bias and tunnel source IP address preservation mechanisms provide the required split-horizon behavior in non-selective or selective AR.¶
Note that if the AR-REPLICATOR implementation keeps the received tunnel source IP address, the use of unicast Reverse Path Forwarding (uRPF) checks in the IP fabric based on the tunnel source IP address MUST be disabled.¶
AR-REPLICATOR nodes attached to the same all-active ES will follow local-bias procedures [RFC8365] as follows:¶
The security considerations in [RFC7432] and [RFC8365] apply to this document. The security considerations related to the Leaf A-D route in [RFC9572] apply too.¶
In addition, the Assisted Replication method introduced by this document may introduce some new risks that could affect the successful delivery of BM traffic. Unicast traffic is not affected by Assisted Replication (although unknown unicast traffic is affected by the procedures for PFLs). The forwarding of BM traffic is modified, and BM traffic from the AR-LEAF nodes will be drawn toward AR-REPLICATORs in the BD. An AR-LEAF will forward BM traffic to its selected AR-REPLICATOR; therefore, an attack on the AR-REPLICATOR could impact the delivery of the BM traffic using that node. Also, an attack on the AR-REPLICATOR and any change to the advertised AR type will modify the selections made by the AR-LEAF nodes. If no other AR-REPLICATOR is selected, the AR-LEAF nodes will be forced to use Ingress Replication forwarding mode, which will impact their performance, since the AR-LEAF nodes are usually NVEs/PEs with poor replication performance.¶
This document introduces the ability of the AR-REPLICATOR to forward traffic received on an overlay tunnel to another overlay tunnel. The reader may determine that this introduces the risk of BM loops -- that is, an AR-LEAF receiving a BM-encapsulated packet that the AR-LEAF originated in the first place due to one or two AR-REPLICATORs "looping" the BM traffic back to the AR-LEAF. Following the procedures provided in this document will prevent these BM loops, since the AR-REPLICATOR will always forward the BM traffic using the correct tunnel IP DA (or the correct VNI in the case of single-IP AR-REPLICATORs), which instructs the remote nodes regarding how to forward the traffic. This is true for both the Non-selective and Selective modes defined in this document. However, incorrect implementation of the procedures provided in this document may lead to those unexpected BM loops.¶
The Selective mode provides a multi-stage replication solution, where proper configuration of all the AR-REPLICATORs will prevent any issues. A mix of mistakenly configured selective and non-selective AR-REPLICATORs in the same BD could theoretically create packet duplication in some AR-LEAFs; however, this document specifies a fallback solution -- falling back to Non-selective mode in cases where the AR-REPLICATORs advertised an inconsistent AR mode.¶
This document allows the AR-REPLICATOR to preserve the tunnel source IP address of the AR-LEAF (as an option) when forwarding BM packets from an overlay tunnel to another overlay tunnel. Preserving the AR-LEAF source IP address makes the local-bias filtering procedures possible for AR-LEAF nodes that are attached to the same ES. If the AR-REPLICATOR does not preserve the AR-LEAF source IP address, AR-LEAF nodes attached to all-active ESs will cause packet duplication on the multihomed CE.¶
The AR-REPLICATOR nodes are, by design, using more bandwidth than PEs [RFC7432] or NVEs [RFC8365] would use. Certain network events or unexpected low performance may exceed the AR-REPLICATOR's local bandwidth and cause service disruption.¶
Finally, PFLs (Section 7) should be used with care. Intentional or unintentional misconfiguration of the BDs on a given leaf node may result in the leaf not receiving the required BM or unknown unicast traffic.¶
IANA has allocated the following Border Gateway Protocol (BGP) parameters:¶
| Value | Meaning | Reference |
|---|---|---|
| 0x0A | Assisted Replication Tunnel | RFC 9574 |
| Value | Name | Reference |
|---|---|---|
| 3-4 | Assisted Replication Type (T) | RFC 9574 |
| 5 | Broadcast and Multicast (BM) | RFC 9574 |
| 6 | Unknown (U) | RFC 9574 |
The authors would like to thank Neil Hart, David Motz, Dai Truong, Thomas Morin, Jeffrey Zhang, Shankar Murthy, and Krzysztof Szarkowicz for their valuable feedback and contributions. Also, thanks to John Scudder for his thorough review, which improved the quality of the document significantly.¶
In addition to the authors listed on the front page, the following people also contributed to this document and should be considered coauthors:¶