draft-ietf-bess-evpn-inter-subnet-forwarding-03.txt   draft-ietf-bess-evpn-inter-subnet-forwarding-04.txt 
L2VPN Workgroup A. Sajassi, Ed. L2VPN Workgroup A. Sajassi, Ed.
INTERNET-DRAFT S. Salam INTERNET-DRAFT S. Salam
Intended Status: Standards Track S. Thoria Intended Status: Standards Track S. Thoria
Cisco Cisco
J. Drake J. Drake
Juniper Juniper
J. Rabadan J. Rabadan
Nokia Nokia
L. Yong
Huawei
Expires: August 8, 2017 February 8, 2017 Expires: January 2, 2019 July 2, 2018
Integrated Routing and Bridging in EVPN Integrated Routing and Bridging in EVPN
draft-ietf-bess-evpn-inter-subnet-forwarding-03 draft-ietf-bess-evpn-inter-subnet-forwarding-04
Abstract Abstract
EVPN provides an extensible and flexible multi-homing VPN solution EVPN provides an extensible and flexible multi-homing VPN solution
for intra-subnet connectivity among hosts/VMs over an MPLS/IP over an MPLS/IP network for intra-subnet connectivity among Tenant
network. However, there are scenarios in which inter-subnet Systems and End Devices that can be physical or virtual. However,
forwarding among hosts/VMs across different IP subnets is required, there are scenarios for which there is a need for a dynamic and
while maintaining the multi-homing capabilities of EVPN. This efficient inter-subnet connectivity among these Tenant Systems and
document describes an Integrated Routing and Bridging (IRB) solution End Devices while maintaining the multi-homing capabilities of EVPN.
based on EVPN to address such requirements. This document describes an Integrated Routing and Bridging (IRB)
solution based on EVPN to address such requirements.
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as other groups may also distribute working documents as
Internet-Drafts. Internet-Drafts.
skipping to change at page 2, line 23 skipping to change at page 2, line 22
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Inter-Subnet Forwarding Scenarios . . . . . . . . . . . . . . . 6 2 EVPN PE Model for IRB Operation . . . . . . . . . . . . . . . . 7
2.1 Switching among IP subnets within a DC . . . . . . . . . . . 7 3 Symmetric and Asymmetric IRB . . . . . . . . . . . . . . . . . 8
2.2 Switching among IP subnets in different DCs without GW . . . 8 3.1 IRB Interface and its MAC & IP addresses . . . . . . . . . . 11
2.3 Switching among IP subnets in different DCs with GW . . . . 8 3.2 Symmetric IRB Procedures . . . . . . . . . . . . . . . . . . 12
2.4 Switching among IP subnets spread across IP-VPN and EVPN 3.2.1 Control Plane - Ingress PE . . . . . . . . . . . . . . . 12
networks with GW . . . . . . . . . . . . . . . . . . . . . . 8 3.2.2 Control Plane - Egress PE . . . . . . . . . . . . . . . 13
3 Default L3 Gateway for Tenant System . . . . . . . . . . . . . . 9 3.2.3 Data Plane - Ingress PE . . . . . . . . . . . . . . . . 14
3.1 Homogeneous Environment . . . . . . . . . . . . . . . . . . 9 3.2.4 Data Plane - Egress PE . . . . . . . . . . . . . . . . . 14
3.2 Heterogeneous Environment . . . . . . . . . . . . . . . . . 10 3.3 Asymmetric IRB Procedures . . . . . . . . . . . . . . . . . 15
4 Operational Models for Asymmetric Inter-Subnet Forwarding . . . 10 3.3.1 Control Plane - Ingress PE . . . . . . . . . . . . . . . 15
4.1 Among EVPN NVEs within a DC . . . . . . . . . . . . . . . . 10 3.3.2 Control Plane - Egress PE . . . . . . . . . . . . . . . 15
4.2 Among EVPN NVEs in Different DCs Without GW . . . . . . . . 11 3.3.3 Data Plane - Ingress PE . . . . . . . . . . . . . . . . 16
4.3 Among EVPN NVEs in Different DCs with GW . . . . . . . . . . 13 3.3.4 Data Plane - Egress PE . . . . . . . . . . . . . . . . . 17
4.4 Among IP-VPN Sites and EVPN NVEs with GW . . . . . . . . . . 14 4 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.5 Use of Centralized Gateway . . . . . . . . . . . . . . . . . 15 4.1 Router's MAC Extended Community . . . . . . . . . . . . . . 17
5 Operational Models for Symmetric Inter-Subnet Forwarding . . . . 16 5 Operational Models for Symmetric Inter-Subnet Forwarding . . . . 18
5.1 IRB forwarding on NVEs for Tenant Systems . . . . . . . . . 16 5.1 IRB forwarding on NVEs for Tenant Systems . . . . . . . . . 18
5.1.1 Control Plane Operation . . . . . . . . . . . . . . . . 17 5.1.1 Control Plane Operation . . . . . . . . . . . . . . . . 19
5.1.2 Data Plane Operation - Inter Subnet . . . . . . . . . . 18 5.1.2 Data Plane Operation - Inter Subnet . . . . . . . . . . 21
5.1.3 TS Move Operation . . . . . . . . . . . . . . . . . . . 19 5.1.3 TS Move Operation . . . . . . . . . . . . . . . . . . . 22
5.2 IRB forwarding on NVEs for Subnets behind Tenant Systems . . 20 5.2 IRB forwarding on NVEs for Subnets behind Tenant Systems . . 23
5.2.1 Control Plane Operation . . . . . . . . . . . . . . . . 22 5.2.1 Control Plane Operation . . . . . . . . . . . . . . . . 24
5.2.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 23 5.2.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 25
6 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 24 6 Inter-Subnet DCI Scenarios . . . . . . . . . . . . . . . . . . 26
6.1 Router's MAC Extended Community . . . . . . . . . . . . . . 24 6.1 Switching among IP subnets in different DCs without GW . . . 27
6.2 Switching among IP subnets in different DCs with GW . . . . 29
7 TS Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . 24 7 TS Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . 31
7.1 TS Mobility & Optimum Forwarding for TS Outbound Traffic . . 24 7.1 TS Mobility & Optimum Forwarding for TS Outbound Traffic . . 31
7.2 TS Mobility & Optimum Forwarding for TS Inbound Traffic . . 24 7.2 TS Mobility & Optimum Forwarding for TS Inbound Traffic . . 31
7.2.1 Mobility without Route Aggregation . . . . . . . . . . . 25 7.2.1 Mobility without Route Aggregation . . . . . . . . . . . 31
8 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 8 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 32
9 Security Considerations . . . . . . . . . . . . . . . . . . . . 25 9 Security Considerations . . . . . . . . . . . . . . . . . . . . 32
10 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 10 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32
11 References . . . . . . . . . . . . . . . . . . . . . . . . . . 25 11 References . . . . . . . . . . . . . . . . . . . . . . . . . . 32
11.1 Normative References . . . . . . . . . . . . . . . . . . . 25 11.1 Normative References . . . . . . . . . . . . . . . . . . . 32
11.2 Informative References . . . . . . . . . . . . . . . . . . 26 11.2 Informative References . . . . . . . . . . . . . . . . . . 32
12 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 26 12 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 33
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 27 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 33
Terminology Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
document are to be interpreted as described in RFC 2119 [RFC2119]. "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
Broadcast Domain: In a bridged network, the broadcast domain AC: Attachment Circuit.
corresponds to a Virtual LAN (VLAN), where a VLAN is typically
represented by a single VLAN ID (VID) but can be represented by
several VIDs where Shared VLAN Learning (SVL) is used per [802.1Q].
EVI : An EVPN instance spanning the Provider Edge (PE) devices ARP: Address Resolution Protocol.
participating in that EVPN
IRB: Integrated Routing and Bridging BD: Broadcast Domain. As per [RFC7432], an EVI consists of a single
or multiple BDs. In case of VLAN-bundle and VLAN-based service models
(see [RFC7432]), a BD is equivalent to an EVI. In case of VLAN-aware
bundle service model, an EVI contains multiple BDs. Also, in this
document, BD and subnet are equivalent terms.
BD Route Target: refers to the Broadcast Domain assigned Route Target
[RFC4364]. In case of VLAN-aware bundle service model, all the BD
instances in the MAC-VRF share the same Route Target.
BT: Bridge Table. The instantiation of a BD in a MAC-VRF, as per
[RFC7432].
DGW: Data Center Gateway.
Ethernet A-D route: Ethernet Auto-Discovery (A-D) route, as per
[RFC7432].
Ethernet NVO tunnel: refers to Network Virtualization Overlay tunnels
with Ethernet payload. Examples of this type of tunnels are VXLAN or
GENEVE.
EVI: EVPN Instance spanning the NVE/PE devices that are participating
on that EVPN, as per [RFC7432].
EVPN: Ethernet Virtual Private Networks, as per [RFC7432].
GRE: Generic Routing Encapsulation.
GW IP: Gateway IP Address.
IPL: IP Prefix Length.
IP NVO tunnel: it refers to Network Virtualization Overlay tunnels
with IP payload (no MAC header in the payload).
IP-VRF: A VPN Routing and Forwarding table for IP routes on an
NVE/PE. The IP routes could be populated by EVPN and IP-VPN address
families. An IP-VRF is also an instantiation of a layer 3 VPN in an
NVE/PE.
IRB: Integrated Routing and Bridging interface. It connects an IP-VRF
to a BD (or subnet).
MAC-VRF: A Virtual Routing and Forwarding table for Media Access MAC-VRF: A Virtual Routing and Forwarding table for Media Access
Control (MAC) addresses on a PE for an EVI Control (MAC) addresses on an NVE/PE, as per [RFC7432]. A MAC-VRF is
also an instantiation of an EVI in an NVE/PE.
Bridge Table: An instantiation of a broadcast domain on a MAC-VRF ML: MAC address length.
IP-VRF: A Virtual Routing and Forwarding table for IP addresses on a ND: Neighbor Discovery Protocol.
PE that is associated with one or more EVIs
IRB Interface: A virtual interface that connects a bridge table in a NVE: Network Virtualization Edge.
MAC-VRF to an IP-VRF in an NVE.
NVE: Network Virtualization Endpoint GENEVE: Generic Network Virtualization Encapsulation, [GENEVE].
TS: Tenant System NVO: Network Virtualization Overlays.
Ethernet NVO tunnel: It refers to Network Virtualization Overlay RT-2: EVPN route type 2, i.e., MAC/IP advertisement route, as defined
tunnels with Ethernet payload. Example of this type of tunnels are in [RFC7432].
VxLAN and NvGRE.
IP NVO tunnel: It refers to Network Virtualization Overlay tunnels RT-5: EVPN route type 5, i.e., IP Prefix route. As defined in Section
with IP payload (no MAC header in the payload). Examples of IP NVO 3 of [EVPN-PREFIX].
tunnels are VxLAN GPE or MPLSoGRE (both with IP payload).
SBD: Supplementary Broadcast Domain. A BD that does not have any ACs,
only IRB interfaces, and it is used to provide connectivity among all
the IP-VRFs of the tenant. The SBD is only required in IP-VRF- to-IP-
VRF use-cases (see Section 4.4.).
SN: Subnet.
TS: Tenant System.
VA: Virtual Appliance.
VNI: Virtual Network Identifier. As in [RFC8365], the term is used as
a representation of a 24-bit NVO instance identifier, with the
understanding that VNI will refer to a VXLAN Network Identifier in
VXLAN, or Virtual Network Identifier in GENEVE, etc. unless it is
stated otherwise.
VTEP: VXLAN Termination End Point, as in [RFC7348].
VXLAN: Virtual Extensible LAN, as in [RFC7348].
This document also assumes familiarity with the terminology of
[RFC7432], [RFC8365] and [RFC7365].
1 Introduction 1 Introduction
EVPN provides an extensible and flexible multi-homing VPN solution EVPN provides an extensible and flexible multi-homing VPN solution
for intra-subnet connectivity among Tenant Systems (TS's) over an over an MPLS/IP network for intra-subnet connectivity among Tenant
MPLS/IP network; where, an IP subnet is represented by an EVI for a Systems (TS's) and End Devices that can be physical or virtual; where
VLAN-based service or by an <EVI, VLAN> for a VLAN-aware bundle an IP subnet is represented by an EVI for a VLAN-based service or by
service. However, there are scenarios where, in addition to intra- an <EVI, VLAN> for a VLAN-aware bundle service. However, there are
subnet forwarding, inter-subnet forwarding is required among TS's scenarios for which there is a need for a dynamic and efficient
across different IP subnets at EVPN PE nodes, also known as EVPN NVE inter-subnet connectivity among these Tenant Systems and End Devices
nodes throughout this document, while maintaining the multi-homing while maintaining the multi-homing capabilities of EVPN. This
capabilities of EVPN. This document describes an Integrated Routing document describes an Integrated Routing and Bridging (IRB) solution
and Bridging (IRB) solution based on EVPN to address such based on EVPN to address such requirements.
requirements.
The inter-subnet communication is traditionally achieved at The inter-subnet communication is traditionally achieved at
centralized L3 Gateway (L3GW) nodes where all the inter-subnet centralized L3 Gateway (L3GW) nodes where all the inter-subnet
communication policies are enforced. When two Tenant Systems (TS's) communication policies are enforced. When two Tenant Systems (TS's)
belonging to two different subnets connected to the same PE node, belonging to two different subnets connected to the same PE node,
wanted to talk to each other, their traffic needed to be back hauled wanted to communicate with each other, their traffic needed to be
from the PE node all the way to the centralized gateway nodes where back hauled from the PE node all the way to the centralized gateway
inter-subnet switching is performed and then back to the PE node. For nodes where inter-subnet switching is performed and then back to the
today's large multi-tenant data center, this scheme is very PE node. For today's large multi-tenant data center, this scheme is
inefficient and sometimes impractical. very inefficient and sometimes impractical.
In order to overcome the drawback of centralized approach, IRB
functionality is needed on the PE nodes (i.e., NVE devices) as close
to TS as possible to avoid hair pinning of user traffic
unnecessarily. Under this design, all traffic between hosts attached
to one NVE can be routed and bridged locally, thus avoiding traffic
hair-pinning issue of the centralized L3GW.
There can be scenarios where both centralized and distributed In order to overcome the drawback of centralized L3GW approach, IRB
approaches may be preferred simultaneously. For example, to allow functionality is needed on the PE nodes (also referred to as EVPN
NVEs to switch inter-subnet traffic belonging to one tenant or one NVEs) attached to TS's in order to avoid inefficient forwarding of
security zone locally; whereas, to back haul inter-subnet traffic tenant traffic (i.e., avoid back-hauling and hair-pinning). A PE with
belonging to two different tenants or security zones to the IRB capability, can not only locally bridged the tenant intra-subnet
centralized gateway nodes and perform switching there after the traffic but also can locally route the tenant inter-subnet traffic on
traffic is subjected to Firewall (FW) or Deep Packet Inspection a packet by packet basis thus meeting the requirements for both intra
(DPI). and inter-subnet forwarding and avoiding non-optimum traffic
forwarding associate with centralized L3GW approach.
Some TS's run non-IP protocols in conjunction with their IP traffic. Some TS's run non-IP protocols in conjunction with their IP traffic.
Therefore, it is important to handle both kinds of traffic optimally Therefore, it is important to handle both kinds of traffic optimally
- e.g., to bridge non-IP traffic and to route IP traffic. - e.g., to bridge non-IP and intra-subnet traffic and to route inter-
subnet IP traffic. Therefore, the solution needs to meet the
Therefore, the solution needs to meet the following requirements: following requirements:
R1: The solution MUST allow for inter-subnet traffic to be locally
switched at NVEs.
R2: The solution MUST allow for both inter-subnet and intra-subnet R1: The solution MUST allow for both inter-subnet and intra-subnet
traffic belonging to the same tenant to be locally routed and bridged traffic belonging to the same tenant to be locally routed and bridged
respectively. The solution MUST provide IP routing for inter-subnet respectively. The solution MUST provide IP routing for inter-subnet
traffic and Ethernet Bridging for intra-subnet traffic. traffic and Ethernet Bridging for intra-subnet traffic.
R3: The solution MUST support bridging of non-IP traffic. R2: The solution MUST support bridging of non-IP traffic.
R4: The solution MUST allow inter-subnet switching to be disabled on R3: The solution MUST allow inter-subnet switching to be disabled on
a per VLAN basis on NVEs where the traffic needs to be back hauled to a per VLAN basis on PEs where the traffic needs to be back hauled to
another node (i.e., for performing FW or DPI functionality). another node (i.e., for performing FW or DPI functionality).
2 Inter-Subnet Forwarding Scenarios 2 EVPN PE Model for IRB Operation
The inter-subnet forwarding scenarios performed by an EVPN NVE can be Since this document discusses IRB operation in relationship to EVPN
divided into the following five categories. The last scenario, along MAC-VRF, IP-VRF, EVI, Bridge Domain (BD), Bridge Table (BT), and IRB
with its corresponding solution, are described in [EVPN-IPVPN- interfaces, it is important to understand the relationship among
INTEROP]. The first four scenarios are covered in this document. these components. Therefore, the following PE model is demonstrated
below to a) describe these components and b) illustrate the
relationship among them.
1. Switching among IP subnets within a DC using EVPN +-------------------------------------------------------------+
| |
| +------------------+ IRB PE |
| Attachment | +------------------+ |
| Circuit(AC1) | | +----------+ | MPLS/NVO tnl
----------------------*Bridge | | +-----
| | | |Table(BT1)| | +-----------+ / \ \
| | | | *---------* |<--> |Eth|
| | | |Eth-Tag x | |IRB1| | \ / /
| | | +----------+ | | | +-----
| | | ... | | IP-VRF1 | |
| | | +----------+ | | RD2/RT2 |MPLS/NVO tnl
| | | |Bridge | | | | +-----
| | | |Table(BT2)| |IRB2| | / \ \
| | | | *---------* |<--> |IP |
----------------------*Eth-Tag y | | +-----------+ \ / /
| AC2 | | +----------+ | +-----
| | | MAC-VRF1 | |
| +-+ RD1/RT1 | |
| +------------------+ |
| |
| |
+-------------------------------------------------------------+
2. Switching among IP subnets in different DCs using EVPN without GW Figure 1: EVPN IRB PE Model
3. Switching among IP subnets in different DCs using EVPN with GW A tenant needing IRB services on a PE, requires an IP Virtual Routing
and Forwarding table (IP-V RF) along with one or more MAC Virtual
Routing and Forwarding tables (MAC-VRFs). An IP-VRF, as defined in
[RFC4364], is the instantiation of an IPVPN in a PE. A MAC-VRF, as
defined in [RFC7432], is the instantiation of an EVI (EVPN Instancce)
in a PE. A MAC-VRF can consists of one or more Bridge Tables (BTs)
where each BT corresponds to a VLAN (broadcast domain - BD). If the
service interface for the EVPN PE is configured in VLAN-Based mode
(i.e., section 6.1 of [RFC7432]), then there is only a single BT per
MAC-VRF (per EVI) - i.e., there is only one tenant VLAN per EVI.
However, if the service interface for the EVPN PE is configured in
VLAN-Aware Bundle mode (i.e., section 6.3 of [RFC7432]), then there
are several BTs per MAC-VRF (per EVI) - i.e., there are several
tenant VLANs per EVI.
4. Switching among IP subnets spread across IP-VPN and EVPN networks Each BT is connected to a IP-VRF via a L3 interface called IRB
with GW interface. Since a single tenant subnet is typically (and in this
document) represented by a VLAN (and thus supported by a single BT),
for a given tenant there are as many BTs as there are subnets and
thus there are also as many IRB interfaces between the tenant IP-VRF
and the associated BTs as shown in the PE model above.
5. Switching among IP subnets spread across IP-VPN and EVPN networks IP-VRF is identified by its corresponding route target and route
without GW distinguisher and MAC-VRF is also identified by its corresponding
route target and route distinguisher. If operating in EVPN VLAN-Based
mode, then a receiving PE that receives an EVPN route with MAC-VRF
route target can identify the corresponding BT; however, if operating
in EVPN VLAN-Aware Bundle mode, then the receiving PE needs both the
MAC-VRF route target and VLAN ID in order to identify the
corresponding BT.
In the above scenario, the term "GW" refers to the case where a node 3 Symmetric and Asymmetric IRB
situated at the WAN edge of the data center network behaves as a
default gateway (GW) for all the destinations that are outside the
data center. The absence of GW refers to the scenario where NVEs
within a data center maintain individual (host) routes that are
outside of the data center.
In the case (4), the WAN edge node also performs route aggregation This document defines and describes two types of IRB solutions -
for all the destinations within its own data center, and acts as an namely symmetric and asymmetric IRB. In symmetric IRB as its name
interworking unit between EVPN and IP VPN (it implements both EVPN implies, the lookup operation is symmetric at both ingress and egress
and IP-VPN functionality). PEs - i.e., both ingress and egress PEs perform lookups on both TS's
MAC and IP addresses - i.e., ingress PE performs lookup on
destination TS's MAC address followed by its IP address and egress PE
performs lookup on destination TS's IP address followed by its MAC
address as depicted in figure 2.
+---+ Enterprise Site 1 Ingress PE Egress PE
|PE1|----- H1 +-------------------+ +------------------+
+---+ | | | |
/ | +-> IP-VFF ----|---->---|-----> IP-VRF -+ |
,---------. Enterprise Site 2 | | | | | |
,' `. +---+ | BT1 BT2 | | BT3 BT2 |
,---------. /( MPLS/IP )---|PE2|----- H2 | | | | | |
' DCN 3 `./ `. Core ,' +---+ | ^ | | v |
`-+------+' `-+------+' | | | | | |
__/__ / / \ \ +-------------------+ +------------------+
:NVE4 : +---+ \ \ ^ |
'-----' ,----|GW |. \ \ | |
| ,' +---+ `. ,---------. TS1->-+ +->-TS2
TS6 ( DCN 1 ) ,' `. Figure 2: Symmetric IRB
`. ,' ( DCN 2 )
`-+------+' `. ,'
__/__ `-+------+'
:NVE1 : __/__ __\__
'-----' :NVE2 : :NVE3 :
| | '-----' '-----'
TS1 TS2 | | |
TS3 TS4 TS5
Figure 2: Interoperability Use-Cases In symmetric IRB as shown in figure-2, the inter-subnet forwarding
between two PEs is done between their associated IP-VRFs. Therefore,
the tunnel connecting these IP-VRFs can be either IP-only tunnel (in
case of MPLS or GENEVE encapsulation) or Ethernet NVO tunnel (in case
of VxLAN encapsulation). If it is Ethernet NOV tunnel, the TS's IP
packet is encapsulated in an Ethernet header consisting of ingress
and egress PEs MAC addresses - i.e., there is no need for ingress PE
to use the destination TS's MAC address. Therefore, in symmetric IRB,
there is no need for the ingress PE to hold destination TS's IP and
MAC association in its ARP table. Each PE participating in symmetric
IRB only maintains ARP entries for locally connected hosts and
maintain MAC-VRFs/BTs for only locally configured subnets.
In what follows, we will describe scenarios 1 through 4 in more In asymmetric IRB, the lookup operation is asymmetric and the ingress
detail. PE performs three lookups; whereas the egress PE performs a single
lookup - i.e., the ingress PE performs lookups on destination TS's
MAC address, followed by its IP address, followed by its MAC address
again; whereas, the egress PE performs just a single lookup on
destination TS's MAC address as depicted in figure 3 below.
2.1 Switching among IP subnets within a DC Ingress PE Egress PE
+-------------------+ +------------------+
| | | |
| +-> IP-VFF -> | | IP-VRF |
| | | | | |
| BT1 BT2 | | BT3 BT2 |
| | | | | | | |
| | +--|--->----|--------------+ | |
| | | | v |
+-------------------+ +----------------|-+
^ |
| |
TS1->-+ +->-TS2
Figure 3: Asymmetric IRB
In this scenario, connectivity is required between TS's in the same In asymmetric IRB as shown in figure-2, the inter-subnet forwarding
data center, where those hosts belong to different IP subnets. All between two PEs is done between their associated MAC-VRFs/BTs.
these subnets belong to the same tenant or are part of the same IP Therefore, the MPLS or NVO tunnel used for inter-subnet forwarding
VPN. Each subnet is associated with a single EVI (or <EVI,VLAN>) MUST be of type Ethernet. Since at the egress PE only MAC lookup is
realized by a collection of MAC-VRFs (one per NVE) residing on the performed (e.g., no IP lookup), the TS's IP packet needs to be
NVEs configured for that EVI. encapsulated with the destination TS's MAC address. In order for
ingress PE to perform such encapsulation, it needs to maintain TS's
IP and MAC address association in its ARP table. Furthermore, it
needs to maintain destination TS's MAC address in the corresponding
BT even though it does not have the corresponding subnet locally
configured. In other words, each PE participating in asymmetric IRB
MUST maintain ARP entries for remote hosts (hosts connected to other
PEs) as well as maintaining MAC-VRFs/BTs for subnets that are not
locally present on that PE.
As an example, consider TS3 and TS5 of Figure 2 above. Assume that The following subsection defines the control and data planes
connectivity is required between these two TS's where TS3 belongs to procedures for symmetric and asymmetric IRB on ingress and egress
the IP-subnet 3 (SN3) whereas TS5 belongs to the IP-subnet 5 (SN5). PEs. The following figure is used in description of these procedures
Both SN3 and SN5 subnets belong to the same tenant. NVE2 has an EVI3 where it shows a single IP-VRF and a number of BTs on each PE for a
associated with the SN3 and this EVI is represented by a MAC-VRF given tenant. The IP-VRF of the tenant (i.e., IP-VRF1) is connected
which is associated with an IP-VRF (for that tenant) via an IRB to each BT via its associated IRB interface. Each BT on a PE is
interface. NVE3 respectively has an EVI5 associated with the SN5 and associated with a unique VLAN (e.g., with a BD) where in turn is
this EVI is represented by an MAC-VRF which is associated with the associated with a single MAC-VRF in case of VLAN-Based mode or a
same IP-VRF via a different IRB interface. number of BTs can be associated with a single MAC-VRF in case of
VLAN-Aware Bundle mode. Whether the service interface on a PE is
VLAN-Based or VLAN-Aware Bundle mode does not impact the IRB
operation and procedures. It only impacts the setting of Ethernet tag
field in EVPN routes as described in [RFC7432].
2.2 Switching among IP subnets in different DCs without GW PE 1 +---------+
+-------------+ | |
TS1-----| MACx| | | PE2
(IP1/M1) |(BT1) | | | +-------------+
TS5-----| \ | | MPLS/ | |MACy (BT3) |-----TS3
(IP5/M5) |Mx/IPx \ | | VxLAN/ | | / | (IP3/M3)
| (IP-VRF1)|----| NVGRE |---|(IP-VRF1) |
| / | | | | \ |
TS2-----|(BT2) / | | | | (BT1) |-----TS4
(IP2/M2) | | | | | | (IP4/M4)
+-------------+ | | +-------------+
| |
+---------+
This case is similar to that of section 2.1 above albeit for the fact Figure 4: IRB forwarding
that the TS's belong to different data centers that are
interconnected over a WAN (e.g. MPLS/IP PSN). The data centers in
question here are seamlessly interconnected to the WAN, i.e., the WAN
edge devices do not maintain any TS-specific addresses in the
forwarding path - e.g., there is no WAN edge GW(s) between these DCs.
As an example, consider TS3 and TS6 of Figure 2 above. Assume that 3.1 IRB Interface and its MAC & IP addresses
connectivity is required between these two TS's where TS3 belongs to
the SN3 whereas TS6 belongs to the SN6. NVE2 has an EVI3 associated
with SN3 and NVE4 has an EVI6 associated with the SN6. Both SN3 and
SN6 are part of the same IP-VRF.
2.3 Switching among IP subnets in different DCs with GW To support inter-subnet forwarding on a PE, the PE acts as an IP
Default Gateway from the perspective of the attached Tenant Systems
where default gateway MAC and IP addresses are configured on each IRB
interface associated with its subnet and falls into one of the
following two options:
In this scenario, connectivity is required between TS's in different 1. All the PEs for a given tenant subnet use the same anycast default
data centers, and those hosts belong to different IP subnets. What gateway IP and MAC addresses . On each PE, this default gateway IP
makes this case different from that of Section 2.2 is that at least and MAC addresses correspond to the IRB interface connecting the BT
one of the data centers has a gateway as the WAN edge switch. Because associated with the tenant's <EVI, VLAN> to the corresponding
of that, the NVE's IP-VRF within that data center need not maintain tenant's IP-VRF.
(host) routes to individual TS's outside of that data center.
As an example, consider a tenant with TS1 and TS5 of Figure 2 above. 2. Each PE for a given tenant subnet uses the same anycast default
Assume that connectivity is required between these two TS's where TS1 gateway IP address but its own MAC address. These MAC addresses are
belongs to the SN1 whereas TS5 belongs to the SN5. NVE3 has an EVI5 aliased to the same anycast default gateway IP address through the
associated with the SN5 and this EVI is represented by the MAC-VRF use of the Default Gateway extended community as specified in [EVPN],
which is connected to the IP-VRF via an IRB interface. NVE1 has an which is carried in the EVPN MAC/IP Advertisement routes. On each PE,
EVI1 associated with the SN1 and this EVI is represented by the MAC- this default gateway IP address along with its associated MAC
VRF which is connected to the IP-VRF representing the same tenant. addresses correspond to the IRB interface connecting the BT
Due to the gateway at the edge of DCN 1, NVE1's IP-VRF does not need associated with the tenant's <EVI, VLAN> to the corresponding
to have the address of TS5 but instead it has a default route in its tenant's IP-VRF.
IP-VRF with the next-hop being the GW.
2.4 Switching among IP subnets spread across IP-VPN and EVPN networks It is worth noting that if the applications that are running on the
with GW TS's are employing or relying on any form of MAC security, then
either the first model (i.e. using anycast MAC address) should be
used to ensure that the applications receive traffic from the same
IRB interface MAC address that they are sending to, or if the second
model is used, then the IRB interface MAC address MUST be the one
used in the initial ARP reply for that TS.
In this scenario, connectivity is required between TS's in a data Although both of these options are equally applicable to both
center and hosts in an enterprise site that belongs to a given IP- symmetric and asymmetric IRB, the option-1 is recommended because of
VPN. The NVE within the data center is an EVPN NVE, whereas the the ease of anycast MAC address provisioning on not only the IRB
enterprise site has an IP-VPN PE. Furthermore, the data center in interface associated with a given subnet across all the PEs
question has a gateway as the WAN edge switch. Because of that, the corresponding to that EVI but also on all IRB interfaces associated
NVE in the data center does not need to maintain individual IP with all the tenant's subnets across all the PEs corresponding to all
prefixes advertised by enterprise sites (by IP-VPN PEs). the EVIs for that tenant. Furthermore, it simplifies the operation as
there is no need for Default Gateway extended community advertisement
and its associated MAC aliasing procedure.
As an example, consider end-station H1 and TS2 of Figure 2. Assume When a TS sends an ARP request to the PE that is attached to, the ARP
that connectivity is required between the end-station and the TS, request is sent for the IP address of the IRB interface associated
where TS2 belongs to the SN2 that is realized using EVPN, whereas H1 with the TS's subnet. For example, in figure 4, TS1 is configured
belongs to an IP VPN site connected to PE1 (PE1 maintains an IP-VRF with the anycast IPx address as its default gateway IP address and
associated with that IP VPN). NVE1 has an EVI2 associated with the thus when it sends an ARP request for IPx (IP address of the IRB
SN2. Moreover, EVI2 on NVE1 is connected to an IP-VRF associated with interface for BT1), the PE1 sends an ARP reply with the MACx which is
that IP VPN. PE1 originates a VPN-IP route that covers H1. The the MAC address of that IRB interface.
gateway at the edge of DCN1 performs interworking function between
IP-VPN and EVPN. As a result of this, a default route in the IP-VRF
on the NVE1, pointing to the gateway as the next hop, and a route to
the TS2 (or maybe SN2) on the PE1's IP-VRF are sufficient for the
connectivity between H1 and TS2. In this scenario, the NVE1's IP-VRF
does not need to maintain a route to H1 because it has the default
route to the gateway.
3 Default L3 Gateway for Tenant System In addition to anycast addresses, IRB interfaces can be configured
with non-anycast IP addresses for the purpose of OAM (such as
traceroute/ping to these interfaces) for both symmetric and
asymmetric IRB. These IP addresses need to be distributed as VPN
routes when PEs operating in symmetric IRB mode. However, they don't
need to be distributed if the PEs are operating in asymmetric IRB
mode and the IRB interfaces are configured with individual MACs.
3.1 Homogeneous Environment 3.2 Symmetric IRB Procedures
This is an environment where all NVEs to which an EVPN instance could 3.2.1 Control Plane - Ingress PE
potentially be attached (or moved), perform inter-subnet switching.
Therefore, inter-subnet traffic can be locally switched by the EVPN
NVE connecting the TS's belonging to different subnets.
To support such inter-subnet forwarding, the NVE behaves as an IP When a PE (e.g., PE1 in figure 4 above) learns MAC and IP address of
Default Gateway from the perspective of the attached TS's. Two models a TS (via an ARP request), it adds the MAC address to the
are possible: corresponding MAC-VRF/BT of that tenant's subnet and adds the IP
address to the IP-VRF for that tenant. Furthermore, it adds this TS's
MAC and IP address association to its ARP table. It then builds an
EVPN MAC/IP Advertisement route (type 2) as follow and advertises it
to other PEs participating in that tenant's VPN.
1. All the NVEs of a given EVPN instance use the same anycast default - The Length field of the BGP EVPN NLRI for an EVPN MAC/IP
gateway IP address and the same anycast default gateway MAC address. Advertisement route MUST be either 40 (if IPv4 address is carried) or
On each NVE, this default gateway IP/MAC address correspond to the 52 (if IPv6 address is carried).
IRB interface connecting the MAC-VRF of that EVI to the corresponding
IP-VRF.
2. Each NVE of a given EVPN instance uses its own default gateway IP - Route Distinguisher (RD), Ethernet Segment Identifier, Ethernet Tag
and MAC addresses, and these addresses are aliased to the same ID, MAC Address Length, MAC Address, IP Address Length, IP Address,
conceptual gateway through the use of the Default Gateway extended and MPLS Label1 fields MUST be used as defined in [RFC7432] and
community as specified in [EVPN], which is carried in the EVPN MAC [RFC8365].
Advertisement routes. On each NVE, this default gateway IP/MAC
address correspond to the IRB interface connecting the MAC-VRF of
that EVI to the corresponding IP-VRF.
Both of these models enable a packet forwarding paradigm for both - The MPLS Label2 field is set to either an MPLS label or a VNI
symmetric and asymmetric IRB forwarding. In case of asymmetric IRB, a corresponding to the tenant's IP-VRF. In case of an MPLS label, this
packet is forwarded through the MAC-VRF followed by the IP-VRF on the field is encoded as 3 octets, where the high-order 20 bits contain
ingress NVE, and then forwarded through the the MAC-VRF on the egress the label value.
(disposition) NVE. The egress NVE merely needs to perform a lookup in
the associated MAC-VRF and forward the Ethernet frames unmodified,
i.e. without rewriting the source MAC address. This is different
from symmetric IRB forwarding where a packet is forwarded through the
MAC-VRF followed by the IP-VRF on the ingress NVE, and then forwarded
through the IP-VRF followed by the MAC-VRF on the egress NVE.
It is worth noting that if the applications that are running on the Just as in [RFC7432], the RD, Ethernet Tag ID, MAC Address Length,
TS's are employing or relying on any form of MAC security, then the MAC Address, IP Address Length, and IP Address fields are part of
first model (i.e. using anycast addresses) would be required to the route key used by BGP to compare routes. The rest of the fields
ensure that the applications receive traffic from the same source MAC are not part of the route key.
address that they are sending to.
3.2 Heterogeneous Environment This route is advertised along with the following two extended
communities:
1) Tunnel Type Extended Community
2) Router's MAC Extended Community
For large data centers with thousands of servers and ToR (or Access) For symmetric IRB mode, Router's MAC EC is needed to carry the PE's
switches, some of them may not have the capability of maintaining or overlay MAC address which is used for IP-VRF to IP-VRF communications
enforcing policies for inter-subnet switching. Even though policies with Ethernet NVO tunnel. If MPLS or IP-only NVO tunnel is used, then
among multiple subnets belonging to same tenant can be simpler, hosts there is no need to send Router's MAC Extended Community along with
belonging to one tenant can also send traffic to peers belonging to this route.
different tenants or security zones. In such scenarios, a WAN edge PE
(e.g., L3GW) may not only need to enforce policies for communication
among subnets belonging to a single tenant, but also it may need to
know how to handle traffic destined towards peers in different
tenants. Therefore, there can be a mixed environment where an NVE
performs inter-subnet switching for some EVPN instances and the L3GW
for others.
4 Operational Models for Asymmetric Inter-Subnet Forwarding This route MUST be advertised with two route targets - one
corresponding to the MAC-VRF of the tenant's subnet and another
corresponding to the tenant's IP-VRF.
4.1 Among EVPN NVEs within a DC 3.2.2 Control Plane - Egress PE
When an EVPN MAC/IP advertisement route is received by a NVE, the IP When a PE (e.g., PE2 in figure 4 above) receives this EVPN MAC/IP
address associated with the route is used to populate the IP-VRF Advertisement route advertisement, it performs the following:
table, whereas the MAC address associated with the route is used to
populate both the MAC-VRF table, as well as the adjacency associated
with the IP route in the IP-VRF table (i.e., ARP table).
When an Ethernet frame is received by an ingress NVE, it performs a - Using MAC-VRF route target, it identifies the corresponding MAC-
lookup on the destination MAC address in the associated MAC-VRF for VRF. If the MAC-VRF exists (e.g., it is locally configured) then it
that EVI. If the MAC address corresponds to its IRB Interface MAC imports the MAC address into it. Otherwise, it does not import the
address, the ingress NVE deduces that the packet MUST be inter-subnet MAC address.
routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup identifies an adjacency that
contains a MAC rewrite and in turn the next-hop (i.e., egress) NVE to
which the packet must be forwarded and the associated MPLS label
stack. The MAC rewrite holds the MAC address associated with the
destination host (as populated by the EVPN MAC route), instead of the
MAC address of the next-hop NVE. The ingress NVE then rewrites the
destination MAC address in the packet with the address specified in
the adjacency. It also rewrites the source MAC address with its IRB
Interface MAC address. The ingress NVE, then, forwards the frame to
the next-hop (i.e. egress) NVE after encapsulating it with the MPLS
label stack. Note that this label stack includes the LSP label as
well as the EVPN label that was advertised by the egress NVE. When
the MPLS encapsulated packet is received by the egress NVE, it uses
the EVPN label to identify the MAC-VRF table. It then performs a MAC
lookup in that table, which yields the outbound interface to which
the Ethernet frame must be forwarded. Figure 2 below depicts the
packet flow, where NVE1 and NVE2 are the ingress and egress NVEs,
respectively.
NVE1 NVE2 - Using IP-VRF route target, it identifies the corresponding IP-VRF
+------------+ +------------+ and imports the IP address into it.
| | | |
|(MAC - (IP | |(IP - (MAC |
| VRF) VRF)| | VRF) VRF)|
| | | | | | | |
+------------+ +------------+
^ v ^ V
| | | |
TS1->-+ +-->--------------+ +->-TS2
Figure 2: Inter-Subnet Forwarding Among EVPN NVEs within a DC The inclusion of MPLS label2 field in this route signals to the
receiving PE that this route is for symmetric IRB mode and MPLS
label2 needs to be installed in forwarding path to identify the
corresponding IP-VRF.
Note that the forwarding behavior on the egress NVE is similar to If the receiving PE receives this route with both the MAC-VRF and IP-
EVPN intra-subnet forwarding. In other words, all the packet VRF route targets but the MAC/IP Advertisement route does not include
processing associated with the inter-subnet forwarding semantics is MPLS label2 field and if the receiving PE does not support asymmetric
confined to the ingress NVE and that is why it is called Asymmetric IRB mode, then if it has the corresponding MAC-VRF, it only imports
IRB. the MAC address; otherwise, if it doesn't have the corresponding MAC-
VRF, it MUST treat the route as withdraw [RFC7606].
It should also be noted that [EVPN] provides different level of 3.2.3 Data Plane - Ingress PE
granularity for the EVPN label. Besides identifying bridge domain
table, it can be used to identify the egress interface or a
destination MAC address on that interface. If EVPN label is used for
egress interface or destination MAC address identification, then no
MAC lookup is needed in the egress EVI and the packet can be directly
forwarded to the egress interface just based on EVPN label lookup.
4.2 Among EVPN NVEs in Different DCs Without GW When an Ethernet frame is received by an ingress PE (e.g., PE1 in
figure 4 above), the PE uses the AC ID (e.g., VLAN ID) to identify
the associated MAC-VRF/BT and it performs a lookup on the destination
MAC address. If the MAC address corresponds to its IRB Interface MAC
address, the ingress PE deduces that the packet must be inter-subnet
routed. Hence, the ingress PE performs an IP lookup in the associated
IP-VRF table. The lookup identifies BGP next hop of egress PE along
with the tunnel/encapsulation type and the associated MPLS/VNI
values.
When an EVPN MAC advertisement route is received by a NVE, the IP If the tunnel type is that of MPLS or IP-only NVO tunnel, then TS's
address associated with the route is used to populate the IP-VRF IP packet is sent over the tunnel without any Ethernet header.
table, whereas the MAC address associated with the route is used to However, if the tunnel type is that of Eternet NVO tunnel, then an
populate both the MAC-VRF table, as well as the adjacency associated Ethernet header needs to be added to the TS's IP packet. The source
with the IP route in the IP-VRF table (i.e., ARP table). MAC address of this Ethernet header is set to the ingress PE's router
MAC address and the destination MAC address of this Ethernet header
is set to the egress PE's router MAC address. The MPLS VPN label or
VNI fields are set accordingly and the packet is forwarded to the
egress PE.
When an Ethernet frame is received by an ingress NVE, it performs a If case of NVO tunnel encapsulation, the outer source IP address is
lookup on the destination MAC address in the associated EVI. If the set to the ingress PE's BGP next-hop address and outer destination IP
MAC address corresponds to its IRB Interface MAC address, the ingress address is set to the egress PE's BGP next-hop address.
NVE deduces that the packet MUST be inter-subnet routed. Hence, the
ingress NVE performs an IP lookup in the associated IP-VRF table. The
lookup identifies an adjacency that contains a MAC rewrite and in
turn the next-hop (i.e. egress) Gateway to which the packet must be
forwarded along with the associated MPLS label stack. The MAC rewrite
holds the MAC address associated with the destination host (as
populated by the EVPN MAC route), instead of the MAC address of the
next-hop Gateway. The ingress NVE then rewrites the destination MAC
address in the packet with the address specified in the adjacency. It
also rewrites the source MAC address with its IRB Interface MAC
address. The ingress NVE, then, forwards the frame to the next-hop
(i.e. egress) Gateway after encapsulating it with the MPLS label
stack.
Note that this label stack includes the LSP label as well as an EVPN 3.2.4 Data Plane - Egress PE
label. The EVPN label could be either advertised by the ingress
Gateway, if inter-AS option B is used, or advertised by the egress
NVE, if inter-AS option C is used. When the MPLS encapsulated packet
is received by the ingress Gateway, the processing again differs
depending on whether inter-AS option B or option C is employed: in
the former case, the ingress Gateway swaps the EVPN label in the
packets with the EVPN label value received from the egress Gateway.
In the latter case, the ingress Gateway does not modify the EVPN
label and performs normal label switching on the LSP label.
Similarly on the egress Gateway, for option B, the egress Gateway
swaps the EVPN label with the value advertised by the egress NVE.
Whereas, for option C, the egress Gateway does not modify the EVPN
label, and performs normal label switching on the LSP label. When the
MPLS encapsulated packet is received by the egress NVE, it uses the
EVPN label to identify the bridge-domain table. It then performs a
MAC lookup in that table, which yields the outbound interface to
which the Ethernet frame must be forwarded. Figure 3 below depicts
the packet flow.
NVE1 GW1 GW2 NVE2 When the tenant's MPLS or NVO encapsulated packet is received over an
+------------+ +------------+ +------------+ +------------+ MPLS or NVO tunnel by the egress PE, the egress PE removes NVO tunnel
| | | | | | | | encapsulation and uses the VPN MPLS label (for MPLS encapsulation) or
|(MAC - (IP | | [LS] | | [LS] | |(IP - (MAC | VNI (for VxLAN encapsulation) to identify the IP-VRF in which IP
| VRF) VRF)| | | | | | VRF) VRF)| lookup needs to be performed.
| | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V
| | | | | | | |
TS1->-+ +-->--------+ +------------+ +---------------+ +->-TS2
Figure 3: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs The lookup identifies a local adjacency to the IRB interface
without GW associated with the egress subnet's MAC-VRF/BT.
4.3 Among EVPN NVEs in Different DCs with GW The egress PE gets the destination TS's MAC address for that TS's IP
address from its ARP table, it encapsulates the packet with that
destination MAC address and a source MAC address corresponding to
that IRB interface and sends the packet to its destination subnet
MAC-VRF/BT.
In this scenario, the NVEs within a given data center do not have The destination MAC address lookup in the MAC-VRF/BT results in local
entries for the MAC/IP addresses of hosts in remote data centers. adjacency (e.g., local interface) over which the Ethernet frame is
Rather, the NVEs have a default IP route pointing to the WAN gateway sent on.
for each VRF. This is accomplished by the WAN gateway advertising for
a given EVPN that spans multiple DC a default VPN-IP route that is
imported by the NVEs of that VPN that are in the gateway's own DC.
When an Ethernet frame is received by an ingress NVE, it performs a 3.3 Asymmetric IRB Procedures
lookup on the destination MAC address in the associated MAC-VRF
table. If the MAC address corresponds to the IRB Interface MAC
address, the ingress NVE deduces that the packet MUST be inter-subnet
routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup, in this case, matches the
default host route which points to the local WAN gateway. The ingress
NVE then rewrites the destination MAC address in the packet with the
router's MAC address of the local WAN gateway. It also rewrites the
source MAC address with its own IRB Interface MAC address. The
ingress NVE, then, forwards the frame to the WAN gateway after
encapsulating it with the MPLS label stack. Note that this label
stack includes the LSP label as well as the label for default host
route that was advertised by the local WAN gateway. When the MPLS
encapsulated packet is received by the local WAN gateway, it uses the
default host route label to identify the IP-VRF table. It then
performs an IP lookup in that table. The lookup identifies an
adjacency that contains a MAC rewrite and in turn the remote WAN
gateway (of the remote data center) to which the packet must be
forwarded along with the associated MPLS label stack. The MAC rewrite
holds the MAC address associated with the ultimate destination host
(as populated by the EVPN MAC route). The local WAN gateway then
rewrites the destination MAC address in the packet with the address
specified in the adjacency. It also rewrites the source MAC address
with its router's MAC address. The local WAN gateway, then, forwards
the frame to the remote WAN gateway after encapsulating it with the
MPLS label stack. Note that this label stack includes the LSP label
as well as a EVPN label that was advertised by the remote WAN
gateway. When the MPLS encapsulated packet is received by the remote
WAN gateway, it simply swaps the EVPN label and forwards the packet
to the egress NVE. This implies that the GW1 needs to keep the remote
host MAC addresses along with the corresponding EVPN labels in the
adjacency entries of the IP-VRF table (i.e., its ARP table). The
remote WAN gateway then forward the packet to the egress NVE. The
egress NVE then performs a MAC lookup in the MAC-VRF (identified by
the received EVPN label) to determine the outbound port to send the
traffic on.
Figure 4 below depicts the forwarding model. 3.3.1 Control Plane - Ingress PE
NVE1 GW1 GW2 NVE2 When a PE (e.g., PE1 in figure 4 above) learns MAC and IP address of
+------------+ +------------+ +------------+ +------------+ a TS (via an ARP request), it populates its MAC-VRF/BT, IP-VRF, and
| | | | | | | | ARP table just as in the case for symmetric IRB. It then builds an
|(MAC - (IP | |(IP - (MAC | | [LS] | |(IP - (MAC | EVPN MAC/IP Advertisement route (type 2) as follow and advertises it
| VRF) VRF)| | VRF) VRF)| | | | | | VRF) VRF)| to other PEs participating in that tenant's VPN.
| | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V
| | | | | | | |
TS1->-+ +-->-----+ +---------------+ +---------------+ +->-TS2
Figure 4: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs - The Length field of the BGP EVPN NLRI for an EVPN MAC/IP
with GW Advertisement route MUST be either 37 (if IPv4 address is carried) or
49 (if IPv6 address is carried).
4.4 Among IP-VPN Sites and EVPN NVEs with GW - Route Distinguisher (RD), Ethernet Segment Identifier, Ethernet Tag
ID, MAC Address Length, MAC Address, IP Address Length, IP Address,
and MPLS Label1 fields MUST be used as defined in [RFC7432] and
[RFC8365].
In this scenario, the NVEs within a given data center do not have - The MPLS Label2 field MUST NOT be included in this route.
entries for the IP addresses of hosts in remote enterprise sites.
Rather, the NVEs have a default IP route pointing the WAN gateway for
each IP-VRF.
When an Ethernet frame is received by an ingress NVE, it performs a Just as in [RFC7432], the RD, Ethernet Tag ID, MAC Address Length,
lookup on the destination MAC address in the associated MAC-VRF MAC Address, IP Address Length, and IP Address fields are part of
table. If the MAC address corresponds to the IRB Interface MAC the route key used by BGP to compare routes. The rest of the fields
address, the ingress NVE deduces that the packet MUST be inter-subnet are not part of the route key.
routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup, in this case, matches the
default route which points to the local WAN gateway. The ingress NVE
then rewrites the destination MAC address in the packet with the
router's MAC address of the local WAN gateway. It also rewrites the
source MAC address with its own IRB Interface MAC address. The
ingress NVE, then, forwards the frame to the local WAN gateway after
encapsulating it with the MPLS label stack. Note that this label
stack includes the LSP label as well as the default host route label
that was advertised by the local WAN gateway. When the MPLS
encapsulated packet is received by the local WAN gateway, it uses the
default host route label to identify the IP-VRF table. It then
performs an IP lookup in that table. The lookup identifies the next
hop ASBR to which the packet must be forwarded. The local gateway in
this case strips the Ethernet encapsulation and perform an IP lookup
in its IP-VRF and forwards the IP packet to the ASBR using a label
stack comprising of an LSP label and an IP-VPN label that was
advertised by the ASBR. When the MPLS encapsulated packet is received
by the ASBR, it simply swaps the IP-VPN label with the one advertised
by the egress PE. The ASBR then forwards the packet to the egress PE.
The egress PE then performs an IP lookup in the IP-VRF (identified by
the received IP-VPN label) to determine where to forward the traffic.
Figure 5 below depicts the forwarding model. This route is advertised along with the following extended
communitiy:
1) Tunnel Type Extended Community
NVE1 GW1 ASBR NVE2 For asymmetric IRB mode, Router's MAC EC is not needed because
+------------+ +------------+ +------------+ +------------+ forwarding is performed using destination TS's MAC address which is
| | | | | | | | carried in this route advertisement.
|(MAC - (IP | |(IP - (MAC | | [LS] | | (IP |
| VRF) VRF)| | VRF) VRF)| | | | | | VRF)|
| | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V
| | | | | | | |
TS1->-+ +-->-----+ +--------------+ +---------------+ +->-H1
Figure 5: Inter-Subnet Forwarding Among IP-VPN Sites and EVPN NVEs This route MUST always be advertised with MAC-VRF route target. It
with GW MAY also be advertised with a second route target corresponding to
the IP-VRF. If only MAC-VRF route target is used, then the receiving
PE uses the MAC-VRF route target to identify the corresponding IP-VRF
- i.e., many MAC-VRF route targets map to the same IP-VRF for a given
tenant.
4.5 Use of Centralized Gateway 3.3.2 Control Plane - Egress PE
When a PE (e.g., PE2 in figure 4 above) receives this EVPN MAC/IP
Advertisement route advertisement, it performs the following:
In this scenario, the NVEs within a given data center need to forward - Using MAC-VRF route target, it identifies the corresponding MAC-VRF
traffic in L2 to a centralized L3GW for a number of reasons: a) they and imports the MAC address into it. For asymmetric IRB mode, it is
don't have IRB capabilities or b) they don't have required policy for assumed that all PEs participating in a tenant's VPN are configured
switching traffic between different tenants or security zones. The with all subnets and corresponding MAC-VRFs/BTs even if there are no
centralized L3GW performs both the IRB function for switching traffic locally attached TS's for some of these subnets. The reason for this
among different EVPN instances as well as it performs interworking is because ingress PE needs to do forwarding based on destination
function when the traffic needs to be switched between IP-VPN sites TS's MAC address and does proper NVO tunnel encapsulation which are
and EVPN instances. property of a lookup in MAC-VRF/BT. An implementation may choose to
consolidate the lookup at the ingress PE's IP-VRF with the lookup at
the ingress PE's destination subnet MAC-VRF. Consideration for such
consolidation of lookups is outside the scope of this document.
- Using MAC-VRF route target, it identifies the corresponding ARP
table for the tenant and it adds an entry to the ARP table for the
TS's MAC and IP address association. It should be noted that the
tenant's ARP table at the receiving PE is identified by all the MAC-
VRF route targets for that tenant. If IP-VRF route target is included
with this route advertisement, then it MAY be used for the
identification of tenant's ARP table.
For asymmetric IRB mode, the MPLS label2 field SHOULD not be included
in the route; however, if the receiving PE receives this route with
the MPLS label2 field, then it SHOULD ignore it.
3.3.3 Data Plane - Ingress PE
When an Ethernet frame is received by an ingress PE (e.g., PE1 in
figure 4 above), the PE uses the AC ID (e.g., VLAN ID) to identify
the associated MAC-VRF/BT and it performs a lookup on the destination
MAC address. If the MAC address corresponds to its IRB Interface MAC
address, the ingress PE deduces that the packet must be inter-subnet
routed. Hence, the ingress PE performs an IP lookup in the associated
IP-VRF table. The lookup identifies a local adjacency to the IRB
interface associated with the egress subnet's MAC-VRF/BT.
The ingress PE gets the destination TS's MAC address for that TS's IP
address from its ARP table, it encapsulates the packet with that
destination MAC address and a source MAC address corresponding to
that IRB interface and sends the packet to its destination subnet
MAC-VRF/BT.
The destination MAC address lookup in the MAC-VRF/BT results in BGP
next hop address of egress PE. The ingress PE encapsulates the packet
using Ethernet NVO tunnel of the choice (e.g., VxLAN or GENEVE) and
sends the packet to the egress PE. Since the packet forwarding is
between ingress PE's MAC-VRF/BT and egress PE's MAC-VRF/BT, the
packet encapsulation procedures follows that of [RFC7432] for MPLS
and [RFC8365] for VxLAN encapsulations.
3.3.4 Data Plane - Egress PE
When a tenant's Ethernet frame is received over an NVO tunnel by the
egress PE, the egress PE removes NVO tunnel encapsulation and uses
the VPN MPLS label (for MPLS encapsulation) or VNI (for VxLAN
encapsulation) to identify the MAC-VRF/BT in which MAC lookup needs
to be performed.
The MAC lookup results in local adjacency (e.g., local interface)
over which the packet needs to get sent.
Note that the forwarding behavior on the egress PE is the same as
EVPN intra-subnet forwarding described in [RFC7432] for MPLS and
[RFC8365] for VxLAN networks. In other words, all the packet
processing associated with the inter-subnet forwarding semantics is
confined to the ingress PE.
It should also be noted that [RFC7432] provides different level of
granularity for the EVPN label. Besides identifying bridge domain
table, it can be used to identify the egress interface or a
destination MAC address on that interface. If EVPN label is used for
egress interface or individual MAC address identification, then no
MAC lookup is needed in the egress PE for MPLS encapsulation and the
packet can be directly forwarded to the egress interface just based
on EVPN label lookup.
4 BGP Encoding
This document defines one new BGP Extended Community for EVPN.
4.1 Router's MAC Extended Community
A new EVPN BGP Extended Community called Router's MAC is introduced
here. This new extended community is a transitive extended community
with the Type field of 0x06 (EVPN) and the Sub-Type of 0x03. It may
be advertised along with BGP Encapsulation Extended Community define
in section 4.5 of [TUNNEL-ENCAP].
The Router's MAC Extended Community is encoded as an 8-octet value as
follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type=0x06 | Sub-Type=0x03 | Router's MAC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Router's MAC Cont'd |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Router's MAC Extended Community
This extended community is used to carry the PE's MAC address for
symmetric IRB scenarios and it is sent with RT-2.
5 Operational Models for Symmetric Inter-Subnet Forwarding 5 Operational Models for Symmetric Inter-Subnet Forwarding
The following sections describe several main symmetric IRB forwarding The following sections describe two main symmetric IRB forwarding
scenarios. scenarios (within a DC - i.e., intra-DC) along with their
corresponding procedures. In the following scenarios, without loss of
generality, it is assumed that a given tenant is represented by a
single IP-VPN instance. Therefore, on a given PE, a tenant is
represented by a single IP-VRF table and one or more MAC-VRF tables.
5.1 IRB forwarding on NVEs for Tenant Systems 5.1 IRB forwarding on NVEs for Tenant Systems
This section covers the symmetric IRB procedures for the scenario This section covers the symmetric IRB procedures for the scenario
where each Tenant System (TS) is attached to one or more NVEs and its where each Tenant System (TS) is attached to one or more NVEs and its
host IP and MAC addresses are learned by the attached NVEs and are host IP and MAC addresses are learned by the attached NVEs and are
distributed to all other NVEs that are interested in participating in distributed to all other NVEs that are interested in participating in
both intra-subnet and inter-subnet communications with that TS. both intra-subnet and inter-subnet communications with that TS.
In this scenario, for a given tenant (e.g., an IP-VPN instance), an In this scenario, for a given tenant, an NVE has typically one MAC-
NVE has typically one MAC-VRF for each tenant's subnet (VLAN) that is VRF for each tenant's subnet (VLAN) that is configured for, assuming
configured for, Assuming VLAN-based service which is typically the VLAN-based service which is typically the case for VxLAN and NVGRE
case for VxLAN and NVGRE encapsulation, each MAC-VRF consists of a encapsulation and each MAC-VRF consists of a single bridge domain. In
single bridge domain. In case of MPLS encapsulation with VLAN-aware case of MPLS encapsulation with VLAN-aware bundling, then each MAC-
bundling, then each MAC-VRF consists of multiple bridge domains (one VRF consists of multiple bridge domains (one bridge domain per VLAN).
bridge domain per VLAN). The MAC-VRFs on an NVE for a given tenant The MAC-VRFs on an NVE for a given tenant are associated with an IP-
are associated with an IP-VRF corresponding to that tenant (or IP-VPN VRF corresponding to that tenant (or IP-VPN instance) via their IRB
instance) via their IRB interfaces. interfaces.
Each NVE MUST support QoS, Security, and OAM policies per IP-VRF Each NVE MUST support QoS, Security, and OAM policies per IP-VRF
to/from the core network. This is not to be confused with the QoS, to/from the core network. This is not to be confused with the QoS,
Security, and OAM policies per Attachment Circuits (AC) to/from the Security, and OAM policies per Attachment Circuits (AC) to/from the
Tenant Systems. How this requirement is met is an implementation Tenant Systems. How this requirement is met is an implementation
choice and it is outside the scope of this document. choice and it is outside the scope of this document.
Since VxLAN and NVGRE encapsulations require inner Ethernet header Since VxLAN and NVGRE encapsulations require inner Ethernet header
(inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address (inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address
cannot be used, the ingress NVE's MAC address is used as inner MAC cannot be used, the ingress NVE's MAC address is used as inner MAC
SA. The NVE's MAC address is the device MAC address and it is common SA. The NVE's MAC address is the device MAC address and it is common
across all MAC-VRFs and IP-VRFs. This MAC address is advertised using across all MAC-VRFs and IP-VRFs. This MAC address is advertised using
the new EVPN Router's MAC Extended Community (section 6.1). the new EVPN Router's MAC Extended Community (section 6.1).
Figure below illustrates this scenario where a given tenant (e.g., an Figure 6 below illustrates this scenario where a given tenant (e.g.,
IP-VPN instance) has three subnets represented by MAC-VRF1, MAC-VRF2, an IP-VPN instance) has three subnets represented by MAC-VRF1, MAC-
and MAC-VRF3 across two NVEs. There are five TS's that are associated VRF2, and MAC-VRF3 across two NVEs. There are five TS's that are
with these three MAC-VRFs - i.e., TS1, TS4, and TS5 are sitting on associated with these three MAC-VRFs - i.e., TS1, TS4, and TS5 are
the same subnet (e.g., same MAC-VRF/VLAN);where, TS1 and TS5 are sitting on the same subnet (e.g., same MAC-VRF/VLAN);where, TS1 and
associated with MAC-VRF1 on NVE1, TS4 is associated with MAC-VRF1 on TS5 are associated with MAC-VRF1 on NVE1, TS4 is associated with MAC-
NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is associated VRF1 on NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is
with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are in turn associated with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are
associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on NVE2 are in turn associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on
associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4 exchange NVE2 are associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4
traffic with each other, only L2 forwarding (bridging) part of the exchange traffic with each other, only L2 forwarding (bridging) part
IRB solution is exercised because all these TS's sit on the same of the IRB solution is exercised because all these TS's sit on the
subnet. However, when TS1 wants to exchange traffic with TS2 or TS3 same subnet. However, when TS1 wants to exchange traffic with TS2 or
which belong to different subnets, then both bridging and routing TS3 which belong to different subnets, then both bridging and routing
parts of the IRB solution are exercised. The following subsections parts of the IRB solution are exercised. The following subsections
describe the control and data planes operations for this IRB scenario describe the control and data planes operations for this IRB scenario
in details. in details.
NVE1 +---------+ NVE1 +---------+
+-------------+ | | +-------------+ | |
TS1-----| MACx| | | NVE2 TS1-----| MACx| | | NVE2
(IP1/M1) |(MAC- | | | +-------------+ (IP1/M1) |(MAC- | | | +-------------+
TS5-----| VRF1)\ | | MPLS/ | |MACy (MAC- |-----TS3 TS5-----| VRF1)\ | | MPLS/ | |MACy (MAC- |-----TS3
(IP5/M5) | \ | | VxLAN/ | | / VRF3) | (IP3/M3) (IP5/M5) | \ | | VxLAN/ | | / VRF3) | (IP3/M3)
skipping to change at page 17, line 28 skipping to change at page 19, line 46
TS2-----|(MAC- / | | | | (MAC- |-----TS4 TS2-----|(MAC- / | | | | (MAC- |-----TS4
(IP2/M2) | VRF2) | | | | VRF1) | (IP4/M4) (IP2/M2) | VRF2) | | | | VRF1) | (IP4/M4)
+-------------+ | | +-------------+ +-------------+ | | +-------------+
| | | |
+---------+ +---------+
Figure 6: IRB forwarding on NVEs for Tenant Systems Figure 6: IRB forwarding on NVEs for Tenant Systems
5.1.1 Control Plane Operation 5.1.1 Control Plane Operation
Each NVE advertises a Route Type-2 (RT-2, MAC/IP Advertisement Route) Each NVE advertises a MAC/IP Advertisement route (i.e., Route Type 2)
for each of its TS's with the following field set: for each of its TS's with the following field set:
- RD and ESI per [EVPN] - RD and ESI per [EVPN]
- Ethernet Tag = 0; assuming VLAN-based service - Ethernet Tag = 0; assuming VLAN-based service
- MAC Address Length = 48 - MAC Address Length = 48
- MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example - MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example
- IP Address Length = 32 or 128 - IP Address Length = 32 or 128
- IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example - IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example
- Label-1 = MPLS Label or VNID corresponding to MAC-VRF - Label-1 = MPLS Label or VNID corresponding to MAC-VRF
- Label-2 = MPLS Label or VNID corresponding to IP-VRF - Label-2 = MPLS Label or VNID corresponding to IP-VRF
Each NVE advertises an RT-2 route with two Route Targets (one Each NVE advertises an RT-2 route with two Route Targets (one
corresponding to its MAC-VRF and the other corresponding to its IP- corresponding to its MAC-VRF and the other corresponding to its IP-
VRF. Furthermore, the RT-2 is advertised with two BGP Extended VRF. Furthermore, the RT-2 is advertised with two BGP Extended
Communities. The first BGP Extended Community identifies the tunnel Communities. The first BGP Extended Community identifies the tunnel
type per section 4.5 of [TUNNEL-ENCAP] and the second BGP Extended type per section 4.5 of [TUNNEL-ENCAP] and the second BGP Extended
Community includes the MAC address of the NVE (e.g., MACx for NVE1 or Community includes the MAC address of the NVE (e.g., MACx for NVE1 or
MACy for NVE2) as defined in section 6.1. This second Extended MACy for NVE2) as defined in section 6.1. This second Extended
Community (for the MAC address of NVE) is only required when Ethernet Community (for the MAC address of NVE) is only required when Ethernet
NVO tunnel type is used. If IP NVO tunnel type is used, then there is NVO tunnel type is used. If IP NVO tunnel type is used, then there is
no need to send this second Extended Community. no need to send this second Extended Community. It should be noted
that IP NVO tunnel type is only applicable to symmetric IRB
procedures.
Upon receiving this advertisement, the receiving NVE performs the Upon receiving this advertisement, the receiving NVE performs the
following: following:
- It uses Route Targets corresponding to its MAC-VRF and IP-VRF for - It uses Route Targets corresponding to its MAC-VRF and IP-VRF for
identifying these tables and subsequently importing this route into identifying these tables and subsequently importing this route into
them. them.
- It imports the MAC address into the MAC-VRF with BGP Next Hop - It imports the MAC address from MAC/IP Advertisement route into the
address as underlay tunnel destination address (e.g., VTEP DA for MAC-VRF with BGP Next Hop address as underlay tunnel destination
VxLAN encapsulation) and Label-1 as VNID for VxLAN encapsulation or address (e.g., VTEP DA for VxLAN encapsulation) and Label-1 as VNID
EVPN label for MPLS encapsulation. for VxLAN encapsulation or EVPN label for MPLS encapsulation.
- If the route carries the new Router's MAC Extended Community, and - If the route carries the new Router's MAC Extended Community, and
if the receiving NVE is using Ethernet NVO tunnel, then the receiving if the receiving NVE is using Ethernet NVO tunnel, then the receiving
NVE imports the IP address into IP-VRF with NVE's MAC address (from NVE imports the IP address into IP-VRF with NVE's MAC address (from
the new Router's MAC Extended Community) as inner MAC DA and BGP Next the new Router's MAC Extended Community) as inner MAC DA and BGP Next
Hop address as underlay tunnel destination address, VTEP DA for VxLAN Hop address as underlay tunnel destination address, VTEP DA for VxLAN
encapsulation and Label-2 as IP-VPN VNID for VxLAN encapsulation. encapsulation and Label-2 as IP-VPN VNID for VxLAN encapsulation.
- If the receiving NVE is going to use MPLS encapsulation, then the - If the receiving NVE is going to use MPLS encapsulation, then the
receiving NVE imports the IP address into IP-VRF with BGP Next Hop receiving NVE imports the IP address into IP-VRF with BGP Next Hop
address as underlay tunnel destination address, and Label-2 as IP-VPN address as underlay tunnel destination address, and Label-2 as IP-VPN
label for MPLS encapsulation. label for MPLS encapsulation.
If the receiving NVE receives a RT-2 with only a single Route Target If the receiving NVE receives a RT-2 with only a single Route Target
corresponding to IP-VRF and Label-1, then it must discard this route corresponding to IP-VRF and Label-1, or if it receives a RT-2 with
and log an error. If the receiving NVE receives a RT-2 with only a only a single Route Target corresponding to MAC-VRF but with both
single Route Target corresponding to MAC-VRF but with both Label-1 Label-1 and Label-2, or if it receives a RT-2 with MAC Address Length
and Label-2, then it must discard this route and log an error. If the of zero, then it must not import it to either IP-VRF or MAC-VRF and
receiving NVE receives a RT-2 with MAC Address Length of zero, then it must log an error.
it must discard this route and log an error.
5.1.2 Data Plane Operation - Inter Subnet 5.1.2 Data Plane Operation - Inter Subnet
The following description of the data-plane operation describes just The following description of the data-plane operation describes just
the logical functions and the actual implementation may differ. Lets the logical functions and the actual implementation may differ. Lets
consider data-plane operation when TS1 in subnet-1 (MAC-VRF1) on NVE1 consider data-plane operation when TS1 in subnet-1 (MAC-VRF1) on NVE1
wants to send traffic to TS3 in subnet-3 (MAC-VRF3) on NVE2. wants to send traffic to TS3 in subnet-3 (MAC-VRF3) on NVE2.
- TS1 send a packet with MAC DA corresponding to the MAC-VRF1 IRB - NVE1 receives a packet with MAC DA corresponding to the MAC-VRF1
interface on NVE1 (the interface between MAC-VRF1 and IP-VRF1), and IRB interface on NVE1 (the interface between MAC-VRF1 and IP-VRF1),
VLAN-tag corresponding to MAC-VRF1. and VLAN-tag corresponding to MAC-VRF1.
- Upon receiving the packet, the NVE1 uses VLAN-tag to identify the - Upon receiving the packet, the NVE1 uses VLAN-tag to identify the
MAC-VRF1. It then looks up the MAC DA and forwards the frame to its MAC-VRF1. It then looks up the MAC DA and forwards the frame to its
IRB interface. IRB interface.
- The Ethernet header of the packet is stripped and the packet is - The Ethernet header of the packet is stripped and the packet is
fed to the IP-VRF where IP lookup is performed on the destination fed to the IP-VRF where IP lookup is performed on the destination
address. This lookup yields an outgoing interface and the required address. This lookup yields an outgoing interface and the required
encapsulation. If the encapsulation is for Ethernet NVO tunnel, then encapsulation. If the encapsulation is for Ethernet NVO tunnel, then
it includes a MAC address to be used as inner MAC DA, an IP address it includes a MAC address to be used as inner MAC DA, an IP address
to be used as VTEP DA, and a VPN-ID to be used as VNID. to be used as VTEP DA, and a VPN-ID to be used as VNID. The inner MAC
SA and VTEP SA is set to NVE's MAC and IP addresses respectively. If
- The packet is then encapsulated with the proper header based on it is a MPLS encapsulation, then corresponding EVPN and LSP labels
the above info. The inner MAC SA and VTEP SA is set to NVE's MAC and are added to the packet. The packet is then forwarded to the egress
IP addresses respectively. The packet is then forwarded to the egress
NVE. NVE.
- On the egress NVE, if the packet arrives on Ethernet NOV tunnel - On the egress NVE, if the packet arrives on Ethernet NVO tunnel
(e.g., it is VxLAN encapsulated), then the VxLAN header is removed. (e.g., it is VxLAN encapsulated), then the VxLAN header is removed.
Since the inner MAC DA is the egress NVE's MAC address, the egress Since the inner MAC DA is the egress NVE's MAC address, the egress
NVE knows that it needs to perform an IP lookup. It uses VNID to NVE knows that it needs to perform an IP lookup. It uses VNID to
identify the IP-VRF table and then performs an IP lookup for the identify the IP-VRF table. If the packet is MPLS encapsulated, then
destination TS (TS3) which results in access-facing IRB interface the EVPN label lookup identifies the IP-VRF table. Next, an IP lookup
over which the packet is sent. Before sending the packet over this is performed for the destination TS (TS3) which results in access-
interface, the ARP table is consulted to get the destination TS's MAC facing IRB interface over which the packet is sent. Before sending
address. the packet over this interface, the ARP table is consulted to get the
destination TS's MAC address.
- The IP packet is encapsulated with an Ethernet header with MAC SA - The IP packet is encapsulated with an Ethernet header with MAC SA
set to that of IRB interface MAC address and MAC DA set to that of set to that of IRB interface MAC address and MAC DA set to that of
destination TS (TS3) MAC address. The packet is sent to the destination TS (TS3) MAC address. The packet is sent to the
corresponding MAC-VRF3 and after a lookup of MAC DA, is forwarded to corresponding MAC-VRF3 and after a lookup of MAC DA, is forwarded to
the destination TS (TS3) over the corresponding interface. the destination TS (TS3) over the corresponding interface.
In this symmetric IRB scenario, inter-subnet traffic between NVEs In this symmetric IRB scenario, inter-subnet traffic between NVEs
will always use the IP-VRF VNID/MPLS label. For instance, traffic will always use the IP-VRF VNID/MPLS label. For instance, traffic
from TS2 to TS4 will be encapsulated by NVE1 using NVE2's IP-VRF from TS2 to TS4 will be encapsulated by NVE1 using NVE2's IP-VRF
skipping to change at page 20, line 8 skipping to change at page 22, line 29
the MAC mobility for a mixed of L2 and L3 connectivity (aka IRB). In the MAC mobility for a mixed of L2 and L3 connectivity (aka IRB). In
order to place the emphasis on the differences between L2-only versus order to place the emphasis on the differences between L2-only versus
L2-and-L3 use cases, the incremental procedure is described for L2-and-L3 use cases, the incremental procedure is described for
single-homed TS with the expectation that the reader can easily single-homed TS with the expectation that the reader can easily
extrapolate multi-homed TS based on the procedures described in extrapolate multi-homed TS based on the procedures described in
section 15 of [EVPN]. section 15 of [EVPN].
Lets consider TS1 in figure-6 above where it moves from NVE1 to NVE2. Lets consider TS1 in figure-6 above where it moves from NVE1 to NVE2.
In such move, NVE2 discovers IP1/MAC1 of TS1 and realizes that it is In such move, NVE2 discovers IP1/MAC1 of TS1 and realizes that it is
a MAC move and it advertises a MAC/IP route per section 5.1.1 above a MAC move and it advertises a MAC/IP route per section 5.1.1 above
with MAC Mobility Extended Community. In this IRB use case, both MAC with MAC Mobility Extended Community.
and IP addresses of the TS along with their corresponding VNI/MPLS
labels are included in the EVPN MAC/IP Advertisement route.
Furthermore, besides MAC mobility Extended Community and Route Target
corresponding to the MAC-VRF, the following additional BGP Extended
Communities are advertised along with the MAC/IP Advertisement route:
- Route Target associated with IP-VRF
- Router's MAC Extended Community
- Tunnel Type Extended Community
Since NVE2 learns TS1's MAC/IP addresses locally, it updates its MAC- Since NVE2 learns TS1's MAC/IP addresses locally, it updates its MAC-
VRF1 and IP-VRF1 for TS1 with its local interface. VRF1 and IP-VRF1 for TS1 with its local interface.
If the local learning at NVE1 is performed using control or If the local learning at NVE1 is performed using control or
management planes, then these interactions serve as the trigger for management planes, then these interactions serve as the trigger for
NVE1 to withdraw the MAC/IP addresses associated with TS1. However, NVE1 to withdraw the MAC and IP addresses associated with TS1.
if the local learning at NVE1 is performed using data-plane learning, However, if the local learning at NVE1 is performed using data-plane
then the reception of the MAC/IP Advertisement route (for TS1) from learning, then the reception of the MAC/IP Advertisement route (for
NVE2 with MAC Mobility extended community serve as the trigger for TS1) from NVE2 with MAC Mobility extended community serve as the
NVE1 to withdraw the MAC/IP addresses associated with TS1. trigger for NVE1 to withdraw the MAC and IP addresses associated with
TS1.
All other remote NVE devices upon receiving the MAC/IP advertisement All other remote NVE devices upon receiving the MAC/IP advertisement
route for TS1 from NVE2 with MAC Mobility extended community compare route for TS1 from NVE2 with MAC Mobility extended community compare
the sequence number in this advertisement with the one previously the sequence number in this advertisement with the one previously
received. If the new sequence number is greater than the old one, received. If the new sequence number is greater than the old one,
then they update the MAC/IP addresses of TS1 in their corresponding then they update the MAC/IP addresses of TS1 in their corresponding
MAC-VRFs and IP-VRFs to point to NVE2. Furthermore, upon receiving MAC-VRFs and IP-VRFs to point to NVE2. Furthermore, upon receiving
the MAC/IP withdraw for TS1 from NVE1, these remote PEs perform the the MAC/IP withdraw for TS1 from NVE1, these remote PEs perform the
cleanups for their BGP tables. cleanups for their BGP tables.
5.2 IRB forwarding on NVEs for Subnets behind Tenant Systems 5.2 IRB forwarding on NVEs for Subnets behind Tenant Systems
This section covers the symmetric IRB procedures for the scenario This section covers the symmetric IRB procedures for the scenario
where some Tenant Systems (TS's) support one or more subnets and where some Tenant Systems (TS's) support one or more subnets and
these TS's are associated with one ore more NVEs. Therefore, besides these TS's are associated with one or more NVEs. Therefore, besides
the advertisement of MAC/IP addresses for each TS which can be in the the advertisement of MAC/IP addresses for each TS which can be multi-
presence of All-Active multi-homing, the associated NVE needs to also homed with All-Active redundancy mode, the associated NVE needs to
advertise the subnets behind each TS. also advertise the subnets statically configured on each TS.
The main difference between this scenario and the previous one is the The main difference between this solution and the previous one is the
additional advertisement corresponding to each subnet. These subnet additional advertisement corresponding to each subnet. These subnet
advertisements are accomplished using EVPN IP Prefix route defined in advertisements are accomplished using EVPN IP Prefix route defined in
[EVPN-PREFIX]. These subnet prefixes are advertised with the IP [EVPN-PREFIX]. These subnet prefixes are advertised with the IP
address of their associated TS (which is in overlay address space) as address of their associated TS (which is in overlay address space) as
their next hop. The receiving NVEs perform recursive route resolution their next hop. The receiving NVEs perform recursive route resolution
to resolve the subnet prefix with its associated ingress NVE so that to resolve the subnet prefix with its associated ingress NVE so that
they know which NVE to forward the packets to when they are destined they know which NVE to forward the packets to when they are destined
for that subnet prefix. for that subnet prefix.
The advantage of this recursive route resolution is that when a TS The advantage of this recursive route resolution is that when a TS
skipping to change at page 22, line 32 skipping to change at page 24, line 32
NVE2 NVE2
Figure 7: IRB forwarding on NVEs for Tenant Systems with configured subnets Figure 7: IRB forwarding on NVEs for Tenant Systems with configured subnets
5.2.1 Control Plane Operation 5.2.1 Control Plane Operation
Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in
[EVPN-PREFIX]) for each of its subnet prefixes with the IP address of [EVPN-PREFIX]) for each of its subnet prefixes with the IP address of
its TS as the next hop (gateway address field) as follow: its TS as the next hop (gateway address field) as follow:
- RD per VPN - RD associated with the IP-VRF
- ESI = 0 - ESI = 0
- Ethernet Tag = 0; - Ethernet Tag = 0;
- IP Prefix Length = 32 or 128 - IP Prefix Length = 32 or 128
- IP Prefix = SNi - IP Prefix = SNi
- Gateway Address = IPi; IP address of TS - Gateway Address = IPi; IP address of TS
- Label = 0 - Label = 0
This RT-5 is advertised with a Route Target corresponding to the IP- This RT-5 is advertised with one or more Route Targets that have been
VPN service. configured as "export route targets" of the IP-VRF from which the
route is originated.
Each NVE also advertises an RT-2 (MAC/IP Advertisement Route) along Each NVE also advertises an RT-2 (MAC/IP Advertisement Route) along
with their associated Route Targets and Extended Communities for each with their associated Route Targets and Extended Communities for each
of its TS's exactly as described in section 5.1.1. of its TS's exactly as described in section 5.1.1.
Upon receiving the RT-5 advertisement, the receiving NVE performs the Upon receiving the RT-5 advertisement, the receiving NVE performs the
following: following:
- It uses the Route Target to identify the corresponding IP-VRF - It uses the Route Target to identify the corresponding IP-VRF
- It imports the IP prefix into its corresponding IP-VRF with the IP - It imports the IP prefix into its corresponding IP-VRF that is
address of the associated TS as its next hop. configured with an import RT that is one of the RTs being carried by
the RT-5 route along with the IP address of the associated TS as its
next hop.
Upon receiving the RT-2 advertisement, the receiving NVE imports When receiving the RT-2 advertisement, the receiving NVE imports
MAC/IP addresses of the TS into the corresponding MAC-VRF and IP-VRF MAC/IP addresses of the TS into the corresponding MAC-VRF and IP-VRF
per section 5.1.1. Furthermore, it performs recursive route per section 5.1.1. When both routes exist, recursive route resolution
resolution to resolve the IP prefix (received in RT-5) to its is performed to resolve the IP prefix (received in RT-5) to its
corresponding NVE's IP address (e.g., its BGP next hop). BGP next hop corresponding NVE's IP address (e.g., its BGP next hop). BGP next hop
will be used as underlay tunnel destination address (e.g., VTEP DA will be used as underlay tunnel destination address (e.g., VTEP DA
for VxLAN encapsulation) and Router's MAC will be used as inner MAC for VxLAN encapsulation) and Router's MAC will be used as inner MAC
for VxLAN encapsulation. for VxLAN encapsulation.
5.2.2 Data Plane Operation 5.2.2 Data Plane Operation
The following description of the data-plane operation describes just The following description of the data-plane operation describes just
the logical functions and the actual implementation may differ. Lets the logical functions and the actual implementation may differ. Lets
consider data-plane operation when a host on SN1 sitting behind TS1 consider data-plane operation when a host on SN1 sitting behind TS1
skipping to change at page 24, line 12 skipping to change at page 26, line 13
packet over this interface, the ARP table is consulted to get the packet over this interface, the ARP table is consulted to get the
destination TS (TS3) MAC address. destination TS (TS3) MAC address.
- The IP packet is encapsulated with an Ethernet header with the MAC - The IP packet is encapsulated with an Ethernet header with the MAC
SA set to that of the access-facing IRB interface of the egress NVE SA set to that of the access-facing IRB interface of the egress NVE
(NVE2) and the MAC DA is set to that of destination TS (TS3) MAC (NVE2) and the MAC DA is set to that of destination TS (TS3) MAC
address. The packet is sent to the corresponding MAC-VRF3 and after a address. The packet is sent to the corresponding MAC-VRF3 and after a
lookup of MAC DA, is forwarded to the destination TS (TS3) over the lookup of MAC DA, is forwarded to the destination TS (TS3) over the
corresponding interface. corresponding interface.
6 BGP Encoding 6 Inter-Subnet DCI Scenarios
This document defines one new BGP Extended Community for EVPN. The inter-subnet DCI scenarios can be categorized into the following
four categories. The last two scenarios, along with its corresponding
solution, are described in [EVPN-IPVPN-INTEROP]. The first two
scenarios are covered in this document.
6.1 Router's MAC Extended Community 1. Switching among IP subnets in different DCs using EVPN without GW
A new EVPN BGP Extended Community called Router's MAC is introduced 2. Switching among IP subnets in different DCs using EVPN with GW
here. This new extended community is a transitive extended community
with the Type field of 0x06 (EVPN) and the Sub-Type of 0x03. It may
be advertised along with BGP Encapsulation Extended Community define
in section 4.5 of [TUNNEL-ENCAP].
The Router's MAC Extended Community is encoded as an 8-octet value as 3. Switching among IP subnets spread across IP-VPN and EVPN networks
follows: with GW
0 1 2 3 4. Switching among IP subnets spread across IP-VPN and EVPN networks
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 without GW
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type=0x06 | Sub-Type=0x03 | Router's MAC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Router's MAC Cont'd |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This extended community is used to carry the NVE's MAC address for In the above scenario, the term "GW" refers to the case where a node
symmetric IRB scenarios and it is sent with RT-2 as described in situated at the WAN edge of the data center network behaves as a
section 5.1.1 and 5.2.1. default gateway (GW) for all the destinations that are outside the
data center. The absence of GW refers to the scenario where NVEs
within a data center maintain individual (host) routes that are
outside of the data center.
In the case (3), the WAN edge node also performs route aggregation
for all the destinations within its own data center, and acts as an
interworking unit between EVPN and IP VPN (it implements both EVPN
and IP-VPN functionality).
+---+ Enterprise Site 1
|PE1|----- H1
+---+
/
,---------. Enterprise Site 2
,' `. +---+
,---------. /( MPLS/IP )---|PE2|----- H2
' DCN 3 `./ `. Core ,' +---+
`-+------+' `-+------+'
__/__ / / \ \
:NVE4 : +---+ \ \
'-----' ,----|GW |. \ \
| ,' +---+ `. ,---------.
TS6 ( DCN 1 ) ,' `.
`. ,' ( DCN 2 )
`-+------+' `. ,'
__/__ `-+------+'
:NVE1 : __/__ __\__
'-----' :NVE2 : :NVE3 :
| | '-----' '-----'
TS1 TS2 | | |
TS3 TS4 TS5
Figure 8: Interoperability Use-Cases
In what follows, we will describe scenarios 1 and 2 in more details.
6.1 Switching among IP subnets in different DCs without GW
This case is similar to that of section 2.1 above albeit for the fact
that the TS's belong to different data centers that are
interconnected over a WAN (e.g. MPLS/IP PSN). The data centers in
question here are seamlessly interconnected to the WAN, i.e., the WAN
edge devices do not maintain any TS-specific addresses in the
forwarding path - e.g., there is no WAN edge GW(s) between these DCs.
As an example, consider TS3 and TS6 of Figure 2 above. Assume that
connectivity is required between these two TS's where TS3 belongs to
the SN3 whereas TS6 belongs to the SN6. NVE2 has an EVI3 associated
with SN3 and NVE4 has an EVI6 associated with the SN6. Both SN3 and
SN6 are part of the same IP-VRF.
When an EVPN MAC advertisement route is received by a NVE, the IP
address associated with the route is used to populate the IP-VRF
table, whereas the MAC address associated with the route is used to
populate both the MAC-VRF table, as well as the adjacency associated
with the IP route in the IP-VRF table (i.e., ARP table).
When an Ethernet frame is received by an ingress NVE, it performs a
lookup on the destination MAC address in the associated EVI. If the
MAC address corresponds to its IRB Interface MAC address, the ingress
NVE deduces that the packet MUST be inter-subnet routed. Hence, the
ingress NVE performs an IP lookup in the associated IP-VRF table. The
lookup identifies an adjacency that contains a MAC rewrite and in
turn the next-hop (i.e. egress) Gateway to which the packet must be
forwarded along with the associated MPLS label stack. The MAC rewrite
holds the MAC address associated with the destination host (as
populated by the EVPN MAC route), instead of the MAC address of the
next-hop Gateway. The ingress NVE then rewrites the destination MAC
address in the packet with the address specified in the adjacency. It
also rewrites the source MAC address with its IRB Interface MAC
address for the destination subnet. The ingress NVE, then, forwards
the frame to the next-hop (i.e. egress) Gateway after encapsulating
it with the MPLS label stack.
Note that this label stack includes the LSP label as well as an EVPN
label. The EVPN label could be either advertised by the ingress
Gateway, if inter-AS option B is used, or advertised by the egress
NVE, if inter-AS option C is used. When the MPLS encapsulated packet
is received by the ingress Gateway, the processing again differs
depending on whether inter-AS option B or option C is employed: in
the former case, the ingress Gateway swaps the EVPN label in the
packets with the EVPN label value received from the egress Gateway.
In the latter case, the ingress Gateway does not modify the EVPN
label and performs normal label switching on the LSP label.
Similarly on the egress Gateway, for option B, the egress Gateway
swaps the EVPN label with the value advertised by the egress NVE.
Whereas, for option C, the egress Gateway does not modify the EVPN
label, and performs normal label switching on the LSP label. When the
MPLS encapsulated packet is received by the egress NVE, it uses the
EVPN label to identify the bridge-domain table. It then performs a
MAC lookup in that table, which yields the outbound interface to
which the Ethernet frame must be forwarded. Figure 3 below depicts
the packet flow.
NVE1 ASBR1 ASBR2 NVE2
+------------+ +------------+ +------------+ +------------+
| | | | | | | |
|(MAC - (IP | | [LS] | | [LS] | |(IP - (MAC |
| VRF) VRF)| | | | | | VRF) VRF)|
| | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V
| | | | | | | |
TS1->-+ +-->--------+ +------------+ +---------------+ +->-TS2
Figure 9: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs
without GW
6.2 Switching among IP subnets in different DCs with GW
In this scenario, connectivity is required between TS's in different
data centers, and those hosts belong to different IP subnets. What
makes this case different from that of Section 2.2 is that at least
one of the data centers has a gateway as the WAN edge switch. Because
of that, the NVE's IP-VRF within that data center need not maintain
(host) routes to individual TS's outside of that data center.
As an example, consider a tenant with TS1 and TS5 of Figure 2 above.
Assume that connectivity is required between these two TS's where TS1
belongs to the SN1 whereas TS5 belongs to the SN5. NVE3 has an EVI5
associated with the SN5 and this EVI is represented by the MAC-VRF
which is connected to the IP-VRF via an IRB interface. NVE1 has an
EVI1 associated with the SN1 and this EVI is represented by the MAC-
VRF which is connected to the IP-VRF representing the same tenant.
Due to the gateway at the edge of DCN 1, NVE1's IP-VRF does not need
to have the address of TS5 but instead it has a default route in its
IP-VRF with the next-hop being the GW.
In this scenario, the NVEs within a given data center do not have
entries for the MAC/IP addresses of hosts in remote data centers.
Rather, the NVEs have a default IP route pointing to the WAN gateway
for each VRF. This is accomplished by the WAN gateway advertising for
a given EVPN that spans multiple DC a default VPN-IP route that is
imported by the NVEs of that VPN that are in the gateway's own DC.
When an Ethernet frame is received by an ingress NVE, it performs a
lookup on the destination MAC address in the associated MAC-VRF
table. If the MAC address corresponds to the IRB Interface MAC
address, the ingress NVE deduces that the packet MUST be inter-subnet
routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup, in this case, matches the
default host route which points to the local WAN gateway (GW1). The
ingress NVE (NVE1) then rewrites the destination MAC address in the
packet with the MAC address of core-facing IRB interface of GW1 (not
shown in the figure) or it can rewrite it with the router's MAC
address of GW1. It also rewrites the source MAC address with its own
core-facing IRB Interface's MAC address for the destination subnet
(i.e., the subnet between NVE1 and GW1) or it can rewrite it with its
own router's MAC address of NVE1. The ingress NVE, then, forwards the
frame to GW1 after encapsulating it with the MPLS label stack. Note
that this label stack includes the LSP label as well as the label for
default host route that was advertised by the local WAN gateway. When
the MPLS encapsulated packet is received by GW1, it uses the default
host route MPLS label to identify the core-facing MAC-VRF. It does a
MAC-DA lookup and forwards the packet to the IP-VRF after stripping
the Ethernet header. It then performs an IP lookup in that table. The
lookup identifies an adjacency that contains a MAC rewrite and in
turn the remote WAN gateway (GW2) to which the packet must be
forwarded along with the associated MPLS label stack. The MAC rewrite
holds the MAC address associated with the ultimate destination host
(as populated by the EVPN MAC route). GW1 then rewrites the
destination MAC address in the packet with the address specified in
the adjacency. It also rewrites the source MAC address with the MAC
address of its core-facing IRB interface (not shown in the figure) or
its router's MAC address. GW1, then, forwards the frame to the GW2
after encapsulating it with the MPLS label stack. Note that this
label stack includes the LSP label as well as a EVPN label that was
advertised by GW2. When the MPLS encapsulated packet is received by
GW2, it uses the EVPN label to identify the destination MAC-VRF. It
then performs a MAC-DA lookup and grabs the EVPN label advertised by
NVE2 along with adjacencies info. It then encapsulates the packet
with the corresponding label stack and forwards the packet to NVE2.
It should be noted that no MAC header re-write is performed on GW2.
This implies that both GW1 and GW2 need to keep the remote host MAC
addresses along with the corresponding EVPN labels in their tables.
The egress NVE (NVE2) then upon receiving the packet, performs a MAC
lookup in the MAC-VRF (identified by the received EVPN label) to
determine the outbound port to send the traffic on.
Figure 4 below depicts the forwarding model.
NVE1 GW1 GW2 NVE2
+------------+ +------------+ +------------+ +------------+
| | | | | | | |
|(MAC - (IP | |(IP - (MAC | | (MAC | |(IP - (MAC |
| VRF) VRF)| | VRF) VRF)| | VRF) | | VRF) VRF)|
| | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V
| | | | | | | |
TS1->-+ +-->-----+ +---------------+ +---------------+ +->-TS2
Figure 10: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs
with GW
7 TS Mobility 7 TS Mobility
7.1 TS Mobility & Optimum Forwarding for TS Outbound Traffic 7.1 TS Mobility & Optimum Forwarding for TS Outbound Traffic
Optimum forwarding for the TS outbound traffic, upon TS mobility, can Optimum forwarding for the TS outbound traffic, upon TS mobility, can
be achieved using either the anycast default Gateway MAC and IP be achieved using either the anycast default Gateway MAC and IP
addresses, or using the address aliasing as discussed in [DC- addresses, or using the address aliasing as discussed in [DC-
MOBILITY]. MOBILITY].
skipping to change at page 25, line 34 skipping to change at page 32, line 13
from the WAN into the data center, the source NVE will receive the from the WAN into the data center, the source NVE will receive the
MAC Advertisement route of the target NVE (with the next hop MAC Advertisement route of the target NVE (with the next hop
attribute adjusted depending on which inter-AS option is employed). attribute adjusted depending on which inter-AS option is employed).
The source NVE will then withdraw its original MAC Advertisement The source NVE will then withdraw its original MAC Advertisement
route as a result of evaluating the Sequence Number field of the MAC route as a result of evaluating the Sequence Number field of the MAC
Mobility extended community in the received MAC Advertisement route. Mobility extended community in the received MAC Advertisement route.
This is per the procedures already defined in [EVPN]. This is per the procedures already defined in [EVPN].
8 Acknowledgements 8 Acknowledgements
The authors would like to thank Sami Boutros for his valuable The authors would like to thank Sami Boutros and Jeffrey Zhang for
comments. their valuable comments.
9 Security Considerations 9 Security Considerations
The security considerations discussed in [EVPN] apply to this The security considerations discussed in [EVPN] apply to this
document. document.
10 IANA Considerations 10 IANA Considerations
IANA has allocated a new transitive extended community Type of 0x06 IANA has allocated a new transitive extended community Type of 0x06
and Sub-Type of 0x03 for EVPN Router's MAC Extended Community. and Sub-Type of 0x03 for EVPN Router's MAC Extended Community.
skipping to change at page 26, line 21 skipping to change at page 32, line 46
[TUNNEL-ENCAP] Rosen et al., "The BGP Tunnel Encapsulation [TUNNEL-ENCAP] Rosen et al., "The BGP Tunnel Encapsulation
Attribute", draft-ietf-idr-tunnel-encaps-03, November Attribute", draft-ietf-idr-tunnel-encaps-03, November
2016. 2016.
[EVPN-PREFIX] Rabadan et al., "IP Prefix Advertisement in EVPN", [EVPN-PREFIX] Rabadan et al., "IP Prefix Advertisement in EVPN",
draft-ietf-bess-evpn-prefix-advertisement-03, September, draft-ietf-bess-evpn-prefix-advertisement-03, September,
2016. 2016.
11.2 Informative References 11.2 Informative References
[RFC7606] Chen, E., Scudder, J., Mohapatra, P., and K. Patel,
"Revised Error Handling for BGP UPDATE Messages", RFC 7606, August
2015, <http://www.rfc-editor.org/info/rfc7606>.
[802.1Q] "IEEE Standard for Local and metropolitan area networks - [802.1Q] "IEEE Standard for Local and metropolitan area networks -
Media Access Control (MAC) Bridges and Virtual Bridged Local Area Media Access Control (MAC) Bridges and Virtual Bridged Local Area
Networks", IEEE Std 802.1Q(tm), 2014 Edition, November 2014. Networks", IEEE Std 802.1Q(tm), 2014 Edition, November 2014.
[EVPN-IPVPN-INTEROP] Sajassi et al., "EVPN Seamless Interoperability [EVPN-IPVPN-INTEROP] Sajassi et al., "EVPN Seamless Interoperability
with IP-VPN", draft-sajassi-l2vpn-evpn-ipvpn-interop-01, work in with IP-VPN", draft-sajassi-l2vpn-evpn-ipvpn-interop-01, work in
progress, October, 2012. progress, October, 2012.
[DC-MOBILITY] Aggarwal et al., "Data Center Mobility based on [DC-MOBILITY] Aggarwal et al., "Data Center Mobility based on
BGP/MPLS, IP Routing and NHRP", draft-raggarwa-data-center-mobility- BGP/MPLS, IP Routing and NHRP", draft-raggarwa-data-center-mobility-
05.txt, work in progress, June, 2013. 05.txt, work in progress, June, 2013.
12 Contributors 12 Contributors
In addition to the authors listed on the front page, the following In addition to the authors listed on the front page, the following
co-authors have also contributed to this document: co-authors have also contributed to this document:
Samer Salam
Florin Balus Florin Balus
Cisco Cisco
Yakov Rekhter Yakov Rekhter
Juniper Juniper
Wim Henderickx Wim Henderickx
Nokia Nokia
Linda Dunbar Linda Dunbar
 End of changes. 116 change blocks. 
594 lines changed or deleted 889 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/