draft-ietf-bess-evpn-inter-subnet-forwarding-01.txt   draft-ietf-bess-evpn-inter-subnet-forwarding-02.txt 
L2VPN Workgroup Ali Sajassi L2VPN Workgroup A. Sajassi, Ed.
INTERNET-DRAFT Samer Salam INTERNET-DRAFT S. Salam
Intended Status: Standards Track Samir Thoria Intended Status: Standards Track S. Thoria
Cisco Cisco
Wim Henderickx J. Drake
Jorge Rabadan Yakov Rekhter
Alcatel-Lucent John Drake
Juniper Juniper
Florin Balus J. Rabadan
Nuage Networks Lucy Yong Nokia
Linda Dunbar L. Yong
Dennis Cai Huawei Huawei
Cisco
Expires: April 18, 2016 October 18, 2015 Expires: August 8, 2017 February 8, 2017
Integrated Routing and Bridging in EVPN Integrated Routing and Bridging in EVPN
draft-ietf-bess-evpn-inter-subnet-forwarding-01 draft-ietf-bess-evpn-inter-subnet-forwarding-02
Abstract Abstract
EVPN provides an extensible and flexible multi-homing VPN solution EVPN provides an extensible and flexible multi-homing VPN solution
for intra-subnet connectivity among hosts/VMs over an MPLS/IP for intra-subnet connectivity among hosts/VMs over an MPLS/IP
network. However, there are scenarios in which inter-subnet network. However, there are scenarios in which inter-subnet
forwarding among hosts/VMs across different IP subnets is required, forwarding among hosts/VMs across different IP subnets is required,
while maintaining the multi-homing capabilities of EVPN. This while maintaining the multi-homing capabilities of EVPN. This
document describes an Integrated Routing and Bridging (IRB) solution document describes an Integrated Routing and Bridging (IRB) solution
based on EVPN to address such requirements. based on EVPN to address such requirements.
skipping to change at page 2, line 26 skipping to change at page 2, line 23
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Inter-Subnet Forwarding Scenarios . . . . . . . . . . . . . . . 5 2 Inter-Subnet Forwarding Scenarios . . . . . . . . . . . . . . . 6
2.1 Switching among Subnets within a DC . . . . . . . . . . . . 6 2.1 Switching among IP subnets within a DC . . . . . . . . . . . 7
2.2 Switching among EVIs in different DCs without route 2.2 Switching among IP subnets in different DCs without GW . . . 8
aggregation . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Switching among IP subnets in different DCs with GW . . . . 8
2.3 Switching among EVIs in different DCs with route 2.4 Switching among IP subnets spread across IP-VPN and EVPN
aggregation . . . . . . . . . . . . . . . . . . . . . . . . 7 networks with GW . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Switching among IP-VPN sites and EVIs with route 3 Default L3 Gateway for Tenant System . . . . . . . . . . . . . . 9
aggregation . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1 Homogeneous Environment . . . . . . . . . . . . . . . . . . 9
3 Default L3 Gateway Addressing . . . . . . . . . . . . . . . . . 8 3.2 Heterogeneous Environment . . . . . . . . . . . . . . . . . 10
3.1 Homogeneous Environment . . . . . . . . . . . . . . . . . . 8 4 Operational Models for Asymmetric Inter-Subnet Forwarding . . . 10
3.2 Heterogeneous Environment . . . . . . . . . . . . . . . . . 9 4.1 Among EVPN NVEs within a DC . . . . . . . . . . . . . . . . 10
4 Operational Models for Asymmetric Inter-Subnet Forwarding . . . 9 4.2 Among EVPN NVEs in Different DCs Without GW . . . . . . . . 11
4.1 Among EVPN NVEs within a DC . . . . . . . . . . . . . . . . 9 4.3 Among EVPN NVEs in Different DCs with GW . . . . . . . . . . 13
4.2 Among EVPN NVEs in Different DCs Without Route Aggregation . 10 4.4 Among IP-VPN Sites and EVPN NVEs with GW . . . . . . . . . . 14
4.3 Among EVPN NVEs in Different DCs with Route Aggregation . . 12 4.5 Use of Centralized Gateway . . . . . . . . . . . . . . . . . 15
4.4 Among IP-VPN Sites and EVPN NVEs with Route Aggregation . . 13 5 Operational Models for Symmetric Inter-Subnet Forwarding . . . . 16
4.5 Use of Centralized Gateway . . . . . . . . . . . . . . . . . 14 5.1 IRB forwarding on NVEs for Tenant Systems . . . . . . . . . 16
5 Operational Models for Symmetric Inter-Subnet Forwarding . . . . 15 5.1.1 Control Plane Operation . . . . . . . . . . . . . . . . 17
5.1 IRB forwarding on NVEs for Tenant Systems . . . . . . . . . 15 5.1.2 Data Plane Operation - Inter Subnet . . . . . . . . . . 18
5.1.1 Control Plane Operation . . . . . . . . . . . . . . . . 16 5.1.3 TS Move Operation . . . . . . . . . . . . . . . . . . . 19
5.1.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 17 5.2 IRB forwarding on NVEs for Subnets behind Tenant Systems . . 20
5.1.3 TS Move Operation . . . . . . . . . . . . . . . . . . . 18 5.2.1 Control Plane Operation . . . . . . . . . . . . . . . . 22
5.2.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 23
6 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 24
6.1 Router's MAC Extended Community . . . . . . . . . . . . . . 24
5.2 IRB forwarding on NVEs for Subnets behind Tenant Systems . . 19 7 TS Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.2.1 Control Plane Operation . . . . . . . . . . . . . . . . 21 7.1 TS Mobility & Optimum Forwarding for TS Outbound Traffic . . 24
5.2.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 22 7.2 TS Mobility & Optimum Forwarding for TS Inbound Traffic . . 24
6 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 23 7.2.1 Mobility without Route Aggregation . . . . . . . . . . . 25
6.1 Router's MAC Extended Community . . . . . . . . . . . . . . 23 8 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25
7 TS Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . 23 9 Security Considerations . . . . . . . . . . . . . . . . . . . . 25
7.1 TS Mobility & Optimum Forwarding for TS Outbound Traffic . . 23 10 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25
7.2 TS Mobility & Optimum Forwarding for TS Inbound Traffic . . 23 11 References . . . . . . . . . . . . . . . . . . . . . . . . . . 25
7.2.1 Mobility without Route Aggregation . . . . . . . . . . . 24
7.2.2 Mobility with Route Aggregation . . . . . . . . . . . . 24
8 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24
9 Security Considerations . . . . . . . . . . . . . . . . . . . . 24
10 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
11 References . . . . . . . . . . . . . . . . . . . . . . . . . . 24
11.1 Normative References . . . . . . . . . . . . . . . . . . . 25 11.1 Normative References . . . . . . . . . . . . . . . . . . . 25
11.2 Informative References . . . . . . . . . . . . . . . . . . 25 11.2 Informative References . . . . . . . . . . . . . . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 12 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26
Terminology Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
EVI : EVPN Instance Broadcast Domain: In a bridged network, the broadcast domain
corresponds to a Virtual LAN (VLAN), where a VLAN is typically
represented by a single VLAN ID (VID) but can be represented by
several VIDs where Shared VLAN Learning (SVL) is used per [802.1Q].
EVI : An EVPN instance spanning the Provider Edge (PE) devices
participating in that EVPN
IRB: Integrated Routing and Bridging IRB: Integrated Routing and Bridging
MAC-VRF: A Virtual Routing and Forwarding table for MAC addresses on MAC-VRF: A Virtual Routing and Forwarding table for Media Access
a PE for an EVI Control (MAC) addresses on a PE for an EVI
Bridge Table: An instantiation of a broadcast domain on a MAC-VRF
IP-VRF: A Virtual Routing and Forwarding table for IP addresses on a IP-VRF: A Virtual Routing and Forwarding table for IP addresses on a
PE that is associated with one or more EVIs PE that is associated with one or more EVIs
IRB Interface: A virtual interface that connects the MAC-VRF and the IRB Interface: A virtual interface that connects a bridge table in a
IP-VRF on an NVE. MAC-VRF to an IP-VRF in an NVE.
NVE: Network Virtualization Endpoint NVE: Network Virtualization Endpoint
TS: Tenant System TS: Tenant System
Ethernet NVO tunnel: It refers to Network Virtualization Overlay
tunnels with Ethernet payload. Example of this type of tunnels are
VxLAN and NvGRE.
IP NVO tunnel: It refers to Network Virtualization Overlay tunnels
with IP payload (no MAC header in the payload). Examples of IP NVO
tunnels are VxLAN GPE or MPLSoGRE (both with IP payload).
1 Introduction 1 Introduction
EVPN provides an extensible and flexible multi-homing VPN solution EVPN provides an extensible and flexible multi-homing VPN solution
for intra-subnet connectivity among Tenant Systems (TS's) over an for intra-subnet connectivity among Tenant Systems (TS's) over an
MPLS/IP network. However, there are scenarios where, in addition to MPLS/IP network; where, an IP subnet is represented by an EVI for a
intra-subnet forwarding, inter-subnet forwarding is required among VLAN-based service or by an <EVI, VLAN> for a VLAN-aware bundle
TS's across different IP subnets at the EVPN PE nodes, also known as service. However, there are scenarios where, in addition to intra-
EVPN NVE nodes throughout this document, while maintaining the multi- subnet forwarding, inter-subnet forwarding is required among TS's
homing capabilities of EVPN. This document describes an Integrated across different IP subnets at EVPN PE nodes, also known as EVPN NVE
Routing and Bridging (IRB) solution based on EVPN to address such nodes throughout this document, while maintaining the multi-homing
capabilities of EVPN. This document describes an Integrated Routing
and Bridging (IRB) solution based on EVPN to address such
requirements. requirements.
The inter-subnet communication is traditionally achieved at The inter-subnet communication is traditionally achieved at
centralized L3 Gateway (L3GW) nodes where all the inter-subnet centralized L3 Gateway (L3GW) nodes where all the inter-subnet
communication policies are enforced. When two Tenant Systems (TS's) communication policies are enforced. When two Tenant Systems (TS's)
belonging to two different subnets connected to the same PE node belonging to two different subnets connected to the same PE node,
wanted to talk to each other, their traffic needed to be back hauled wanted to talk to each other, their traffic needed to be back hauled
from the PE node all the way to the centralized gateway nodes where from the PE node all the way to the centralized gateway nodes where
inter-subnet switching is performed and then back to the PE node. For inter-subnet switching is performed and then back to the PE node. For
today's large multi-tenant data center, this scheme is very today's large multi-tenant data center, this scheme is very
inefficient and sometimes impractical. inefficient and sometimes impractical.
In order to overcome the drawback of centralized approach, IRB In order to overcome the drawback of centralized approach, IRB
functionality is needed on the PE nodes (i.e., NVE devices) as close functionality is needed on the PE nodes (i.e., NVE devices) as close
to TS as possible to avoid hair pinning of user traffic to TS as possible to avoid hair pinning of user traffic
unnecessarily. Under this design, all traffic between hosts attached unnecessarily. Under this design, all traffic between hosts attached
to one NVE can be routed and bridged locally, thus avoiding traffic to one NVE can be routed and bridged locally, thus avoiding traffic
hair-pinning issue of the centralized L3GW. hair-pinning issue of the centralized L3GW.
There can be scenarios where both centralized and decentralized There can be scenarios where both centralized and distributed
approaches may be preferred simultaneously. For example, to allow approaches may be preferred simultaneously. For example, to allow
NVEs to switch inter-subnet traffic belonging to one tenant or one NVEs to switch inter-subnet traffic belonging to one tenant or one
security zone locally; whereas, to back haul inter-subnet traffic security zone locally; whereas, to back haul inter-subnet traffic
belonging to two different tenants or security zones to the belonging to two different tenants or security zones to the
centralized gateway nodes and perform switching there after the centralized gateway nodes and perform switching there after the
traffic is subjected to Firewall or Deep Packet Inspection (DPI). traffic is subjected to Firewall (FW) or Deep Packet Inspection
(DPI).
Some TS's run non-IP protocols in conjunction with their IP traffic. Some TS's run non-IP protocols in conjunction with their IP traffic.
Therefore, it is important to handle both kinds of traffic optimally Therefore, it is important to handle both kinds of traffic optimally
- e.g., to bridge non-IP traffic and to route IP traffic. - e.g., to bridge non-IP traffic and to route IP traffic.
Therefore, the solution needs to meet the following requirements: Therefore, the solution needs to meet the following requirements:
R1: The solution MUST allow for inter-subnet traffic to be locally R1: The solution MUST allow for inter-subnet traffic to be locally
switched at NVEs. switched at NVEs.
R2: The solution MUST allow for both inter-subnet and intra-subnet R2: The solution MUST allow for both inter-subnet and intra-subnet
traffic belonging to the same tenant to be locally routed and bridged traffic belonging to the same tenant to be locally routed and bridged
respectively. The solution MUST provide IP routing for inter-subnet respectively. The solution MUST provide IP routing for inter-subnet
traffic and Ethernet Bridging for intra-subnet traffic. traffic and Ethernet Bridging for intra-subnet traffic.
R3: The solution MUST support bridging of non-IP traffic. R3: The solution MUST support bridging of non-IP traffic.
R4: The solution MUST allow inter-subnet switching to be disabled on R4: The solution MUST allow inter-subnet switching to be disabled on
a per VLAN basis on NVEs where the traffic needs to be back hauled to a per VLAN basis on NVEs where the traffic needs to be back hauled to
another node (e.g., for performing FW or DPI functionality). another node (i.e., for performing FW or DPI functionality).
2 Inter-Subnet Forwarding Scenarios 2 Inter-Subnet Forwarding Scenarios
The inter-subnet forwarding scenarios performed by an EVPN NVE can be The inter-subnet forwarding scenarios performed by an EVPN NVE can be
divided into the following five categories. The last scenario, along divided into the following five categories. The last scenario, along
with their corresponding solutions, are described in [EVPN-IPVPN- with its corresponding solution, are described in [EVPN-IPVPN-
INTEROP]. The solutions for the first four scenarios are the focus of INTEROP]. The first four scenarios are covered in this document.
this document.
1. Switching among EVIs (subnets) within a DC 1. Switching among IP subnets within a DC using EVPN
2. Switching among EVIs (subnets) in different DCs without route 2. Switching among IP subnets in different DCs using EVPN without GW
aggregation
3. Switching among EVIs (subnets) in different DCs with route 3. Switching among IP subnets in different DCs using EVPN with GW
aggregation
4. Switching among IP-VPN instance and EVIs with route aggregation 4. Switching among IP subnets spread across IP-VPN and EVPN networks
with GW
5. Switching among IP-VPN instance and EVIs without route aggregation 5. Switching among IP subnets spread across IP-VPN and EVPN networks
without GW
In the above scenario, the term "route aggregation" refers to the In the above scenario, the term "GW" refers to the case where a node
case where a node situated at the WAN edge of the data center network situated at the WAN edge of the data center network behaves as a
behaves as a default gateway for all the destinations that are default gateway (GW) for all the destinations that are outside the
outside the data center. The absence of route aggregation refers to data center. The absence of GW refers to the scenario where NVEs
the scenario where NVEs within a data center maintain individual within a data center maintain individual (host) routes that are
(host) routes that are outside of the data center. outside of the data center.
In the case (4), the WAN edge node also performs route aggregation In the case (4), the WAN edge node also performs route aggregation
for all the destinations within its own data center, and acts as an for all the destinations within its own data center, and acts as an
interworking unit between EVPN and IP VPN (it implements both EVPN interworking unit between EVPN and IP VPN (it implements both EVPN
and IP VPN functionality). and IP-VPN functionality).
+---+ Enterprise Site 1 +---+ Enterprise Site 1
|PE1|----- H1 |PE1|----- H1
+---+ +---+
/ /
,---------. Enterprise Site 2 ,---------. Enterprise Site 2
,' `. +---+ ,' `. +---+
,---------. /( MPLS/IP )---|PE2|----- H2 ,---------. /( MPLS/IP )---|PE2|----- H2
' DCN 3 `./ `. Core ,' +---+ ' DCN 3 `./ `. Core ,' +---+
`-+------+' `-+------+' `-+------+' `-+------+'
skipping to change at page 6, line 33 skipping to change at page 7, line 33
'-----' :NVE2 : :NVE3 : '-----' :NVE2 : :NVE3 :
| | '-----' '-----' | | '-----' '-----'
TS1 TS2 | | | TS1 TS2 | | |
TS3 TS4 TS5 TS3 TS4 TS5
Figure 2: Interoperability Use-Cases Figure 2: Interoperability Use-Cases
In what follows, we will describe scenarios 1 through 4 in more In what follows, we will describe scenarios 1 through 4 in more
detail. detail.
2.1 Switching among Subnets within a DC 2.1 Switching among IP subnets within a DC
In this scenario, connectivity is required between TS's in the same In this scenario, connectivity is required between TS's in the same
data center, where those hosts belong to different IP subnets. All data center, where those hosts belong to different IP subnets. All
these subnets belong to the same tenant or are part of the same IP these subnets belong to the same tenant or are part of the same IP
VPN. Each subnet is associated with a single EVPN instance (EVI) VPN. Each subnet is associated with a single EVI (or <EVI,VLAN>)
realized by a collection of MAC-VRFs (one per NVE) residing on the realized by a collection of MAC-VRFs (one per NVE) residing on the
NVEs configured for that EVI. NVEs configured for that EVI.
As an example, consider TS3 and TS5 of Figure 2 above. Assume that As an example, consider TS3 and TS5 of Figure 2 above. Assume that
connectivity is required between these two TS's where TS3 belongs to connectivity is required between these two TS's where TS3 belongs to
the IP-subnet 3 (SN3) whereas TS5 belongs to the IP-subnet 5 (SN5). the IP-subnet 3 (SN3) whereas TS5 belongs to the IP-subnet 5 (SN5).
Both SN3 and SN5 subnets belong to the same tenant (e.g., are part of Both SN3 and SN5 subnets belong to the same tenant. NVE2 has an EVI3
the same IP-VRF). NVE2 has an EVI3 associated with the SN3 and this associated with the SN3 and this EVI is represented by a MAC-VRF
EVI is represented by a MAC-VRF which is associated with an IP-VRF which is associated with an IP-VRF (for that tenant) via an IRB
(for that IP VPN) via an IRB interface. NVE3 respectively has an EVI5 interface. NVE3 respectively has an EVI5 associated with the SN5 and
associated with the SN5 and this EVI is represented by an MAC-VRF this EVI is represented by an MAC-VRF which is associated with the
which is associated with the same IP-VRF via a different IRB same IP-VRF via a different IRB interface.
interface.
2.2 Switching among EVIs in different DCs without route aggregation 2.2 Switching among IP subnets in different DCs without GW
This case is similar to that of section 2.1 above albeit for the fact This case is similar to that of section 2.1 above albeit for the fact
that the TS's belong to different data centers that are that the TS's belong to different data centers that are
interconnected over a WAN (e.g. MPLS/IP PSN). The data centers in interconnected over a WAN (e.g. MPLS/IP PSN). The data centers in
question here are seamlessly interconnected to the WAN, i.e., the WAN question here are seamlessly interconnected to the WAN, i.e., the WAN
edge devices do not maintain any TS-specific addresses in the edge devices do not maintain any TS-specific addresses in the
forwarding path - e.g., there is no WAN edge GW(s) between these DCs. forwarding path - e.g., there is no WAN edge GW(s) between these DCs.
As an example, consider TS3 and TS6 of Figure 2 above. Assume that As an example, consider TS3 and TS6 of Figure 2 above. Assume that
connectivity is required between these two TS's where TS3 belongs to connectivity is required between these two TS's where TS3 belongs to
the SN3 whereas TS6 belongs to the SN6. NVE2 has an EVI3 associated the SN3 whereas TS6 belongs to the SN6. NVE2 has an EVI3 associated
with SN3 and NVE4 has an EVI6 associated with the SN6. Both SN3 and with SN3 and NVE4 has an EVI6 associated with the SN6. Both SN3 and
SN6 are part of the same IP-VRF. SN6 are part of the same IP-VRF.
2.3 Switching among EVIs in different DCs with route aggregation 2.3 Switching among IP subnets in different DCs with GW
In this scenario, connectivity is required between TS's in different In this scenario, connectivity is required between TS's in different
data centers, and those hosts belong to different IP subnets. What data centers, and those hosts belong to different IP subnets. What
makes this case different from that of Section 2.2 is that (in the makes this case different from that of Section 2.2 is that at least
context of a given IP-VRF) at least one of the data centers in one of the data centers has a gateway as the WAN edge switch. Because
question has a gateway as the WAN edge switch. Because of that, the of that, the NVE's IP-VRF within that data center need not maintain
NVE's IP-VRF within each data center need not maintain (host) routes (host) routes to individual TS's outside of that data center.
to individual TS's outside of the data center.
As an example, consider TS1 and TS5 of Figure 2 above. Assume that As an example, consider a tenant with TS1 and TS5 of Figure 2 above.
connectivity is required between these two TS's where TS1 belongs to Assume that connectivity is required between these two TS's where TS1
the SN1 whereas TS5 belongs to the SN5 thus SN1 and SN5 belong to the belongs to the SN1 whereas TS5 belongs to the SN5. NVE3 has an EVI5
same IP VPN. NVE3 has an EVI5 associated with the SN5 and this EVI is associated with the SN5 and this EVI is represented by the MAC-VRF
represented by the MAC-VRF which is connected to the IP-VRF via an which is connected to the IP-VRF via an IRB interface. NVE1 has an
IRB interface. NVE1 has an EVI1 associated with the SN1 and this EVI EVI1 associated with the SN1 and this EVI is represented by the MAC-
is represented by the MAC-VRF which is connected to the IP-VRF VRF which is connected to the IP-VRF representing the same tenant.
representing the same IP VPN. Due to the gateway at the edge of DCN Due to the gateway at the edge of DCN 1, NVE1's IP-VRF does not need
1, NVE1's IP-VRF does not need to have the address of TS5 but instead to have the address of TS5 but instead it has a default route in its
it has a default route in its IP-VRF with the next-hop being the GW. IP-VRF with the next-hop being the GW.
2.4 Switching among IP-VPN sites and EVIs with route aggregation 2.4 Switching among IP subnets spread across IP-VPN and EVPN networks
with GW
In this scenario, connectivity is required between TS's in a data In this scenario, connectivity is required between TS's in a data
center and hosts in an enterprise site that belongs to a given IP- center and hosts in an enterprise site that belongs to a given IP-
VPN. The NVE within the data center is an EVPN NVE, whereas the VPN. The NVE within the data center is an EVPN NVE, whereas the
enterprise site has an IP-VPN PE. Furthermore, the data center in enterprise site has an IP-VPN PE. Furthermore, the data center in
question has a gateway as the WAN edge switch. Because of that, the question has a gateway as the WAN edge switch. Because of that, the
NVE in the data center does not need to maintain individual IP NVE in the data center does not need to maintain individual IP
prefixes advertised by enterprise sites (by IP-VPN PEs). prefixes advertised by enterprise sites (by IP-VPN PEs).
As an example, consider end-station H1 and TS2 of Figure 2. Assume As an example, consider end-station H1 and TS2 of Figure 2. Assume
skipping to change at page 8, line 18 skipping to change at page 9, line 18
SN2. Moreover, EVI2 on NVE1 is connected to an IP-VRF associated with SN2. Moreover, EVI2 on NVE1 is connected to an IP-VRF associated with
that IP VPN. PE1 originates a VPN-IP route that covers H1. The that IP VPN. PE1 originates a VPN-IP route that covers H1. The
gateway at the edge of DCN1 performs interworking function between gateway at the edge of DCN1 performs interworking function between
IP-VPN and EVPN. As a result of this, a default route in the IP-VRF IP-VPN and EVPN. As a result of this, a default route in the IP-VRF
on the NVE1, pointing to the gateway as the next hop, and a route to on the NVE1, pointing to the gateway as the next hop, and a route to
the TS2 (or maybe SN2) on the PE1's IP-VRF are sufficient for the the TS2 (or maybe SN2) on the PE1's IP-VRF are sufficient for the
connectivity between H1 and TS2. In this scenario, the NVE1's IP-VRF connectivity between H1 and TS2. In this scenario, the NVE1's IP-VRF
does not need to maintain a route to H1 because it has the default does not need to maintain a route to H1 because it has the default
route to the gateway. route to the gateway.
3 Default L3 Gateway Addressing 3 Default L3 Gateway for Tenant System
3.1 Homogeneous Environment 3.1 Homogeneous Environment
This is an environment where all NVEs to which an EVPN instance could This is an environment where all NVEs to which an EVPN instance could
potentially be attached (or moved), perform inter-subnet switching. potentially be attached (or moved), perform inter-subnet switching.
Therefore, inter-subnet traffic can be locally switched by the EVPN Therefore, inter-subnet traffic can be locally switched by the EVPN
NVE connecting the TS's belonging to different subnets. NVE connecting the TS's belonging to different subnets.
To support such inter-subnet forwarding, the NVE behaves as an IP To support such inter-subnet forwarding, the NVE behaves as an IP
Default Gateway from the perspective of the attached TS's. Two models Default Gateway from the perspective of the attached TS's. Two models
are possible: are possible:
1. All the NVEs of a given EVPN instance use the same anycast default 1. All the NVEs of a given EVPN instance use the same anycast default
gateway IP address and the same anycast default gateway MAC address. gateway IP address and the same anycast default gateway MAC address.
On each NVE, this default gateway IP/MAC address correspond to the On each NVE, this default gateway IP/MAC address correspond to the
IRB interface of the MAC-VRF associated with that EVI. IRB interface connecting the MAC-VRF of that EVI to the corresponding
IP-VRF.
2. Each NVE of a given EVPN instance uses its own default gateway IP 2. Each NVE of a given EVPN instance uses its own default gateway IP
and MAC addresses, and these addresses are aliased to the same and MAC addresses, and these addresses are aliased to the same
conceptual gateway through the use of the Default Gateway extended conceptual gateway through the use of the Default Gateway extended
community as specified in [EVPN], which is carried in the EVPN MAC community as specified in [EVPN], which is carried in the EVPN MAC
Advertisement routes. On each NVE, this default gateway IP/MAC Advertisement routes. On each NVE, this default gateway IP/MAC
address correspond to the IRB interface of the MAC-VRF associated address correspond to the IRB interface connecting the MAC-VRF of
with that EVI. that EVI to the corresponding IP-VRF.
Both of these models enable a packet forwarding paradigm for both Both of these models enable a packet forwarding paradigm for both
symmetric and asymmetric IRB forwarding. In case of asymmetric IRB, a symmetric and asymmetric IRB forwarding. In case of asymmetric IRB, a
packet is forwarded through the MAC-VRF followed by the IP-VRF on the packet is forwarded through the MAC-VRF followed by the IP-VRF on the
ingress NVE, and then forwarded through the the MAC-VRF on the egress ingress NVE, and then forwarded through the the MAC-VRF on the egress
(disposition) NVE. The egress NVE merely needs to perform a lookup in (disposition) NVE. The egress NVE merely needs to perform a lookup in
the associated MAC-VRF and forward the Ethernet frames unmodified, the associated MAC-VRF and forward the Ethernet frames unmodified,
i.e. without rewriting the source MAC address. This is different i.e. without rewriting the source MAC address. This is different
from symmetric IRB forwarding where a packet is forwarded through the from symmetric IRB forwarding where a packet is forwarded through the
MAC-VRF followed by the IP-VRF on the ingress NVE, and then forwarded MAC-VRF followed by the IP-VRF on the ingress NVE, and then forwarded
skipping to change at page 9, line 20 skipping to change at page 10, line 21
ensure that the applications receive traffic from the same source MAC ensure that the applications receive traffic from the same source MAC
address that they are sending to. address that they are sending to.
3.2 Heterogeneous Environment 3.2 Heterogeneous Environment
For large data centers with thousands of servers and ToR (or Access) For large data centers with thousands of servers and ToR (or Access)
switches, some of them may not have the capability of maintaining or switches, some of them may not have the capability of maintaining or
enforcing policies for inter-subnet switching. Even though policies enforcing policies for inter-subnet switching. Even though policies
among multiple subnets belonging to same tenant can be simpler, hosts among multiple subnets belonging to same tenant can be simpler, hosts
belonging to one tenant can also send traffic to peers belonging to belonging to one tenant can also send traffic to peers belonging to
different tenants or security zones. In such scenarios, a L3GW may different tenants or security zones. In such scenarios, a WAN edge PE
not only need to enforce policies for communication among subnets (e.g., L3GW) may not only need to enforce policies for communication
belonging to a single tenant, but also it may need to know how to among subnets belonging to a single tenant, but also it may need to
handle traffic destined towards peers in different tenants. know how to handle traffic destined towards peers in different
Therefore, there can be a mixed environment where an NVE performs tenants. Therefore, there can be a mixed environment where an NVE
inter-subnet switching for some EVPN instances and the L3GW for performs inter-subnet switching for some EVPN instances and the L3GW
others. for others.
4 Operational Models for Asymmetric Inter-Subnet Forwarding 4 Operational Models for Asymmetric Inter-Subnet Forwarding
4.1 Among EVPN NVEs within a DC 4.1 Among EVPN NVEs within a DC
When an EVPN MAC advertisement route is received by the NVE, the IP When an EVPN MAC/IP advertisement route is received by a NVE, the IP
address associated with the route is used to populate the IP-VRF address associated with the route is used to populate the IP-VRF
table, whereas the MAC address associated with the route is used to table, whereas the MAC address associated with the route is used to
populate both the MAC-VRF table, as well as the adjacency associated populate both the MAC-VRF table, as well as the adjacency associated
with the IP route in the IP-VRF table. with the IP route in the IP-VRF table (i.e., ARP table).
When an Ethernet frame is received by an ingress NVE, it performs a When an Ethernet frame is received by an ingress NVE, it performs a
lookup on the destination MAC address in the associated MAC-VRF for lookup on the destination MAC address in the associated MAC-VRF for
that EVI. If the MAC address corresponds to its IRB Interface MAC that EVI. If the MAC address corresponds to its IRB Interface MAC
address, the ingress NVE deduces that the packet MUST be inter-subnet address, the ingress NVE deduces that the packet MUST be inter-subnet
routed. Hence, the ingress NVE performs an IP lookup in the routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup identifies both the next-hop associated IP-VRF table. The lookup identifies an adjacency that
(i.e. egress) NVE to which the packet must be forwarded, in addition contains a MAC rewrite and in turn the next-hop (i.e., egress) NVE to
to an adjacency that contains a MAC rewrite and an MPLS label stack. which the packet must be forwarded and the associated MPLS label
The MAC rewrite holds the MAC address associated with the destination stack. The MAC rewrite holds the MAC address associated with the
host (as populated by the EVPN MAC route), instead of the MAC address destination host (as populated by the EVPN MAC route), instead of the
of the next-hop NVE. The ingress NVE then rewrites the destination MAC address of the next-hop NVE. The ingress NVE then rewrites the
MAC address in the packet with the address specified in the destination MAC address in the packet with the address specified in
adjacency. It also rewrites the source MAC address with its IRB the adjacency. It also rewrites the source MAC address with its IRB
Interface MAC address. The ingress NVE, then, forwards the frame to Interface MAC address. The ingress NVE, then, forwards the frame to
the next-hop (i.e. egress) NVE after encapsulating it with the MPLS the next-hop (i.e. egress) NVE after encapsulating it with the MPLS
label stack. Note that this label stack includes the LSP label as label stack. Note that this label stack includes the LSP label as
well as the EVPN label that was advertised by the egress NVE. When well as the EVPN label that was advertised by the egress NVE. When
the MPLS encapsulated packet is received by the egress NVE, it uses the MPLS encapsulated packet is received by the egress NVE, it uses
the EVPN label to identify the MAC-VRF table. It then performs a MAC the EVPN label to identify the MAC-VRF table. It then performs a MAC
lookup in that table, which yields the outbound interface to which lookup in that table, which yields the outbound interface to which
the Ethernet frame must be forwarded. Figure 2 below depicts the the Ethernet frame must be forwarded. Figure 2 below depicts the
packet flow, where NVE1 and NVE2 are the ingress and egress NVEs, packet flow, where NVE1 and NVE2 are the ingress and egress NVEs,
respectively. respectively.
skipping to change at page 10, line 43 skipping to change at page 11, line 44
IRB. IRB.
It should also be noted that [EVPN] provides different level of It should also be noted that [EVPN] provides different level of
granularity for the EVPN label. Besides identifying bridge domain granularity for the EVPN label. Besides identifying bridge domain
table, it can be used to identify the egress interface or a table, it can be used to identify the egress interface or a
destination MAC address on that interface. If EVPN label is used for destination MAC address on that interface. If EVPN label is used for
egress interface or destination MAC address identification, then no egress interface or destination MAC address identification, then no
MAC lookup is needed in the egress EVI and the packet can be directly MAC lookup is needed in the egress EVI and the packet can be directly
forwarded to the egress interface just based on EVPN label lookup. forwarded to the egress interface just based on EVPN label lookup.
4.2 Among EVPN NVEs in Different DCs Without Route Aggregation 4.2 Among EVPN NVEs in Different DCs Without GW
When an EVPN MAC advertisement route is received by the NVE, the IP When an EVPN MAC advertisement route is received by a NVE, the IP
address associated with the route is used to populate the IP-VRF address associated with the route is used to populate the IP-VRF
table, whereas the MAC address associated with the route is used to table, whereas the MAC address associated with the route is used to
populate both the MAC-VRF table, as well as the adjacency associated populate both the MAC-VRF table, as well as the adjacency associated
with the IP route in the IP-VRF table. with the IP route in the IP-VRF table (i.e., ARP table).
When an Ethernet frame is received by an ingress NVE, it performs a When an Ethernet frame is received by an ingress NVE, it performs a
lookup on the destination MAC address in the associated EVI. If the lookup on the destination MAC address in the associated EVI. If the
MAC address corresponds to its IRB Interface MAC address, the ingress MAC address corresponds to its IRB Interface MAC address, the ingress
NVE deduces that the packet MUST be inter-subnet routed. Hence, the NVE deduces that the packet MUST be inter-subnet routed. Hence, the
ingress NVE performs an IP lookup in the associated IP-VRF table. The ingress NVE performs an IP lookup in the associated IP-VRF table. The
lookup identifies both the next-hop (i.e. egress) Gateway to which lookup identifies an adjacency that contains a MAC rewrite and in
the packet must be forwarded, in addition to an adjacency that turn the next-hop (i.e. egress) Gateway to which the packet must be
contains a MAC rewrite and an MPLS label stack. The MAC rewrite holds forwarded along with the associated MPLS label stack. The MAC rewrite
the MAC address associated with the destination host (as populated by holds the MAC address associated with the destination host (as
the EVPN MAC route), instead of the MAC address of the next-hop populated by the EVPN MAC route), instead of the MAC address of the
Gateway. The ingress NVE then rewrites the destination MAC address in next-hop Gateway. The ingress NVE then rewrites the destination MAC
the packet with the address specified in the adjacency. It also address in the packet with the address specified in the adjacency. It
rewrites the source MAC address with its IRB Interface MAC address. also rewrites the source MAC address with its IRB Interface MAC
The ingress NVE, then, forwards the frame to the next-hop (i.e. address. The ingress NVE, then, forwards the frame to the next-hop
egress) Gateway after encapsulating it with the MPLS label stack. (i.e. egress) Gateway after encapsulating it with the MPLS label
stack.
Note that this label stack includes the LSP label as well as an EVPN Note that this label stack includes the LSP label as well as an EVPN
label. The EVPN label could be either advertised by the ingress label. The EVPN label could be either advertised by the ingress
Gateway, if inter-AS option B is used, or advertised by the egress Gateway, if inter-AS option B is used, or advertised by the egress
NVE, if inter-AS option C is used. When the MPLS encapsulated packet NVE, if inter-AS option C is used. When the MPLS encapsulated packet
is received by the ingress Gateway, the processing again differs is received by the ingress Gateway, the processing again differs
depending on whether inter-AS option B or option C is employed: in depending on whether inter-AS option B or option C is employed: in
the former case, the ingress Gateway swaps the EVPN label in the the former case, the ingress Gateway swaps the EVPN label in the
packets with the EVPN label value received from the egress Gateway. packets with the EVPN label value received from the egress Gateway.
In the latter case, the ingress Gateway does not modify the EVPN In the latter case, the ingress Gateway does not modify the EVPN
skipping to change at page 12, line 17 skipping to change at page 13, line 17
| | | | | | | | | | | | | | | |
|(MAC - (IP | | [LS] | | [LS] | |(IP - (MAC | |(MAC - (IP | | [LS] | | [LS] | |(IP - (MAC |
| VRF) VRF)| | | | | | VRF) VRF)| | VRF) VRF)| | | | | | VRF) VRF)|
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V ^ v ^ V ^ V ^ V
| | | | | | | | | | | | | | | |
TS1->-+ +-->--------+ +------------+ +---------------+ +->-TS2 TS1->-+ +-->--------+ +------------+ +---------------+ +->-TS2
Figure 3: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs Figure 3: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs
without Route Aggregation without GW
4.3 Among EVPN NVEs in Different DCs with Route Aggregation 4.3 Among EVPN NVEs in Different DCs with GW
In this scenario, the NVEs within a given data center do not have In this scenario, the NVEs within a given data center do not have
entries for the MAC/IP addresses of hosts in remote data centers. entries for the MAC/IP addresses of hosts in remote data centers.
Rather, the NVEs have a default IP route pointing to the WAN gateway Rather, the NVEs have a default IP route pointing to the WAN gateway
for each VRF. This is accomplished by the WAN gateway advertising for for each VRF. This is accomplished by the WAN gateway advertising for
a given EVPN that spans multiple DC a default VPN-IP route that is a given EVPN that spans multiple DC a default VPN-IP route that is
imported by the NVEs of that EVPN that are in the gateway's own DC. imported by the NVEs of that VPN that are in the gateway's own DC.
When an Ethernet frame is received by an ingress NVE, it performs a When an Ethernet frame is received by an ingress NVE, it performs a
lookup on the destination MAC address in the associated MAC-VRF lookup on the destination MAC address in the associated MAC-VRF
table. If the MAC address corresponds to the IRB Interface MAC table. If the MAC address corresponds to the IRB Interface MAC
address, the ingress NVE deduces that the packet MUST be inter-subnet address, the ingress NVE deduces that the packet MUST be inter-subnet
routed. Hence, the ingress NVE performs an IP lookup in the routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup, in this case, matches the associated IP-VRF table. The lookup, in this case, matches the
default route which points to the local WAN gateway. The ingress NVE default host route which points to the local WAN gateway. The ingress
then rewrites the destination MAC address in the packet with the IRB NVE then rewrites the destination MAC address in the packet with the
Interface MAC address of the local WAN gateway. It also rewrites the router's MAC address of the local WAN gateway. It also rewrites the
source MAC address with its own IRB Interface MAC address. The source MAC address with its own IRB Interface MAC address. The
ingress NVE, then, forwards the frame to the WAN gateway after ingress NVE, then, forwards the frame to the WAN gateway after
encapsulating it with the MPLS label stack. Note that this label encapsulating it with the MPLS label stack. Note that this label
stack includes the LSP label as well as the IP-VPN label that was stack includes the LSP label as well as the label for default host
advertised by the local WAN gateway. When the MPLS encapsulated route that was advertised by the local WAN gateway. When the MPLS
packet is received by the local WAN gateway, it uses the IP-VPN label encapsulated packet is received by the local WAN gateway, it uses the
to identify the IP-VRF table. It then performs an IP lookup in that default host route label to identify the IP-VRF table. It then
table. The lookup identifies both the remote WAN gateway (of the performs an IP lookup in that table. The lookup identifies an
remote data center) to which the packet must be forwarded, in adjacency that contains a MAC rewrite and in turn the remote WAN
addition to an adjacency that contains a MAC rewrite and an MPLS gateway (of the remote data center) to which the packet must be
label stack. The MAC rewrite holds the MAC address associated with forwarded along with the associated MPLS label stack. The MAC rewrite
the ultimate destination host (as populated by the EVPN MAC route). holds the MAC address associated with the ultimate destination host
The local WAN gateway then rewrites the destination MAC address in (as populated by the EVPN MAC route). The local WAN gateway then
the packet with the address specified in the adjacency. It also rewrites the destination MAC address in the packet with the address
rewrites the source MAC address with its IRB Interface MAC address. specified in the adjacency. It also rewrites the source MAC address
with its router's MAC address. The local WAN gateway, then, forwards
The local WAN gateway, then, forwards the frame to the remote WAN the frame to the remote WAN gateway after encapsulating it with the
gateway after encapsulating it with the MPLS label stack. Note that MPLS label stack. Note that this label stack includes the LSP label
this label stack includes the LSP label as well as a EVPN label that as well as a EVPN label that was advertised by the remote WAN
was advertised by the remote WAN gateway. When the MPLS encapsulated gateway. When the MPLS encapsulated packet is received by the remote
packet is received by the remote WAN gateway, it simply swaps the WAN gateway, it simply swaps the EVPN label and forwards the packet
EVPN label and forwards the packet to the egress NVE. This implies to the egress NVE. This implies that the GW1 needs to keep the remote
that the GW1 needs to keep the remote host MAC addresses along with host MAC addresses along with the corresponding EVPN labels in the
the corresponding EVPN labels in the adjacency entries of the IP-VRF adjacency entries of the IP-VRF table (i.e., its ARP table). The
table. The remote WAN gateway then forward the packet to the egress remote WAN gateway then forward the packet to the egress NVE. The
NVE. The egress NVE then performs a MAC lookup in the MAC-VRF egress NVE then performs a MAC lookup in the MAC-VRF (identified by
(identified by the received EVPN label) to determine the outbound the received EVPN label) to determine the outbound port to send the
port to send the traffic on. traffic on.
Figure 4 below depicts the forwarding model. Figure 4 below depicts the forwarding model.
NVE1 GW1 GW2 NVE2 NVE1 GW1 GW2 NVE2
+------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+
| | | | | | | | | | | | | | | |
|(MAC - (IP | |(IP - (MAC | | [LS] | |(IP - (MAC | |(MAC - (IP | |(IP - (MAC | | [LS] | |(IP - (MAC |
| VRF) VRF)| | VRF) VRF)| | | | | | VRF) VRF)| | VRF) VRF)| | VRF) VRF)| | | | | | VRF) VRF)|
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V ^ v ^ V ^ V ^ V
| | | | | | | | | | | | | | | |
TS1->-+ +-->-----+ +---------------+ +---------------+ +->-TS2 TS1->-+ +-->-----+ +---------------+ +---------------+ +->-TS2
Figure 4: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs Figure 4: Inter-Subnet Forwarding Among EVPN NVEs in Different DCs
with Route Aggregation with GW
4.4 Among IP-VPN Sites and EVPN NVEs with Route Aggregation 4.4 Among IP-VPN Sites and EVPN NVEs with GW
In this scenario, the NVEs within a given data center do not have In this scenario, the NVEs within a given data center do not have
entries for the IP addresses of hosts in remote enterprise sites. entries for the IP addresses of hosts in remote enterprise sites.
Rather, the NVEs have a default IP route pointing the WAN gateway for Rather, the NVEs have a default IP route pointing the WAN gateway for
each IP-VRF. each IP-VRF.
When an Ethernet frame is received by an ingress NVE, it performs a When an Ethernet frame is received by an ingress NVE, it performs a
lookup on the destination MAC address in the associated MAC-VRF lookup on the destination MAC address in the associated MAC-VRF
table. If the MAC address corresponds to the IRB Interface MAC table. If the MAC address corresponds to the IRB Interface MAC
address, the ingress NVE deduces that the packet MUST be inter-subnet address, the ingress NVE deduces that the packet MUST be inter-subnet
routed. Hence, the ingress NVE performs an IP lookup in the routed. Hence, the ingress NVE performs an IP lookup in the
associated IP-VRF table. The lookup, in this case, matches the associated IP-VRF table. The lookup, in this case, matches the
default route which points to the local WAN gateway. The ingress NVE default route which points to the local WAN gateway. The ingress NVE
then rewrites the destination MAC address in the packet with the IRB then rewrites the destination MAC address in the packet with the
Interface MAC address of the local WAN gateway. It also rewrites the router's MAC address of the local WAN gateway. It also rewrites the
source MAC address with its own IRB Interface MAC address. The source MAC address with its own IRB Interface MAC address. The
ingress NVE, then, forwards the frame to the local WAN gateway after ingress NVE, then, forwards the frame to the local WAN gateway after
encapsulating it with the MPLS label stack. Note that this label encapsulating it with the MPLS label stack. Note that this label
stack includes the LSP label as well as the IP-VPN label that was stack includes the LSP label as well as the default host route label
advertised by the local WAN gateway. When the MPLS encapsulated that was advertised by the local WAN gateway. When the MPLS
packet is received by the local WAN gateway, it uses the IP-VPN label encapsulated packet is received by the local WAN gateway, it uses the
to identify the VRF table. It then performs an IP lookup in that default host route label to identify the IP-VRF table. It then
table. The lookup identifies the next hop ASBR to which the packet performs an IP lookup in that table. The lookup identifies the next
must be forwarded. The local gateway in this case strips the Ethernet hop ASBR to which the packet must be forwarded. The local gateway in
encapsulation and perform an IP lookup in its IP-VRF and forwards the this case strips the Ethernet encapsulation and perform an IP lookup
IP packet to the ASBR using a label stack comprising of an LSP label in its IP-VRF and forwards the IP packet to the ASBR using a label
and an IP-VPN label that was advertised by the ASBR. When the MPLS stack comprising of an LSP label and an IP-VPN label that was
encapsulated packet is received by the ASBR, it simply swaps the IP- advertised by the ASBR. When the MPLS encapsulated packet is received
VPN label with the one advertised by the egress PE. This implies that by the ASBR, it simply swaps the IP-VPN label with the one advertised
the remote WAN gateway must allocate the VPN label at least at the by the egress PE. The ASBR then forwards the packet to the egress PE.
granularity of a (VRF, egress PE) tuple. The ASBR then forwards the The egress PE then performs an IP lookup in the IP-VRF (identified by
packet to the egress PE. The egress PE then performs an IP lookup in the received IP-VPN label) to determine where to forward the traffic.
the IP-VRF (identified by the received IP-VPN label) to determine
where to forward the traffic.
Figure 5 below depicts the forwarding model. Figure 5 below depicts the forwarding model.
NVE1 GW1 ASBR NVE2 NVE1 GW1 ASBR NVE2
+------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+
| | | | | | | | | | | | | | | |
|(MAC - (IP | |(IP - (MAC | | [LS] | | (IP | |(MAC - (IP | |(IP - (MAC | | [LS] | | (IP |
| VRF) VRF)| | VRF) VRF)| | | | | | VRF)| | VRF) VRF)| | VRF) VRF)| | | | | | VRF)|
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
+------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+ +------------+
^ v ^ V ^ V ^ V ^ v ^ V ^ V ^ V
| | | | | | | | | | | | | | | |
TS1->-+ +-->-----+ +--------------+ +---------------+ +->-H1 TS1->-+ +-->-----+ +--------------+ +---------------+ +->-H1
Figure 5: Inter-Subnet Forwarding Among IP-VPN Sites and EVPN NVEs Figure 5: Inter-Subnet Forwarding Among IP-VPN Sites and EVPN NVEs
with Route Aggregation with GW
4.5 Use of Centralized Gateway 4.5 Use of Centralized Gateway
In this scenario, the NVEs within a given data center need to forward In this scenario, the NVEs within a given data center need to forward
traffic in L2 to a centralized L3GW for a number of reasons: a) they traffic in L2 to a centralized L3GW for a number of reasons: a) they
don't have IRB capabilities or b) they don't have required policy for don't have IRB capabilities or b) they don't have required policy for
switching traffic between different tenants or security zones. The switching traffic between different tenants or security zones. The
centralized L3GW performs both the IRB function for switching traffic centralized L3GW performs both the IRB function for switching traffic
among different EVPN instances as well as it performs interworking among different EVPN instances as well as it performs interworking
function when the traffic needs to be switched between IP-VPN sites function when the traffic needs to be switched between IP-VPN sites
skipping to change at page 15, line 18 skipping to change at page 16, line 18
scenarios. scenarios.
5.1 IRB forwarding on NVEs for Tenant Systems 5.1 IRB forwarding on NVEs for Tenant Systems
This section covers the symmetric IRB procedures for the scenario This section covers the symmetric IRB procedures for the scenario
where each Tenant System (TS) is attached to one or more NVEs and its where each Tenant System (TS) is attached to one or more NVEs and its
host IP and MAC addresses are learned by the attached NVEs and are host IP and MAC addresses are learned by the attached NVEs and are
distributed to all other NVEs that are interested in participating in distributed to all other NVEs that are interested in participating in
both intra-subnet and inter-subnet communications with that TS. both intra-subnet and inter-subnet communications with that TS.
In this scenario, for a given tenant (e.g., an IP-VPN service), an In this scenario, for a given tenant (e.g., an IP-VPN instance), an
NVE has one MAC-VRF for each tenant's subnet (VLAN) that is NVE has typically one MAC-VRF for each tenant's subnet (VLAN) that is
configured for. Assuming VLAN-based service which is typically the configured for, Assuming VLAN-based service which is typically the
case for VxLAN and NVGRE encapsulation, each MAC-VRF consists of a case for VxLAN and NVGRE encapsulation, each MAC-VRF consists of a
single bridge domain. In case of MPLS encapsulation with VLAN-aware single bridge domain. In case of MPLS encapsulation with VLAN-aware
bundling, then each MAC-VRF consists of multiple bridge domains (one bundling, then each MAC-VRF consists of multiple bridge domains (one
bridge domain per VLAN). The MAC-VRFs on an NVE for a given tenant bridge domain per VLAN). The MAC-VRFs on an NVE for a given tenant
are associated with an IP-VRF corresponding to that tenant (or IP-VPN are associated with an IP-VRF corresponding to that tenant (or IP-VPN
service) via their IRB interfaces. instance) via their IRB interfaces.
Each NVE MUST support QoS, Security, and OAM policies per IP-VRF Each NVE MUST support QoS, Security, and OAM policies per IP-VRF
to/from the core network. This is not to be confused with the QoS, to/from the core network. This is not to be confused with the QoS,
Security, and OAM policies per Attachment Circuits (AC) to/from the Security, and OAM policies per Attachment Circuits (AC) to/from the
Tenant Systems. How this requirement is met is an implementation Tenant Systems. How this requirement is met is an implementation
choice and it is outside the scope of this document. choice and it is outside the scope of this document.
Since VxLAN and NVGRE encapsulations require inner Ethernet header Since VxLAN and NVGRE encapsulations require inner Ethernet header
(inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address (inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address
cannot be used, the ingress NVE's MAC address is used as inner MAC cannot be used, the ingress NVE's MAC address is used as inner MAC
SA. The NVE's MAC address is the device MAC address and it is common SA. The NVE's MAC address is the device MAC address and it is common
across all MAC-VRFs and IP-VRFs. This MAC address is advertised using across all MAC-VRFs and IP-VRFs. This MAC address is advertised using
the new EVPN Router's MAC Extended Community (section 6.1). the new EVPN Router's MAC Extended Community (section 6.1).
Figure below illustrates this scenario where a given tenant (e.g., an Figure below illustrates this scenario where a given tenant (e.g., an
IP-VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2, IP-VPN instance) has three subnets represented by MAC-VRF1, MAC-VRF2,
and MAC-VRF3 across two NVEs. There are five TS's that are associated and MAC-VRF3 across two NVEs. There are five TS's that are associated
with these three MAC-VRFs - i.e., TS1, TS5 are associated with MAC- with these three MAC-VRFs - i.e., TS1, TS4, and TS5 are sitting on
VRF1 on NVE1, TS4 is associated with MAC-VRF1 on NVE2, TS2 is the same subnet (e.g., same MAC-VRF/VLAN);where, TS1 and TS5 are
associated with MAC-VRF2 on NVE1, and TS3 is associated with MAC-VRF3 associated with MAC-VRF1 on NVE1, TS4 is associated with MAC-VRF1 on
on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are in turn associated with NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is associated
IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on NVE2 are associated with with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are in turn
IP-VRF1 on NVE2. When TS1, TS5, and TS4 exchange traffic with each associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on NVE2 are
other, only L2 forwarding (bridging) part of the IRB solution is associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4 exchange
exercised because all these TS's sit on the same subnet. However, traffic with each other, only L2 forwarding (bridging) part of the
when TS1 wants to exchange traffic with TS2 or TS3 which belong to IRB solution is exercised because all these TS's sit on the same
different subnets, then both bridging and routing parts of the IRB subnet. However, when TS1 wants to exchange traffic with TS2 or TS3
solution are exercised. The following subsections describe the which belong to different subnets, then both bridging and routing
control and data planes operations for this IRB scenario in details. parts of the IRB solution are exercised. The following subsections
describe the control and data planes operations for this IRB scenario
in details.
NVE1 +---------+ NVE1 +---------+
+-------------+ | | +-------------+ | |
TS1-----| MACx| | | NVE2 TS1-----| MACx| | | NVE2
(IP1/M1) |(MAC- | | | +-------------+ (IP1/M1) |(MAC- | | | +-------------+
TS5-----| VRF1)\ | | MPLS/ | |MACy (MAC- |-----TS3 TS5-----| VRF1)\ | | MPLS/ | |MACy (MAC- |-----TS3
(IP5/M5) | \ | | VxLAN/ | | / VRF3) | (IP3/M3) (IP5/M5) | \ | | VxLAN/ | | / VRF3) | (IP3/M3)
| (IP-VRF1)|----| NVGRE |---|(IP-VRF1) | | (IP-VRF1)|----| NVGRE |---|(IP-VRF1) |
| / | | | | \ | | / | | | | \ |
TS2-----|(MAC- / | | | | (MAC- |-----TS4 TS2-----|(MAC- / | | | | (MAC- |-----TS4
(IP2/M2) | VRF2) | | | | VRF1) | (IP4/M4) (IP2/M2) | VRF2) | | | | VRF1) | (IP4/M4)
+-------------+ | | +-------------+ +-------------+ | | +-------------+
| | | |
+---------+ +---------+
Figure 6: IRB forwarding on NVEs without core-facing IRB Interface Figure 6: IRB forwarding on NVEs for Tenant Systems
5.1.1 Control Plane Operation 5.1.1 Control Plane Operation
Each NVE advertises a Route Type-2 (RT-2, MAC/IP Advertisement Route) Each NVE advertises a Route Type-2 (RT-2, MAC/IP Advertisement Route)
for each of its TS's with the following field set: for each of its TS's with the following field set:
- RD and ESI per [EVPN] - RD and ESI per [EVPN]
- Ethernet Tag = 0; assuming VLAN-based service - Ethernet Tag = 0; assuming VLAN-based service
- MAC Address Length = 48 - MAC Address Length = 48
- MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example - MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example
skipping to change at page 16, line 44 skipping to change at page 17, line 46
- IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example - IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example
- Label-1 = MPLS Label or VNID corresponding to MAC-VRF - Label-1 = MPLS Label or VNID corresponding to MAC-VRF
- Label-2 = MPLS Label or VNID corresponding to IP-VRF - Label-2 = MPLS Label or VNID corresponding to IP-VRF
Each NVE advertises an RT-2 route with two Route Targets (one Each NVE advertises an RT-2 route with two Route Targets (one
corresponding to its MAC-VRF and the other corresponding to its IP- corresponding to its MAC-VRF and the other corresponding to its IP-
VRF. Furthermore, the RT-2 is advertised with two BGP Extended VRF. Furthermore, the RT-2 is advertised with two BGP Extended
Communities. The first BGP Extended Community identifies the tunnel Communities. The first BGP Extended Community identifies the tunnel
type per section 4.5 of [RFC5512] and the second BGP Extended type per section 4.5 of [RFC5512] and the second BGP Extended
Community includes the MAC address of the NVE (e.g., MACx for NVE1 or Community includes the MAC address of the NVE (e.g., MACx for NVE1 or
MACy for NVE2) and it is defined in section 6.1. This second Extended MACy for NVE2) as defined in section 6.1. This second Extended
Community (for the MAC address of NVE) is only required when the Community (for the MAC address of NVE) is only required when Ethernet
tunnel encapsulation is of type VxLAN or NVGRE where an inner MAC NVO tunnel type is used. If IP NVO tunnel type is used, then there is
address is needed. no need to send this second Extended Community.
Upon receiving this advertisement, the receiving NVE performs the Upon receiving this advertisement, the receiving NVE performs the
following: following:
- It uses Route Targets corresponding to its MAC-VRF and IP-VRF for - It uses Route Targets corresponding to its MAC-VRF and IP-VRF for
identifying these tables and subsequently importing this route into identifying these tables and subsequently importing this route into
them. them.
- It imports the MAC address into the MAC-VRF with BGP Next Hop - It imports the MAC address into the MAC-VRF with BGP Next Hop
address as underlay tunnel destination address (e.g., VTEP DA for address as underlay tunnel destination address (e.g., VTEP DA for
VxLAN encapsulation) and Label-1 as VNID for VxLAN encapsulation or VxLAN encapsulation) and Label-1 as VNID for VxLAN encapsulation or
EVPN label for MPLS encapsulation. EVPN label for MPLS encapsulation.
- If the route carries the new Router's MAC Extended Community, and - If the route carries the new Router's MAC Extended Community, and
if the receiving NVE is going to use VxLAN encapsulation, then the if the receiving NVE is using Ethernet NVO tunnel, then the receiving
receiving NVE imports the IP address into IP-VRF with NVE's MAC NVE imports the IP address into IP-VRF with NVE's MAC address (from
address (from the new Router's MAC Extended Community) as inner MAC the new Router's MAC Extended Community) as inner MAC DA and BGP Next
DA and BGP Next Hop address as underlay tunnel destination address, Hop address as underlay tunnel destination address, VTEP DA for VxLAN
VTEP DA for VxLAN encapsulation and Label-2 as IP-VPN VNID for VxLAN encapsulation and Label-2 as IP-VPN VNID for VxLAN encapsulation.
encapsulation.
- If the receiving NVE is going to use MPLS encapsulation, then the - If the receiving NVE is going to use MPLS encapsulation, then the
receiving NVE imports the IP address into IP-VRF with BGP Next Hop receiving NVE imports the IP address into IP-VRF with BGP Next Hop
address as underlay tunnel destination address, and Label-2 as IP-VPN address as underlay tunnel destination address, and Label-2 as IP-VPN
label for MPLS encapsulation. label for MPLS encapsulation.
If the receiving NVE receives a RT-2 with only a single Route Target If the receiving NVE receives a RT-2 with only a single Route Target
corresponding to IP-VRF and Label-1, then it must discard this route corresponding to IP-VRF and Label-1, then it must discard this route
and log an error. If the receiving NVE receives a RT-2 with only a and log an error. If the receiving NVE receives a RT-2 with only a
single Route Target corresponding to MAC-VRF but with both Label-1 single Route Target corresponding to MAC-VRF but with both Label-1
and Label-2, then it must discard this route and log an error. If the and Label-2, then it must discard this route and log an error. If the
receiving NVE receives a RT-2 with MAC Address Length of zero, then receiving NVE receives a RT-2 with MAC Address Length of zero, then
it must discard this route and log an error. it must discard this route and log an error.
5.1.2 Data Plane Operation 5.1.2 Data Plane Operation - Inter Subnet
The following description of the data-plane operation describes just The following description of the data-plane operation describes just
the logical functions and the actual implementation may differ. Lets the logical functions and the actual implementation may differ. Lets
consider data-plane operation when TS1 in subnet-1 (MAC-VRF1) on NVE1 consider data-plane operation when TS1 in subnet-1 (MAC-VRF1) on NVE1
wants to send traffic to TS3 in subnet-3 (MAC-VRF3) on NVE2. wants to send traffic to TS3 in subnet-3 (MAC-VRF3) on NVE2.
- TS1 send a packet with MAC DA corresponding to the MAC-VRF1 IRB - TS1 send a packet with MAC DA corresponding to the MAC-VRF1 IRB
interface on NVE1 (the interface between MAC-VRF1 and IP-VRF1), and interface on NVE1 (the interface between MAC-VRF1 and IP-VRF1), and
VLAN-tag corresponding to MAC-VRF1. VLAN-tag corresponding to MAC-VRF1.
- Upon receiving the packet, the NVE1 uses VLAN-tag to identify the - Upon receiving the packet, the NVE1 uses VLAN-tag to identify the
MAC-VRF1. It then looks up the MAC DA and forwards the frame to its MAC-VRF1. It then looks up the MAC DA and forwards the frame to its
IRB interface. IRB interface.
- The Ethernet header of the packet is stripped and the packet is - The Ethernet header of the packet is stripped and the packet is
fed to the IP-VRF (iVRF) where IP lookup is performed on the fed to the IP-VRF where IP lookup is performed on the destination
destination address. This lookup yields a MAC address to be used as address. This lookup yields an outgoing interface and the required
inner MAC DA for VxLAN/NVGRE encapsulation, an IP address to be used encapsulation. If the encapsulation is for Ethernet NVO tunnel, then
as VTEP DA for VxLAN encap or tunnel label for MPLS encap, and a VPN- it includes a MAC address to be used as inner MAC DA, an IP address
ID to be used as VNID for VxLAN encap or VPN label for MPLS encap. to be used as VTEP DA, and a VPN-ID to be used as VNID.
- The packet is then encapsulated with the proper header based on - The packet is then encapsulated with the proper header based on
the above info. The inner MAC SA and VTEP SA is set to NVE's MAC and the above info. The inner MAC SA and VTEP SA is set to NVE's MAC and
IP addresses respectively. The packet is then forwarded to the egress IP addresses respectively. The packet is then forwarded to the egress
NVE. NVE.
- On the egress NVE, if the packet is VxLAN encapsulated, the VxLAN - On the egress NVE, if the packet arrives on Ethernet NOV tunnel
header is removed. Since the inner MAC DA is the egress NVE's MAC (e.g., it is VxLAN encapsulated), then the VxLAN header is removed.
address, the egress NVE knows that it needs to perform an IP lookup. Since the inner MAC DA is the egress NVE's MAC address, the egress
It uses VNID to identify the IP-VRF (iVRF) table and then performs an NVE knows that it needs to perform an IP lookup. It uses VNID to
IP lookup which results in destination TS (TS3) MAC address and the identify the IP-VRF table and then performs an IP lookup for the
access-facing IRB interface over which the packet is sent. destination TS (TS3) which results in access-facing IRB interface
over which the packet is sent. Before sending the packet over this
interface, the ARP table is consulted to get the destination TS's MAC
address.
- The IP packet is encapsulated with an Ethernet header with MAC SA - The IP packet is encapsulated with an Ethernet header with MAC SA
set to that of NVE2 MAC address(MACy) and MAC DA set to that of set to that of IRB interface MAC address and MAC DA set to that of
destination TS (TS3) MAC address. The packet is sent to the destination TS (TS3) MAC address. The packet is sent to the
corresponding MAC-VRF3 and after a lookup of MAC DA, is forwarded to corresponding MAC-VRF3 and after a lookup of MAC DA, is forwarded to
the destination TS (TS3) over the corresponding interface. the destination TS (TS3) over the corresponding interface.
In this scenario, inter-subnet forwarding traffic between NVEs will In this symmetric IRB scenario, inter-subnet traffic between NVEs
always use the IP-VRF VNID/MPLS label, even if the IP DA belongs to a will always use the IP-VRF VNID/MPLS label. For instance, traffic
subnet defined in both NVEs. For instance, traffic from TS2 to TS4 from TS2 to TS4 will be encapsulated by NVE1 using NVE2's IP-VRF
will be encapsulated by NVE1 using NVE2's IP-VRF VNID/MPLS label as VNID/MPLS label, as long as TS4's host IP is present in NVE1's IP-
opposed to the MAC-VRF1 VNID/MPLS label, as long as TS4's host IP is VRF.
present in NVE1's IP-VRF.
5.1.3 TS Move Operation 5.1.3 TS Move Operation
When a TS move from one NVE to other, it is important that the MAC When a TS move from one NVE to other, it is important that the MAC
mobility procedures are properly executed and the corresponding MAC- mobility procedures are properly executed and the corresponding MAC-
VRF and IP-VRF tables on all participating NVEs are updated. [EVPN] VRF and IP-VRF tables on all participating NVEs are updated. [EVPN]
describes the MAC mobility procedures for L2-only services for both describes the MAC mobility procedures for L2-only services for both
single-homed TS and All-Active multi-homed TS . This section single-homed TS and multi-homed TS. This section describes the
describes the incremental procedures and BGP Extended Communities incremental procedures and BGP Extended Communities needed to handle
needed to handle the MAC mobility for a mixed of L2 and L3 services the MAC mobility for a mixed of L2 and L3 connectivity (aka IRB). In
known as Integrated Routing and Bridging - IRB. In order to place order to place the emphasis on the differences between L2-only versus
the emphasis on the differences between L2-only versus L2-and-L3 use L2-and-L3 use cases, the incremental procedure is described for
cases, the incremental procedure is described for single-homed TS single-homed TS with the expectation that the reader can easily
with the expectation that the reader can easily extrapolate multi- extrapolate multi-homed TS based on the procedures described in
homed TS based on the procedures described in section 15 of [EVPN]. section 15 of [EVPN].
Lets consider TS1 in figure-6 above where it moves from NVE1 to NVE2. Lets consider TS1 in figure-6 above where it moves from NVE1 to NVE2.
In such move, NVE2 discovers IP1/MAC1 of TS1 and realizes that it is In such move, NVE2 discovers IP1/MAC1 of TS1 and realizes that it is
a MAC move and it advertises a MAC/IP route per section 5.1.1 above a MAC move and it advertises a MAC/IP route per section 5.1.1 above
with MAC Mobility Extended Community. In this IRB use case, both MAC with MAC Mobility Extended Community. In this IRB use case, both MAC
and IP addresses of the TS are included in the EVPN MAC/IP and IP addresses of the TS along with their corresponding VNI/MPLS
Advertisement route as oppose to L2-only use case where only the MAC labels are included in the EVPN MAC/IP Advertisement route.
address of the TS is included. Furthermore, besides MAC mobility Furthermore, besides MAC mobility Extended Community and Route Target
Extended Community and Route Target corresponding to the MAC-VRF, the corresponding to the MAC-VRF, the following additional BGP Extended
following additional BGP Extended Communities are advertised along Communities are advertised along with the MAC/IP Advertisement route:
with the MAC/IP Advertisement route:
- Route Target associated with IP-VRF - Route Target associated with IP-VRF
- Router's MAC Extended Community - Router's MAC Extended Community
- Tunnel Type Extended Community - Tunnel Type Extended Community
Since NVE2 learns TS1's MAC/IP addresses locally, it updates its MAC- Since NVE2 learns TS1's MAC/IP addresses locally, it updates its MAC-
VRF1 and IP-VRF1 for TS1 with its local interface. VRF1 and IP-VRF1 for TS1 with its local interface.
If the local learning at NVE1 is performed using control or If the local learning at NVE1 is performed using control or
management planes, then these interactions serve as the trigger for management planes, then these interactions serve as the trigger for
NVE1 to withdraw the MAC/IP addresses associated with TS1. However, NVE1 to withdraw the MAC/IP addresses associated with TS1. However,
if the local learning at NVE1 is performed using data-plane learning, if the local learning at NVE1 is performed using data-plane learning,
then the reception of the MAC/IP Advertisement route for TS1 with MAC then the reception of the MAC/IP Advertisement route (for TS1) from
Mobility extended community serve as the trigger for NVE1 to withdraw NVE2 with MAC Mobility extended community serve as the trigger for
the MAC/IP addresses associated with TS1. NVE1 to withdraw the MAC/IP addresses associated with TS1.
All other remote NVE devices upon receiving the MAC/IP advertisement All other remote NVE devices upon receiving the MAC/IP advertisement
route for TS1 from NVE2 with MAC Mobility extended community compare route for TS1 from NVE2 with MAC Mobility extended community compare
the sequence number in this advertisement with the one previously the sequence number in this advertisement with the one previously
received. If the new sequence number is greater than the old one, received. If the new sequence number is greater than the old one,
then they update the MAC/IP addresses of TS1 in their corresponding then they update the MAC/IP addresses of TS1 in their corresponding
MAC-VRFs and IP-VRFs to point to NVE2. Furthermore, upon receiving MAC-VRFs and IP-VRFs to point to NVE2. Furthermore, upon receiving
the MAC/IP withdraw for TS1 from NVE1, these remote PEs perform the the MAC/IP withdraw for TS1 from NVE1, these remote PEs perform the
cleanups for their BGP tables. cleanups for their BGP tables.
skipping to change at page 20, line 29 skipping to change at page 21, line 29
IP-VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2, IP-VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2,
and MAC-VRF3 across two NVEs. There are four TS's associated with and MAC-VRF3 across two NVEs. There are four TS's associated with
these three MAC-VRFs - i.e., TS1, TS5 are connected to MAC-VRF1 on these three MAC-VRFs - i.e., TS1, TS5 are connected to MAC-VRF1 on
NVE1, TS2 is connected to MAC-VRF2 on NVE1, TS3 is connected to MAC- NVE1, TS2 is connected to MAC-VRF2 on NVE1, TS3 is connected to MAC-
VRF3 on NVE2, and TS4 is connected to MAC-VRF1 on NVE2. TS1 has two VRF3 on NVE2, and TS4 is connected to MAC-VRF1 on NVE2. TS1 has two
subnet prefixes (SN1 and SN2) and TS3 has a single subnet prefix, subnet prefixes (SN1 and SN2) and TS3 has a single subnet prefix,
SN3. The MAC-VRFs on each NVE are associated with their corresponding SN3. The MAC-VRFs on each NVE are associated with their corresponding
IP-VRF using their IRB interfaces. When TS4 and TS1 exchange intra- IP-VRF using their IRB interfaces. When TS4 and TS1 exchange intra-
subnet traffic, only L2 forwarding (bridging) part of the IRB subnet traffic, only L2 forwarding (bridging) part of the IRB
solution is used (i.e., the traffic only goes through their MAC- solution is used (i.e., the traffic only goes through their MAC-
VRFs); however, when TS4 wants to forward traffic to SN1 or SN2 VRFs); however, when TS3 wants to forward traffic to SN1 or SN2
sitting behind TS1 (inter-subnet traffic), then both bridging and sitting behind TS1 (inter-subnet traffic), then both bridging and
routing parts of the IRB solution are exercised (i.e., the traffic routing parts of the IRB solution are exercised (i.e., the traffic
goes through the corresponding MAC-VRFs and IP-VRFs). The following goes through the corresponding MAC-VRFs and IP-VRFs). The following
subsections describe the control and data planes operations for this subsections describe the control and data planes operations for this
IRB scenario in details. IRB scenario in details.
NVE1 +----------+ NVE1 +----------+
SN1--+ +-------------+ | | SN1--+ +-------------+ | |
|--TS1-----|(MAC- \ | | | |--TS1-----|(MAC- \ | | |
SN2--+ IP1/M1 | VRF1) \ | | | SN2--+ IP1/M1 | VRF1) \ | | |
| (IP-VRF)|---| | | (IP-VRF)|---| |
| / | | | | / | | |
TS2-----|(MAC- / | | MPLS/ | TS2-----|(MAC- / | | MPLS/ |
IP2/M2 | VRF2) | | VxLAN/ | IP2/M2 | VRF2) | | VxLAN/ |
+-------------+ | NVGRE | +-------------+ | NVGRE |
+-------------+ | | +-------------+ | |
SN3--+--TS3-----|(MAC-\ | | | SN3--+--TS3-----|(MAC-\ | | |
IP3/M3 | VRF3)\ | | | IP3/M3 | VRF3)\ | | |
| (iVRF)|---| | | (IP-VRF)|---| |
| / | | | | / | | |
TS4-----|(MAC- / | | | TS4-----|(MAC- / | | |
IP4/M4 | VRF1) | | | IP4/M4 | VRF1) | | |
+-------------+ +----------+ +-------------+ +----------+
NVE2 NVE2
Figure 7: IRB forwarding on NVEs with core-facing IRB Interface Figure 7: IRB forwarding on NVEs for Tenant Systems with configured subnets
5.2.1 Control Plane Operation 5.2.1 Control Plane Operation
Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in
[EVPN-PREFIX]) for each of its subnet prefixes with the IP address of [EVPN-PREFIX]) for each of its subnet prefixes with the IP address of
its TS as the next hop (gateway address field) as follow: its TS as the next hop (gateway address field) as follow:
- RD per VPN - RD per VPN
- ESI = 0 - ESI = 0
- Ethernet Tag = 0; - Ethernet Tag = 0;
skipping to change at page 22, line 46 skipping to change at page 23, line 46
- The packet is then encapsulated with the proper header based on - The packet is then encapsulated with the proper header based on
the above info and is forwarded to the egress NVE (NVE2). the above info and is forwarded to the egress NVE (NVE2).
- On the egress NVE (NVE2), assuming the packet is VxLAN - On the egress NVE (NVE2), assuming the packet is VxLAN
encapsulated, the VxLAN and the inner Ethernet headers are removed encapsulated, the VxLAN and the inner Ethernet headers are removed
and the resultant IP packet is fed to the IP-VRF associated with that and the resultant IP packet is fed to the IP-VRF associated with that
the VNID. the VNID.
- Next, a lookup is performed based on IP DA (which is in SN3) in the - Next, a lookup is performed based on IP DA (which is in SN3) in the
associated IP-VRF of NVE2. The IP lookup yields the destination TS associated IP-VRF of NVE2. The IP lookup yields the access-facing IRB
(TS3) MAC address and the access-facing IRB interface over which the interface over which the packet needs to be sent. Before sending the
packet needs to be sent. packet over this interface, the ARP table is consulted to get the
destination TS (TS3) MAC address.
- The IP packet is encapsulated with an Ethernet header with the MAC - The IP packet is encapsulated with an Ethernet header with the MAC
SA set to that of the access-facing IRB interface of the egress NVE SA set to that of the access-facing IRB interface of the egress NVE
(NVE2) and the MAC DA is set to that of destination TS (TS3) MAC (NVE2) and the MAC DA is set to that of destination TS (TS3) MAC
address. The packet is sent to the corresponding MAC-VRF3 and after a address. The packet is sent to the corresponding MAC-VRF3 and after a
lookup of MAC DA, is forwarded to the destination TS (TS3) over the lookup of MAC DA, is forwarded to the destination TS (TS3) over the
corresponding interface. corresponding interface.
6 BGP Encoding 6 BGP Encoding
skipping to change at page 24, line 20 skipping to change at page 25, line 20
- source NVE refers to the NVE behind which the TS used to reside - source NVE refers to the NVE behind which the TS used to reside
prior to the TS mobility event. prior to the TS mobility event.
- target NVE refers to the new NVE behind which the TS has moved - target NVE refers to the new NVE behind which the TS has moved
after the mobility event. after the mobility event.
7.2.1 Mobility without Route Aggregation 7.2.1 Mobility without Route Aggregation
In this scenario, when a target NVE detects that a MAC mobility event In this scenario, when a target NVE detects that a MAC mobility event
has occurred, it initiates the MAC mobility handshake in BGP as has occurred, it initiates the MAC mobility handshake in BGP as
specified in [EVPN]. The WAN Gateways, acting as ASBRs in this case, specified in section 5.1.3. The WAN Gateways, acting as ASBRs in this
re-advertise the MAC route of the target NVE with the MAC Mobility case, re-advertise the MAC route of the target NVE with the MAC
extended community attribute unmodified. Because the WAN Gateway for Mobility extended community attribute unmodified. Because the WAN
a given data center re-advertises BGP routes received from the WAN Gateway for a given data center re-advertises BGP routes received
into the data center, the source NVE will receive the MAC from the WAN into the data center, the source NVE will receive the
Advertisement route of the target NVE (with the next hop attribute MAC Advertisement route of the target NVE (with the next hop
adjusted depending on which inter-AS option is employed). The source attribute adjusted depending on which inter-AS option is employed).
NVE will then withdraw its original MAC Advertisement route as a The source NVE will then withdraw its original MAC Advertisement
result of evaluating the Sequence Number field of the MAC Mobility route as a result of evaluating the Sequence Number field of the MAC
extended community in the received MAC Advertisement route. This is Mobility extended community in the received MAC Advertisement route.
per the procedures already defined in [EVPN]. This is per the procedures already defined in [EVPN].
7.2.2 Mobility with Route Aggregation
This section will be completed in the next revision.
8 Acknowledgements 8 Acknowledgements
The authors would like to thank Sami Boutros for his valuable The authors would like to thank Sami Boutros for his valuable
comments. comments.
9 Security Considerations 9 Security Considerations
The security considerations discussed in [RFC7432] apply to this
document.
10 IANA Considerations 10 IANA Considerations
IANA has allocated a new transitive extended community Type of 0x06 IANA has allocated a new transitive extended community Type of 0x06
and Sub-Type of 0x03 for EVPN Router's MAC Extended Community. and Sub-Type of 0x03 for EVPN Router's MAC Extended Community.
11 References 11 References
11.1 Normative References 11.1 Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
11.2 Informative References 11.2 Informative References
[EVPN] Sajassi et al., "BGP MPLS Based Ethernet VPN", draft-ietf- [EVPN] Sajassi et al., "BGP MPLS Based Ethernet VPN", draft-ietf-
l2vpn-evpn-04.txt, work in progress, July, 2014. l2vpn-evpn-04.txt, work in progress, July, 2014.
skipping to change at page 25, line 25 skipping to change at page 26, line 24
with IP-VPN", draft-sajassi-l2vpn-evpn-ipvpn-interop-01, work in with IP-VPN", draft-sajassi-l2vpn-evpn-ipvpn-interop-01, work in
progress, October, 2012. progress, October, 2012.
[DC-MOBILITY] Aggarwal et al., "Data Center Mobility based on [DC-MOBILITY] Aggarwal et al., "Data Center Mobility based on
BGP/MPLS, IP Routing and NHRP", draft-raggarwa-data-center-mobility- BGP/MPLS, IP Routing and NHRP", draft-raggarwa-data-center-mobility-
05.txt, work in progress, June, 2013. 05.txt, work in progress, June, 2013.
[EVPN-PREFIX] Rabadan et al., "IP Prefix Advertisement in EVPN", [EVPN-PREFIX] Rabadan et al., "IP Prefix Advertisement in EVPN",
draft-rabadan-l2vpn-evpn-prefix-advertisement-02, July, 2014. draft-rabadan-l2vpn-evpn-prefix-advertisement-02, July, 2014.
12 Contributors
In addition to the authors listed on the front page, the following
co-authors have also contributed to this document:
Samer Salam
Florin Balus
Cisco
Yakov Rekhter
Juniper
Wim Henderickx
Nokia
Linda Dunbar
Huawei
Dennis Cai
Alibaba
Authors' Addresses Authors' Addresses
Ali Sajassi Ali Sajassi (Editor)
Cisco Cisco
Email: sajassi@cisco.com Email: sajassi@cisco.com
Samer Salam Samer Salam
Cisco Cisco
Email: ssalam@cisco.com Email: sslam@cisco.com
Yakov Rekhter Samir Thoria
Juniper Networks Cisco
Email: yakov@juniper.net Email: sthoria@cisco.com
John E. Drake John E. Drake
Juniper Networks Juniper Networks
Email: jdrake@juniper.net Email: jdrake@juniper.net
Lucy Yong Lucy Yong
Huawei Technologies Huawei Technologies
Email: lucy.yong@huawei.com Email: lucy.yong@huawei.com
Linda Dunbar
Huawei Technologies
Email: linda.dunbar@huawei.com
Wim Henderickx Jorge Rabadan
Alcatel-Lucent Nokia
Email: wim.henderickx@alcatel-lucent.com Email: jorge.rabadan@nokia.com
Florin Balus
Alcatel-Lucent
Email: Florin.Balus@alcatel-lucent.com
Samir Thoria
Cisco
Email: sthoria@cisco.com
 End of changes. 84 change blocks. 
292 lines changed or deleted 323 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/