draft-ietf-nvo3-hpvr2nve-cp-req-17.txt   rfc8394.txt 
NVO3 Working Group Y. Li Internet Engineering Task Force (IETF) Y. Li
INTERNET-DRAFT D. Eastlake Request for Comments: 8394 D. Eastlake 3rd
Intended Status: Informational Huawei Technologies Category: Informational Huawei Technologies
L. Kreeger ISSN: 2070-1721 L. Kreeger
Arrcus, Inc Arrcus, Inc.
T. Narten T. Narten
IBM IBM
D. Black D. Black
Dell EMC Dell EMC
Expires: September 14, 2018 March 13, 2018 May 2018
Split Network Virtualization Edge (Split-NVE) Control Plane Requirements Split Network Virtualization Edge (Split-NVE) Control-Plane Requirements
draft-ietf-nvo3-hpvr2nve-cp-req-17
Abstract Abstract
In a Split Network Virtualization Edge (Split-NVE) architecture, the In the Split Network Virtualization Edge (Split-NVE) architecture,
functions of the NVE (Network Virtualization Edge) are split across a the functions of the NVE are split across a server and a piece of
server and an external network equipment which is called an external external network equipment that is called an "External NVE". The
NVE. The server-resident control plane functionality resides in server-resident control-plane functionality resides in control
control software, which may be part of hypervisor or container software, which may be part of hypervisor or container-management
management software; for simplicity, this document refers to the software; for simplicity, this document refers to the hypervisor as
hypervisor as the location of this software. the "location" of this software.
Control plane protocol(s) between a hypervisor and its associated
external NVE(s) are used by the hypervisor to distribute its virtual
machine networking state to the external NVE(s) for further handling.
This document illustrates the functionality required by this type of
control plane signaling protocol and outlines the high level
requirements. Virtual machine states as well as state transitioning
are summarized to help clarify the protocol requirements.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the One or more control-plane protocols between a hypervisor and its
provisions of BCP 78 and BCP 79. associated External NVE(s) are used by the hypervisor to distribute
its virtual-machine networking state to the External NVE(s) for
further handling. This document illustrates the functionality
required by this type of control-plane signaling protocol and
outlines the high-level requirements. Virtual-machine states as well
as state transitioning are summarized to help clarify the protocol
requirements.
Internet-Drafts are working documents of the Internet Engineering Status of This Memo
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months This document is not an Internet Standards Track specification; it is
and may be updated, replaced, or obsoleted by other documents at any published for informational purposes.
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This document is a product of the Internet Engineering Task Force
http://www.ietf.org/1id-abstracts.html (IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are candidates for any level of Internet
Standard; see Section 2 of RFC 7841.
The list of Internet-Draft Shadow Directories can be accessed at Information about the current status of this document, any errata,
http://www.ietf.org/shadow.html and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8394.
Copyright and License Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction ....................................................3
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1. Terminology ................................................4
1.2 Target Scenarios . . . . . . . . . . . . . . . . . . . . . 6 1.2. Target Scenarios ...........................................6
2. VM Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . 8 2. VM Lifecycle ....................................................7
2.1 VM Creation Event . . . . . . . . . . . . . . . . . . . . . 8 2.1. VM Creation Event ..........................................8
2.2 VM Live Migration Event . . . . . . . . . . . . . . . . . . 9 2.2. VM Live Migration Event ....................................8
2.3 VM Termination Event . . . . . . . . . . . . . . . . . . . . 10 2.3. VM Termination Event .......................................9
2.4 VM Pause, Suspension and Resumption Events . . . . . . . . . 10 2.4. VM Pause, Suspension, and Resumption Events ...............10
3. Hypervisor-to-NVE Control Plane Protocol Functionality . . . . 11 3. Hypervisor-to-NVE Control-Plane Protocol Functionality .........10
3.1 VN Connect and Disconnect . . . . . . . . . . . . . . . . . 11 3.1. VN_Connect and VN_Disconnect ..............................10
3.2 TSI Associate and Activate . . . . . . . . . . . . . . . . . 13 3.2. TSI Associate and Activate ................................12
3.3 TSI Disassociate and Deactivate . . . . . . . . . . . . . . 15 3.3. TSI De-Associate and Deactivate ...........................15
4. Hypervisor-to-NVE Control Plane Protocol Requirements . . . . . 16 4. Hypervisor-to-NVE Control-Plane Protocol Requirements ..........16
5. VDP Applicability and Enhancement Needs . . . . . . . . . . . . 17 5. VDP Applicability and Enhancement Needs ........................17
6. Security Considerations . . . . . . . . . . . . . . . . . . . . 19 6. Security Considerations ........................................19
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 20 7. IANA Considerations ............................................20
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 20 8. References .....................................................21
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.1. Normative References ......................................21
8.1 Normative References . . . . . . . . . . . . . . . . . . . 20 8.2. Informative References ....................................22
8.2 Informative References . . . . . . . . . . . . . . . . . . 21 Appendix A. VDP Illustrations (per IEEE 802.1Q) (for Information
Appendix A. IEEE 802.1Q VDP Illustration (For information only) . 21 Only) .................................................23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24 Acknowledgements ..................................................25
Authors' Addresses ................................................26
1. Introduction 1. Introduction
In the Split-NVE architecture shown in Figure 1, the functionality of In the Split Network Virtualization Edge (Split-NVE) architecture
the NVE (Network Virtualization Edge) is split across an end device shown in Figure 1, the functionality of the NVE is split across an
supporting virtualization and an external network device which is end device supporting virtualization and an external network device
called an external NVE. The portion of the NVE functionality located that is called an "External NVE". The portion of the NVE
on the end device is called the tNVE (terminal-side NVE) and the functionality located on the end device is called the "tNVE"
portion located on the external NVE is called the nNVE (network-side (terminal-side NVE), and the portion located on the External NVE is
NVE) in this document. Overlay encapsulation/decapsulation functions called the "nNVE" (network-side NVE) in this document. Overlay
are normally off-loaded to the nNVE on the external NVE. encapsulation/decapsulation functions are normally offloaded to the
nNVE on the External NVE.
The tNVE is normally implemented as a part of hypervisor or container
and/or virtual switch in an virtualized end device. This document
uses the term "hypervisor" throughout when describing the Split-NVE
scenario where part of the NVE functionality is off-loaded to a
separate device from the "hypervisor" that contains a VM (Virtual
Machine) connected to a VN (Virutal Network). In this context, the
term "hypervisor" is meant to cover any device type where part of the
NVE functionality is off-loaded in this fashion, e.g.,a Network
Service Appliance or Linux Container.
The NVO3 problem statement [RFC7364], discusses the needs for a
control plane protocol (or protocols) to populate each NVE with the
state needed to perform the required functions. In one scenario, an
NVE provides overlay encapsulation/decapsulation packet forwarding
services to Tenant Systems (TSs) that are co-resident within the NVE
on the same End Device (e.g. when the NVE is embedded within a
hypervisor or a Network Service Appliance). In such cases, there is
no need for a standardized protocol between the hypervisor and NVE,
as the interaction is implemented via software on a single device.
While in the Split-NVE architecture scenarios, as shown in figure 2
to figure 4, control plane protocol(s) between a hypervisor and its
associated external NVE(s) are required for the hypervisor to
distribute the virtual machines networking states to the NVE(s) for
further handling. The protocol is an NVE-internal protocol and runs
between tNVE and nNVE logical entities. This protocol is mentioned in
the NVO3 problem statement [RFC7364] and appears as the third work
item.
Virtual machine states and state transitioning are summarized in this
document showing events where the NVE needs to take specific actions.
Such events might correspond to actions the control plane signaling
protocol(s) need to take between tNVE and nNVE in the Split-NVE
scenario. The high level requirements to be fulfilled are stated.
+------------ Split-NVE ---------+ +------------ Split-NVE ---------+
| | | |
| | | |
+-----------------|-----+ | +-----------------|-----+ |
| +---------------|----+| | | +---------------|----+| |
| | +--+ \|/ || | | | +--+ \|/ || |
| | |V |TSI +-------+ || +------|-------------+ | | |V |TSI +-------+ || +------|-------------+
| | |M |-----+ | || | \|/ | | | |M |-----+ | || | \|/ |
| | +--+ | | || |+--------+ | | | +--+ | | || |+--------+ |
| | +--+ | tNVE | ||-------------------|| | | | | +--+ | tNVE | ||-------------------|| | |
| | |V |TSI | | || || nNVE | | | | |V |TSI | | || || nNVE | |
| | |M |-----| | || || | | | | |M |-----| | || || | |
| | +--+ +-------+ || |+--------+ | | | +--+ +-------+ || |+--------+ |
| | || +--------------------+ | | || +--------------------+
| +-----Hypervisor-----+| | +-----Hypervisor-----+|
+-----------------------+ +-----------------------+
End Device External NVE End Device External NVE
Figure 1 Split-NVE structure Figure 1: Split-NVE Structure
This document uses VMs as an example of Tenant Systems (TSs) in order The tNVE is normally implemented as a part of a hypervisor or
to describe the requirements, even though a VM is just one type of container and/or a virtual switch in a virtualized end device. This
Tenant System that may connect to a VN. For example, a service document uses the term "hypervisor" throughout when describing the
instance within a Network Service Appliance is another type of TS, as Split-NVE scenario where part of the NVE functionality is offloaded
are systems running on an OS-level virtualization technologies like to a separate device from the "hypervisor" that contains a VM
(Virtual Machine) connected to a VN (Virtual Network). In this
context, the term "hypervisor" is meant to cover any device type
where part of the NVE functionality is offloaded in this fashion,
e.g., a Network Service Appliance or Linux Container.
The Network Virtualization over Layer 3 (NVO3) problem statement
[RFC7364] discusses the need for a control-plane protocol (or
protocols) to populate each NVE with the state needed to perform the
required functions. In one scenario, an NVE provides overlay
encapsulation/decapsulation packet-forwarding services to Tenant
Systems that are co-resident within the NVE on the same end device
(e.g., when the NVE is embedded within a hypervisor or a Network
Service Appliance). In such cases, there is no need for a
standardized protocol between the hypervisor and the NVE, as the
interaction is implemented via software on a single device. However,
in the Split-NVE architecture scenarios shown in Figures 2 through 4
(see Section 1.2), one or more control-plane protocols between a
hypervisor and its associated External NVE(s) are required for the
hypervisor to distribute the VM's networking states to the NVE(s) for
further handling. The protocol is an NVE-internal protocol and runs
between tNVE and nNVE logical entities. This protocol is mentioned
in the "third work area" text in Section 4.5 of the NVO3 problem
statement [RFC7364].
VM states and state transitioning are summarized in this document,
showing events where the NVE needs to take specific actions. Such
events might correspond to actions that the control-plane signaling
protocol or protocols need to take between the tNVE and the nNVE in
the Split-NVE scenario. The high-level requirements to be fulfilled
are listed in Section 4.
To describe the requirements, this document uses VMs as an example of
Tenant Systems, even though a VM is just one type of Tenant System
that may connect to a VN. For example, a service instance within a
Network Service Appliance is another type of Tenant System, as are
systems running on OS-level virtualization technologies like
containers. The fact that VMs have lifecycles (e.g., can be created containers. The fact that VMs have lifecycles (e.g., can be created
and destroyed, can be moved, and can be started or stopped) results and destroyed, can be moved, and can be started or stopped) results
in a general set of protocol requirements, most of which are in a general set of protocol requirements, most of which are
applicable to other forms of TSs although not all of the requirements applicable to other forms of Tenant Systems, although not all of the
are applicable to all forms of TSs. requirements are applicable to all forms of Tenant Systems.
Section 2 describes VM states and state transitioning in the VM's Section 2 describes VM states and state transitioning in the VM's
lifecycle. Section 3 introduces Hypervisor-to-NVE control plane lifecycle. Section 3 introduces hypervisor-to-NVE control-plane
protocol functionality derived from VM operations and network events. protocol functionality derived from VM operations and network events.
Section 4 outlines the requirements of the control plane protocol to Section 4 outlines the requirements of the control-plane protocol to
achieve the required functionality. achieve the required functionality.
1.1 Terminology 1.1. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in
14 [RFC2119] [RFC8174] when, and only when, they appear in all BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
This document uses the same terminology as found in [RFC7365]. This This document uses the same terminology as the terminology found in
section defines additional terminology used by this document. [RFC7365]. This section defines additional terminology used by this
document.
Split-NVE: a type of NVE (Network Virtualization Edge) where the Split-NVE: A type of NVE (Network Virtualization Edge) where the
functionalities are split across an end device supporting functionalities are split across an end device supporting
virtualization and an external network device. virtualization and an external network device.
tNVE: the portion of Split-NVE functionalities located on the end tNVE: Terminal-side NVE. The portion of Split-NVE functionalities
device supporting virtualization. It interacts with a tenant system located on the end device supporting virtualization. The tNVE
through an internal interface in the end device. interacts with a Tenant System through an internal interface in
the end device.
nNVE: the portion of Split-NVE functionalities located on the network nNVE: Network-side NVE. The portion of Split-NVE functionalities
device that is directly or indirectly connected to the end device located on the network device that is directly or indirectly
holding the corresponding tNVE. nNVE normally performs encapsulation connected to the end device that contains the corresponding tNVE.
to and decapsulation from the overlay network. The nNVE normally performs encapsulation to and decapsulation from
the overlay network.
External NVE: the physical network device holding the nNVE External NVE: The physical network device that contains the nNVE.
Hypervisor: the logical collection of software, firmware and/or Hypervisor: The logical collection of software, firmware, and/or
hardware that allows the creation and running of server or service hardware that allows the creation and running of server or service
appliance virtualization. tNVE is located under a Hypervisor. appliance virtualization. The tNVE is located under a hypervisor.
Hypervisor is loosely used in this document to refer to the end The term "hypervisor" is loosely used in this document to refer to
device supporting the virtualization. For simplicity, we also use the end device supporting the virtualization. For simplicity, we
Hypervisor to represent both hypervisor and container. also use the term "hypervisor" to represent both the hypervisor
and the container.
Container: Please refer to Hypervisor. For simplicity this document Container: Please see "Hypervisor:" above.
use the term hypervisor to represent both hypervisor and container.
VN Profile: Meta data associated with a VN (Virtual Network) that is VN Profile: Metadata that is associated with a VN and applied to any
applied to any attachment point to the VN. That is, VAP (Virtual attachment point to the VN (i.e., VAP (Virtual Access Point)
Access Point) properties that are applied to all VAPs associated with properties that are applied to all VAPs associated with a given VN
a given VN and used by an NVE when ingressing/egressing packets and used by an NVE when ingressing/egressing packets to/from a
to/from a specific VN. Meta data could include such information as specific VN). Metadata could include such information as Access
ACLs, QoS settings, etc. The VN Profile contains parameters that Control Lists (ACLs) and QoS settings. The VN Profile contains
apply to the VN as a whole. Control protocols between the NVE and parameters that apply to the VN as a whole. Control protocols
NVA (Network Virtualization Authority) could use the VN ID or VN Name between the NVE and the NVA (Network Virtualization Authority)
to obtain the VN Profile. could use the VN ID or VN Name to obtain the VN Profile.
VSI: Virtual Station Interface. [IEEE 802.1Q] VSI: Virtual Station Interface. See [IEEE802.1Q].
VDP: VSI Discovery and Configuration Protocol [IEEE 802.1Q] VDP: VSI Discovery and Configuration Protocol. See [IEEE802.1Q].
1.2 Target Scenarios 1.2. Target Scenarios
In the Split-NVE architecture, an external NVE can provide an offload In the Split-NVE architecture, an External NVE can provide offloading
of the encapsulation / decapsulation functions and network policy of the encapsulation/decapsulation functions and network policy
enforcement as well as the VN Overlay protocol overhead. This enforcement as well as offloading of overhead from the VN overlay
offloading may improve performance and/or save resources in the End protocol. This offloading may improve performance and/or save
Device (e.g. hypervisor) using the external NVE. resources in the end device (e.g., hypervisor) using the
External NVE.
The following figures give example scenarios of a Split-NVE Figures 2 through 4 give example scenarios for the Split-NVE
architecture. architecture.
Hypervisor Access Switch Hypervisor Access Switch
+------------------+ +-----+-------+ +------------------+ +-----+-------+
| +--+ +-------+ | | | | | +--+ +-------+ | | | |
| |VM|---| | | VLAN | | | | |VM|---| | | VLAN | | |
| +--+ | tNVE |---------+ nNVE| +--- Underlying | +--+ | tNVE |---------+ nNVE| +--- Underlying
| +--+ | | | Trunk | | | Network | +--+ | | | Trunk | | | Network
| |VM|---| | | | | | | |VM|---| | | | | |
| +--+ +-------+ | | | | | +--+ +-------+ | | | |
+------------------+ +-----+-------+ +------------------+ +-----+-------+
Figure 2 Hypervisor with an External NVE
Hypervisor L2 Switch Figure 2: Hypervisor with an External NVE
Hypervisor L2 Switch
+---------------+ +-----+ +----+---+ +---------------+ +-----+ +----+---+
| +--+ +----+ | | | | | | | +--+ +----+ | | | | | |
| |VM|---| | |VLAN | |VLAN | | | | |VM|---| | |VLAN | |VLAN | | |
| +--+ |tNVE|-------+ +-----+nNVE| +--- Underlying | +--+ |tNVE|-------+ +-----+nNVE| +--- Underlying
| +--+ | | |Trunk| |Trunk| | | Network | +--+ | | |Trunk| |Trunk| | | Network
| |VM|---| | | | | | | | | |VM|---| | | | | | | |
| +--+ +----+ | | | | | | | +--+ +----+ | | | | | |
+---------------+ +-----+ +----+---+ +---------------+ +-----+ +----+---+
Figure 3 Hypervisor with an External NVE
connected through an Ethernet Access Switch
Network Service Appliance Access Switch Figure 3: Hypervisor with an External NVE
+--------------------------+ +-----+-------+ Connected through an Ethernet Access Switch
| +------------+ | \ | | | |
| |Net Service |----| \ | | | |
| |Instance | | \ | VLAN | | |
| +------------+ |tNVE| |------+nNVE | +--- Underlying
| +------------+ | | | Trunk| | | Network
| |Net Service |----| / | | | |
| |Instance | | / | | | |
| +------------+ | / | | | |
+--------------------------+ +-----+-------+
Figure 4 Physical Network Service Appliance with an External NVE
Tenant Systems connect to external NVEs via a Tenant System Interface Network Service Appliance Access Switch
(TSI). The TSI logically connects to the external NVE via a Virtual +-----------------------------+ +-----+-------+
Access Point (VAP) [RFC8014]. The external NVE may provide Layer 2 or | +---------------+ | \ | | | |
Layer 3 forwarding. In the Split-NVE architecture, the external NVE | |Network Service|----| \ | | | |
may be able to reach multiple MAC and IP addresses via a TSI. An IP | |Instance | | \ | VLAN | | |
address can be in either IPv4 or IPv6 format. For example, Tenant | +---------------+ |tNVE| |------+nNVE | +--- Underlying
Systems that are providing network services (such as transparent | +---------------+ | | | Trunk| | | Network
firewall, load balancer, or VPN gateway) are likely to have a complex | |Network Service|----| / | | | |
address hierarchy. This implies that if a given TSI disassociates | |Instance | | / | | | |
from one VN, all the MAC and/or IP addresses are also disassociated. | +---------------+ | / | | | |
There is no need to signal the deletion of every MAC or IP when the +-----------------------------+ +-----+-------+
TSI is brought down or deleted. In the majority of cases, a VM will
be acting as a simple host that will have a single TSI and single MAC Figure 4: Physical Network Service Appliance with an External NVE
and IP visible to the external NVE.
Tenant Systems connect to External NVEs via a Tenant System Interface
(TSI). The TSI logically connects to the External NVE via a VAP
[RFC8014]. The External NVE may provide Layer 2 or Layer 3
forwarding. In the Split-NVE architecture, the External NVE may be
able to reach multiple Media Access Control (MAC) addresses and IP
addresses via a TSI. An IP address can be in either IPv4 or IPv6
format. For example, Tenant Systems that are providing network
services (such as a transparent firewall, load balancer, or VPN
gateway) are likely to have a complex address hierarchy. This
implies that if a given TSI de-associates from one VN, all the MAC
and/or IP addresses are also de-associated. There is no need to
signal the deletion of every MAC or IP address when the TSI is
brought down or deleted. In the majority of cases, a VM will be
acting as a simple host that will have a single TSI as well as a
single MAC and IP address visible to the External NVE.
Figures 2 through 4 show the use of VLANs to separate traffic for Figures 2 through 4 show the use of VLANs to separate traffic for
multiple VNs between the tNVE and nNVE; VLANs are not strictly multiple VNs between the tNVE and the nNVE; VLANs are not strictly
necessary if only one VN is involved, but multiple VNs are expected necessary if only one VN is involved, but multiple VNs are expected
in most cases. Hence this draft assumes the presence of VLANs. in most cases. Hence, this document assumes the presence of VLANs.
2. VM Lifecycle 2. VM Lifecycle
Figure 2 of [RFC7666] shows the state transition of a VM. Some of the Figure 2 of [RFC7666] shows the states and transitions of a VM. Some
VM states are of interest to the external NVE. This section of the VM states are of interest to the External NVE. This section
illustrates the relevant phases and events in the VM lifecycle. Note illustrates the relevant phases and events in the VM lifecycle. Note
that the following subsections do not give an exhaustive traversal of that the following subsections do not give exhaustive descriptions of
VM lifecycle state. They are intended as the illustrative examples VM lifecycle states. Rather, they are intended as illustrative
which are relevant to Split-NVE architecture, not as prescriptive examples that are relevant to the Split-NVE architecture and not as
text; the goal is to capture sufficient detail to set a context for prescriptive text; the goal is to capture sufficient detail to set a
the signaling protocol functionality and requirements described in context for the signaling-protocol functionality and requirements
the following sections. described in the following sections.
2.1 VM Creation Event 2.1. VM Creation Event
The VM creation event causes the VM state transition from Preparing The VM creation event causes the VM state to transition from the
to Shutdown and then to Running [RFC7666]. The end device allocates "preparing" state to the "shutdown" state and then to the "running"
and initializes local virtual resources like storage in the VM state [RFC7666]. The end device allocates and initializes local
Preparing state. In the Shutdown state, the VM has everything ready virtual resources like storage in the VM's preparing state. In the
except that CPU execution is not scheduled by the hypervisor and VM's shutdown state, the VM has everything ready, except that CPU
memory is not resident in the hypervisor. The transition from the execution is not scheduled by the hypervisor and the VM's memory is
Shutdown state to the Running state normally requires human action or not resident in the hypervisor. The transition from the shutdown
a system triggered event. Running state indicates the VM is in the state to the running state normally requires human action or a
normal execution state. As part of transitioning the VM to the system-triggered event. The running state indicates that the VM is
Running state, the hypervisor must also provision network in the normal execution state. As part of transitioning the VM to
the running state, the hypervisor must also provision network
connectivity for the VM's TSI(s) so that Ethernet frames can be sent connectivity for the VM's TSI(s) so that Ethernet frames can be sent
and received correctly. Initially, when Running, no ongoing and received correctly. Initially, when in the running state, no
migration, suspension or shutdown is in process. ongoing migration, suspension, or shutdown is in process.
In the VM creation phase, the VM's TSI has to be associated with the In the VM creation phase, the VM's TSI has to be associated with the
external NVE. Association here indicates that hypervisor and the External NVE. "Association" here indicates that the hypervisor and
external NVE have signaled each other and reached some agreement. the External NVE have signaled each other and reached some form of
Relevant networking parameters or information have been provisioned agreement. Relevant networking parameters or information have been
properly. The External NVE should be informed of the VM's TSI MAC provisioned properly. The External NVE should be informed of the
address and/or IP address. In addition to external network VM's TSI MAC address and/or IP address. In addition to external
connectivity, the hypervisor may provide local network connectivity network connectivity, the hypervisor may provide local network
between the VM's TSI and other VM's TSI that are co-resident on the connectivity between the VM's TSI and TSIs for other VMs that are
same hypervisor. When the intra- or inter-hypervisor connectivity is co-resident on the same hypervisor. When the intra- or
extended to the external NVE, a locally significant tag, e.g. VLAN inter-hypervisor connectivity is extended to the External NVE, a
ID, should be used between the hypervisor and the external NVE to locally significant tag, e.g., VLAN ID, should be used between the
differentiate each VN's traffic. Both the hypervisor and external NVE hypervisor and the External NVE to differentiate each VN's traffic.
sides must agree on that tag value for traffic identification, Both the hypervisor and External NVE sides must agree on that tag
isolation, and forwarding. value for traffic identification, isolation, and forwarding.
The external NVE may need to do some preparation before it signals The External NVE may need to do some preparation before it signals
successful association with the TSI. Such preparation may include successful association with the TSI. Such preparation may include
locally saving the states and binding information of the tenant locally saving the states and binding information of the TSI and its
system interface and its VN, communicating with the NVA for network VN or communicating with the NVA for network provisioning.
provisioning, etc.
Tenant System interface association should be performed before the VM A TSI association should be performed before the VM enters the
enters the Running state, preferably in the Shutdown state. If running state, preferably in the shutdown state. If the association
association with an external NVE fails, the VM should not go into the with an External NVE fails, the VM should not go into the running
Running state. state.
2.2 VM Live Migration Event 2.2. VM Live Migration Event
Live migration is sometimes referred to as "hot" migration in that, Live migration is sometimes referred to as "hot" migration in that,
from an external viewpoint, the VM appears to continue to run while from an external viewpoint, the VM appears to continue to run while
being migrated to another server (e.g., TCP connections generally being migrated to another server (e.g., TCP connections generally
survive this class of migration). In contrast, "cold" migration survive this class of migration). In contrast, "cold" migration
consists of shutting down VM execution on one server and restarting consists of shutting down VM execution on one server and restarting
it on another. For simplicity, the following abstract summary of live it on another. For simplicity, the following abstract summary of
migration assumes shared storage, so that the VM's storage is live migration assumes shared storage, so that the VM's storage is
accessible to the source and destination servers. Assume VM live accessible to the source and destination servers. Assume that the VM
migrates from hypervisor 1 to hypervisor 2. Such a migration event "live migrates" from hypervisor 1 to hypervisor 2. Such a migration
involves state transitions on both source hypervisor 1 and event involves state transitions on both source hypervisor 1 and
destination hypervisor 2. The VM state on source hypervisor 1 destination hypervisor 2. The VM state on source hypervisor 1
transits from Running to Migrating and then to Shutdown [RFC7666]. transitions from the running state to the "migrating" state and then
The VM state on destination hypervisor 2 transits from Shutdown to to the shutdown state [RFC7666]. The VM state on destination
Migrating and then Running. hypervisor 2 transitions from the shutdown state to the migrating
state and then to the running state.
The external NVE connected to destination hypervisor 2 has to The External NVE connected to destination hypervisor 2 has to
associate the migrating VM's TSI with it by discovering the TSI's MAC associate the migrating VM's TSI with itself (i.e., the External NVE)
and/or IP addresses, its VN, locally significant VLAN ID if any, and by discovering the TSI's MAC and/or IP addresses, discovering its VN,
provisioning other network related parameters of the TSI. The discovering its locally significant VLAN ID (if any), and
external NVE may be informed about the VM's peer VMs, storage devices provisioning other network-related parameters of the TSI. The
and other network appliances with which the VM needs to communicate External NVE may be informed about the VM's peer VMs, storage
or is communicating. The migrated VM on destination hypervisor 2 devices, and other network appliances with which the VM needs to
should not go to Running state until all the network provisioning and communicate or is communicating. The migrated VM on destination
binding has been done. hypervisor 2 should not go to the running state until all the network
provisioning and binding have been done.
The states of VM on the source and destination hypervisors both are The VM state on both the source hypervisor and the destination
Migrating during transfer of migration execution. The migrating VM hypervisor will be the migrating state during the transfer of VM
should not be in Running state at the same time on the source execution. The migrating VM should not be in the running state at
hypervisor and destination hypervisor during migration. The VM on the the same time on the source hypervisor and destination hypervisor
source hypervisor does not transition into Shutdown state until the during migration. The VM on the source hypervisor does not
VM successfully enters the Running state on the destination transition to the shutdown state until the VM successfully enters the
hypervisor. It is possible that the VM on the source hypervisor stays running state on the destination hypervisor. It is possible that the
in Migrating state for a while after the VM on the destination VM on the source hypervisor stays in the migrating state for a while
hypervisor enters Running state. after the VM on the destination hypervisor enters the running state.
2.3 VM Termination Event 2.3. VM Termination Event
A VM termination event is also referred to as "powering off" a VM. A A VM termination event is also referred to as "powering off" a VM. A
VM termination event leads to its state becoming Shutdown. There are VM termination event leads to the VM's transition to the shutdown
two possible causes of VM termination [RFC7666]. One is the normal state. Per [RFC7666], there are two possible causes of VM
"power off" of a running VM; the other is that the VM has been termination:
migrated to another hypervisor and the VM image on the source
hypervisor has to stop executing and be shutdown.
In VM termination, the external NVE connecting to that VM needs to 1. A running VM has undergone a normal "power-off".
deprovision the VM, i.e. delete the network parameters associated
with that VM. In other words, the external NVE has to de-associate 2. The VM has been migrated to another hypervisor, and the VM image
on the source hypervisor has to stop executing and be shut down.
In VM termination, the External NVE connecting to that VM needs to
deprovision the VM, i.e., delete the network parameters associated
with that VM. In other words, the External NVE has to de-associate
the VM's TSI. the VM's TSI.
2.4 VM Pause, Suspension and Resumption Events 2.4. VM Pause, Suspension, and Resumption Events
A VM pause event leads to the VM transiting from Running state to A VM pause event leads to the VM transitioning from the running state
Paused state. The Paused state indicates that the VM is resident in to the "paused" state. The paused state indicates that the VM is
memory but no longer scheduled to execute by the hypervisor resident in memory but that CPU execution is not scheduled by the
[RFC7666]. The VM can be easily re-activated from Paused state to hypervisor [RFC7666]. The VM can be easily reactivated from the
Running state. paused state to the running state.
A VM suspension event leads to the VM transiting from Running state A VM suspension event leads to the VM transitioning from the running
to Suspended state. A VM resumption event leads to the VM transiting state to the "suspended" state. A VM resumption event leads to the
state from Suspended state to Running state. Suspended state means VM transitioning from the suspended state to the running state. In
the memory and CPU execution state of the virtual machine are saved the suspended state, the memory and CPU execution state of the VM are
to persistent store. During this state, the virtual machine is not saved to persistent storage. During this state, CPU execution for
scheduled to execute by the hypervisor [RFC7666]. the VM is not scheduled by the hypervisor [RFC7666].
In the Split-NVE architecture, the external NVE should not In the Split-NVE architecture, the External NVE should not
disassociate the paused or suspended VM as the VM can return to de-associate the paused or suspended VM, as the VM can return to the
Running state at any time. running state at any time.
3. Hypervisor-to-NVE Control Plane Protocol Functionality 3. Hypervisor-to-NVE Control-Plane Protocol Functionality
The following subsections show illustrative examples of the state The following subsections show illustrative examples of the state
transitions of an external NVE which are relevant to Hypervisor-to- transitions of an External NVE that are relevant to hypervisor-to-NVE
NVE Signaling protocol functionality. It should be noted this is not signaling-protocol functionality. Note: This is not prescriptive
prescriptive text for the full state machine. text for the full state machine.
3.1 VN Connect and Disconnect 3.1. VN_Connect and VN_Disconnect
In the Split-NVE scenario, a protocol is needed between the End In the Split-NVE scenario, a protocol is needed between the end
Device (e.g. Hypervisor) and the external NVE it is using in order to device (e.g., hypervisor) and the External NVE it is using, in order
make the external NVE aware of the changing VN membership to make the External NVE aware of the changing VN membership
requirements of the Tenant Systems within the End Device. requirements of the Tenant Systems within the end device.
A key driver for using a protocol rather than using static A key driver for using a protocol rather than using static
configuration of the external NVE is because the VN connectivity configuration of the External NVE is that the VN connectivity
requirements can change frequently as VMs are brought up, moved, and requirements can change frequently as VMs are brought up, moved, and
brought down on various hypervisors throughout the data center or brought down on various hypervisors throughout the data center or
external cloud. external cloud.
+---------------+ Receive VN_connect; +-------------------+ Figure 5 shows the state transition for a VAP on the External NVE.
|VN_Disconnected| return Local_Tag value |VN_Connected | An NVE that supports the hypervisor-to-NVE control-plane protocol
+---------------+ for VN if successful; +-------------------+ should support one instance of the state machine for each active VN.
|VN_ID; |-------------------------->|VN_ID; | The state transition on the External NVE is normally triggered by
|VN_State= | |VN_State=connected;| events and behaviors on the hypervisor-facing side. Some of the
|disconnected; | |Num_TSI_Associated;| interleaved interactions between the NVE and the NVA will be
| |<--Receive VN_disconnect---|Local_Tag; | illustrated to better explain the whole procedure, while other
+---------------+ |VN_Context; | interactions will not be shown.
+-------------------+
Figure 5. State Transition Example of a VAP Instance +----------------+ Receive VN_Connect; +----------------------+
on an External NVE |VN_Disconnected | return Local_Tag value |VN_Connected |
+----------------+ for VN if successful; +----------------------+
|VN_ID; |-------------------------->|VN_ID; |
|VN_State= | |VN_State=VN_Connected;|
|VN_Disconnected;| |Num_TSI_Associated; |
| |<--Receive VN_Disconnect---|Local_Tag; |
+----------------+ |VN_Context; |
+----------------------+
Figure 5 shows the state transition for a VAP on the external NVE. An Figure 5: State Transition Example of a VAP Instance
NVE that supports the hypervisor to NVE control plane protocol should on an External NVE
support one instance of the state machine for each active VN. The
state transition on the external NVE is normally triggered by the
hypervisor-facing side events and behaviors. Some of the interleaved
interaction between NVE and NVA will be illustrated to better explain
the whole procedure; while others of them may not be shown.
The external NVE must be notified when an End Device requires The External NVE must be notified when an end device requires a
connection to a particular VN and when it no longer requires connection to a particular VN and when it no longer requires a
connection. Connection clean up for the failed devices should be connection. Connection cleanup for the failed devices should be
employed which is out of the scope of the protocol specified in this employed. Note that this topic is out of scope for the protocol
document. specified in this document.
In addition, the external NVE should provide a local tag value for In addition, the External NVE should provide a local tag value for
each connected VN to the End Device to use for exchanging packets each connected VN to the end device to use for exchanging packets
between the End Device and the external NVE (e.g. a locally between the end device and the External NVE (e.g., a locally
significant [IEEE 802.1Q] tag value). How "local" the significance is significant tag value per [IEEE802.1Q]). How "local" the
depends on whether the Hypervisor has a direct physical connection to significance is depends on whether
the external NVE (in which case the significance is local to the
physical link), or whether there is an Ethernet switch (e.g. a blade 1. the hypervisor has a direct physical connection to the
switch) connecting the Hypervisor to the NVE (in which case the External NVE (in which case the significance is local to the
significance is local to the intervening switch and all the links physical link) or
connected to it).
2. there is an Ethernet switch (e.g., a blade switch) connecting the
hypervisor to the NVE (in which case the significance is local to
the intervening switch and all the links connected to it).
These VLAN tags are used to differentiate between different VNs as These VLAN tags are used to differentiate between different VNs as
packets cross the shared access network to the external NVE. When the packets cross the shared-access network to the External NVE. When
external NVE receives packets, it uses the VLAN tag to identify their the External NVE receives packets, it uses the VLAN tag to identify
VN coming from a given TSI, strips the tag, adds the appropriate their VN coming from a given TSI, strips the tag, adds the
overlay encapsulation for that VN, and sends it towards the appropriate overlay encapsulation for that VN, and sends it towards
corresponding remote NVE across the underlying IP network. the corresponding remote NVE across the underlying IP network.
The Identification of the VN in this protocol could either be through The Identification of the VN in this protocol could be through either
a VN Name or a VN ID. A globally unique VN Name facilitates a VN Name or a VN ID. A globally unique VN Name facilitates
portability of a Tenant's Virtual Data Center. Once an external NVE portability of a tenant's virtual data center. Once an External NVE
receives a VN connect indication, the NVE needs a way to get a VN receives a VN_Connect message, the NVE needs a way to get a
Context allocated (or receive the already allocated VN Context) for a VN_Context allocated (or to receive the already-allocated VN_Context)
given VN Name or ID (as well as any other information needed to for a given VN Name or VN ID (as well as any other information needed
transmit encapsulated packets). How this is done is the subject of to transmit encapsulated packets). How this is done is the subject
the NVE-to-NVA protocol which are part of work items 1 and 2 in of the NVE-to-NVA protocol; see the "first two areas of work" text in
[RFC7364]. The external NVE needs to synchronize the mapping Section 4.5 of [RFC7364]. The External NVE needs to synchronize the
information of the local tag and VN Name or VN ID with NVA. mapping information of the local tag and VN Name or VN ID with
the NVA.
The VN_connect message can be explicit or implicit. Explicit means The VN_Connect message can be explicit or implicit. "Explicit" means
the hypervisor sends a request message explicitly for the connection that the hypervisor sends a request message explicitly for the
to a VN. Implicit means the external NVE receives other messages, connection to a VN. "Implicit" means that the External NVE receives
e.g. very first TSI associate message (see the next subsection) for a other messages, e.g., the very first TSI Associate message (see the
given VN, that implicitly indicate its interest in connecting to a next subsection) for a given VN, that implicitly indicate its
VN. interest in connecting to a VN.
A VN_disconnect message indicates that the NVE can release all the A VN_Disconnect message indicates that the NVE can release all the
resources for that disconnected VN and transit to VN_disconnected resources for that disconnected VN and transition to the
state. The local tag assigned for that VN can possibly be reclaimed VN_Disconnected state. The local tag assigned for that VN can
for use by another VN. possibly be reclaimed for use by another VN.
3.2 TSI Associate and Activate 3.2. TSI Associate and Activate
Typically, a TSI is assigned a single MAC address and all frames Typically, a TSI is assigned a single MAC address, and all frames
transmitted and received on that TSI use that single MAC address. As transmitted and received on that TSI use that single MAC address. As
mentioned earlier, it is also possible for a Tenant System to mentioned earlier, it is also possible for a Tenant System to
exchange frames using multiple MAC addresses or packets with multiple exchange frames using multiple MAC addresses or packets with multiple
IP addresses. IP addresses.
Particularly in the case of a TS that is forwarding frames or packets Particularly in the case of a Tenant System that is forwarding frames
from other TSs, the external NVE will need to communicate the mapping or packets from other Tenant Systems, the External NVE will need to
between the NVE's IP address on the underlying network and ALL the communicate the mapping between the NVE's IP address on the
addresses the TS is forwarding on behalf of the corresponding VN to underlying network and ALL the addresses the Tenant System is
the NVA. forwarding on behalf of the corresponding VN to the NVA.
The NVE has two ways it can discover the tenant addresses for which The NVE has two ways it can discover the tenant addresses for which
frames are to be forwarded to a given End Device (and ultimately to frames are to be forwarded to a given end device (and ultimately to
the TS within that End Device). the Tenant System within that end device).
1. It can glean the addresses by inspecting the source addresses in 1. It can glean the addresses by inspecting the source addresses in
packets it receives from the End Device. packets it receives from the end device.
2. The hypervisor can explicitly signal the address associations of 2. The hypervisor can explicitly signal the address associations of
a TSI to the external NVE. An address association includes all the a TSI to the External NVE. An address association includes all
MAC and/or IP addresses possibly used as source addresses in a packet the MAC and/or IP addresses possibly used as source addresses in
sent from the hypervisor to external NVE. The external NVE may a packet sent from the hypervisor to the External NVE. The
further use this information to filter the future traffic from the External NVE may further use this information to filter the
hypervisor. future traffic from the hypervisor.
To use the second approach above, the "hypervisor-to-NVE" protocol To use the second approach above, the control-plane protocol running
must support End Devices communicating new tenant addresses between the hypervisor and the NVE must support end devices
associations for a given TSI within a given VN. communicating new tenant-address associations for a given TSI within
a given VN.
Figure 6 shows the example of a state transition for a TSI connecting Figure 6 shows an example of a state transition for a TSI connecting
to a VAP on the external NVE. An NVE that supports the hypervisor to to a VAP on the External NVE. An NVE that supports the hypervisor-
NVE control plane protocol may support one instance of the state to-NVE control-plane protocol may support one instance of the state
machine for each TSI connecting to a given VN. machine for each TSI connecting to a given VN.
disassociate +--------+ disassociate De-Associate +--------+ De-Associate
+--------------->| Init |<--------------------+ +--------------->| Init |<--------------------+
| +--------+ | | +--------+ |
| | | | | | | |
| | | | | | | |
| +--------+ | | +--------+ |
| | | | | | | |
| associate | | activate | | Associate | | Activate |
| +-----------+ +-----------+ | | +-----------+ +-----------+ |
| | | | | | | |
| | | | | | | |
| \|/ \|/ | | \|/ \|/ |
+--------------------+ +---------------------+ +--------------------+ +---------------------+
| Associated | | Activated | | Associated | | Activated |
+--------------------+ +---------------------+ +--------------------+ +---------------------+
|TSI_ID; | |TSI_ID; | |TSI_ID; | |TSI_ID; |
|Port; |-----activate---->|Port; | |Port; |-----Activate---->|Port; |
|VN_ID; | |VN_ID; | |VN_ID; | |VN_ID; |
|State=associated; | |State=activated ; |-+ |State=Associated; | |State=Activated; |-+
+-|Num_Of_Addr; |<---deactivate ---|Num_Of_Addr; | | +-|Num_Of_Addr; |<---Deactivate ---|Num_Of_Addr; | |
| |List_Of_Addr; | |List_Of_Addr; | | | |List_Of_Addr; | |List_Of_Addr; | |
| +--------------------+ +---------------------+ | | +--------------------+ +---------------------+ |
| /|\ /|\ | | /|\ /|\ |
| | | | | | | |
+---------------------+ +-------------------+ +---------------------+ +-------------------+
add/remove/updt addr; add/remove/updt addr; add/remove/updt addr; add/remove/updt addr;
or update port; or update port; or update port; or update port;
Figure 6 State Transition Example of a TSI Instance Figure 6: State Transition Example of a TSI Instance
on an External NVE on an External NVE
The Associated state of a TSI instance on an external NVE indicates The Associated state of a TSI instance on an External NVE indicates
all the addresses for that TSI have already associated with the VAP that all the addresses for that TSI have already associated with the
of the external NVE on a given port e.g. on port p for a given VN but VAP of the External NVE on a given port, e.g., on port p for a given
no real traffic to and from the TSI is expected and allowed to pass VN, but no real traffic to and from the TSI is expected and allowed
through. An NVE has reserved all the necessary resources for that to pass through. An NVE has reserved all the necessary resources for
TSI. An external NVE may report the mappings of its underlay IP that TSI. An External NVE may report the mappings of its underlay IP
address and the associated TSI addresses to NVA and relevant network address and the associated TSI addresses to the NVA, and relevant
nodes may save such information to their mapping tables but not their network nodes may save such information to their mapping tables but
forwarding tables. An NVE may create ACL or filter rules based on the not their forwarding tables. An NVE may create ACLs or filter rules
associated TSI addresses on that attached port p but not enable them based on the associated TSI addresses on that attached port p but not
yet. The local tag for the VN corresponding to the TSI instance enable them yet. The local tag for the VN corresponding to the TSI
should be provisioned on port p to receive packets. instance should be provisioned on port p to receive packets.
The VM migration event (discussed section 2) may cause the hypervisor The VM migration event (discussed in Section 2) may cause the
to send an associate message to the NVE connected to the destination hypervisor to send an Associate message to the NVE connected to the
hypervisor of the migration. A VM creation event may also cause to destination hypervisor of the migration. A VM creation event may
the same practice. also trigger the same scenario.
The Activated state of a TSI instance on an external NVE indicates The Activated state of a TSI instance on an External NVE indicates
that all the addresses for that TSI are functioning correctly on a that all the addresses for that TSI are functioning correctly on a
given port e.g. port p and traffic can be received from and sent to given port, e.g., port p, and traffic can be received from and sent
that TSI via the NVE. The mappings of the NVE's underlay IP address to that TSI via the NVE. The mappings of the NVE's underlay IP
and the associated TSI addresses should be put into the forwarding address and the associated TSI addresses should be added to the
table rather than the mapping table on relevant network nodes. ACL or forwarding table rather than the mapping table on relevant network
filter rules based on the associated TSI addresses on the attached nodes. ACLs or filter rules based on the associated TSI addresses on
port p in the NVE are enabled. The local tag for the VN corresponding the attached port p on the NVE are enabled. The local tag for the VN
to the TSI instance must be provisioned on port p to receive packets. corresponding to the TSI instance must be provisioned on port p to
receive packets.
The Activate message makes the state transit from Init or Associated The Activate message makes the state transition from Init or
to Activated. VM creation, VM migration, and VM resumption events Associated to Activated. VM creation, VM migration, and VM
discussed in Section 4 may trigger sending the Activate message from resumption events (discussed in Section 2) may trigger sending the
the hypervisor to the external NVE. Activate message from the hypervisor to the External NVE.
TSI information may get updated in either the Associated or Activated TSI information may get updated in either the Associated state or the
state. The following are considered updates to the TSI information: Activated state. The following are considered updates to the TSI
add or remove the associated addresses, update the current associated information: add or remove the associated addresses, update the
addresses (for example updating IP for a given MAC), and update the current associated addresses (for example, update the IP address for
NVE port information based on where the NVE receives messages. Such a given MAC address), and update the NVE port information based on
updates do not change the state of TSI. When any address associated where the NVE receives messages. Such updates do not change the
with a given TSI changes, the NVE should inform the NVA to update the state of the TSI. When any address associated with a given TSI
mapping information for NVE's underlying address and the associated changes, the NVE should inform the NVA to update the mapping
TSI addresses. The NVE should also change its local ACL or filter information for the NVE's underlying address and the associated TSI
settings accordingly for the relevant addresses. Port information addresses. The NVE should also change its local ACLs or filter
settings accordingly for the relevant addresses. Port information
updates will cause the provisioning of the local tag for the VN updates will cause the provisioning of the local tag for the VN
corresponding to the TSI instance on new port and removal from the corresponding to the TSI instance on the new port and removal from
old port. the old port.
3.3 TSI Disassociate and Deactivate 3.3. TSI De-Associate and Deactivate
Disassociate and deactivate behaviors are conceptually the reverse of De-Associate and Deactivate behaviors are conceptually the reverse of
associate and activate. Associate and Activate.
From Activated state to Associated state, the external NVE needs to From the Activated state to the Associated state, the External NVE
make sure the resources are still reserved but the addresses needs to make sure the resources are still reserved but the addresses
associated to the TSI are not functioning. No traffic to or from the associated with the TSI are not functioning. No traffic to or from
TSI is expected or allowed to pass through. For example, the NVE the TSI is expected or allowed to pass through. For example, the NVE
needs to tell the NVA to remove the relevant addresses mapping needs to tell the NVA to remove the relevant information regarding
information from forwarding and routing tables. ACL and filtering address mapping from the forwarding and routing tables. ACLs and
rules regarding the relevant addresses should be disabled. filter rules regarding the relevant addresses should be disabled.
From Associated or Activated state to the Init state, the NVE From the Associated or Activated state to the Init state, the NVE
releases all the resources relevant to TSI instances. The NVE should releases all the resources relevant to TSI instances. The NVE should
also inform the NVA to remove the relevant entries from mapping also inform the NVA to remove the relevant entries from the mapping
table. ACL or filtering rules regarding the relevant addresses should table. ACLs or filter rules regarding the relevant addresses should
be removed. Local tag provisioning on the connecting port on NVE be removed. Local tag provisioning on the connecting port on the NVE
should be cleared. should be cleared.
A VM suspension event (discussed in section 2) may cause the relevant A VM suspension event (discussed in Section 2) may cause the relevant
TSI instance(s) on the NVE to transit from Activated to Associated TSI instance(s) on the NVE to transition from the Activated state to
state. the Associated state.
A VM pause event normally does not affect the state of the relevant A VM pause event normally does not affect the state of the relevant
TSI instance(s) on the NVE as the VM is expected to run again soon. TSI instance(s) on the NVE, as the VM is expected to run again soon.
A VM shutdown event will normally cause the relevant TSI instance(s) A VM shutdown event will normally cause the relevant TSI instance(s)
on the NVE to transition to Init state from Activated state. All on the NVE to transition to the Init state from the Activated state.
resources should be released. All resources should be released.
A VM migration will cause the TSI instance on the source NVE to leave A VM migration will cause the TSI instance on the source NVE to leave
Activated state. When a VM migrates to another hypervisor connecting the Activated state. When a VM migrates to another hypervisor
to the same NVE, i.e. source and destination NVE are the same, NVE connecting to the same NVE, i.e., the source and destination NVE are
should use TSI_ID and incoming port to differentiate two TSI the same, the NVE should use the TSI_ID and the incoming port to
instances. differentiate two TSI instances.
Although the triggering messages for the state transition shown in Although the triggering messages for the state transition shown in
Figure 6 does not indicate the difference between a VM Figure 6 do not indicate the difference between a VM
creation/shutdown event and a VM migration arrival/departure event, creation/shutdown event and a VM migration arrival/departure event,
the external NVE can make optimizations if it is given such the External NVE can make optimizations if it is given such
information. For example, if the NVE knows the incoming activate information. For example, if the NVE knows that the incoming
message is caused by migration rather than VM creation, some Activate message is caused by migration rather than VM creation, some
mechanisms may be employed or triggered to make sure the dynamic mechanisms may be employed or triggered to make sure the dynamic
configurations or provisionings on the destination NVE are the same configurations or provisionings on the destination NVE are the same
as those on the source NVE for the migrated VM. For example an IGMP as those on the source NVE for the migrated VM. For example, an IGMP
query [RFC2236] can be triggered by the destination external NVE to query [RFC2236] can be triggered by the destination External NVE to
the migrated VM so that VM is forced to send an IGMP report to the the migrated VM so that the VM is forced to send an IGMP report to
multicast router. Then a multicast router can correctly route the the multicast router. The multicast router can then correctly route
multicast traffic to the new external NVE for those multicast groups the multicast traffic to the new External NVE for those multicast
the VM joined before the migration. groups the VM joined before the migration.
4. Hypervisor-to-NVE Control Plane Protocol Requirements 4. Hypervisor-to-NVE Control-Plane Protocol Requirements
Req-1: The protocol MUST support a bridged network connecting End Req-1: The protocol MUST support a bridged network connecting end
Devices to the External NVE. devices to the External NVE.
Req-2: The protocol MUST support multiple End Devices sharing the Req-2: The protocol MUST support multiple end devices sharing the
same External NVE via the same physical port across a bridged same External NVE via the same physical port across a
network. bridged network.
Req-3: The protocol MAY support an End Device using multiple external Req-3: The protocol MAY support an end device using multiple
NVEs simultaneously, but only one external NVE for each VN. External NVEs simultaneously, but only one External NVE for
each VN (active-standby External NVE case for a VN).
Req-4: The protocol MAY support an End Device using multiple external Req-4: The protocol MAY support an end device using multiple
NVEs simultaneously for the same VN. External NVEs simultaneously for the same VN (active-active
External NVE case for a VN).
Req-5: The protocol MUST allow the End Device to initiate a request Req-5: The protocol MUST allow the end device to initiate a request
to its associated External NVE to be connected/disconnected to a to its associated External NVE to be connected/disconnected
given VN. to a given VN.
Req-6: The protocol MUST allow an External NVE initiating a request Req-6: The protocol MUST allow an External NVE initiating a request
to its connected End Devices to be disconnected from a given VN. to its connected end devices to be disconnected from a
given VN.
Req-7: When a TS attaches to a VN, the protocol MUST allow for an End Req-7: When a Tenant System attaches to a VN, the protocol MUST
Device and its external NVE to negotiate one or more locally- allow for an end device and its External NVE to negotiate
significant tag(s) for carrying traffic associated with a specific VN one or more locally significant tags for carrying traffic
(e.g., [IEEE 802.1Q] tags). associated with a specific VN (e.g., tags per [IEEE802.1Q]).
Req-8: The protocol MUST allow an End Device initiating a request to Req-8: The protocol MUST allow an end device initiating a request
associate/disassociate and/or activate/deactive some or all to associate/de-associate and/or activate/deactivate some or
address(es) of a TSI instance to a VN on an NVE port. all addresses of a TSI instance to a VN on an NVE port.
Req-9: The protocol MUST allow the External NVE initiating a request Req-9: The protocol MUST allow the External NVE initiating a
to disassociate and/or deactivate some or all address(es) of a TSI request to de-associate and/or deactivate some or all
instance to a VN on an NVE port. addresses of a TSI instance to a VN on an NVE port.
Req-10: The protocol MUST allow an End Device initiating a request to Req-10: The protocol MUST allow an end device initiating a request
add, remove or update address(es) associated with a TSI instance on to add, remove, or update address(es) associated with a TSI
the external NVE. Addresses can be expressed in different formats, instance on the External NVE. Addresses can be expressed in
for example, MAC, IP or pair of IP and MAC. different formats -- for example, MAC, IP, or IP-MAC pair.
Req-11: The protocol MUST allow the External NVE and the connected Req-11: The protocol MUST allow the External NVE and the connected
End Device to authenticate each other. end device to authenticate each other.
Req-12: The protocol MUST be able to run over L2 links between the Req-12: The protocol MUST be able to run over Layer 2 links between
End Device and its External NVE. the end device and its External NVE.
Req-13: The protocol SHOULD support the End Device indicating if an Req-13: The protocol SHOULD support an end device that indicates
associate or activate request from it is the result of a VM hot that an Associate or Activate request from the end device is
migration event. the result of a VM hot migration event.
5. VDP Applicability and Enhancement Needs 5. VDP Applicability and Enhancement Needs
Virtual Station Interface (VSI) Discovery and Configuration Protocol The Virtual Station Interface (VSI) Discovery and Configuration
(VDP) [IEEE 802.1Q] can be the control plane protocol running between Protocol (VDP) [IEEE802.1Q] can be the control-plane protocol running
the hypervisor and the external NVE. Appendix A illustrates VDP for between the hypervisor and the External NVE. Appendix A provides
the reader's information. informative VDP illustrations for the reader.
VDP facilitates the automatic discovery and configuration of Edge VDP facilitates the automatic discovery and configuration of Edge
Virtual Bridging (EVB) stations and Edge Virtual Bridging (EVB) Virtual Bridging (EVB) stations and EVB bridges. An EVB station is
bridges. An EVB station is normally an end station running multiple normally an end station running multiple VMs. In this document, it
VMs. It is conceptually equivalent to a hypervisor in this document. is considered conceptually equivalent to a hypervisor. An EVB bridge
An EVB bridge is conceptually equivalent to the external NVE. is conceptually equivalent to the External NVE.
VDP is able to pre-associate/associate/de-associate a VSI on an EVB VDP is able to pre-associate/associate/de-associate a VSI on an EVB
station with a port on the EVB bridge. A VSI is approximately the station with a port on the EVB bridge. In the context of this
concept of a virtual port by which a VM connects to the hypervisor in document, a VSI is conceptually approximate to a virtual port by
this document's context. The EVB station and the EVB bridge can reach which a VM connects to the hypervisor. The EVB station and the EVB
agreement on VLAN ID(s) assigned to a VSI via VDP message exchange. bridge can reach agreement on VLAN ID(s) assigned to a VSI via a VDP
Other configuration parameters can be exchanged via VDP as well. VDP message exchange. Other configuration parameters can be exchanged
is carried over the Edge Control Protocol(ECP) [IEEE 802.1Q] which via VDP as well. VDP is carried over the Edge Control Protocol (ECP)
provides a reliable transportation over a layer 2 network. [IEEE802.1Q], which provides reliable transportation over a Layer 2
network.
VDP protocol needs some extensions to fulfill the requirements listed VDP needs some extensions to fulfill the requirements listed in
in this document. Table 1 shows the needed extensions and/or Section 4 of this document. Table 1 shows the needed extensions
clarifications in the NVO3 context. and/or clarifications in the NVO3 context.
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
| Req | Supported | remarks | | Req | Supported | Remarks |
| | by VDP? | | | | by VDP? | |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
| Req-1| | | | Req-1| | |
+------+ |Needs extension. Must be able to send to a | +------+ |Needs extension. Must be able to send to a |
| Req-2| |specific unicast MAC and should be able to send| | Req-2| |specific unicast MAC, and should be able to |
+------+ Partially |to a non-reserved well known multicast address | +------+ Partially |send to a non-reserved well-known multicast |
| Req-3| |other than the nearest customer bridge address.| | Req-3| |address other than the nearest customer bridge |
+------+ | | +------+ |address. |
| Req-4| | | | Req-4| | |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
| Req-5| Yes |VN is indicated by GroupID | | Req-5| Yes |The VN is indicated by GroupID. |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
| Req-6| Yes |Bridge sends De-Associate | | Req-6| Yes |The bridge sends a De-Associate. |
+------+-----------+------------------------+----------------------+ +------+-----------+------------------------+----------------------+
| | |VID==NULL in request and bridge returns the | | | |VID==NULL in the request. The bridge returns |
| Req-7| Yes |assigned value in response or specify GroupID | | | |the assigned VLAN ID (VID) value in the |
| | |in request and get VID assigned in returning | | Req-7| Yes |response. GroupID, which is optionally present|
| | |response. Multiple VLANs per group are allowed.| | | |in the request, is equivalent to the VN ID in |
| | |the context of NVO3. Multiple VLANs per group |
| | |are allowed. |
+------+-----------+------------------------+----------------------+ +------+-----------+------------------------+----------------------+
| | | requirements | VDP equivalence | | | | Requirements | VDP Equivalent |
| | +------------------------+----------------------+ | | +------------------------+----------------------+
| | | associate/disassociate|pre-asso/de-associate | | Req-8| Partially | Associate/De-Associate |Pre-Assoc/De-Associate|
| Req-8| Partially | activate/deactivate |associate/de-associate| | | | Activate/Deactivate |Associate/De-Associate|
| | +------------------------+----------------------| | | +------------------------+----------------------|
| | |Needs extension to allow associate->pre-assoc | | | |Needs extension to allow Associate->Pre-Assoc. |
+------+-----------+------------------------+----------------------+ +------+-----------+------------------------+----------------------+
| Req-9| Yes | VDP bridge initiates de-associate | | Req-9| Yes |The VDP bridge initiates a De-Associate. |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
|Req-10| Partially |Needs extension for IPv4/IPv6 address. Add a | |Req-10| Partially |Needs extension for an IPv4/IPv6 address. |
| | |new "filter info format" type. | | | |Add a new "filter information format" type. |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
|Req-11| No |Out-of-band mechanism is preferred, e.g. MACSec| | | |An out-of-band mechanism is preferred, e.g., |
| | |or 802.1X. Implicit authentication based on | | | |MACsec or 802.1X. Implicit authentication |
| | |control of physical connectivity exists in VDP | |Req-11| No |based on control of physical connectivity |
| | |when the External NVE connects to the End | | | |exists in VDP when the External NVE connects to|
| | |Device directly and is reachable with the | | | |the end device directly and is reachable with |
| | |nearest customer bridge address. | | | |the nearest customer bridge address. |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
|Req-12| Yes |L2 protocol naturally | |Req-12| Yes |VDP naturally runs on the Layer 2 protocol. |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
| | |M bit for migrated VM on destination hypervisor| | | |A migration event may cause the M-bit to be set|
| | |and S bit for that on source hypervisor. | | | |to 1 in the VDP request to the migration |
|Req-13| Partially |It is indistinguishable when M/S is 0 between | | | |destination hypervisor and the S-bit to be set |
| | |no guidance and events not caused by migration | | | |to 1 in the VDP request to the migration source|
| | |where NVE may act differently. Needs new | | | |hypervisor. However, a setting of M-bit = 0 or|
| | |New bits for migration indication in new | |Req-13| Partially |S-bit = 0 can indicate that no information is |
| | |"filter info format" type. | | | |available regarding migration or that the |
| | |events in question are not caused by migration.|
| | |To fully meet the requirement, this ambiguity |
| | |would need to be fixed so that migration or no |
| | |migration could be safely inferred from the |
| | |M-bit or S-bit settings. |
+------+-----------+-----------------------------------------------+ +------+-----------+-----------------------------------------------+
Table 1 Compare VDP with the requirements
Simply adding the ability to carry layer 3 addresses, VDP can serve Table 1: Comparison of Split-NVE Requirements and VDP Capabilities
the Hypervisor-to-NVE control plane functions pretty well. Other
extensions are the improvement of the protocol capabilities for
better fit in an NVO3 network.
6. Security Considerations By simply adding the ability to carry Layer 3 addresses as per
Req-10, VDP can provide most of the hypervisor-to-NVE control-plane
functionality required.
6. Security Considerations
External NVEs must ensure that only properly authorized Tenant External NVEs must ensure that only properly authorized Tenant
Systems are allowed to join and become a part of any particular Systems are allowed to join and become a part of any particular VN.
Virtual Network. In some cases, tNVE may want to connect to the nNVE In some cases, the tNVE may want to connect to the nNVE for
for provisioning purposes. This may require that the tNVE provisioning purposes. This may require that the tNVE authenticate
authenticate the nNVE in addition to the nNVE authenticating the the nNVE in addition to the nNVE authenticating the tNVE. If a
tNVE. If a secure channel is required between tNVE and nNVE to carry secure channel is required between the tNVE and the nNVE to carry the
encrypted split-NVE control plane protocol, then existing mechanisms encrypted Split-NVE control-plane protocol, then existing mechanisms
such as MACsec [IEEE 802.1AE] can be used. In some deployments, such as MACsec [IEEE802.1AE] can be used. In some deployments,
authentication may be implicit based on control of physical authentication may be implicit, based on control of physical
connectivity, e.g., if the nNVE is located in the bridge that is connectivity, e.g., if the nNVE is located in the bridge that is
directly connected to the server that contains the tNVE. Use of directly connected to the server that contains the tNVE. The use of
"nearest customer bridge address" in VDP [IEEE 802.1Q] is an example the "nearest customer bridge address" in VDP [IEEE802.1Q] is an
where this sort of implicit authentication is possible, although example of where this sort of implicit authentication is possible,
explicit authentication also applies in that case. although explicit authentication also applies in that case.
As the control plane protocol results in configuration changes for As the control-plane protocol results in configuration changes for
both the tNVE and nNVE, tNVE and nNVE implementations should log all both the tNVE and the nNVE, tNVE and nNVE implementations should log
state changes, including those described in Section 3. all state changes, including those described in Section 3.
Implementations should also log significant protocol events, such as Implementations should also log significant protocol events, such as
establishment or loss of control plane protocol connectivity between the establishment or loss of control-plane protocol connectivity
the tNVE and nNVE and authentication results. between the tNVE and the nNVE, as well as authentication results.
In addition, external NVEs will need appropriate mechanisms to ensure In addition, External NVEs will need appropriate mechanisms to ensure
that any hypervisor wishing to use the services of an NVE is properly that any hypervisor wishing to use the services of an NVE is properly
authorized to do so. One design point is whether the hypervisor authorized to do so. One design point is whether the hypervisor
should supply the external NVE with necessary information (e.g., VM should
addresses, VN information, or other parameters) that the external NVE
uses directly, or whether the hypervisor should only supply a VN ID
and an identifier for the associated VM (e.g., its MAC address), with
the external NVE using that information to obtain the information
needed to validate the hypervisor-provided parameters or obtain
related parameters in a secure manner. The former approach can be
used in a trusted environment so that the external NVE can directly
use all the information retrieved from the hypervisor for local
configuration. It saves the effort on the external NVE side from
information retrieval and/or validation. The latter approach gives
more reliable information as the external NVE needs to retrieve them
from some management system database. Especially some network related
parameters like VLAN IDs can be passed back to hypervisor to be used
as a more authoritative provisioning. However in certain cases, it is
difficult or inefficient for an external NVE to have access or query
on some information to those management systems. Then the external
NVE has to obtain those information from hypervisor.
7. IANA Considerations 1. supply the External NVE with necessary information (e.g., VM
addresses, VN information, or other parameters) that the
External NVE uses directly or
No IANA action is required. 2. only supply a VN ID and an identifier for the associated VM
(e.g., its MAC address), with the External NVE using that
information to obtain the information needed to validate the
hypervisor-provided parameters or obtain related parameters in a
secure manner.
8. Acknowledgements The former approach can be used in a trusted environment so that the
External NVE can directly use all the information retrieved from the
hypervisor for local configuration. It relieves the External NVE
side of effort related to information retrieval and/or validation.
The latter approach gives more reliable information, as the
External NVE needs to retrieve it from a management-system database.
In particular, some network-related parameters, such as VLAN IDs, can
be passed back to the hypervisor to be used as a form of provisioning
that is more authoritative. However, in certain cases it is
difficult or inefficient for an External NVE to be granted rights to
access or query information on those management systems. The
External NVE then has to obtain the information from the hypervisor.
This document was initiated based on the merger of the drafts draft- 7. IANA Considerations
kreeger-nvo3-hypervisor-nve-cp, draft-gu-nvo3-tes-nve-mechanism, and
draft-kompella-nvo3-server2nve. Thanks to all the co-authors and
contributing members of those drafts.
The authors would like to specially thank Lucy Yong and Jon Hudson This document has no IANA actions.
for their generous help in improving this document.
8. References 8. References
8.1 Normative References 8.1. Normative References
[IEEE802.1Q]
IEEE, "IEEE Standard for Local and metropolitan area
networks--Bridges and Bridged Networks", IEEE Standard
802.1Q-2014, DOI 10.1109/IEEESTD.2014.6991462.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. [RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y.
Rekhter, "Framework for DC Network Virtualization", Rekhter, "Framework for Data Center (DC) Network
October 2014. Virtualization", RFC 7365, DOI 10.17487/RFC7365,
October 2014, <https://www.rfc-editor.org/info/rfc7365>.
[RFC7666] Asai H., MacFaden M., Schoenwaelder J., Shima K., Tsou T., [RFC7666] Asai, H., MacFaden, M., Schoenwaelder, J., Shima, K., and
"Management Information Base for Virtual Machines T. Tsou, "Management Information Base for Virtual Machines
Controlled by a Hypervisor", October 2015. Controlled by a Hypervisor", RFC 7666,
DOI 10.17487/RFC7666, October 2015,
<https://www.rfc-editor.org/info/rfc7666>.
[RFC8014] Black, D., Hudson, J., Kreeger, L., Lasserre, M., Narten, [RFC8014] Black, D., Hudson, J., Kreeger, L., Lasserre, M., and T.
T., "An Architecture for Data-Center Network Narten, "An Architecture for Data-Center Network
Virtualization over Layer 3 (NVO3)", December 2016. Virtualization over Layer 3 (NVO3)", RFC 8014,
DOI 10.17487/RFC8014, December 2016,
<https://www.rfc-editor.org/info/rfc8014>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in
Key Words ", BCP 14, RFC 8174, May 2017. RFC 2119 Key Words", BCP 14, RFC 8174,
DOI 10.17487/RFC8174, May 2017,
<https://www.rfc-editor.org/info/rfc8174>.
[IEEE 802.1Q] IEEE, "Media Access Control (MAC) Bridges and Virtual 8.2. Informative References
Bridged Local Area Networks", IEEE Std 802.1Q-2014,
November 2014.
8.2 Informative References [IEEE802.1AE]
IEEE, "IEEE Standard for Local and Metropolitan Area
Networks: Media Access Control (MAC) Security",
IEEE Standard 802.1AE-2006,
DOI 10.1109/IEEESTD.2006.245590.
[RFC2236] Fenner, W., "Internet Group Management Protocol, Version [NVO3-HYPERVISOR-NVE-CP]
2", RFC 2236, November 1997. Kreeger, L., Narten, T., and D. Black, "Network
Virtualization Hypervisor-to-NVE Overlay Control Protocol
Requirements", Work in Progress, draft-kreeger-nvo3-
hypervisor-nve-cp-01, February 2013.
[RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally [NVO3-TES-NVE]
Unique IDentifier (UUID) URN Namespace", RFC 4122, July Yingjie, G. and L. Yizhou, "The mechanism and signalling
2005. between TES and NVE", Work in Progress, draft-gu-nvo3-tes-
nve-mechanism-01, October 2012.
[RFC7364] Narten, T., Gray, E., Black, D., Fang, L., Kreeger, L., and [NVO3-VM-NVE]
M. Napierala, "Problem Statement: Overlays for Network Kompella, K., Rekhter, Y., Morin, T., and D. Black,
Virtualization", October 2014. "Signaling Virtual Machine Activity to the Network
Virtualization Edge", Work in Progress, draft-kompella-
nvo3-server2nve-02, April 2013.
[IEEE 802.1AE] IEEE, "MAC Security (MACsec)", IEEE Std 802.1AE-2006, [RFC2236] Fenner, W., "Internet Group Management Protocol,
August 2006. Version 2", RFC 2236, DOI 10.17487/RFC2236, November 1997,
<https://www.rfc-editor.org/info/rfc2236>.
Appendix A. IEEE 802.1Q VDP Illustration (For information only) [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally
Unique IDentifier (UUID) URN Namespace", RFC 4122,
DOI 10.17487/RFC4122, July 2005,
<https://www.rfc-editor.org/info/rfc4122>.
The VDP (VSI Discovery and Discovery and Configuration Protocol, [RFC7364] Narten, T., Ed., Gray, E., Ed., Black, D., Fang, L.,
clause 41 of [IEEE 802.1Q]) can be considered as a controlling Kreeger, L., and M. Napierala, "Problem Statement:
protocol running between the hypervisor and the external bridge. VDP Overlays for Network Virtualization", RFC 7364,
association TLV structure are formatted as shown in Figure A.1. DOI 10.17487/RFC7364, October 2014,
<https://www.rfc-editor.org/info/rfc7364>.
Appendix A. VDP Illustrations (per IEEE 802.1Q) (for Information Only)
VDP (the VSI Discovery and Discovery and Configuration Protocol; see
Clause 41 of [IEEE802.1Q]) can be considered as a controlling
protocol running between the hypervisor and the external bridge. The
VDP association TLV structure is formatted as shown in Figure 7.
+--------+--------+------+-----+--------+------+------+------+------+ +--------+--------+------+-----+--------+------+------+------+------+
|TLV type|TLV info|Status|VSI |VSI Type|VSI ID|VSI ID|Filter|Filter| |TLV Type|TLV Info|Status|VSI |VSI Type|VSI ID|VSI ID|Filter|Filter|
| |string | |Type |Version |Format| |Info |Info | | |String | |Type |Version |Format| |Info |Info |
| |length | |ID | | | |format| | | |Length | |ID | | | |Format| |
+--------+--------+------+-----+--------+------+------+------+------+ +--------+--------+------+-----+--------+------+------+------+------+
| | |<----VSI type&instance----->|<--Filter--->| | | |<--VSI Type and instance--->|<--Filter--->|
| | |<-------------VSI attributes------------->| | | |<-------------VSI attributes------------->|
|<--TLV header--->|<-----------TLV information string ------------->| |<--TLV header--->|<-----------TLV information string ------------->|
Figure A.1: VDP association TLV Figure 7: VDP Association TLV
There are basically four TLV types. There are basically four TLV types.
1. Pre-associate: Pre-associate is used to pre-associate a VSI 1. Pre-Associate: The Pre-Associate is used to Pre-Associate a VSI
instance with a bridge port. The bridge validates the request and instance with a bridge port. The bridge validates the request
returns a failure Status in case of errors. Successful pre-associate and returns a failure status in the case of errors. A successful
does not imply that the indicated VSI Type or provisioning will be Pre-Associate does not imply that the indicated VSI Type or
applied to any traffic flowing through the VSI. The pre-associate provisioning will be applied to any traffic flowing through the
enables faster response to an associate, by allowing the bridge to VSI. By allowing the bridge to obtain the VSI Type prior to an
obtain the VSI Type prior to an association. association, the Pre-Associate enables faster response to an
Associate.
2. Pre-associate with resource reservation: Pre-associate with 2. Pre-Associate with Resource Reservation: The Pre-Associate with
Resource Reservation involves the same steps as Pre-associate, but on Resource Reservation involves the same steps as those for the
success it also reserves resources in the bridge to prepare for a Pre-Associate, but on success it also reserves resources in the
subsequent Associate request. bridge to prepare for a subsequent Associate request.
3. Associate: Associate creates and activates an association between 3. Associate: The Associate request creates and activates an
a VSI instance and a bridge port. An bridge allocates any required association between a VSI instance and a bridge port. A bridge
bridge resources for the referenced VSI. The bridge activates the allocates any required bridge resources for the referenced VSI.
configuration for the VSI Type ID. This association is then applied The bridge activates the configuration for the VSI Type ID. This
to the traffic flow to/from the VSI instance. association is then applied to the traffic flow to/from the VSI
instance.
4. De-associate: The de-associate is used to remove an association 4. De-Associate: The De-Associate is used to remove an association
between a VSI instance and a bridge port. Pre-associated and between a VSI instance and a bridge port. Pre-associated and
associated VSIs can be de-associated. De-associate releases any associated VSIs can be de-associated. The De-Associate releases
resources that were reserved as a result of prior associate or pre- any resources that were reserved as a result of prior Associate
Associate operations for that VSI instance. or Pre-Associate operations for that VSI instance.
De-associate can be initiated by either side and the other types can The De-Associate can be initiated by either side, and the other types
only be initiated by the server side. can only be initiated by the server side.
Some important flag values in VDP Status field: Some important flag values in the VDP Status field are as follows:
1. M-bit (Bit 5): Indicates that the user of the VSI (e.g., the VM) 1. M-bit (Bit 5): M-bit = 1: indicates that the user of the VSI
is migrating (M-bit = 1) or provides no guidance on the migration of (e.g., the VM) is migrating. M-bit = 0: no indication of whether
the user of the VSI (M-bit = 0). The M-bit is used as an indicator the VSI user is migrating. The M-bit is used as an indicator
relative to the VSI that the user is migrating to. relative to the VSI to which the user is migrating.
2. S-bit (Bit 6): Indicates that the VSI user (e.g., the VM) is 2. S-bit (Bit 6): S-bit = 1: indicates that the VSI user (e.g., the
suspended (S-bit = 1) or provides no guidance as to whether the user VM) is suspended. S-bit = 0: no indication of whether the VSI
of the VSI is suspended (S-bit = 0). A keep-alive Associate request user is suspended. A keep-alive Associate request with S-bit = 1
with S-bit = 1 can be sent when the VSI user is suspended. The S-bit can be sent when the VSI user is suspended. The S-bit is used as
is used as an indicator relative to the VSI that the user is an indicator relative to the VSI from which the user is
migrating from. migrating.
The filter information format currently defines 4 types. Each of the The filter information format currently defines four types.
filter information is shown in details as follows. Information for each of these types is shown in detail in Figures 8
through 11. "PCP" stands for Priority Code Point [IEEE802.1Q]. The
PCP value, if specified, is used by the EVB station as the default
PCP value associated with the VSI and VID. The filter information
contains a PCP Significant (PS) bit associated with each PCP field,
indicating whether the PCP field carries a PCP value (binary 1) or
does not carry a PCP value (binary 0).
1. VID Filter Info format +----------+-------+--------+--0------+
+---------+------+-------+--------+ | # of | PS | PCP | VID |
| #of | PS | PCP | VID | |entries |(1 bit)|(3 bits)|(12 bits)|
|entries |(1bit)|(3bits)|(12bits)| |(2 octets)| | | |
|(2octets)| | | | +----------+-------+--------+---------+
+---------+------+-------+--------+ |<---Repeated per entry--->|
|<--Repeated per entry->|
Figure A.2 VID Filter Info format Figure 8: VID Filter Information Format
2. MAC/VID Filter Info format +----------+--------------+-------+--------+---------+
+---------+--------------+------+-------+--------+ | # of | MAC address | PS | PCP | VID |
| #of | MAC address | PS | PCP | VID | |entries | (6 octets) |(1 bit)|(3 bits)|(12 bits)|
|entries | (6 octets) |(1bit)|(3bits)|(12bits)| |(2 octets)| | | | |
|(2octets)| | | | | +----------+--------------+-------+--------+---------+
+---------+--------------+------+-------+--------+ |<----------Repeated per entry----------->|
|<--------Repeated per entry---------->|
Figure A.3 MAC/VID filter format Figure 9: MAC/VID Filter Information Format
3. GroupID/VID Filter Info format +----------+--------------+-------+--------+---------+
+---------+--------------+------+-------+--------+ | # of | GroupID | PS | PCP | VID |
| #of | GroupID | PS | PCP | VID | |entries | (4 octets) |(1 bit)|(3 bits)|(12 bits)|
|entries | (4 octets) |(1bit)|(3bits)|(12bits)| |(2 octets)| | | | |
|(2octets)| | | | | +----------+--------------+-------+--------+---------+
+---------+--------------+------+-------+--------+ |<----------Repeated per entry----------->|
|<--------Repeated per entry---------->|
Figure A.4 GroupID/VID filter format Figure 10: GroupID/VID Filter Information Format
4. GroupID/MAC/VID Filter Info format +----------+----------+-------------+-------+--------+---------+
+---------+----------+-------------+------+-----+--------+ | # of | GroupID | MAC address | PS | PCP | VID |
| #of | GroupID | MAC address | PS | PCP | VID | |entries |(4 octets)| (6 octets) |(1 bit)|(3 bits)|(12 bits)|
|entries |(4 octets)| (6 octets) |(1bit)|(3b )|(12bits)| |(2 octets)| | | | | |
|(2octets)| | | | | | +----------+----------+-------------+-------+--------+---------+
+---------+----------+-------------+------+-----+--------+ |<---------------Repeated per entry---------------->|
|<-------------Repeated per entry------------->|
Figure A.5 GroupID/MAC/VID filter format Figure 11: GroupID/MAC/VID Filter Information Format
The null VID can be used in the VDP Request sent from the station to The null VID can be used in the VDP Request sent from the station to
the external bridge. Use of the null VID indicates that the set of the external bridge. The null VID indicates that the set of VID
VID values associated with the VSI is expected to be supplied by the values associated with the VSI is expected to be supplied by the
bridge. The set of VID values is returned to the station via the VDP bridge. The set of VID values is returned to the station via the VDP
Response. The returned VID value can be a locally significant value. Response. The returned VID values can be locally significant values.
When GroupID is used, it is equivalent to the VN ID in NVO3. GroupID When GroupID is used, it is equivalent to the VN ID in NVO3. GroupID
will be provided by the station to the bridge. The bridge maps will be provided by the station to the bridge. The bridge maps
GroupID to a locally significant VLAN ID. GroupID to a locally significant VLAN ID.
The VSI ID in VDP association TLV that identify a VM can be one of The VSI ID in the VDP association TLV that identifies a VM can be in
the following format: IPV4 address, IPV6 address, MAC address, UUID one of the following formats: IPv4 address, IPv6 address, MAC
[RFC4122], or locally defined. address, Universally Unique Identifier (UUID) [RFC4122], or locally
defined.
Acknowledgements
This document was initiated based on the merger of the following
documents: [NVO3-HYPERVISOR-NVE-CP], [NVO3-TES-NVE], and
[NVO3-VM-NVE]. Thanks to all the coauthors and contributing members
of those documents.
The authors would like to specially thank Lucy Yong and Jon Hudson
for their generous help in improving this document.
Authors' Addresses Authors' Addresses
Yizhou Li Yizhou Li
Huawei Technologies Huawei Technologies
101 Software Avenue, 101 Software Avenue
Nanjing 210012 Nanjing 210012
China China
Phone: +86-25-56625409 Phone: +86-25-56625409
EMail: liyizhou@huawei.com Email: liyizhou@huawei.com
Donald Eastlake Donald Eastlake 3rd
Huawei R&D USA Huawei R&D USA
155 Beaver Street 155 Beaver Street
Milford, MA 01757 USA Milford, MA 01757
United States of America
Phone: +1-508-333-2270 Phone: +1-508-333-2270
EMail: d3e3e3@gmail.com Email: d3e3e3@gmail.com
Lawrence Kreeger Lawrence Kreeger
Arrcus, Inc Arrcus, Inc.
Email: lkreeger@gmail.com Email: lkreeger@gmail.com
Thomas Narten Thomas Narten
IBM IBM
Email: narten@us.ibm.com Email: narten@us.ibm.com
David Black David Black
Dell EMC Dell EMC
176 South Street, 176 South Street
Hopkinton, MA 01748 USA Hopkinton, MA 01748
United States of America
Email: david.black@dell.com Email: david.black@dell.com
 End of changes. 193 change blocks. 
748 lines changed or deleted 828 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/