Network Working Group E.
Crabbe, Ed.Crabbe Internet-Draft Intended status: Standard Track L. Yong, Ed.Yong Huawei USA X. Xu, Ed.Xu Huawei Technologies T. Herbert Google Expires: AprilAugust 2015 October 27, 2014 Generic UDPFebruary 11, 2015 GRE-in-UDP Encapsulation for IP Tunneling draft-ietf-tsvwg-gre-in-udp-encap-03draft-ietf-tsvwg-gre-in-udp-encap-04 Abstract This document describes a method of encapsulating arbitrary protocolsnetwork protocol packets within GRE and UDP headers. In this encapsulation, the source UDP port maycan be used as an entropy field for purposes of load balancingbalancing, while the payloadprotocol may beof the encapsulated packet in the GRE payload is identified by the GRE Protocol Type. Usage restrictions apply to GRE-in-UDP usage for traffic that is not congestion controlled and to UDP zero checksum usage with IPv6. Status of This Document This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 27,August 11, 2015. Copyright Notice Copyright (c) 20142015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction...................................................3 1.1. Applicability Statement...................................3 2. Terminology....................................................4 2.1. Requirements Language.....................................4 3. Procedures.....................................................4Encapsulation in UDP...........................................4 3.1. IP header.................................................7 3.2. UDP checksum usageheader................................................7 3.2.1. Source Port..........................................7 3.2.2. Destination port.....................................7 3.2.3. Checksum.............................................7 3.2.4. Length...............................................8 3.3. GRE header................................................8 4. UDP Checksum Handling..........................................8 4.1. UDP Checksum with IPv6..............................5 3.2.IPv4....................................8 4.2. UDP Checksum with IPv6....................................9 4.2.1. Middlebox Considerations for IPv6 UDP Zero Checksums......7 3.3. GRE-in-UDP Encapsulation Format...........................8 4. Encapsulation Considerations..................................10Checksums12 5. Congestion Considerations.....................................11Encapsulation Process Procedures..............................12 5.1. Packet Fragmentation.....................................13 5.2. Differentiated services..................................13 6. Backward Compatibility........................................13Congestion Considerations.....................................14 7. IANA Considerations...........................................13Backward Compatibility........................................15 8. Security Considerations.......................................13 8.1. Vulnerability............................................13IANA Considerations...........................................16 9. Acknowledgements..............................................14Security Considerations.......................................16 9.1. Vulnerability............................................16 10. Contributors.................................................14Acknowledgements.............................................17 11. References...................................................16 11.1.Contributors.................................................17 12. References...................................................19 12.1. Normative References....................................16 11.2.References....................................19 12.2. Informative References..................................16 12.References..................................19 13. Authors' Addresses...........................................17Addresses...........................................20 1. Introduction Load balancing, or more specifically,specifically statistical multiplexing of traffic using Equal Cost Multi-Path (ECMP) and/or Link Aggregation Groups (LAGs) in IP networks is a widely used technique for creating higher capacity networks out of lower capacity links. Most existing routers in IP networks are already capable of distributing IP traffic flows over ECMP paths and/or LAGs on the basis of a hash function performed on flow invariant fields in IP packet headers and their payload protocol headers. Specifically, when the IP payload is a User Datagram Protocol (UDP)[RFC0768](UDP)[RFC768] or Transmission Control Protocol (TCP) [RFC793] packet, router hash functions frequently operate on the five-tuple of thesource IP address, thedestination IP address, thesource port, thedestination port, and theprotocol/next-header Several tunnelingencapsulation techniques are in common usecommonly used in IP networks, such as Generic Routing Encapsulation (GRE) [RFC2784], MPLS [RFC4023] and L2TPv3 [RFC3931]. GRE is an increasingly popular encapsulation choice, especially in environments where MPLS is unavailable or unnecessary.choice. Unfortunately, use of common GRE endpoints may reduce the entropy available for use in load balancing, especially in environments where the GRE Key field [RFC2890] is not readily available for use as entropy in forwarding decisions. This document defines a generic GRE-in-UDP encapsulation for tunneling arbitrarynetwork protocol payloadspackets across an IP network environment where ECMP or LAGs are used.network. The GRE header provides payload protocol de-multiplexing by way of it'stype as an EtherType in the protocol type field [RFC2784] while[RFC2784][GREIPV6], and the UDP header provides additional entropy by way of it'sits source port. This encapsulation method requires no changes to the transit IP network. Hash functions in most existing IP routers may utilize and benefit from the use of a GRE-in-UDP tunnel iswithout needing any change or upgrade to their ECMP implementation. The encapsulation mechanism is applicable to a variety of IP networks including Data Center and wide area networks. 1.1. Applicability Statement ItGRE encapsulation is recommendedwidely used for many applications. For example, to use GRE-in-UDP encapsulation withinredirect IP traffic to traverse a Service Provider (SP)different path instead of the default path in an operator network, to tunnel private network and/or DCtraffic over a public network where the congestion control isby use of public IP network addresses, or to tunnel IPv6 traffic over an IPv4 network, etc. When encapsulating GRE in UDP, encapsulated traffic will be treated as a UDP application, not as a concern. However the encapsulation can apply to ISP networks and/or Internet. Some environments requestGRE application, in an IP network. Thus GRE-in-UDP applications must meet UDP tunnel to run more functions than others.requirements as specified in [RFC5405]. This may constrain GRE-in-UDP tunnel usage in certain applications and/or environments. See Section 6. GRE-in-UDP encapsulation may be used to tunnel theencapsulate already tunneled traffic, i.e. tunnel-in-tunnel. The tunneled traffic may use GRE-in-UDPGRE-in- UDP or other tunnel encapsulation. In this case, GRE-in-UDP tunnel end points treat other tunnel endpoints as of the end hosts for the traffic and do not differentiate such end hosts from other end hosts. The use case and applicability for a GRE-in-UDP tunnel egress and stacked tunnel egress terminate on the same IP address is for further study. 2. Terminology2. Terminology The terms defined in [RFC768][RFC768][RFC2784] are used in this document. 2.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 3. Procedures When a tunnel ingress device conforming to this document receives a packet, the ingress MUST encapsulate the packetEncapsulation in UDP and GRE headers and set the destination portGRE-in-UDP encapsulation format is shown as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 IPv4 Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of the UDP headerService| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to [TBD] Section 6. The ingress device must also insert the payload protocol type in theLive |Protcol=17(UDP)| Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = XXXX | Dest Port = TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ GRE Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| |K|S| Reserved0 | Ver | Protocol Type field. The ingress device SHOULD set the UDP source port based on flow invariant fields from the payload header. In the case that ingress is unable to get the flow entropy from the payload header, it should set a randomly selected constant value for UDP source port to avoid payload packet flow reordering. The value, for example, may be simply a result of boot-up time. How a tunnel ingress generates entropy from the payload is outside the scope of this document. The tunnel ingress MUST encode its own IP address as the source IP address and| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (optional) | Reserved1 (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Key (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1 UDP+GRE Headers in IPv4 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 IPv6 Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| Traffic Class | Flow Label | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Length | NxtHdr=17(UDP)| Hop Limit | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Outer Source IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Outer Destination IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = XXXX | Dest Port = TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ GRE Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| |K|S| Reserved0 | Ver | Protocol Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (optional) | Reserved1 (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Key (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2 UDP+GRE Headers in IPv6 The contents of the egress tunnel endpointIP, UDP, and GRE headers that are relevant in this encapsulation are described below. 3.1. IP header An encapsulator MUST encode its own IP address as the source IP address and the decapsulator's IP address as the destination IP address. The TTL field in the IP header must be set to a value appropriate for delivery of the encapsulated packet to the tunnel egress endpoint. Whenpeer of the tunnel egress receivesencapsulation. 3.2. UDP header 3.2.1. Source Port The UDP source port contains a packet, it must remove16-bit entropy value that is generated by the outer UDP and GRE headers. Section 5 describesencapsulator to identify a flow for the error handling when this entity is not instantiated atencapsulated packet. The port value SHOULD be within the tunnel egress. For IPv4 UDP encapsulation,ephemeral port range. IANA suggests this field is RECOMMENDEDrange to be 49152 to 65535, where the high order two bits of the port are set to zero becauseone. This provides fourteen bits of entropy for the IPv4 header includesinner flow identifier. In the case that an encapsulator is unable to derive flow entropy from the payload header, it should set a checksum, and userandomly selected constant value for UDP source port to avoid payload packet flow reordering. The source port value for a flow set by an encapsulator MAY change over the lifetime of the UDP checksum is optional with IPv4, unless checksum protectionencapsulated flow. For instance, an encapsulator may change the assignment for Denial of tunneledService (DOS) mitigation or as a means to effect routing through the ECMP network. An encapsulator SHOULD NOT change the source port selected for a flow more than once every thirty seconds. How an encapsulator generates entropy from the payload is important, see Section 6. For IPv6 UDP encapsulation,outside the IPv6 header does not include a checksum, soscope of this field MUST contain a UDP checksum that MUST be used as specified in [RFC0768] and [RFC2460] unless onedocument. 3.2.2. Destination port The destination port of the exceptions that allows use ofUDP zero-checksum mode (as specified in [RFC6935]) applies. Seeheader is set the GRE/UDP port (TBD) (see Section 3.1 for specification of these exceptions and additional requirements that apply when8). 3.2.3. Checksum The UDP zero-checksum modeis used for GRE-in-UDP traffic over IPv6.The tunnel ingress mayset the GRE Key Present, Sequence Number Present,and Checksum Present bitsprocessed per [RFC768] and associated fields in the GRE header defined by [RFC2784][RFC1122] for IPv4, and [RFC2890]. 3.1. UDP[RFC2460] for IPv6. Requirements for checksum usage with IPv6 Whenhandling and use of zero UDP is used over IPv6,checksums are detailed in section 4. 3.2.4. Length The usage of this field is in accordance with the current UDP specification in [RFC768]. This length will include the UDP header (eight bytes), GRE header, and the GRE payload (encapsulated packet). 3.3. GRE header An encapsulator sets the protocol type (EtherType) of the packet being encapsulated in the GRE Protocol Type field. An encapsulator may set the GRE Key Present, Sequence Number Present, and Checksum Present bits and associated fields in the GRE header as defined by [RFC2784] and [RFC2890]. The GRE checksum is relied uponMAY be enabled to protect the IPv6GRE header from corruption,and MUST be used unlesspayload. An encapsulator SHOULD NOT enable both the requirements in [RFC 6935]GRE checksum and [RFC 6936] for useUDP checksum simultaneously as this would be mostly redundant. Since the UDP checksum covers more of the packet including the GRE header and payload, the UDP zero-checksum modeSHOULD have preference to using GRE checksum. 4. UDP Checksum Handling 4.1. UDP Checksum with a tunnel protocol are satisfied. Therefore,IPv4 For UDP in IPv4, the UDP checksum MUST be implemented and MUST be usedprocessed as specified in accordance with [RFC0768][RFC768] and [RFC2460][RFC1122] for GRE inboth transmit and receive. An encapsulator MAY set the UDP traffic over IPv6 unless onechecksum to zero for performance or implementation considerations. The IPv4 header includes a checksum which protects against mis-delivery of the following exceptions appliespacket due to corruption of IP addresses. The UDP checksum potentially provides protection against corruption of the UDP header, GRE header, and GRE payload. Enabling or disabling the additional requirements stated below are complied with. In addition,use of checksums is a deployment consideration that should take into account the risk and effects of packet corruption, and whether the packets in the network are already adequately protected by other, possibly stronger mechanisms such as the Ethernet CRC. When a decapsulator receives a packet, the UDP checksum with IPv6field MUST be processed. If the default configuration of all GRE-in-UDP implementations. There are two exceptions that allow use ofUDP zero-checksum mode for IPv6 with GRE-in-UDP, subject tochecksum is non-zero, the additional requirements stated below in this section. The two exceptions are: o Use of GRE-in-UDP withindecapsulator MUST verify the checksum before accepting the packet. By default a single service providerdecapsularor SHOULD accept UDP packets with a zero checksum. A node MAY be configured to disallow zero checksums per [RFC1122]; this may be done selectively, for instance disallowing zero checksums from certain hosts that utilizes careful provisioning (e.g., rate limiting at the entriesare known to be sending over paths subject to packet corruption. If verification of a non-zero checksum fails, a decapsulator lacks the network while over-provisioning network capacity)capability to ensure against congestion and that actively monitors encapsulated traffic for errors;verify a non-zero checksum, or o Use of GRE-in-UDP withina limited number of service providers who closely cooperate in orderpacket with a zero-checksum was received and the decapsulator is configured to jointly provide this same careful provisioningdisallow, the packet MUST be dropped and monitoring. As such, foran event MAY be logged. 4.2. UDP Checksum with IPv6 For UDP in IPv6, the UDP checksum for GRE-in-UDPMUST be usedprocessed as specified in [RFC0768][RFC768] and [RFC2460] over the general Internet, and over non-cooperating ISPs, even if each non-cooperating ISP independently satisfies the first exceptionfor both transmit and receive. When UDP zero-checksum mode usage with GRE-in-UDPis used over IPv6 within the ISP's own network. Section 5 of RFC6936 [RFC6936] specifiesIPv6, the additional requirements that implementation ofUDP zero-checksum overchecksum is relied upon to protect both the IPv6 and UDP headers from corruption, and so MUST compliant with. To compliantused with it,the following additional requirements apply to GRE-in-UDP implementation and use of UDP zero-checksum mode over IPv6:exceptions: a. AUse of GRE-in-UDP implementation MUST comply with all requirements specifiedin Section 4networks under single administrative control (such as within a single operator's network) where it is known (perhaps through knowledge of [RFC6936]equipment types and with requirement 1 specified in Section 5 of [RFC6936]. b. A GRE-in-UDP receiver MUST checklower layer checks) that the sourcepacket corruption is exceptionally unlikely and destination IPv6 addresses are valid forwhere the GRE-in-UDP tunnel and discard anyoperator is willing to take the risk of undetected packet for which this check fails. c. Acorruption. b. Use of GRE-in-UDP sender SHOULDin networks under single administrative control (such as within a single operator's network) where it is judged through observational measurements (perhaps of historic or current traffic flows that use different IPv6 addresses for each GRE-in-UDP tunnela non-zero checksum) that uses UDP zero-checksum mode in order to strengthenthe receiver's checklevel of the IPv6 source address. When thispacket corruption is not possible, ittolerably low and where the operator is RECOMMENDEDwilling to use each source IPv6 address for as few UDP zero-checksum mode MPLS-in-UDP tunnels as is feasible. d.take the risk of undetected packet corruption. c. Use of GRE-in-UDP senderfor traffic delivery for applications that are tolerant of misdelivered or corrupted packets (perhaps through higher layer checksum, validation, and receiver MUST agree the key(s) used overretransmission or transmission redundancy) where the tunnel. The sender MUST insert a keyoperator is willing to rely on GRE header, and the receiver MUST check ifthe key in GRE header is valid forapplications using the tunnel and drop invalid packet. e. A GRE-in-UDP receiver node SHOULD only enableto survive any corrupt packets. For these exceptions, the use ofUDP zero-checksum mode on a single UDP port and SHOULD NOT support any othercan be used. However the use of the UDP zero-checksum mode on any other UDP port. f. A GRE-in-UDP sender SHOULD send GRE keepalive messages with a zero UDP checksum. GRE-in-UDP receiver that discovers an appreciable loss rate for keepalive packets MAY terminatemust meet the tunnel. g. GRE keepalive messages SHOULD include both UDP datagrams with a checksumrequirements specified in [RFC6935] and datagrams with a zero UDP checksum. This will enable[RFC6936] as well at the remote endpointadditional requirements stated below. These exceptions may also be extended to distinguish between a path failure andthe droppinguse of datagrams withGRE-in-UDP within a zero UDP checksum. h. Any middlebox support for MPLS-in-UDP with UDP zero-checksum mode for IPv6 MUST comply with requirements 1 and 8-10 in Section 5set of RFC 6936. (Editor note: the design team and authors need further discuss above requirements text) The above requirements are intendedclosely cooperating network administrations (such as network operators who have agreed to bework together in additionorder to jointly provide specific services). As such, for IPv6, the requirementsUDP checksum for GRE-in-UDP MUST be used as specified in [RFC2460] as modified by [RFC6935][RFC768] and [RFC2460] for tunnels that span multiple networks whose network administrations do not cooperate closely, even if each non-cooperating network administration independently satisfies one or more of the requirements specified in [RFC6936].exceptions for UDP zero-checksum mode usage with GRE-in-UDP over IPv6 does not include anIPv6. The following additional integrity check because the aboverequirements in combination with the exceptions that restrictapply to implementation and use of UDP zero-checksum mode to well-managed networks should not significantly increase the rate of corruption of UDP/GRE- encapsulated traffic by comparison to GRE-encapsulated trafficfor GRE-in-UDP over similar well-managed networks and because GRE does not accumulate incorrect state as a consequenceIPv6: a. Use of GRE header corruption. Editor Note:the UDP checksum with IPv6 MUST be the default configuration of all GRE-in-UDP implementations. b. The preceding paragraph addressesGRE-in-UDP implementation MUST comply with all requirements 2-4specified in Section 54 of [RFC 6936]. Requirement 5 in that section is addressed by the requirement e in this section. Requirements 6 and 7 in that section are covered by the requirements f[RFC6936] and g in this section. Requirement 8-10 in that section is addressed by thewith requirement h1 specified in this section. In summary,Section 5 of [RFC6936]. c. A decapsulator SHOULD only allow the use of UDP zero-checksum mode for IPv6 on a single received UDP Destination Port. The motivation for this requirement is allowed to be used with GRE-in-UDP when onepossible corruption of the two exceptions specified above applies, providedUDP destination port, which may cause packet delivery to the wrong UDP port. If that additional requirements stated above are complied with. Otherwiseother UDP port requires the UDP checksumchecksum, the mis-delivered packet will be discarded d. By default a decapsulator MUST disallow receipt of GRE-in-UDP packets with zero UDP checksums with IPv6. Zero checksums May selectively be usedenabled for IPv6 as specified in [RFC0768]certain source address. A decapsulator MUST check that the source and [RFC2460]. This entire sectiondestination IPv6 addresses are valid for the GRE-in-UDP tunnel on which the packet was received if that tunnel uses UDP zero-checksum mode and its requirements apply only todiscard any packet for which this check fails. e. An encapsulator SHOULD use ofdifferent IPv6 addresses for each GRE- in-UDP tunnel that uses UDP zero-checksum mode for IPv6; they can be avoided by usingregardless of the decapsulator in order to strengthen the decapsulator's check of the IPv6 source address (i.e., the same IPv6 source address SHOULD NOT be used with more than one IPv6 destination address, independent of whether that destination address is a unicast or multicast address). When this is not possible, it is RECOMMENDED to use each source IPv6 address for as few UDP checksumzero-checksum mode GRE-in-UDP tunnels as specified in [RFC0768] and [RFC2460]. 3.2. Middlebox Considerationsis feasible. f. Any middlebox support for GRE-in-UDP with UDP zero-checksum mode for IPv6 MUST comply with requirements 1 and 8-10 in Section 5 of [RFC6936].[RFC6936]. g. Measures SHOULD be taken to prevent IPv6 traffic with zero UDP Zero Checksumschecksums from "escaping" to the general Internet; see Section 6 for examples of such measures. h. IPv6 datagramstraffic with azero UDP checksum will notchecksums MUST be passedactively monitored for errors by any middlebox that validates the checksum based on [RFC2460] or that updatesthe network operator. i. The use a zero UDP checksum field, such as NATs or firewalls. Changing this behavior would require such middleboxes to be updated to correctly handle datagrams with zeroshould present the equivalent risk of undetected packet corruption when sending similar packet using GRE-in-IPv6 without UDP and without GRE checksums. The GRE-in-UDP encapsulation does not providej. If a mechanism to safely fall back to usingpacket with a non-zero checksum when a path change occurs redirectingis received, the checksum MUST be verified before accepting the packet. This is regardless of whether a tunnel over a path that includes a middlebox that discards IPv6 datagramsencapsulator and decapsulator have been configured with a zeroUDP checksum. In this casezero-checksum mode. The above requirements do not change either the GRE-in-UDP tunnel will be black-holedrequirements specified in [RFC2460] as modified by that middlebox. Recommended changes[RFC6935] or the requirements specified in [RFC6936]. The requirement to allow firewalls, NATs and other middleboxescheck the source IPv6 address in addition to support usethe destination IPv6 address, plus the strong recommendation against reuse of ansource IPv6 zeroaddresses among GRE-in-UDP tunnels collectively provide some mitigation for the absence of UDP checksum are described in Section 5 of [RFC6936]. 3.3. GRE-in-UDP Encapsulation Format The formatcoverage of the GRE-in-UDP encapsulation for both IPv4 andIPv6 outer headersheader. Additional assurance is shownprovided by the restrictions in the following figures: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 IPv4 Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Typeabove exceptions that limit usage of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TimeIPv6 UDP zero-checksum mode to Live |Protcol=17(UDP)| Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = XXXX | Dest Port = TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+well-managed networks for which GRE Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| |K|S| Reserved0 | Ver | Protocol Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (optional) | Reserved1 (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Key (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1 UDP+GRE IPv4 headers 0 1 2 3 0 1 2encapsulated packet corruption has not been a problem in practice. Hence GRE-in-UDP is suitable for transmission over lower layers in the well-managed networks that are allowed by the exceptions stated above and the rate of corruption of the inner IP packet on such networks is not expected to increase by comparison to GRE traffic that is not encapsulated in UDP. For these reasons, GRE-in-UDP does not provide an additional integrity check except when GRE checksum is used when UDP zero-checksum mode is used with IPv6, and this design is in accordance with requirements 2, 3 4and 5 6 7 8 9 0 1 2 3 4specified in Section 5 6 7 8 9 0 1 2 3of [RFC6936]. GRE does not accumulate incorrect state as a consequence of GRE header corruption. A corrupt GRE results in either packet discard or forwarding of the packet without accumulation of GRE state. GRE checksum MAY be used for protecting GRE header and payload. Active monitoring of GRE-in-UDP traffic for errors is REQUIRED as occurrence of errors will result in some accumulation of error information outside the protocol for operational and management purposes. This design is in accordance with requirement 4 specified in Section 5 of [RFC6936]. The remaining requirements specified in Section 5 of [RFC6936] are inapplicable to GRE-in-UDP. Requirements 6 and 7 8 9 0 1do not apply because GRE does not have a GRE-generic control feedback mechanism. Requirements 8-10 are middlebox requirements that do not apply to GRE-in-UDP tunnel endpoints, but see Section 3.2 for further middlebox discussion. In summary, UDP zero-checksum mode for IPv6 Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| Traffic Class | Flow Label | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Length | NxtHdr=17(UDP)| Hop Limit | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Outer Sourceis allowed to be used with GRE-in-UDP when one of the three exceptions specified above applies, provided that additional requirements stated above are complied with. Otherwise the UDP checksum MUST be used for IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Outer Destinationas specified in [RFC768] and [RFC2460]. 4.2.1. Middlebox Considerations for IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+UDP Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = XXXX | Dest Port = TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Zero Checksums IPv6 datagrams with a zero UDP Length |checksum will not be passed by any middlebox that validates the checksum based on [RFC2460] or that updates the UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ GRE Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| |K|S| Reserved0 | Ver | Protocol Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum (optional) | Reserved1 (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Key (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2 UDP+GRE IPv6 headerschecksum field, such as NATs or firewalls. Changing this behavior would require such middleboxes to be updated to correctly handle datagrams with zero UDP checksums. The total overhead increase forGRE-in-UDP encapsulation does not provide a mechanism to safely fall back to using a checksum when a path change occurs redirecting a UDP+GREtunnel withoutover a path that includes a middlebox that discards IPv6 datagrams with a zero UDP checksum. In this case the GRE-in-UDP tunnel will be black-holed by that middlebox. Recommended changes to allow firewalls, NATs and other middleboxes to support use of optionalan IPv6 zero UDP checksum are described in Section 5 of [RFC6936]. 5. Encapsulation Process Procedures This GRE-in-UDP encapsulation allows packets to be forwarded through "GRE-UDP tunnels". When performing GRE-in-UDP encapsulation by the encapsulator, the entropy value would be generated by the encapsulator and then be filled in the Source Port field of the UDP header. The Destination Port field is set to a value (TBD) allocated by IANA to indicate that the UDP tunnel payload is a GRE packet. The Protocol Type header field in GRE fields, representing the lowest total overhead increase,header is 32 bytes inset to the caseEtherType value corresponding to the protocol of IPv4 and 52 bytes inthe caseencapsulated packet. Intermediate routers, upon receiving these UDP encapsulated packets, could balance these packets based on the hash of IPv6. The total overhead increase for a UDP+GRE tunnel with usethe five-tuple of GRE Key, Sequence and Checksum Fields, representingUDP packets. Upon receiving these UDP encapsulated packets, the highest total overhead increase,decapsulator would decapsulate them by removing the UDP headers and then process them accordingly. Note: Each UDP tunnel is 44 bytes inunidirectional, as GRE-in-UDP traffic is sent to the case of IPv4IANA-allocated UDP Destination Port, and 64 bytesin the case of IPv6. 4. Encapsulation Considerations GRE-in-UDP encapsulationparticular, is never sent back to any port used for single tunnel mechanism where both GRE andas a UDP header are required. The mechanism allows the tunneledSource Port (which serves solely as a source of entropy). This is at odds with a common middlebox (e.g., firewall) assumption that bidirectional traffic uses a common pair of UDP ports. As a result, arranging to bepass bidirectional GRE-in-UDP traffic through middleboxes may require separate configuration for each direction of traffic. GRE-in-UDP allows encapsulation of unicast, broadcast, or multicast traffic. Entropy may be generated from the header of tunneledencapsulated unicast or broadcast/multicast packets at tunnel ingress.an encapsulator. The mapping mechanism between the tunneledencapsulated multicast traffic and the multicast capability in the IP network is transparent and independent to the encapsulation and is otherwise outside the scope of this document. The tunnel ingress5.1. Packet Fragmentation Regarding Fragmentation, an encapsulator SHOULD perform thefragmentation [GREMTU] on a packet before theencapsulation and factor in both GRE and UDP header bytes in the effective Maximum Transmission Unit (MTU) size. Not performing the fragmentation will cause the packets exceeding network MTU size to be dropped or fragmented in the network. The tunnel ingressAn encapsulator MUST use the same source UDP port for all packet fragments to ensure that the transit routers will forward the packet fragments on the same path. An operator should factor in the addition overheadadditional bytes of overhead when considering an MTU size for the payload to reduce the likelihood of fragmentation. 5.2. Differentiated services To ensure thethat tunneled traffic gets the same treatment over the IP network, prior to the encapsulation process, tunnel ingressan encapsulator should process the payload to get the proper parameters to fill into the IP header such as DiffServ [RFC2983]. Tunnel end points that support ECN MUST use the method described in [RFC6040] for ECN marking propagation. This process is outside of the scope of this document. Note that the IPv6 header [RFC2460] contains a flow label field that may be used for load balancing in an IPv6 network [RFC6438]. Thus in an IPv6 network, either GRE-in-UDP or flow labels may be used for improving load balancing performance. Use of GRE-in-UDP encapsulation provides a unified hardware implementation for load balancing in an IP network independent ofsuch as DiffServ [RFC2983]. Encapsulation end points that support ECN must use the IP version(s)method described in use. However IPv6 network require performing[RFC6040] for ECN marking propagation. This process is outside of the UDP checksum, which may impact network performance and user experience. Thus, a flow label based load balancing may be a better approach in an IPv6 network. 5.scope of this document. 6. Congestion Considerations Section 3.1.3 of RFC 5405[RFC5405] discussed the congestion implications of UDP tunnels. As discussed in RFC 5405,[RFC5405], because other flows can share the path with one or more UDP tunnels, congestion control [RFC2914] needs to be considered. A major motivation for encapsulating GRE in UDPGRE-in-UDP encapsulation is to provide a generic UDP tunnel protocol totunnel a network protocol over IP network and improve the use of multipath (such as Equal Cost MultiPath,ECMP) in cases where traffic is to traverse routers which are able to hash on UDP Port and IP address. As such, in many cases this may reduce the occurrence of congestion and improve usage of available network capacity. However, it is also necessary to ensure that the network, including applications that use the network, responds appropriately in more difficult cases, such as when link or equipment failures have reduced the available capacity. The impact of congestion must be considered both in terms of the effect on the rest of the network of aover which packets are sent in UDP tunnel that is consuming excessive capacity,tunnels, and in terms of the effect on the flows using thethat are sent by UDP tunnels. The potential impact of congestion from a UDP tunnel depends upon what sort of traffic is carried over the tunnel, as well as the path of the tunnel. GRE in UDP as a generic UDP tunnel mechanism can beencapsulation is widely used to carry a wide range of network protocolprotocols and traffic. If tunneledIn many cases GRE encapsulation is used to carry IP traffic. IP traffic is alreadygenerally assumed to be congestion controlled, GRE in UDPand thus a tunnel carrying general IP traffic (as might be expected to be carried across the Internet) generally does not need additional congestion control mechanisms. As specified in RFC 5405: IP-based"IP-based traffic is generally assumed to be congestion-controlled, i.e., it is assumed that the transport protocols generating IP-based traffic at the sender already employ mechanisms that are sufficient to address congestion on the path. Consequently, a tunnel carrying.carrying IP-based traffic should already interact appropriately with other traffic sharing the path, and specific congestion control mechanisms for the tunnel are not necessary.necessary." For this reason, where GRE in UDPGRE-in-UDP tunneling is used to carry IP traffic that is known to be congestion controlled, the tunnelUDP tunnels MAY be used across any combination ofwithin a single service provider, multiple cooperating service providers,network or across the general Internet.multiple networks, with cooperating network operators. Internet IP traffic is generally assumed to be congestion-controlled. However, GRE in UDPGRE-in-UDP tunneling iscan be also used in many casesto carry traffic that is not necessarily congestion controlled. In such cases service providers and data centernetwork operators may avoid congestion by careful provisioning of their networks, by rate limiting of user data traffic, and/or by using Traffic Engineering tools to monitor the network segments and dynamically steers traffic away from the potentialpotentially congested link in time.links. For this reason, where the GRE payload traffic is not congestion controlled, GRE in UDPGRE-in-UDP tunnels MUST only be used within a single service provideroperator's network that utilizes careful provisioning (e.g., rate limiting at the entries of the network while over-provisioning network capacity) to ensure against congestion, or within a limited number of service providers whonetworks whose operators closely cooperate in order to jointly provide this same careful provisioning. As such, GRE in UDPGRE-in-UDP MUST NOT be used over the general Internet, or over non-cooperating ISPs,network operators, to carry traffic that is not congestion- controlled.congestion-controlled. Measures SHOULD be taken to prevent non-congestion-controlled GRE- over-UDPin-UDP traffic from "escaping" to the general Internet, e.g.: o physicalPhysical or logical isolation of the links carrying GRE-over-UDPGRE-in-UDP from the general Internet,Internet. o deploymentDeployment of packet filters that block the UDP ports assigned for GRE-over-UDP,GRE-in-UDP. o impositionImposition of restrictions on GRE-over-UDPGRE-in-UDP traffic by software tools used to set up GRE-over-UDPGRE-in-UDP tunnels between specific end systems (as might be used within a single data center), andcenter). o useUse of a "Managed Circuit Breaker" for the tunneled traffic as described in [I-D.-tsvwg-circuit-breaker]. [Editor: the text in this section was derived from the text for mpls-in-udp. More work necessary to make general for this] 6.[CB]. 7. Backward Compatibility It is assumed that tunnel ingress routers must be upgraded in order to support the encapsulations described in this document. No change is required at transit routers to support forwarding of the encapsulation described in this document. If a router that is intended for use as a tunnel egressdecapsulator does not support theor enable GRE-in-UDP encapsulation described in this document, it will not be listening on destination port [TBD].(TBD). In these cases, the router will conform to normal UDP processing and respond to the tunnel ingressan encapsulator with an ICMP message indicating "port unreachable" according to [RFC792]. Upon receiving this ICMP message, the tunnel ingressnode MUST NOT continue to use GRE-in-UDP encapsulation toward this tunnel egresspeer without management intervention. 7.8. IANA Considerations IANA is requested to make the following allocation: Service Name: GRE-in-UDP Transport Protocol(s): UDP Assignee: IESG <firstname.lastname@example.org> Contact: IETF Chair <email@example.com> Description: GRE-in-UDP Encapsulation Reference: [This.I-D] Port Number: TBD Service Code: N/A Known Unauthorized Uses: N/A Assignment Notes: N/A 8.9. Security Considerations 184.108.40.206. Vulnerability Neither UDP nor GRE encapsulation effects security for the payload protocol. When using GRE-in-UDP, Network Security in a network is the same asmostly equivalent to that of a network using GRE. Use of ICMP for signaling of the GRE-in-UDP encapsulation capability adds a security concern. Upon receiving an ICMP message and before taking an action on it, the ingress MUST validate the IP address originating against tunnel egress address and MUST evaluate the packet header returned in the ICMP payload to ensure the source port is the one used for this tunnel. The mechanism for performing this validation is out of the scope of this document. In an instance where the UDP srcsource port is not set based on the flow invariant fields from the payload header, a random port SHOULD be selected in order to minimize the vulnerability to off-path attacks. [RFC6056][RFC6056]. The random port may also be periodically changed to mitigate certain denial of service attacks. How the srcsource port randomization occurs is outside scope of this document. Using one standardized value in UDP destination port for an encapsulation indication may increase the vulnerability of off-path attack. To overcome this, tunnel egressan alternate port may request tunnel ingress using a differentbe agreed upon to use between an encapsulator and specific value [RFC6056] in UDP destination port for the GRE-in-UDP encapsulation indication.decapsulator [RFC6056]. How the tunnelencapsulator end points communicate the value is outside scope of this document. This document does not require that the tunnel egressdecapsulator validates the IP source address of the tunneled packets (with the exception that the IPv6 source address MUST be validated when UDP zero-checksum mode is used with IPv6), but it should be understood that failure to do so presupposes that there is effective destination-based (or a combination of source-based and destination-based) filtering at the boundaries. 9.10. Acknowledgements Authors like to thank Vivek Kumar, Ron Bonica, Joe Touch, Ruediger Geib, Gorry Fairhurst, David Black,Lar Edds, Lloyd, and many others for their review and valuable input on this draft. Thank the design team led by David Black (members: Ross Callon, Gorry Fairhurst, Xiaohu Xu, Lucy Yong) to efficiently work out the descriptions for the congestion considerations and IPv6 UDP zero checksum. 10.11. Contributors The following people all contributed significantly to this document and are listed below in alphabetical order: David Black EMC Corporation 176 South Street Hopkinton, MA 01748 USA Email: firstname.lastname@example.org Ross Callon Juniper Networks 10 Technology Park Drive Westford, MA 01886 USA Email: email@example.com David Black EMC Corporation 176 South Street Hopkinton, MA 01748 USA Email: firstname.lastname@example.orgJohn E. Drake Juniper Networks Email: email@example.com Gorry Fairhurst University of Aberdeen Email: firstname.lastname@example.org Yongbing Fan China Telecom Guangzhou, China. Phone: +86 20 38639121 Email: email@example.com Adrian Farrel Juniper Networks Email: firstname.lastname@example.org Vishwas Manral Hewlett-Packard Corp. 3000 Hanover St, Palo Alto. Email: email@example.com Carlos Pignataro Cisco Systems 7200-12 Kit Creek Road Research Triangle Park, NC 27709 USA EMail: firstname.lastname@example.org Yongbing Fan China Telecom Guangzhou, China. Phone: +86 20 38639121 Email: email@example.com 11.12. References 220.127.116.11. Normative References [RFC768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, August 1980. [RFC791] DARPA, "Internet Protocol", RFC791, September 1981[RFC1122] Braden, R., "Requirements for Internet Hosts -- Communication Layers", RFC1122, October 1989. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC2119, March 1997. [RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D., and P. Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, March 2000. [RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE", RFC2890, September 2000. [RFC2983] Black, D., "Differentiated Services and Tunnels", RFC2983, October 2000. [RFC5405] Eggert, L., "Unicast UDP Usage Guideline for Application Designers", RFC5405, November 2008. [RFC6040] Briscoe, B., "Tunneling of Explicit Congestion Notification", RFC6040, November 20102010. [RFC6438] Carpenter, B., Amante, S., "Using the IPv6 Flow Label for Equal Cost Multipath Routing and Link Aggregation in tunnels", RFC6438, November, 2011 11.2.2011. [RFC6935] Eubanks, M., Chimento, P., and M. Westerlund, "IPv6 and UDP Checksums for Tunneled Packets", RFC 6935, April 2013. [RFC6936] Fairhurst, G. and M. Westerlund, "Applicability Statement for the Use of IPv6 UDP Datagrams with Zero Checksums", RFC 6936, April 2013. 12.2. Informative References [RFC792] Postel, J., "Internet Control Message Protocol", STD 5, RFC 792, September 1981. [RFC793] DARPA, "Transmission Control Protocol", RFC793, September 1981. [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", RFC 2460, December 1998. [RFC2914] Floyd, S.,"Congestion Control Principles", RFC2914, September 2000. [RFC3931] Lau, J., Townsley, M., and I. Goyret, "Layer Two Tunneling Protocol - Version 3 (L2TPv3)", RFC 3931, March 2005. [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, "Encapsulating MPLS in IP or Generic Routing Encapsulation (GRE)", RFC 4023, March 2005. [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, February 2006. [RFC4884] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, "Extended ICMP to Support Multi-Part Messages", RFC 4884, April 2007.[RFC6056] Larsen, M. and Gont, F., "Recommendations for Transport- Protocol Port Randomization", RFC6056, January 2011 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and L. Yong, "The Use of Entropy Labels2011. [GREIPV6] Pignataro, C., el al, "IPv6 Support for Generic Routing Encapsulation (GRE)", draft-ietf-intarea-gre-ipv6-02, work in MPLS Forwarding", RFC 6790, November 2012.progress. [GREMTU] Bonica, R., "A Fragmentation Strategy for Generic Routing Encapsulation (GRE)", draft-bonica-intara-gre-mtu,draft-ietf-intarea-gre-mtu, work in progressprogress. [CB] Fairhurst, G., "Network Transport Circuit Breakers", draft-fairhurst-tsvwg-circuit-breaker-01, work in progress 12.progress. 13. Authors' Addresses Edward Crabbe (editor)Email: firstname.lastname@example.org Lucy Yong (editor)Huawei Technologies, USA Email: email@example.com Xiaohu Xu (editor)Huawei Technologies, Beijing, China Email: firstname.lastname@example.org Gorry's comments - give an example of random constant value selection for UDP source port in the case where tunnel ingress can't get flow entropy - use "MUST" instead of "SHOULD" for requesting use of UDP checksum in IPv6 network - more concise text for congestion description; use some text in [RFC5405] - State what consequence without doing fragmentation - tunnel ingress actions upon receiving an ICMP msg - tunnel-in-tunnel case - CB does not describe the protocol to support CB, only the mechanism. UDP report protocol may be good fit.Tom Herbert Google 1600 Amphitheatre Parkway Mountain View, CA Email : email@example.com