draft-ietf-tsvwg-ecn-tunnel-02.txt   draft-ietf-tsvwg-ecn-tunnel-03.txt 
Transport Area Working Group B. Briscoe Transport Area Working Group B. Briscoe
Internet-Draft BT Internet-Draft BT
Intended status: Standards Track March 24, 2009 Updates: 3168, 4301 July 24, 2009
Expires: September 25, 2009 (if approved)
Intended status: Standards Track
Expires: January 25, 2010
Tunnelling of Explicit Congestion Notification Tunnelling of Explicit Congestion Notification
draft-ietf-tsvwg-ecn-tunnel-02 draft-ietf-tsvwg-ecn-tunnel-03
Status of this Memo Status of This Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 25, 2009. This Internet-Draft will expire on January 25, 2010.
Copyright Notice Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info). publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. and restrictions with respect to this document.
Abstract Abstract
This document redefines how the explicit congestion notification This document redefines how the explicit congestion notification
(ECN) field of the IP header should be constructed on entry to and (ECN) field of the IP header should be constructed on entry to and
exit from any IP in IP tunnel. On encapsulation it brings all IP in exit from any IP in IP tunnel. On encapsulation it updates RFC3168
IP tunnels (v4 or v6) into line with the way RFC4301 IPsec tunnels to bring all IP in IP tunnels (v4 or v6) into line with RFC4301 IPsec
now construct the ECN field. On decapsulation it redefines how the ECN processing. On decapsulation it updates both RFC3168 and RFC4301
ECN field in the forwarded IP header should be calculated for two to add new behaviours for previously unused combinations of inner and
previously invalid combinations of incoming inner and outer headers, outer header. The new rules propagate the ECN field whether it is
in order that these combinations may be usefully employed in future used to signal one or two severity levels of congestion, whereas
standards actions. It includes a thorough analysis of the reasoning before they propagated only one. Tunnel endpoints can be updated in
for these changes and the implications. any order without affecting pre-existing uses of the ECN field
(backward compatible). Nonetheless, operators wanting to support two
severity levels (e.g. for pre-congestion notification--PCN) can
require compliance with this new specification. A thorough analysis
of the reasoning for these changes and the implications is included.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2. Document Roadmap . . . . . . . . . . . . . . . . . . . . . 9 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 10
2. Requirements Language . . . . . . . . . . . . . . . . . . . . 9 3. Summary of Pre-Existing RFCs . . . . . . . . . . . . . . . . . 11
3. Summary of Pre-Existing RFCs . . . . . . . . . . . . . . . . . 10 3.1. Encapsulation at Tunnel Ingress . . . . . . . . . . . . . 11
3.1. Encapsulation at Tunnel Ingress . . . . . . . . . . . . . 10
3.2. Decapsulation at Tunnel Egress . . . . . . . . . . . . . . 12 3.2. Decapsulation at Tunnel Egress . . . . . . . . . . . . . . 12
4. New ECN Tunnelling Rules . . . . . . . . . . . . . . . . . . . 13 4. New ECN Tunnelling Rules . . . . . . . . . . . . . . . . . . . 13
4.1. Default Tunnel Ingress Behaviour . . . . . . . . . . . . . 14 4.1. Default Tunnel Ingress Behaviour . . . . . . . . . . . . . 13
4.2. Default Tunnel Egress Behaviour . . . . . . . . . . . . . 14 4.2. Default Tunnel Egress Behaviour . . . . . . . . . . . . . 14
4.3. Design Principles for Future Non-Default Schemes . . . . . 16 4.3. Encapsulation Modes . . . . . . . . . . . . . . . . . . . 16
5. Backward Compatibility . . . . . . . . . . . . . . . . . . . . 17 4.4. Single Mode of Decapsulation . . . . . . . . . . . . . . . 17
5.1. Non-Issues Upgrading Any Tunnel Decapsulation . . . . . . 18 5. Changes from Earlier RFCs . . . . . . . . . . . . . . . . . . 18
5.2. Non-Issues for RFC4301 IPsec Encapsulation . . . . . . . . 18 5.1. Changes to RFC4301 ECN processing . . . . . . . . . . . . 18
5.3. Upgrading Other IP in IP Tunnel Encapsulators . . . . . . 19 5.2. Changes to RFC3168 ECN processing . . . . . . . . . . . . 19
6. Changes from Earlier RFCs . . . . . . . . . . . . . . . . . . 20 5.3. Motivation for Changes . . . . . . . . . . . . . . . . . . 19
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 5.3.1. Motivation for Changing Encapsulation . . . . . . . . 20
8. Security Considerations . . . . . . . . . . . . . . . . . . . 21 5.3.2. Motivation for Changing Decapsulation . . . . . . . . 21
9. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 23 6. Backward Compatibility . . . . . . . . . . . . . . . . . . . . 23
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24 6.1. Non-Issues Updating Decapsulation . . . . . . . . . . . . 23
11. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 25 6.2. Non-Update of RFC4301 IPsec Encapsulation . . . . . . . . 23
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.3. Update to RFC3168 Encapsulation . . . . . . . . . . . . . 24
12.1. Normative References . . . . . . . . . . . . . . . . . . . 25 7. Design Principles for Future Non-Default Schemes . . . . . . . 24
12.2. Informative References . . . . . . . . . . . . . . . . . . 25 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26
Appendix A. Design Constraints . . . . . . . . . . . . . . . . . 28 9. Security Considerations . . . . . . . . . . . . . . . . . . . 26
A.1. Security Constraints . . . . . . . . . . . . . . . . . . . 28 10. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 27
A.2. Control Constraints . . . . . . . . . . . . . . . . . . . 30 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 28
A.3. Management Constraints . . . . . . . . . . . . . . . . . . 31 12. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 28
Appendix B. Relative Placement of Tunnelling and In-Path Load 13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Regulation . . . . . . . . . . . . . . . . . . . . . 32 13.1. Normative References . . . . . . . . . . . . . . . . . . . 29
B.1. Identifiers and In-Path Load Regulators . . . . . . . . . 32 13.2. Informative References . . . . . . . . . . . . . . . . . . 29
B.2. Non-Dependence of Tunnelling on In-path Load Regulation . 33 Appendix A. Early ECN Tunnelling RFCs . . . . . . . . . . . . . . 31
B.3. Dependence of In-Path Load Regulation on Tunnelling . . . 34 Appendix B. Design Constraints . . . . . . . . . . . . . . . . . 32
Appendix C. Contribution to Congestion across a Tunnel . . . . . 37 B.1. Security Constraints . . . . . . . . . . . . . . . . . . . 32
Appendix D. Why Not Propagating ECT(1) on Decapsulation B.2. Control Constraints . . . . . . . . . . . . . . . . . . . 34
Impedes PCN . . . . . . . . . . . . . . . . . . . . . 38 B.3. Management Constraints . . . . . . . . . . . . . . . . . . 35
D.1. Alternative Ways to Introduce the New Decapsulation Appendix C. Contribution to Congestion across a Tunnel . . . . . 36
Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Appendix D. Why Losing ECT(1) on Decapsulation Impedes PCN . . . 36
Appendix E. Why Resetting CE on Encapsulation Impedes PCN . . . . 40 Appendix E. Why Resetting ECN on Encapsulation Impedes PCN . . . 38
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 40 Appendix F. Compromise on Decap with ECT(1) Inner and ECT(0)
Outer . . . . . . . . . . . . . . . . . . . . . . . . 39
Request to the RFC Editor (to be removed on publication):
In the RFC index, RFC3168 should be identified as an update to
RFC2481, RFC2401 and RFC2003. RFC4301 should be identified as an
update to RFC3168.
Changes from previous drafts (to be removed by the RFC Editor) Changes from previous drafts (to be removed by the RFC Editor)
Full text differences between IETF draft versions are available at Full text differences between IETF draft versions are available at
<http://tools.ietf.org/wg/tsvwg/draft-ietf-tsvwg-ecn-tunnel/>, and <http://tools.ietf.org/wg/tsvwg/draft-ietf-tsvwg-ecn-tunnel/>, and
between earlier individual draft versions at between earlier individual draft versions at
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ecn-tunnel> <http://www.briscoe.net/pubs.html#ecn-tunnel>
From ietf-01 to ietf-02 (current): From ietf-02 to ietf-03 (current):
* Functional changes:
+ Corrected errors in recap of previous RFCs, which wrongly
stated the different decapsulation behaviours of RFC3168 &
RFC4301 with a Not-ECT inner header. This also required
corrections to the "Changes from Earlier RFCs" and the
Motivations for these changes.
+ Mandated that any future standards action SHOULD NOT use the
ECT(0) codepoint as an indication of congestion, without
giving strong reasons.
+ Added optional alarm when decapsulating ECT(1) outer,
ECT(0), but noted it would need to be disabled for
2-severity level congestion (e.g. PCN).
* Structural changes:
+ Removed Document Roadmap which merely repeated the Contents
(previously Section 1.2).
+ Moved "Changes from Earlier RFCs" (Section 5) before
Section 6 on Backward Compatibility and internally organised
both by RFC, rather than by ingress then egress.
+ Moved motivation for changing existing RFCs (Section 5.3) to
after the changes are specified.
+ Moved informative "Design Principles for Future Non-Default
Schemes" after all the normative sections.
+ Added Appendix A on early history of ECN tunnelling RFCs.
+ Removed specialist appendix on "Relative Placement of
Tunnelling and In-Path Load Regulation" (Appendix D in the
-02 draft)
+ Moved and updated specialist text on "Compromise on Decap
with ECT(1) Inner and ECT(0) Outer" from Security
Considerations to Appendix F
* Textual changes:
+ Simplified vocabulary for non-native-english speakers
+ Simplified Introduction and defined regularly used terms in
an expanded Terminology section.
+ More clearly distinguished statically configured tunnels
from dynamic tunnel endpoint discovery, before explaining
operating modes.
+ Simplified, cut-down and clarified throughout
+ Updated references.
From ietf-01 to ietf-02:
* Scope reduced from any encapsulation of an IP packet to solely * Scope reduced from any encapsulation of an IP packet to solely
IP in IP tunnelled encapsulation. Consequently changed title IP in IP tunnelled encapsulation. Consequently changed title
and removed whole section 'Design Guidelines for New and removed whole section 'Design Guidelines for New
Encapsulations of Congestion Notification' (to be included in a Encapsulations of Congestion Notification' (to be included in a
future companion informational document). future companion informational document).
* Included a new normative decapsulation rule for ECT(0) inner * Included a new normative decapsulation rule for ECT(0) inner
and ECT(1) outer that had previously only been outlined in the and ECT(1) outer that had previously only been outlined in the
non-normative appendix 'Comprehensive Decapsulation Rules'. non-normative appendix 'Comprehensive Decapsulation Rules'.
skipping to change at page 4, line 42 skipping to change at page 6, line 12
this change would streamline PCN. New text on the logic of this change would streamline PCN. New text on the logic of
the resulting decap rules added. the resulting decap rules added.
* If inner/outer is Not-ECT/ECT(0), changed decapsulation to * If inner/outer is Not-ECT/ECT(0), changed decapsulation to
propagate Not-ECT rather than drop the packet; and added propagate Not-ECT rather than drop the packet; and added
reasoning. reasoning.
* Considerably restructured: * Considerably restructured:
+ "Design Constraints" analysis moved to an appendix + "Design Constraints" analysis moved to an appendix
(Appendix A); (Appendix B);
+ Added Section 3 to summarise relevant existing RFCs; + Added Section 3 to summarise relevant existing RFCs;
+ Structured Section 4 and Section 5 into subsections. + Structured Section 4 and Section 6 into subsections.
+ Added tables to sections on old and new rules, for precision + Added tables to sections on old and new rules, for precision
and comparison. and comparison.
+ Moved Section 4.3 on Design Principles to the end of the + Moved Section 7 on Design Principles to the end of the
section specifying the new default normative tunnelling section specifying the new default normative tunnelling
behaviour. Rewritten and shifted text on identifiers and behaviour. Rewritten and shifted text on identifiers and
in-path load regulators to Appendix B.1. in-path load regulators to Appendix B.1 [deleted in revision
-03].
From ietf-00 to ietf-01: From ietf-00 to ietf-01:
* Identified two additional alarm states in the decapsulation * Identified two additional alarm states in the decapsulation
rules (Figure 4) if ECT(X) in outer and inner contradict each rules (Figure 4) if ECT(X) in outer and inner contradict each
other. other.
* Altered Comprehensive Decapsulation Rules (Appendix D) so that * Altered Comprehensive Decapsulation Rules (Appendix D) so that
ECT(0) in the outer no longer overrides ECT(1) in the inner. ECT(0) in the outer no longer overrides ECT(1) in the inner.
Used the term 'Comprehensive' instead of 'Ideal'. And Used the term 'Comprehensive' instead of 'Ideal'. And
considerably updated the text in this appendix. considerably updated the text in this appendix.
* Added Appendix D.1 to weigh up the various ways the * Added Appendix D.1 (removed again in a later revision) to weigh
Comprehensive Decapsulation Rules might be introduced. This up the various ways the Comprehensive Decapsulation Rules might
replaces the previous contradictory statements saying complex be introduced. This replaces the previous contradictory
backwards compatibility interactions would be introduced while statements saying complex backwards compatibility interactions
also saying there would be no backwards compatibility issues. would be introduced while also saying there would be no
backwards compatibility issues.
* Updated references. * Updated references.
From briscoe-01 to ietf-00: From briscoe-01 to ietf-00:
* Re-wrote Appendix C giving much simpler technique to measure * Re-wrote Appendix C giving much simpler technique to measure
contribution to congestion across a tunnel. contribution to congestion across a tunnel.
* Added discussion of backward compatibility of the ideal * Added discussion of backward compatibility of the ideal
decapsulation scheme in Appendix D decapsulation scheme in Appendix D
* Updated references. Minor corrections & clarifications * Updated references. Minor corrections & clarifications
throughout. throughout.
From -00 to -01: From briscoe-00 to briscoe-01:
* Related everything conceptually to the uniform and pipe models * Related everything conceptually to the uniform and pipe models
of RFC2983 on Diffserv Tunnels, and completely removed the of RFC2983 on Diffserv Tunnels, and completely removed the
dependence of tunnelling behaviour on the presence of any in- dependence of tunnelling behaviour on the presence of any in-
path load regulation by using the [1 - Before] [2 - Outer] path load regulation by using the [1 - Before] [2 - Outer]
function placement concepts from RFC2983; function placement concepts from RFC2983;
* Added specific cases where the existing standards limit new * Added specific cases where the existing standards limit new
proposals, particularly Appendix E; proposals, particularly Appendix E;
skipping to change at page 6, line 4 skipping to change at page 7, line 25
dependence of tunnelling behaviour on the presence of any in- dependence of tunnelling behaviour on the presence of any in-
path load regulation by using the [1 - Before] [2 - Outer] path load regulation by using the [1 - Before] [2 - Outer]
function placement concepts from RFC2983; function placement concepts from RFC2983;
* Added specific cases where the existing standards limit new * Added specific cases where the existing standards limit new
proposals, particularly Appendix E; proposals, particularly Appendix E;
* Added sub-structure to Introduction (Need for Rationalisation, * Added sub-structure to Introduction (Need for Rationalisation,
Roadmap), added new Introductory subsection on "Scope" and Roadmap), added new Introductory subsection on "Scope" and
improved clarity; improved clarity;
* Added Design Guidelines for New Encapsulations of Congestion * Added Design Guidelines for New Encapsulations of Congestion
Notification; Notification;
* Considerably clarified the Backward Compatibility section * Considerably clarified the Backward Compatibility section
(Section 5); (Section 6);
* Considerably extended the Security Considerations section * Considerably extended the Security Considerations section
(Section 8); (Section 9);
* Summarised the primary rationale much better in the * Summarised the primary rationale much better in the
conclusions; conclusions;
* Added numerous extra acknowledgements; * Added numerous extra acknowledgements;
* Added Appendix E. "Why resetting CE on encapsulation harms * Added Appendix E. "Why resetting CE on encapsulation harms
PCN", Appendix C. "Contribution to Congestion across a Tunnel" PCN", Appendix C. "Contribution to Congestion across a Tunnel"
and Appendix D. "Ideal Decapsulation Rules"; and Appendix D. "Ideal Decapsulation Rules";
* Re-wrote Appendix B.2, explaining how tunnel encapsulation no * Re-wrote Appendix B [deleted in a later revision], explaining
longer depends on in-path load-regulation (changed title from how tunnel encapsulation no longer depends on in-path load-
"In-path Load Regulation" to "Non-Dependence of Tunnelling on regulation (changed title from "In-path Load Regulation" to
In-path Load Regulation"), but explained how an in-path load "Non-Dependence of Tunnelling on In-path Load Regulation"), but
regulation function must be carefully placed with respect to explained how an in-path load regulation function must be
tunnel encapsulation (in a new sub-section entitled "Dependence carefully placed with respect to tunnel encapsulation (in a new
of In-Path Load Regulation on Tunnelling"). sub-section entitled "Dependence of In-Path Load Regulation on
Tunnelling").
1. Introduction 1. Introduction
This document redefines how the explicit congestion notification Explicit congestion notification (ECN [RFC3168]) allows a forwarding
(ECN) field [RFC3168] in the IP header should be constructed for all element to notify the onset of congestion without having to drop
IP in IP tunnelling. Previously, tunnel endpoints blocked visibility packets. Instead it can explicitly mark a proportion of packets in
of transitions of the ECN field except the minimum necessary to allow the 2-bit ECN field in the IP header (Table 1 recaps the ECN
the basic ECN mechanism to work. Three main change are defined, one codepoints).
on entry to and two on exit from any IP in IP tunnel. The newly
specified behaviours make all transitions to the ECN field visible
across tunnel end-points, so tunnels no longer restrict new uses of
the ECN field that were not envisaged when ECN was first designed.
The immediate motivation for opening up the ECN behaviour of tunnels
is because otherwise they impede the introduction of pre-congestion
notification (PCN [I-D.ietf-pcn-marking-behaviour]) in networks with
tunnels (Appendix E explains why). But these changes are not just
intended to ease the introduction of PCN; care has been taken to
ensure the resulting ECN tunnelling behaviour is simple and generic
for other potential future uses.
Given this is a change to behaviour at 'the neck of the hourglass',
an extensive analysis of the trade-offs between control, management
and security constraints has been conducted in order to minimise
unexpected side-effects both now and in the future. Care has also
been taken to ensure the changes are fully backwards compatible with
all previous tunnelling behaviours.
The ECN protocol allows a forwarding element to notify the onset of
congestion of its resources without having to drop packets. Instead
it can explicitly mark a proportion of packets by setting the
congestion experienced (CE) codepoint in the 2-bit ECN field in the
IP header (see Table 1 for a recap of the ECN codepoints).
+------------------+----------------+---------------------------+
| Binary codepoint | Codepoint name | Meaning |
+------------------+----------------+---------------------------+
| 00 | Not-ECT | Not ECN-capable transport |
| 01 | ECT(1) | ECN-capable transport |
| 10 | ECT(0) | ECN-capable transport |
| 11 | CE | Congestion experienced |
+------------------+----------------+---------------------------+
Table 1: Recap of Codepoints of the ECN Field [RFC3168] in the IP The outer header of an IP packet can encapsulate one or more IP
Header headers for tunnelling. A forwarding element using ECN to signify
congestion will only mark the immediately visible outer IP header.
When a tunnel decapsulator later removes this outer header, it
follows rules to propagate congestion markings by combining the ECN
fields of the inner and outer IP header into one outgoing IP header.
The outer header of an IP packet can encapsulate one (or more) This document updates those rules for IPsec [RFC4301] and non-IPsec
additional IP headers tunnelled within it. A forwarding element that [RFC3168] tunnels to add new behaviours for previously unused
is using ECN to signify congestion will only mark the outer IP header combinations of inner and outer header. It also updates the tunnel
that is immediately visible to it. When a tunnel decapsulator later ingress behaviour of RFC3168 to match that of RFC4301. The updated
removes this outer header, it must follow rules to ensure the marking rules are backward compatible with RFC4301 and RFC3168 when
is propagated into the IP header being forwarded onwards, otherwise interworking with any other tunnel endpoint complying with any
congestion notifications will disappear into a black hole leading to earlier specification.
potential congestion collapse.
The rules for constructing the ECN field to be forwarded after tunnel When ECN and its tunnelling was defined in RFC3168, only the minimum
decapsulation ensure this happens, but they are not wholly necessary changes to the ECN field were propagated through tunnel
straightforward, and neither are the rules for encapsulating one IP endpoints--just enough for the basic ECN mechanism to work. This was
header in another on entry to a tunnel. The factor that has due to concerns that the ECN field might be toggled to communicate
introduced most complication at both ends of a tunnel has been the between a secure site and someone on the public Internet--a covert
possibility that the ECN field might be used as a covert channel to channel. This was because a mutable field like ECN cannot be
compromise the integrity of an IPsec tunnel. protected by IPsec's integrity mechanisms--it has to be able to
change as it traverses the Internet.
A common use for IPsec is to create a secure tunnel between two Nonetheless, the latest IPsec architecture [RFC4301] considers a
secure sites across the public Internet. A field like ECN that can bandwidth limit of 2 bits per packet on a covert channel makes it a
change as it traverses the Internet cannot be covered by IPsec's manageable risk. Therefore, for simplicity, an RFC4301 ingress
integrity mechanisms. Therefore, the ECN field might be toggled copies the whole ECN field to encapsulate a packet. It also
(with two bits per packet) to communicate between a secure site and dispenses with the two modes of RFC3168, one which partially copied
someone on the public Internet--a covert channel. the ECN field, and the other which blocked all propagation of ECN
changes.
Over the years covert channel restrictions have been added to the Unfortunately, this entirely reasonable sequence of standards actions
design of ECN (with consequent backward compatibility complications). resulted in a perverse outcome; non-IPsec tunnels (RFC3168) blocked
However the latest IPsec architecture [RFC4301] takes the view that the 2-bit covert channel, while IPsec tunnels (RFC4301) did not--at
simplicity is more important than closing off the covert channel least not at the ingress. At the egress, both IPsec and non-IPsec
threat, which it deems manageable given its bandwidth is limited to tunnels still partially restricted propagation of the full ECN field.
two bits per packet.
As a result, an unfortunate sequence of standards actions has left us The trigger for the changes in this document was the introduction of
with nearly the worst of all possible combinations of outcomes, pre-congestion notification (PCN [I-D.ietf-pcn-marking-behaviour]) to
despite the best endeavours of everyone concerned. The new IPsec the IETF standards track. PCN needs the ECN field to be copied at a
architecture [RFC4301] only updates the earlier specification of ECN tunnel ingress and it needs four states of congestion signalling to
tunnelling behaviour [RFC3168] for the case of IPsec tunnels. For be propagated at the egress, but pre-existing tunnels only propagate
the case of non-IPsec tunnels the earlier RFC3168 specification still three in the ECN field.
applies. At the time RFC3168 was standardised, covert channels
through the ECN field were restricted, whether or not IPsec was being
used. The perverse position now is that non-IPsec tunnels restrict
covert channels, while IPsec tunnels don't.
Actually, this statement needs some qualification. IPsec tunnels This document draws on currently unused (CU) combinations of inner
only don't restrict the ECN covert channel at the ingress. At the and outer headers to add tunnelling of four-state congestion
tunnel egress, the presumption that the ECN covert channel should be signalling to RFC3168 and RFC4301. Operators of tunnels who
restricted has not been removed from any tunnelling specifications, specifically want to support four states can require that all their
whether IPsec or not. tunnels comply with this specification. Nonetheless, all tunnel
endpoint implementations (RFC4301, RFC3168, RFC2481, RFC2401,
RFC2003) can safely be updated to this new specification as part of
general code maintenance. This will gradually add support for four
congestion states to the Internet. Existing three state schemes will
continue to work as before.
Now that these historic 2-bit covert channel constraints are impeding At the same time as harmonising covert channel constraints, the
the introduction of PCN, this specification is designed to remove opportunity has been taken to draw together diverging tunnel
them and at the same time streamline the whole ECN behaviour for the specifications into a single consistent behaviour. Then any tunnel
future. can be deployed unilaterally, and it will support the full range of
congestion control and management schemes without any modes or
configuration. Further, any host or router can expect the ECN field
to behave in the same way, whatever type of tunnel might intervene in
the path.
1.1. Scope 1.1. Scope
This document only concerns wire protocol processing at tunnel This document only concerns wire protocol processing of the ECN field
endpoints and makes no changes or recommendations concerning at tunnel endpoints and makes no changes or recommendations
algorithms for congestion marking or congestion response. concerning algorithms for congestion marking or congestion response.
This document specifies common, default ECN field processing at This document specifies common ECN field processing at encapsulation
encapsulation and decapsulation for any IP in IP tunnelling. It and decapsulation for any IP in IP tunnelling, whether IPsec or non-
applies irrespective of whether IPv4 or IPv6 is used for either of IPsec tunnels. It applies irrespective of whether IPv4 or IPv6 is
the inner and outer headers. It applies to all Diffserv per-hop used for either of the inner and outer headers. It applies for
packets with any destination address type, whether unicast or
multicast. It applies as the default for all Diffserv per-hop
behaviours (PHBs), unless stated otherwise in the specification of a behaviours (PHBs), unless stated otherwise in the specification of a
PHB. It is intended to be a good trade off between somewhat PHB. It is intended to be a good trade off between somewhat
conflicting security, control and management requirements. conflicting security, control and management requirements.
Nonetheless, if necessary, an alternate congestion encapsulation
behaviour can be introduced as part of the definition of an alternate
congestion marking scheme used by a specific Diffserv PHB (see S.5 of
[RFC3168] and [RFC4774]). When designing such new encapsulation
schemes, the principles in Section 4.3 should be followed as closely
as possible. There is no requirement for a PHB to state anything
about ECN tunnelling behaviour if the new default behaviour is
sufficient.
[RFC2983] is a comprehensive primer on differentiated services and [RFC2983] is a comprehensive primer on differentiated services and
tunnels. Given ECN raises similar issues to differentiated services tunnels. Given ECN raises similar issues to differentiated services
when interacting with tunnels, useful concepts introduced in RFC2983 when interacting with tunnels, useful concepts introduced in RFC2983
are used throughout, with brief recaps of the explanations where are used throughout, with brief recaps of the explanations where
necessary. necessary.
1.2. Document Roadmap 2. Terminology
The body of the document focuses solely on standards actions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
impacting implementation. Appendices record the analysis that "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
motivates and justifies these actions. The whole document is document are to be interpreted as described in RFC 2119 [RFC2119].
organised as follows:
o Section 3 recaps relevant existing RFCs and explains exactly why Table 1 recaps the names of the ECN codepoints [RFC3168].
changes are needed, referring to Appendix D and Appendix E in
order to explain in detail why current tunnelling behaviours
impede PCN deployment, at egress and ingress respectively.
o Section 4 uses precise standards terminology to specify the new +------------------+----------------+---------------------------+
ECN tunnelling behaviours. It refers to Appendix A for analysis | Binary codepoint | Codepoint name | Meaning |
of the trade-offs between security, control and management design +------------------+----------------+---------------------------+
constraints that led to these particular standards actions. | 00 | Not-ECT | Not ECN-capable transport |
| 01 | ECT(1) | ECN-capable transport |
| 10 | ECT(0) | ECN-capable transport |
| 11 | CE | Congestion experienced |
+------------------+----------------+---------------------------+
o Extending the new IPsec tunnel ingress behaviour to all IP in IP Table 1: Recap of Codepoints of the ECN Field [RFC3168] in the IP
tunnels requires consideration of backwards compatibility, which Header
is covered in Section 5 and detailed changes from earlier RFCs are
brought together in Section 6.
o Finally, a number of security considerations are discussed and Further terminology used within this document:
conclusions are drawn.
o Additional specialist issues are deferred to appendices in Encapsulator: The tunnel endpoint function that adds an outer IP
addition to those already referred to above, in particular header to tunnel a packet (also termed the 'ingress tunnel
Appendix B discusses specialist tunnelling issues that could arise endpoint' or just the 'ingress' where the context is clear).
when ECN is fed back to a load regulation function on a middlebox,
rather than at the source of the path.
2. Requirements Language Decapsulator: The tunnel endpoint function that removes an outer IP
header from a tunnelled packet (also termed the 'egress tunnel
endpoint' or just the 'egress' where the context is clear).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", Incoming header: The header of an arriving packet before
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this encapsulation.
document are to be interpreted as described in RFC 2119 [RFC2119].
Outer header: The header added to encapsulate a tunnelled packet.
Inner header: The header encapsulated by the outer header.
Outgoing header: The header constructed by the decapsulator using
logic that combines the fields in the outer and inner headers.
Copying ECN: On encapsulation, setting the ECN field of the new
outer header to be a copy of the ECN field in the incoming header.
Zeroing ECN: On encapsulation, clearing the ECN field of the new
outer header to Not-ECT ("00").
Resetting ECN: On encapsulation, setting the ECN field of the new
outer header to be a copy of the ECN field in the incoming header
except the outer ECN field is set to the ECT(0) codepoint if the
incoming ECN field is CE ("11").
3. Summary of Pre-Existing RFCs 3. Summary of Pre-Existing RFCs
This section is informative not normative. It merely recaps pre- This section is informative not normative, as it recaps pre-existing
existing RFCs to help motivate changing these behaviours. Earlier RFCs. Earlier relevant RFCs that were either experimental or
relevant RFCs that were either experimental or incomplete with incomplete with respect to ECN tunnelling (RFC2481, RFC2401 and
respect to ECN tunnelling (RFC2481, RFC2401 and RFC2003) are not RFC2003) are briefly outlined inAppendix A. The question of whether
discussed, although the backwards compatibility considerations in tunnel implementations used in the Internet comply with any of these
Section 5 take them into account. The question of whether tunnel RFCs is not discussed.
implementations used in the Internet comply with any of these RFCs is
also not discussed.
3.1. Encapsulation at Tunnel Ingress 3.1. Encapsulation at Tunnel Ingress
The controversy at tunnel ingress has been over whether to propagate At the encapsulator, the controversy has been over whether to
information about congestion experienced on the path upstream of the propagate information about congestion experienced on the path so far
tunnel ingress into the outer header of the tunnel. into the outer header of the tunnel.
Specifically, RFC3168 says that, if a tunnel fully supports ECN Specifically, RFC3168 says that, if a tunnel fully supports ECN
(termed a 'full-functionality' ECN tunnel in [RFC3168]), the tunnel (termed a 'full-functionality' ECN tunnel in [RFC3168]), the
ingress must not copy a CE marking from the inner header into the encapsulator must not copy a CE marking from the inner header into
outer header that it creates. Instead the tunnel ingress must set the outer header that it creates. Instead the encapsulator must set
the outer header to ECT(0) (i.e. codepoint 10) if the ECN field is the outer header to ECT(0) if the ECN field is marked CE in the
marked CE (codepoint 11) in the arriving IP header. We term this arriving IP header. We term this 'resetting' a CE codepoint.
'resetting' a CE codepoint.
However, the new IPsec architecture in [RFC4301] reverses this rule, However, the new IPsec architecture in [RFC4301] reverses this rule,
stating that the tunnel ingress must simply copy the ECN field from stating that the encapsulator must simply copy the ECN field from the
the arriving to the outer header. The main purpose of the present incoming header to the outer header.
specification is to carry the new behaviour of IPsec over to all IP
in IP tunnels, so all tunnel ingress nodes consistently copy the ECN
field.
RFC3168 also provided a Limited Functionality mode that turns off ECN RFC3168 also provided a Limited Functionality mode that turns off ECN
processing over the scope of the tunnel. This is necessary if the processing over the scope of the tunnel by setting the outer header
ingress does not know whether the tunnel egress supports propagation to Not-ECT ("00"). Then such packets will be dropped to indicate
of ECN markings. Neither Limited Functionality mode nor Full congestion rather than marked with ECN. This is necessary for the
Functionality mode are used in RFC4301 IPsec. ingress to interwork with legacy decapsulators ([RFC2481], [RFC2401]
and [RFC2003]) that do not propagate ECN markings added to the outer
header. Otherwise such legacy decapsulators would throw away
congestion notifications before they reached the transport layer.
Neither Limited Functionality mode nor Full Functionality mode are
used by an RFC4301 IPsec encapsulator, which simply copies the
incoming ECN field into the outer header. An earlier key-exchange
phase ensures an RFC4301 ingress will not have to interwork with a
legacy egress that does not support ECN.
These pre-existing behaviours are summarised in Figure 1. These pre-existing behaviours are summarised in Figure 1.
+-----------------+-----------------------------------------------+ +-----------------+-----------------------------------------------+
| Incoming Header | Outgoing Outer Header | | Incoming Header | Outgoing Outer Header |
| (also equal to +---------------+---------------+---------------+ | (also equal to +---------------+---------------+---------------+
| Outgoing Inner | RFC3168 ECN | RFC3168 ECN | RFC4301 IPsec | | Outgoing Inner | RFC3168 ECN | RFC3168 ECN | RFC4301 IPsec |
| Header) | Limited | Full | | | Header) | Limited | Full | |
| | Functionality | Functionality | | | | Functionality | Functionality | |
+-----------------+---------------+---------------+---------------+ +-----------------+---------------+---------------+---------------+
| Not-ECT | Not-ECT | Not-ECT | Not-ECT | | Not-ECT | Not-ECT | Not-ECT | Not-ECT |
| ECT(0) | Not-ECT | ECT(0) | ECT(0) | | ECT(0) | Not-ECT | ECT(0) | ECT(0) |
| ECT(1) | Not-ECT | ECT(1) | ECT(1) | | ECT(1) | Not-ECT | ECT(1) | ECT(1) |
| CE | Not-ECT | ECT(0) | CE e| | CE | Not-ECT | ECT(0) | CE |
+-----------------+---------------+---------------+---------------+ +-----------------+---------------+---------------+---------------+
Figure 1: IP in IP Encapsulation: Recap of Pre-existing Behaviours Figure 1: IP in IP Encapsulation: Recap of Pre-existing Behaviours
For encapsulation, the specification in Section 4 below brings all IP
in IP tunnels (v4 or v6) into line with the way IPsec tunnels
[RFC4301] now construct the ECN field, except where a legacy tunnel
egress might not understand ECN at all. This removes the now
redundant full functionality mode in the middle column of Figure 1.
Wherever possible it ensures that the outer header reveals any
congestion experienced so far on the whole path, not just since the
last tunnel ingress.
Why does it matter if we have different ECN encapsulation behaviours
for IPsec and non-IPsec tunnels? A general answer is that gratuitous
inconsistency constrains the available design space and makes it
harder to design networks and new protocols that work predictably.
But there is also a specific need not to reset the CE codepoint. The
standards track proposal for excess rate pre-congestion notification
(PCN [I-D.ietf-pcn-marking-behaviour]) only works correctly in the
presence of RFC4301 IPsec encapsulation or [RFC5129] MPLS
encapsulation, but not with RFC3168 IP in IP encapsulation
(Appendix E explains why). The PCN architecture
[I-D.ietf-pcn-architecture] states that the regular RFC3168 rules for
IP in IP tunnelling of the ECN field should not be used for PCN. But
if non-IPsec tunnels are already present within a network to which
PCN is being added, that is not particularly helpful advice.
The present specification provides a clean solution to this problem,
so that network operators who want to use PCN and tunnels can specify
that all tunnel endpoints in a PCN region need to be upgraded to
comply with this specification. Also, whether using PCN or not, as
more tunnel endpoints comply with this specification, it should make
ECN behaviour simpler, faster and more predictable.
To ensure copying rather than resetting CE on ingress will not cause
unintended side-effects, Appendix A assesses whether either harm any
security, control or management functions. It finds that resetting
CE makes life difficult in a number of directions, while copying CE
harms nothing (other than opening a low bit-rate covert channel
vulnerability which the IETF Security Area now deems is manageable).
3.2. Decapsulation at Tunnel Egress 3.2. Decapsulation at Tunnel Egress
Both RFC3168 and RFC4301 specify the decapsulation behaviour RFC3168 and RFC4301 specify the decapsulation behaviour summarised in
summarised in Figure 2. The ECN field in the outgoing header is set Figure 2. The ECN field in the outgoing header is set to the
to the codepoint at the intersection of the appropriate incoming codepoint at the intersection of the appropriate incoming inner
inner header (row) and incoming outer header (column). header (row) and incoming outer header (column).
+------------------+----------------------------------------------+ +---------+------------------------------------------------+
| Incoming Inner | Incoming Outer Header | |Incoming | Incoming Outer Header |
| Header +---------+------------+------------+----------+ | Inner +---------+------------+------------+------------+
| | Not-ECT | ECT(0) | ECT(1) | CE | | Header | Not-ECT | ECT(0) | ECT(1) | CE |
+------------------+---------+------------+------------+----------+ +---------+---------+------------+------------+------------+
| Not-ECT | Not-ECT | drop(!!!)| drop(!!!)| drop(!!!)| RFC3168->| Not-ECT | Not-ECT |Not-ECT |Not-ECT | drop |
RFC4301->| Not-ECT | Not-ECT |Not-ECT |Not-ECT |Not-ECT |
| ECT(0) | ECT(0) | ECT(0) | ECT(0) | CE | | ECT(0) | ECT(0) | ECT(0) | ECT(0) | CE |
| ECT(1) | ECT(1) | ECT(1) | ECT(1) | CE | | ECT(1) | ECT(1) | ECT(1) | ECT(1) | CE |
| CE | CE | CE | CE | CE | | CE | CE | CE | CE | CE |
+------------------+---------+------------+------------+----------+ +---------+---------+------------+------------+------------+
| Outgoing Header | | Outgoing Header |
+----------------------------------------------+ +------------------------------------------------+
Figure 2: IP in IP Decapsulation; Recap of Pre-existing Behaviour Figure 2: IP in IP Decapsulation; Recap of Pre-existing Behaviour
The behaviour in the table derives from the logic given in RFC3168, The behaviour in the table derives from the logic given in RFC3168
briefly recapped as follows: and RFC4301, briefly recapped as follows:
o On decapsulation, if the inner ECN field is Not-ECT but the outer
ECN field is anything except Not-ECT the decapsulator must drop
the packet. Drop is mandated because known legal protocol
transitions should not be able to lead to these cases (indicated
in the table by '(!!!)'), therefore the decapsulator may also
raise an alarm;
o In all other cases, the outgoing ECN field is set to the more o On decapsulation, if the inner ECN field is Not-ECT the outer is
severe marking of the outer and inner ECN fields, where the discarded. RFC3168 (but not RFC4301) also specified that the
ranking of severity from highest to lowest is CE, ECT, Not-ECT; decapsulator must drop a packet with a Not-ECT inner and CE in the
outer.
o ECT(0) and ECT(1) are considered of equal severity (indicated by o In all other cases, if the outer is CE, the outgoing ECN field is
just 'ECT' in the rank order above). Where the inner and outer set to CE, but otherwise the outer is ignored and the inner is
ECN fields are both ECT but they differ, the packet is forwarded used for the outgoing ECN field.
with the codepoint of the inner ECN field, which prevents ECT
codepoints being used for a covert channel.
The specification for decapsulation in Section 4 fixes two problems RFC3168 also made it an auditable event for an IPsec tunnel "if the
with this pre-existing behaviour: ECN Field is changed inappropriately within an IPsec tunnel...".
Inappropriate changes were not specifically enumerated. RFC4301 did
not mention inappropriate ECN changes.
o Firstly, forwarding the codepoint of the inner header in the cases 4. New ECN Tunnelling Rules
where both inner and outer are different values of ECT effectively
implies that any distinction between ECT(0) and ECT(1) cannot be
introduced in the future wherever a tunnel might be deployed.
Therefore, the currently specified tunnel decapsulation behaviour
unnecessarily wastes one of four codepoints (effectively wasting
half a bit) in the IP (v4 & v6) header. As explained in
Appendix A.1, the original reason for not using the outer ECT
codepoints for onward forwarding was to limit the covert channel
across a decapsulator to 1 bit per packet. However, now that the
IETF Security Area has deemed that a 2-bit covert channel through
an encapsulator is a manageable risk, the same should be true for
a decapsulator.
As well as being a general future-proofing issue, this problem is The standards actions below in Section 4.1 (ingress encapsulation)
immediately pressing for standardisation of pre-congestion and Section 4.2 (egress decapsulation) define new default ECN tunnel
notification (PCN). PCN solutions generally require three processing rules for any IP packet (v4 or v6) with any Diffserv
encoding states in addition to Not-ECT: one for 'not marked' and codepoint.
two increasingly severe levels of marking. Although the ECN field
gives sufficient codepoints for these three states, they cannot
all be used for PCN because a change between ECT(0) and ECT(1) in
any tunnelled packet would be lost when the outer header was
decapsulated, dangerously discarding congestion signalling. A
number of wasteful or convoluted work-rounds to this problem are
being considered for standardisation by the PCN working group (see
Appendix D), but by far the simplest approach is just to remove
the covert channel blockages from tunnelling behaviour, that are
now deemed unnecessary anyway. Not only will this streamline PCN
standardisation, but it could also streamline other future uses of
these codepoints.
o Secondly, mandating drop is not always a good idea just because a If absolutely necessary, an alternate congestion encapsulation
combination of headers seems invalid. There are many cases where behaviour can be introduced as part of the definition of an alternate
it has become nearly impossible to deploy new standards because congestion marking scheme used by a specific Diffserv PHB (see S.5 of
legacy middleboxes drop packets carrying header values they don't [RFC3168] and [RFC4774]). When designing such new encapsulation
expect. Where possible, the new decapsulation behaviour specified schemes, the principles in Section 7 should be followed. However,
in Section 4 below is more liberal in its response to unexpected alternate ECN tunnelling schemes are NOT RECOMMENDED as the
combinations of headers. deployment burden of handling exceptional PHBs in implementations of
all affected tunnels should not be underestimated. There is no
requirement for a PHB definition to state anything about ECN
tunnelling behaviour if the default behaviour in the present
specification is sufficient.
4. New ECN Tunnelling Rules 4.1. Default Tunnel Ingress Behaviour
The ECN tunnel processing rules below in Section 4.1 (ingress Two modes of encapsulation are defined here; `normal mode' and
encapsulation) and Section 4.2 (egress decapsulation) are the default `compatibility mode', which is for backward compatibility with tunnel
for a packet with any DSCP. If required, different ECN encapsulation decapsulators that do not understand ECN. Section 4.3 explains why
rules MAY be defined as part of the definition of an appropriate two modes are necessary and specifies the circumstances in which it
Diffserv PHB using the guidelines that follow in Section 4.3. is sufficient to solely implement normal mode. Note that these are
However, the deployment burden of handling exceptional PHBs in modes of the ingress tunnel endpoint only, not the whole tunnel.
implementations of all affected tunnels and lower layer link
protocols should not be underestimated.
4.1. Default Tunnel Ingress Behaviour Whatever the mode, an encapsulator forwards the inner header without
changing the ECN field.
A tunnel ingress compliant with this specification MUST implement a In normal mode an encapsulator compliant with this specification MUST
`normal mode'. It might also need to implement a `compatibility construct the outer encapsulating IP header by copying the 2-bit ECN
mode' for backward compatibility with legacy tunnel egresses that do field of the incoming IP header. In compatibility mode it clears the
not understand ECN (see Section 5 for when compatibility mode is ECN field in the outer header to the Not-ECT codepoint. These rules
required). Note that these are modes of the ingress tunnel endpoint are tabulated for convenience in Figure 3.
only, not the tunnel as a whole.
Whatever the mode, the tunnel ingress forwards the inner header
without changing the ECN field. In normal mode a tunnel ingress
compliant with this specification MUST construct the outer
encapsulating IP header by copying the 2-bit ECN field of the
arriving IP header. In compatibility mode it clears the ECN field in
the outer header to the Not-ECT codepoint. These rules are tabulated
for convenience in Figure 3.
+-----------------+-------------------------------+ +-----------------+-------------------------------+
| Incoming Header | Outgoing Outer Header | | Incoming Header | Outgoing Outer Header |
| (also equal to +---------------+---------------+ | (also equal to +---------------+---------------+
| Outgoing Inner | Compatibility | Normal | | Outgoing Inner | Compatibility | Normal |
| Header) | Mode | Mode | | Header) | Mode | Mode |
+-----------------+---------------+---------------+ +-----------------+---------------+---------------+
| Not-ECT | Not-ECT | Not-ECT | | Not-ECT | Not-ECT | Not-ECT |
| ECT(0) | Not-ECT | ECT(0) | | ECT(0) | Not-ECT | ECT(0) |
| ECT(1) | Not-ECT | ECT(1) | | ECT(1) | Not-ECT | ECT(1) |
| CE | Not-ECT | CE | | CE | Not-ECT | CE |
+-----------------+---------------+---------------+ +-----------------+---------------+---------------+
Figure 3: New IP in IP Encapsulation Behaviours Figure 3: New IP in IP Encapsulation Behaviours
Compatibility mode is the same per packet behaviour as the ingress An ingress in compatibility mode encapsulates packets identically to
end of RFC3168's limited functionality mode. Normal mode is the same an ingress in RFC3168's limited functionality mode. An ingress in
per packet behaviour as the ingress end of RFC4301 IPsec. normal mode encapsulates packets identically to an RFC4301 IPsec
ingress.
4.2. Default Tunnel Egress Behaviour 4.2. Default Tunnel Egress Behaviour
To decapsulate the inner header at the tunnel egress, a compliant To decapsulate the inner header at the tunnel egress, a compliant
tunnel egress MUST set the outgoing ECN field to the codepoint at the tunnel egress MUST set the outgoing ECN field to the codepoint at the
intersection of the appropriate incoming inner header (row) and outer intersection of the appropriate incoming inner header (row) and outer
header (column) in Figure 4. header (column) in Figure 4 (the IPv4 header checksum also changes
whenever the ECN field is changed). There is no need for more than
+------------------+----------------------------------------------+ one mode of decapsulation, as these rules cater for all known
| Incoming Inner | Incoming Outer Header | requirements.
| Header +---------+------------+------------+----------+ +---------+------------------------------------------------+
| | Not-ECT | ECT(0) | ECT(1) | CE | |Incoming | Incoming Outer Header |
+------------------+---------+------------+------------+----------+ | Inner +---------+------------+------------+------------+
| Header | Not-ECT | ECT(0) | ECT(1) | CE |
+---------+---------+------------+------------+------------+
| Not-ECT | Not-ECT |Not-ECT(!!!)| drop(!!!)| drop(!!!)| | Not-ECT | Not-ECT |Not-ECT(!!!)| drop(!!!)| drop(!!!)|
| ECT(0) | ECT(0) | ECT(0) | ECT(1) | CE | | ECT(0) | ECT(0) | ECT(0) | ECT(1)(!!!)| CE |
| ECT(1) | ECT(1) | ECT(1)(!!!)| ECT(1) | CE | | ECT(1) | ECT(1) | ECT(1)(!!!)| ECT(1) | CE |
| CE | CE | CE | CE(!!!)| CE | | CE | CE | CE | CE(!!!)| CE |
+------------------+---------+------------+------------+----------+ +---------+---------+------------+------------+------------+
| Outgoing Header | | Outgoing Header |
+----------------------------------------------+ +------------------------------------------------+
Unexpected combinations are indicated by '(!!!)'
Figure 4: New IP in IP Decapsulation Behaviour Figure 4: New IP in IP Decapsulation Behaviour
This table for decapsulation behaviour is derived from the following This table for decapsulation behaviour is derived from the following
logic: logic:
o If the inner ECN field is Not-ECT the decapsulator MUST NOT o If the inner ECN field is Not-ECT the decapsulator MUST NOT
propagate any other ECN codepoint in the outer header onwards. propagate any other ECN codepoint onwards. This is because the
This is because the inner Not-ECT marking is set by transports inner Not-ECT marking is set by transports that use drop as an
that would not understand the ECN protocol. Instead: indication of congestion and would not understand or respond to
any other ECN codepoint [RFC4774]. In addition:
* If the inner ECN field is Not-ECT and the outer ECN field is * If the inner ECN field is Not-ECT and the outer ECN field is
ECT(1) or CE the decapsulator MUST drop the packet. ECT(1) or CE the decapsulator MUST drop the packet.
Reasoning: these combinations of codepoints either imply some
illegal protocol transition has occurred within the tunnel, or
that some locally defined mechanism is being used within the
tunnel that might be signalling congestion. In either case,
the only appropriate signal to the transport is a packet drop.
It would have been nice to allow packets with ECT(1) in the
outer to be forwarded, but drop has had to be mandated in case
future multi-level ECN schemes are defined. Then ECT(1) and CE
can be used in the future to signify two levels of congestion
severity.
* If the inner ECN field is Not-ECT and the outer ECN field is * If the inner ECN field is Not-ECT and the outer ECN field is
ECT(0) or Not-ECT the decapsulator MUST forward the packet with ECT(0) or Not-ECT the decapsulator MUST forward the outgoing
the ECN field cleared to Not-ECT. packet with the ECN field cleared to Not-ECT.
Reasoning: Although no known legal protocol transition would
lead to ECT(0) in the outer and Not-ECT in the inner, no known
or proposed protocol uses ECT(0) as a congestion signal either.
Therefore in this case the packet can be forwarded rather than
dropped, which will allow future standards actions to use this
combination.
o In all other cases, the outgoing ECN field is set to the more * This specification mandates that any future standards action
severe marking of the outer and inner ECN fields, where the SHOULD NOT use the ECT(0) codepoint as an indication of
ranking of severity from highest to lowest is CE, ECT(1), ECT(0), congestion, without giving strong reasons, given the above rule
Not-ECT; forwards an ECT(0) outer as Not-ECT.
o There are cases where no currently legal transition in any current o In all other cases where the inner supports ECN, the outgoing ECN
or previous ECN tunneling specification would result in certain field is set to the more severe marking of the outer and inner ECN
combinations of inner and outer ECN fields. These cases are fields, where the ranking of severity from highest to lowest is
indicated in Figure 4 by '(!!!)'). In these cases, the CE, ECT(1), ECT(0), Not-ECT. This in no way precludes cases where
decapsulator SHOULD log the event and MAY also raise an alarm, but ECT(1) and ECT(0) have the same severity;
not so often that the illegal combinations would amplify into a
flood of alarm messages. o Certain combinations of inner and outer ECN fields cannot result
from any currently used transition in any current or previous ECN
tunneling specification. These cases are indicated in Figure 4 by
'(!!!)'). In these cases, the decapsulator SHOULD log the event
and MAY also raise an alarm. Alarms should be rate-limited so
that the illegal combinations will not amplify into a flood of
alarm messages. It MUST be possible to suppress alarms or
logging, e.g. if it becomes apparent that a combination that
previously was not used has started to be used for legitimate
purposes such as a new standards action. An example is an ECT(0)
inner combined with an ECT(1) outer, which is proposed as a legal
combination for PCN [I-D.ietf-pcn-3-in-1-encoding], so an operator
that deploys support for PCN should turn off logging and alarms in
this case.
The above logic allows for ECT(0) and ECT(1) to both represent the The above logic allows for ECT(0) and ECT(1) to both represent the
same severity of congestion marking (e.g. "not congestion marked"). same severity of congestion marking (e.g. "not congestion marked").
But it also allows future schemes to be defined where ECT(1) is a But it also allows future schemes to be defined where ECT(1) is a
more severe marking than ECT(0). This approach is discussed in more severe marking than ECT(0). This approach is discussed in
Appendix D and in the discussion of the ECN nonce [RFC3540] in Appendix D and in the discussion of the ECN nonce [RFC3540] in
Section 8. Section 9, which in turn refers to Appendix F.
4.3. Design Principles for Future Non-Default Schemes 4.3. Encapsulation Modes
This section is informative not normative. Section 4.1 introduces two encapsulation modes, normal mode and
compatibility mode, defining their encapsulation behaviour (i.e.
header copying or zeroing respectively). Note that these are modes
of the ingress tunnel endpoint only, not the tunnel as a whole.
S.5 of RFC3168 permits the Diffserv codepoint (DSCP)[RFC2474] to A tunnel ingress MUST at least implement `normal mode' and, if it
'switch in' different behaviours for marking the ECN field, just as might be used with legacy tunnel egress nodes (RFC2003, RFC2401 or
it switches in different per-hop behaviours (PHBs) for scheduling. RFC2481 or the limited functionality mode of RFC3168), it MUST also
Therefore here we give guidance for designing possibly different implement `compatibility mode' for backward compatibility with tunnel
marking schemes. egresses that do not propagate explicit congestion notifications
[RFC4774]. If the egress does support propagation of ECN (full
functionality mode of RFC3168 or RFC4301 or the present
specification), the ingress SHOULD use normal mode, in order to
support ECN where possible.
In one word the guidance is "Don't". If a scheme requires tunnels to We can categorise the way that an ingress tunnel endpoint is paired
implement special processing of the ECN field for certain DSCPs, it with an egress as either:
is highly unlikely that every implementer of every tunnel will want
to add the required exception and that operators will want to deploy
the required configuration options. Therefore it is highly likely
that some tunnels within a network will not implement this special
case. Therefore, designers should avoid non-default tunnelling
schemes if at all possible.
That said, if a non-default scheme for processing the ECN field is static: those paired together by prior configuration or;
really required, the following guidelines may prove useful in its
design:
o For any new scheme, a tunnel ingress should not set the ECN field dynamically discovered: those paired together by some form of tunnel
of the outer header if it cannot guarantee that any corresponding endpoint discovery, typically driven by the path taken by arriving
tunnel egress will understand how to handle such an ECN field. packets.
o On encapsulation in any new scheme, an outer header capable of Static: Some implementations of encapsulator might be constrained to
carrying congestion markings should reflect accumulated congestion be statically deployed, and constrained to never be paired with a
since the last interface designed to regulate load (see legacy decapsulator (RFC2003, RFC2401 or RFC2481 or the limited
Appendix A.2 for the definition of a Load Regulator, which is functionality mode of RFC3168). In such a case, only normal mode
usually but not always the data source). This implies that new needs to be implemented.
schemes for tunnelling congestion notification should copy
congestion notification into the outer header of each new
encapsulating header that supports it.
Reasoning: The constraints from the three perspectives of For instance, RFC4301-compatible IPsec tunnel endpoints invariably
security, control and management in Appendix A are somewhat in use IKEv2 [RFC4306] for key exchange, which was introduced alongside
tension as to whether a tunnel ingress should copy congestion RFC4301. Therefore both endpoints of an RFC4301 tunnel can be sure
markings into the outer header it creates or reset them. From the that the other end is RFC4301-compatible, because the tunnel is only
control perspective either copying or resetting works for existing formed after IKEv2 key management has completed, at which point both
arrangements, but copying has more potential for simplifying ends will be RFC4301-compliant by definition. Further, an RFC4301
control. From the management perspective copying is preferable. encapsulator behaves identically to the normal mode of the present
From the security perspective resetting is preferable but copying specification and does not need to implement compatibility mode as it
is now considered acceptable given the bandwidth of a 2-bit covert will never interact with legacy ECN tunnels.
channel can be managed. Therefore, on balance, copying is simpler
and more useful than resetting and does minimal harm.
o For any new scheme, a tunnel egress should not forward any ECN Dynamic Discovery: This specification does not require or recommend
codepoint if the arriving inner header implies the transport will dynamic discovery and it does not define how dynamic negotiation
not understand how to process it. might be done, but it recognises that proprietary tunnel endpoint
discovery protocols exist. It therefore sets down some constraints
on discovery protocols to ensure safe interworking.
o On decapsulation in any new scheme, if a combination of inner and If dynamic tunnel endpoint discovery might pair an ingress with a
outer headers is encountered that should not have been possible, legacy egress (RFC2003, RFC2401 or RFC2481 or the limited
this event should be logged and an alarm raised. But the packet functionality mode of RFC3168), the ingress MUST implement both
should still be forwarded with a safe codepoint setting if at all normal and compatibility mode. If the tunnel discovery process is
possible. This increases the chances of 'forward compatibility' arranged to only ever find a tunnel egress that propagates ECN
with possible future protocol extensions. (RFC3168 full functionality mode, RFC4301 or this present
specification), then a tunnel ingress can be complaint with the
present specification without implementing compatibility mode.
o On decapsulation in any new scheme, the ECN field that the tunnel If a compliant tunnel ingress is discovering an egress, it MUST send
egress forwards should reflect the more severe congestion marking packets in compatibility mode in case the egress it discovers is a
of the arriving inner and outer headers. legacy egress. If, through the discovery protocol, the egress
indicates that it is compliant with the present specification, with
RFC4301 or with RFC3168 full functionality mode, the ingress can
switch itself into normal mode. If the egress denies compliance with
any of these or returns an error that implies it does not understand
a request to work to any of these ECN specifications, the tunnel
ingress MUST remain in compatibility mode.
5. Backward Compatibility An ingress cannot claim compliance with this specification simply by
disabling ECN processing across the tunnel (i.e. only implementing
compatibility mode). It is true that such a tunnel ingress is at
least safe with the ECN behaviour of any egress it may encounter, but
it does not meet the aim of introducing ECN support to tunnels.
Note: in RFC3168, a whole tunnel was considered in one of two modes: Implementation note: if a compliant node is the ingress for multiple
limited functionality or full functionality. The new modes defined tunnels, a mode setting will need to be stored for each tunnel
in this specification are only modes of the tunnel ingress. The new ingress. However, if a node is the egress for multiple tunnels, none
tunnel egress behaviour has only one mode and doesn't need to know of the tunnels will need to store a mode setting, because a compliant
what mode the ingress is in. egress can only be in one mode.
5.1. Non-Issues Upgrading Any Tunnel Decapsulation 4.4. Single Mode of Decapsulation
This specification only changes the egress per-packet calculation of A compliant decapsulator only has one mode of operation. However, if
the ECN field for combinations of inner and outer headers that have a complaint egress is implemented to be dynamically discoverable, it
so far not been used in any IETF protocols. Therefore, a tunnel may need to respond to discovery requests from various types of
egress complying with any previous specification (RFC4301, both modes legacy tunnel ingress. This specification does not define how
of RFC3168, both modes of RFC2481, RFC2401 and RFC2003) can be dynamic negotiation might be done by (proprietary) discovery
upgraded to comply with this new decapsulation specification without protocols, but it sets down some constraints to ensure safe
any backwards compatibility issues. interworking.
The proposed tunnel egress behaviour also requires no additional mode Through the discovery protocol, a tunnel ingress compliant with the
or option configuration at the ingress or egress nor any additional present specification might ask if the egress is compliant with the
negotiation with the ingress. A compliant tunnel egress merely needs present specification, with RFC4301 or with RFC3168 full
to implement the one behaviour in Section 4. The reduction to one functionality mode. Or an RFC3168 tunnel ingress might try to
mode at the egress has no backwards compatibility issues, because negotiate to use limited functionality or full functionality mode
previously the egress produced the same output whichever mode the [RFC3168]. In all these cases, a decapsulating tunnel egress
tunnel was in. compliant with this specification MUST agree to any of these
requests, since it will behave identically in all these cases.
These new decapsulation rules have been defined in such a way that If no ECN-related mode is requested, a compliant tunnel egress MUST
congestion control will still work safely if any of the earlier continue without raising any error or warning as its egress behaviour
versions of ECN processing are used unilaterally at the encapsulating is compatible with all the legacy ingress behaviours that do not
ingress of the tunnel (any of RFC2003, RFC2401, either mode of negotiate capabilities.
RFC2481, either mode of RFC3168, RFC4301 and this present
specification). If a tunnel ingress tries to negotiate to use
limited functionality mode or full functionality mode [RFC3168], a
decapsulating tunnel egress compliant with this specification MUST
agree to either request, as its behaviour will be the same in both
cases.
For 'forward compatibility', a compliant tunnel egress SHOULD raise a For 'forward compatibility', a compliant tunnel egress SHOULD raise a
warning about any requests to enter modes it doesn't recognise, but warning alarm about any requests to enter modes it does not
it can continue operating. If no ECN-related mode is requested, a recognise, but it SHOULD continue operating.
compliant tunnel egress can continue without raising any error or
warning as its egress behaviour is compatible with all the legacy
ingress behaviours that don't negotiate capabilities.
5.2. Non-Issues for RFC4301 IPsec Encapsulation 5. Changes from Earlier RFCs
The new normal mode of ingress behaviour defined above (Section 4.1) 5.1. Changes to RFC4301 ECN processing
brings all IP in IP tunnels into line with [RFC4301]. If one end of
an IPsec tunnel is compliant with [RFC4301], the other end is
guaranteed to also be RFC4301-compliant (there could be corner cases
where manual keying is used, but they will be set aside here).
Therefore the new normal ingress behaviour introduces no backward
compatibility isses with IKEv2 [RFC4306] IPsec [RFC4301] tunnels, and
no need for any new modes, options or configuration.
5.3. Upgrading Other IP in IP Tunnel Encapsulators Ingress: An RFC4301 IPsec encapsulator is not changed at all by the
present specification
At the tunnel ingress, this specification effectively extends the Egress: The new decapsulation behaviour in Figure 4 updates RFC4301.
scope of RFC4301's ingress behaviour to any IP in IP tunnel. If any However, it solely updates combinations of inner and outer that
other IP in IP tunnel ingress (i.e. not RFC4301 IPsec) is upgraded to have never been used on the Internet, even though they were
be compliant with this specification, it has to cater for the defined in RFC4301 for completeness. Therefore, the present
possibility that it is talking to a legacy tunnel egress that may not specification adds new behaviours to RFC4301 decapsulation without
know how to process the ECN field. If ECN capable outer headers were altering existing behaviours. The following specific updates have
sent towards a legacy (e.g. [RFC2003]) egress, it would most likely been made:
simply disregard the outer headers, dangerously discarding
information about congestion experienced within the tunnel. ECN-
capable traffic sources would not see any congestion feedback and
instead continually ratchet up their share of the bandwidth without
realising that cross-flows from other ECN sources were continually
having to ratchet down.
This specification introduces no new backward compatibility issues * The outer, not the inner, is propagated when the outer is
when a compliant ingress talks with a legacy egress, but it has to ECT(1) and the inner is ECT(0);
provide similar sfaeguards to those already defined in RFC3168.
Therefore, to comply with this specification, a tunnel ingress that
does not always know the ECN capability of its tunnel egress MUST
implement a 'normal' mode and a 'compatibility' mode, and for safety
it MUST initiate each negotiated tunnel in compatibility mode.
However, a tunnel ingress can be compliant even if it only implements * A packet with Not-ECT in the inner and an outer of ECT(1) or CE
the 'normal mode' of encapsulation behaviour, but only as long as it is dropped rather than forwarded as Not-ECT;
is designed or configured so that all possible tunnel egress nodes it
will ever talk to will have at least full ECN functionality
(complying with either RFC3168 full functionality mode, RFC4301 or
this present specification).
Before switching to normal mode, a compliant tunnel ingress that does * Certain combinations of inner and outer ECN field have been
not know the egress ECN capability MUST negotiate with the tunnel identified as currently unused. These can trigger logging
egress. If the egress says it is compliant with this specification and/or raise alarms.
or with RFC3168 full functionality mode, the ingress puts itself into
normal mode. If the egress denies compliance with all of these or
doesn't understand the question, the tunnel ingress MUST remain in
compatibility mode.
The encapsulation rules for normal mode and compatibility mode are Modes: RFC4301 does not need modes and is not updated by the modes
defined in Section 4 (i.e. header copying or zeroing respectively). in the present specification. The normal mode of encapsulation is
unchanged from RFC4301 encapsulation and an RFC4301 IPsec ingress
will never need compatibility mode as explained in Section 4.3
(except in one corner-case described below).
One corner case can exist where an RFC4301 ingress does not use
IKEv2, but uses manual keying instead. Then an RFC4301 ingress
could conceivably be configured to tunnel to an egress with
limited functionality ECN handling. Strictly, for this corner-
case, the requirement to use compatibility mode in this
specification updates RFC4301. However, this is such a remote
possibility that in general RFC4301 IPsec implementations are NOT
REQUIRED to implement compatibility mode.
An ingress cannot claim compliance with this specification simply by 5.2. Changes to RFC3168 ECN processing
disabling ECN processing across the tunnel (only implementing
compatibility mode). Although such a tunnel ingress is at least safe
with the ECN behaviour of any egress it may encounter (any of
RFC2003, RFC2401, either mode of RFC2481 and RFC3168's limited
functionality mode), it doesn't meet the aim of introducing ECN.
Therefore, a compliant tunnel ingress MUST at least implement `normal Ingress: On encapsulation, the new rule in Figure 3 that a normal
mode' and, if it might be used with arbitrary tunnel egress nodes, it mode tunnel ingress copies any ECN field into the outer header
MUST also implement `compatibility mode'. updates the ingress behaviour of RFC3168. Nonetheless, the new
compatibility mode is identical to the limited functionality mode
of RFC3168.
Implementation note: if a compliant node is the ingress for multiple Egress: The new decapsulation behaviour in Figure 4 updates RFC3168.
tunnels, a mode setting will need to be stored for each tunnel However, the present specification solely updates combinations of
ingress. However, if a node is the egress for multiple tunnels, none inner and outer that have never been used on the Internet, even
of the tunnels will need to store a mode setting, because a compliant though they were defined in RFC3168 for completeness. Therefore,
egress can only be in one mode. the present specification adds new behaviours to RFC3168
decapsulation without altering existing behaviours. The following
specific updates have been made:
6. Changes from Earlier RFCs * The outer, not the inner, is propagated when the outer is
ECT(1) and the inner is ECT(0);
On encapsulation, the rule that a normal mode tunnel ingress MUST * A packet with Not-ECT in the inner and an outer of ECT(1) is
copy any ECN field into the outer header is a change to the ingress dropped rather than forwarded as Not-ECT;
behaviour of RFC3168, but it is the same as the rules for IPsec
tunnels in RFC4301.
On decapsulation, the rules for calculating the outgoing ECN field at * Certain combinations of inner and outer ECN field have been
a tunnel egress are similar to the full functionality mode of ECN in identified as currently unused. These can trigger logging
RFC3168 and to RFC4301, with the following exceptions: and/or raise alarms.
o The outer, not the inner, is propagated when the outer is ECT(1) Modes: RFC3168 defines a (required) limited functionality mode and
and the inner is ECT(0); an (optional) full functionality mode for a tunnel. In RFC3168,
modes applied to both ends of the tunnel, while in the present
specification, modes are only used at the ingress--a single egress
behaviour covers all cases. The normal mode of encapsulation
updates the encapsulation behaviour of the full functionality mode
of RFC3168. The compatibility mode of encapsulation is identical
to the encapsulation behaviour of the limited functionality mode
of RFC3168. The constraints on how tunnel discovery protocols set
modes in Section 4.3 and Section 4.4 are an update to RFC3168.
o A packet with Not-ECT in the inner may be forwarded as Not-ECT 5.3. Motivation for Changes
rather than dropped, if the outer is ECT(0);
o The following extra illegal combinations have been identified, An overriding goal is to ensure the same ECN signals can mean the
which may require logging and/or an alarm: outer ECT(1) with inner same thing whatever tunnels happen to encapsulate an IP packet flow.
CE; outer ECT(0) with inner ECT(1) This removes gratuitous inconsistency, which otherwise constrains the
available design space and makes it harder to design networks and new
protocols that work predictably.
The rules for how a tunnel establishes whether the egress has full 5.3.1. Motivation for Changing Encapsulation
functionality ECN capabilities are an update to RFC3168. For all the
typical cases, RFC4301 is not updated by the ECN capability check in
this specification, because a typical RFC4301 tunnel ingress will
have already established that it is talking to an RFC4301 tunnel
egress (e.g. if it uses IKEv2). However, there may be some corner
cases (e.g. manual keying) where an RFC4301 tunnel ingress talks with
an egress with limited functionality ECN handling. Strictly, for
such corner cases, the requirement to use compatibility mode in this
specification updates RFC4301, but this is unlikely to be necessary
to implement for this corner case in practice.
The optional ECN Tunnel field in the IPsec security association The normal mode in Section 4 updates RFC3168 to make all IP in IP
database (SAD) and the optional ECN Tunnel Security Association encapsulation of the ECN field consistent--consistent with the way
Attribute defined in RFC3168 are no longer needed. The security both RFC4301 IPsec [RFC4301] and IP in MPLS or MPLS in MPLS
association (SA) has no policy on ECN usage, because all RFC4301 encapsulation [RFC5129] construct the ECN field.
tunnels now support ECN without any policy choice.
RFC3168 defines a (required) limited functionality mode and an Compatibility mode has also been defined so a non-RFC4301 ingress can
(optional) full functionality mode for a tunnel, but RFC4301 doesn't still switch to using drop across a tunnel for backwards
need modes. In this specification only the ingress might need two compatibility with legacy decapsulators that do not propagate ECN
modes: a normal mode (required) and a compatibility mode (required in correctly.
some scenarios, optional in others). The egress needs only one mode
which correctly handles any ingress ECN behaviour.
Additional changes to the RFC Index (to be removed by the RFC Editor): The trigger that motivated this update to RFC3168 encapsulation was a
standards track proposal for pre-congestion notification (PCN
[I-D.ietf-pcn-marking-behaviour]). PCN excess rate marking only
works correctly if the ECN field is copied on encapsulation (as in
RFC4301 and RFC5129); it does not work if ECN is reset (as in
RFC3168). This is because PCN excess rate marking depends on the
outer header revealing any congestion experienced so far on the whole
path, not just since the last tunnel ingress (see Appendix E for a
full explanation).
In the RFC index, RFC3168 should be identified as an update to PCN allows a network operator to add flow admission and termination
RFC2003. RFC4301 should be identified as an update to RFC3168. for inelastic traffic at the edges of a Diffserv domain, but without
any per-flow mechanisms in the interior and without the generous
provisioning typical of Diffserv, aiming to significantly reduce
costs. The PCN architecture [RFC5559] states that RFC3168 IP in IP
tunnelling of the ECN field cannot be used for any tunnel ingress in
a PCN domain. Prior to the present specification, this left a stark
choice between not being able to use PCN for inelastic traffic
control or not being able to use the many tunnels already deployed
for Mobile IP, VPNs and so forth.
This specification updates RFC3168 and RFC4301. The present specification provides a clean solution to this problem,
so that network operators who want to use both PCN and tunnels can
specify that every tunnel ingress in a PCN region must comply with
this latest specification.
7. IANA Considerations Rather than allow tunnel specifications to fragment further into one
for PCN, one for IPsec and one for other tunnels, the opportunity has
been taken to consolidate the diverging specifications back into a
single tunnelling behaviour. Resetting ECN was originally motivated
by a covert channel concern that has been deliberately set aside in
RFC4301 IPsec. Therefore the reset behaviour of RFC3168 is an
anomaly that we do not need to keep. Copying ECN on encapsulation is
anyway simpler than resetting. So, as more tunnel endpoints comply
with this single consistent specification, encapsulation will be
simpler as well as more predictable.
This memo includes no request to IANA. Appendix B assesses whether copying rather than resetting CE on
ingress will cause any unintended side-effects, from the three
perspectives of security, control and management. In summary this
analysis finds that:
8. Security Considerations o From the control perspective either copying or resetting works for
existing arrangements, but copying has more potential for
simplifying control and resetting breaks at least one proposal
already on the standards track.
Appendix A.1 discusses the security constraints imposed on ECN tunnel o From the management and monitoring perspective copying is
processing. The new rules for ECN tunnel processing (Section 4) preferable.
trade-off between security (covert channels) and congestion
monitoring & control. In fact, ensuring congestion markings are not
lost is itself another aspect of security, because if we allowed
congestion notification to be lost, any attempt to enforce a response
to congestion would be much harder.
If alternate congestion notification semantics are defined for a o From the traffic security perspective (enforcing congestion
certain PHB (e.g. the pre-congestion notification architecture control, mitigating denial of service etc) copying is preferable.
[I-D.ietf-pcn-architecture]), the scope of the alternate semantics
might typically be bounded by the limits of a Diffserv region or
regions, as envisaged in [RFC4774]. The inner headers in tunnels
crossing the boundary of such a Diffserv region but ending within the
region can potentially leak the external congestion notification
semantics into the region, or leak the internal semantics out of the
region. [RFC2983] discusses the need for Diffserv traffic
conditioning to be applied at these tunnel endpoints as if they are
at the edge of the Diffserv region. Similar concerns apply to any
processing or propagation of the ECN field at the edges of a Diffserv
region with alternate ECN semantics. Such edge processing must also
be applied at the endpoints of tunnels with one end inside and the
other outside the domain. [I-D.ietf-pcn-architecture] gives specific
advice on this for the PCN case, but other definitions of alternate
semantics will need to discuss the specific security implications in
each case.
With the decapsulation rules as they stood in RFC3168 and RFC4301, a o From the information security perspective resetting is preferable,
small part of the protection of the ECN nonce [RFC3540] was but the IETF Security Area now considers copying acceptable given
compromised. The new decapsulation rules do not solve this problem. the bandwidth of a 2-bit covert channel can be managed.
The minor problem is as follows: The ECN nonce was defined to enable Therefore there are two points against resetting CE on ingress while
the data source to detect if a CE marking had been applied then copying CE causes no harm (other than opening a 2-bit covert channel
subsequently removed. The source could detect this by weaving a that is deemed manageable).
pseudo-random sequence of ECT(0) and ECT(1) values into a stream of
packets, which is termed an ECN nonce. By the decapsulation rules in
RFC3168 and RFC4301, if the inner and outer headers carry
contradictory ECT values only the inner header is preserved for
onward forwarding. So if a CE marking added to the outer ECN field
in a tunnel has been illegally (or accidentally) suppressed by a
subsequent node in the tunnel, the decapsulator will revert the ECN
field to its value before tampering, hiding all evidence of the crime
from the onward feedback loop. We chose not to close this minor
loophole for all the following reasons:
1. This loophole is only applicable in the corner case where the 5.3.2. Motivation for Changing Decapsulation
attacker controls a network node downstream of a congested node
in the same tunnel;
2. In tunnelling scenarios, the ECN nonce is already vulnerable to The specification for decapsulation in Section 4 fixes three problems
suppression by nodes downstream of a congested node in the same with the pre-existing behaviours of both RFC3168 and RFC4301:
tunnel, if they can copy the ECT value in the inner header to the
outer header (any node in the tunnel can do this if the inner
header is not encrypted, and an IPsec tunnel egress can do it
whether or not the tunnel is encrypted);
3. Although the new decapsulation behaviour removes evidence of 1. The pre-existing rules prevented the introduction of alternate
congestion suppression from the onward feedback loop, the ECN semantics to signal more than one severity level of
decapsulator itself can at least detect that congestion within congestion [RFC4774], [RFC5559]. The four states of the 2-bit
the tunnel has been suppressed; ECN field provide room for signalling two severity levels in
addition to not-congested and not-ECN-capable states. But, the
pre-existing rules assumed that two of the states (ECT(0) and
ECT(1)) are always equivalent. This unnecessarily restricts the
use of one of four codepoints (half a bit) in the IP (v4 & v6)
header. The new rules are designed to work in either case;
whether ECT(1) is more severe than or equivalent to ECT(0).
4. The ECN nonce [RFC3540] currently has experimental status and As explained in Appendix B.1, the original reason for not
there has been no evidence that anyone has implemented it beyond forwarding the outer ECT codepoints was to limit the covert
the author's prototype. channel across a decapsulator to 1 bit per packet. However, now
that the IETF Security Area has deemed that a 2-bit covert
channel through an encapsulator is a manageable risk, the same
should be true for a decapsulator.
We could have fixed this loophole by specifying that the outer header As well as being useful for general future-proofing, this problem
should always be propagated onwards if inner and outer are both ECT. is immediately pressing for standardisation of pre-congestion
Although this would close the minor loophole in the nonce, it would notification (PCN), which uses two severity levels of congestion.
raise a minor safety issue if multilevel ECN or PCN were used. A If a congested queue used ECT(1) in the outer header to signal
less severe marking in the inner header would override a more severe more severe congestion than ECT(0), the pre-existing
one in the outer. Both are corner cases so it is difficult to decide decapsulation rules would have thrown away this congestion
which is more important: signal, preventing tunnelled traffic from ever knowing that it
should reduce its load.
1. The loophole in the nonce is only for a minor case of one tunnel The PCN working group has had to consider a number of wasteful or
node attacking another in the same tunnel; convoluted work-rounds to this problem (see Appendix D). But by
far the simplest approach is just to remove the covert channel
blockages from tunnelling behaviour--now deemed unnecessary
anyway. Then network operators that want to support two
congestion severity-levels for PCN can specify that every tunnel
egress in a PCN region must comply with this latest
specification.
2. The severity inversion for multilevel congestion notification Not only does this make two congestion severity-levels available
would not result from any legal codepoint transition. for PCN standardisation, but also for other potential uses of the
extra ECN codepoint (e.g. [VCP]).
We decided safety against misconfiguration was slightly more 2. Cases are documented where a middlebox (e.g. a firewall) drops
important than securing against an attack that has little, if any, packets with header values that were currently unused (CU) when
clear motivation. the box was deployed, often on the grounds that anything
unexpected might be an attack. This tends to bar future use of
CU values. The new decapsulation rules specify optional logging
and/or alarms for specific combinations of inner and outer header
that are currently unused. The aim is to give implementers a
recourse other than drop if they are concerned about the security
of CU values. It recognises legitimate security concerns about
CU values but still eases their future use. If the alarms are
interpreted as an attack (e.g. by a management system) the
offending packets can be dropped. But alarms can be turned off
if these combinations come into use (e.g. a through a future
standards action).
If a legacy security policy configures a legacy tunnel ingress to 3. While reviewing currently unused combinations of inner and outer,
negotiate to turn off ECN processing, a compliant tunnel egress will the opportunity was taken to define a single consistent behaviour
agree to a request to turn off ECN processing but it will actually for the cases with a Not-ECT inner header but a different outer.
still copy CE markings from the outer to the forwarded header. RFC3168 and RFC4301 had diverged in this respect. These
Although the tunnel ingress 'I' in Figure 5 (Appendix A.1) will set combinations should not result from known Internet protocols.
all ECN fields in outer headers to Not-ECT, 'M' could still toggle CE So, for safety, it was decided to drop a packet if the outer
on and off to communicate covertly with 'B', because we have carries codepoints CE or ECT(1) that respectively signal
specified that 'E' only has one mode regardless of what mode it says congestion or could potentially signal congestion in a scheme
it has negotiated. We could have specified that 'E' should have a progressing through the IETF [I-D.ietf-pcn-3-in-1-encoding].
limited functionality mode and check for such behaviour. But we Given an inner of Not-ECT implies the transport only understands
decided not to add the extra complexity of two modes on a compliant drop as a signal of congestion, this was the safest course of
tunnel egress merely to cater for a legacy security concern that is action.
now considered manageable.
9. Conclusions Problems 2 & 3 alone would not warrant a change to decapsulation, but
it was decided they are worth fixing and making consistent at the
same time as decapsulation code is changed to fix problem 1 (two
congestion severity-levels).
This document updates the ingress tunnelling encapsulation of RFC3168 6. Backward Compatibility
ECN for all IP in IP tunnels to bring it into line with the new
behaviour in the IPsec architecture of RFC4301. It copies rather A tunnel endpoint compliant with the present specification is
than resets a congestion experienced (CE) marking when creating outer backward compatible when paired with any tunnel endpoint compliant
with any previous tunnelling RFC, whether RFC4301, RFC3168 (see
Section 3) or the earlier RFCs summarised in Appendix A (RFC2481,
RFC2401 and RFC2003). Each case is enumerated below.
6.1. Non-Issues Updating Decapsulation
At the egress, this specification only augments the per-packet
calculation of the ECN field (RFC3168 and RFC4301) for combinations
of inner and outer headers that have so far not been used in any IETF
protocols.
Therefore, all other things being equal, if an RFC4301 IPsec egress
is updated to comply with the new rules, it will still interwork with
any RFC4301 compliant ingress and the packet outputs will be
identical to those it would have output before (fully backward
compatible).
And, all other things being equal, if an RFC3168 egress is updated to
comply with the same new rules, it will still interwork with any
ingress complying with any previous specification (both modes of
RFC3168, both modes of RFC2481, RFC2401 and RFC2003) and the packet
outputs will be identical to those it would have output before (fully
backward compatible).
A compliant tunnel egress merely needs to implement the one behaviour
in Section 4 with no additional mode or option configuration at the
ingress or egress nor any additional negotiation with the ingress.
The new decapsulation rules have been defined in such a way that
congestion control will still work safely if any of the earlier
versions of ECN processing are used unilaterally at the encapsulating
ingress of the tunnel (any of RFC2003, RFC2401, either mode of
RFC2481, either mode of RFC3168, RFC4301 and this present
specification).
6.2. Non-Update of RFC4301 IPsec Encapsulation
An RFC4301 IPsec ingress can comply with this new specification
without any update and it has no need for any new modes, options or
configuration. So, all other things being equal, it will continue to
interwork identically with any egress it worked with before (fully
backward compatible).
6.3. Update to RFC3168 Encapsulation
The encapsulation behaviour of the new normal mode copies the ECN
field whereas RFC3168 full functionality mode reset it. However, all
other things being equal, if RFC3168 ingress is updated to the
present specification, the outgoing packets from any tunnel egress
will still be unchanged. This is because all variants of tunnelling
at either end (RFC4301, both modes of RFC3168, both modes of RFC2481,
RFC2401, RFC2003 and the present specification) have always
propagated an incoming CE marking through the inner header and onward
into the outgoing header, whether the outer header is reset or
copied. Therefore, If the tunnel is considered as a black box, the
packets output from any egress will be identical with or without an
update to the ingress. Nonetheless, if packets are observed within
the black box (between the tunnel endpoints), CE markings copied by
the updated ingress will be visible within the black box, whereas
they would not have been before. Therefore, the update to
encapsulation can be termed 'black-box backwards compatible' (i.e.
identical unless you look inside the tunnel).
This specification introduces no new backward compatibility issues
when a compliant ingress talks with a legacy egress, but it has to
provide similar safeguards to those already defined in RFC3168.
RFC3168 laid down rules to ensure that an RFC3168 ingress turns off
ECN (limited functionality mode) if it is paired with a legacy egress
(RFC 2481, RFC2401 or RFC2003), which would not propagate ECN
correctly. The present specification carries forward those rules
(Section 4.3). It uses compatibility mode whenever RFC3168 would
have used limited functionality mode, and their per-packet behaviours
are identical. Therefore, all other things being equal, an ingress
using the new rules will interwork with any legacy tunnel egress in
exactly the same way as an RFC3168 ingress (still black-box backward
compatible).
7. Design Principles for Future Non-Default Schemes
This section is informative not normative.
S.5 of RFC3168 permits the Diffserv codepoint (DSCP)[RFC2474] to
'switch in' alternative behaviours for marking the ECN field, just as
it switches in different per-hop behaviours (PHBs) for scheduling.
[RFC4774] gives best current practice for designing such alternative
ECN semantics and very briefly mentions that tunnelling should be
considered. Here we give additional guidance on designing alternate
ECN semantics that would also require alternate tunnelling semantics.
In one word the guidance is "Don't". If a scheme requires tunnels to
implement special processing of the ECN field for certain DSCPs, it
is highly unlikely that every implementer of every tunnel will want
to add the required exception and that operators will want to deploy
the required configuration options. Therefore it is highly likely
that some tunnels within a network will not implement the required
special case. Therefore, designers of new protocols should avoid
non-default tunnelling schemes if at all possible.
That said, if a non-default scheme for tunnelling the ECN field is
really required, the following guidelines may prove useful in its
design:
On encapsulation in any new scheme:
1. The ECN field of the outer header should be cleared to Not-ECT
("00") unless it is guaranteed that the corresponding tunnel
egress will correctly propagate congestion markings introduced
across the tunnel in the outer header.
2. If it has established that ECN will be correctly propagated,
an encapsulator should also copy incoming congestion
notification into the outer header. The general principle
here is that the outer header should reflect congestion
accumulated along the whole upstream path, not just since the
tunnel ingress (Appendix B.3 on management and monitoring
explains).
In some circumstances (e.g. pseudowires, PCN), the whole path
is divided into segments, each with its own congestion
notification and feedback loop. In these cases, the function
that regulates load at the start of each segment will need to
reset congestion notification for its segment. Often the
point where congestion notification is reset will also be
located at the start of a tunnel. However, the resetting
function should be thought of as being applied to packets
after the encapsulation function--two logically separate
functions even though they might run on the same physical box.
Then the code module doing encapsulation can keep to the
copying rule and the load regulator module can reset
congestion, without any code in either module being
conditional on whether the other is there.
On decapsulation in any new scheme:
1. If the arriving inner header is Not-ECT it implies the
transport will not understand other ECN codepoints. If the
outer header carries an explicit congestion marking, the
packet should be dropped--the only indication of congestion
the transport will understand. If the outer carries any other
ECN codepoint the packet can be forwarded, but only as Not-
ECT.
2. If the arriving inner header is other than Not-ECT, the ECN
field that the tunnel egress forwards should reflect the more
severe congestion marking of the arriving inner and outer
headers. headers.
It also specifies new rules that update both RFC3168 and RFC4301 for 3. If a combination of inner and outer headers is encountered
calculating the outgoing ECN field on tunnel decapsulation. The new that is not currently used in known standards, this event
rules update egress behaviour for two specific combinations of inner should be logged and an alarm raised. This is a preferable
and outer header that have no current legal usage, but will now be approach to dropping currently unused combinations in case
possible to use in future standards actions, rather than being wasted they represent an attack. The new scheme should try to define
by current tunnelling behaviour. a way to forward such packets, but only if a safe outgoing
codepoint can be defined.
The new rules propagate changes to the ECN field across tunnel end- 8. IANA Considerations
points that were previously blocked due to a perceived covert channel
vulnerability. The new IPsec architecture deems the two-bit covert
channel that the ECN field opens up is a manageable threat, so these
new rules bring all IP in IP tunnelling into line with this new more
permissive attitude. The result is a single specification for all
future tunnelling of ECN, whether IPsec or not. Then equipment can
be specified against a single ECN behaviour and ECN markings can have
a well-defined meaning wherever they are measured in a network. This
new certainty will enable new uses of the ECN field that would
otherwise be confounded by ambiguity.
The immediate motivation for making these changes is to allow the This memo includes no request to IANA.
introduction of multi-level pre-congestion notification (PCN). But
great care has been taken to ensure the resulting ECN tunnelling
behaviour is simple and generic for other potential future uses.
The change to encapsulation has been analysed from the three 9. Security Considerations
perspectives of security, control and management. They are somewhat
in tension as to whether a tunnel ingress should copy congestion
markings into the outer header it creates or reset them. From the
control perspective either copying or resetting works for existing
arrangements, but copying has more potential for simplifying control
and resetting breaks at least one proposal already on the standards
track. From the management and monitoring perspective copying is
preferable. From the network security perspective (theft of service
etc) copying is preferable. From the information security
perspective resetting is preferable, but the IETF Security Area now
considers copying acceptable given the bandwidth of a 2-bit covert
channel can be managed. Therefore there are no points against
copying and a number against resetting CE on ingress.
The only downside of the changes to decapsulation is that the same Appendix B.1 discusses the security constraints imposed on ECN tunnel
2-bit covert channel is opened up as at the ingress, but this is now processing. The new rules for ECN tunnel processing (Section 4)
deemed to be a manageable threat. The changes at decapsulation have trade-off between information security (covert channels) and
been found to be free of any backwards compatibility issues. congestion monitoring & control. In fact, ensuring congestion
markings are not lost is itself another aspect of security, because
if we allowed congestion notification to be lost, any attempt to
enforce a response to congestion would be much harder.
10. Acknowledgements Specialist security issues:
Tunnels intersecting Diffserv regions with alternate ECN semantics:
If alternate congestion notification semantics are defined for a
certain Diffserv PHB, the scope of the alternate semantics might
typically be bounded by the limits of a Diffserv region or
regions, as envisaged in [RFC4774] (e.g. the pre-congestion
notification architecture [RFC5559]). The inner headers in
tunnels crossing the boundary of such a Diffserv region but ending
within the region can potentially leak the external congestion
notification semantics into the region, or leak the internal
semantics out of the region. [RFC2983] discusses the need for
Diffserv traffic conditioning to be applied at these tunnel
endpoints as if they are at the edge of the Diffserv region.
Similar concerns apply to any processing or propagation of the ECN
field at the edges of a Diffserv region with alternate ECN
semantics. Such edge processing must also be applied at the
endpoints of tunnels with one end inside and the other outside the
domain. [RFC5559] gives specific advice on this for the PCN case,
but other definitions of alternate semantics will need to discuss
the specific security implications in each case.
ECN nonce tunnel coverage: The new decapsulation rules improve the
coverage of the ECN nonce [RFC3540] relative to the previous rules
in RFC3168 and RFC4301. However, nonce coverage is still not
perfect, as this would have led to a safety problem in another
case. Both are corner-cases, so discussion of the compromise
between them is deferred to Appendix F.
Covert channel not turned off: A legacy (RFC3168) tunnel ingress
could ask an RFC3168 egress to turn off ECN processing as well as
itself turning off ECN. An egress compliant with the present
specification will agree to such a request from a legacy ingress,
but it relies on the ingress solely sending Not-ECT in the outer.
If the egress receives other ECN codepoints in the outer it will
process them as normal, so it will actually still copy congestion
markings from the outer to the outgoing header. Referring for
example to Figure 5 (Appendix B.1), although the tunnel ingress
'I' will set all ECN fields in outer headers to Not-ECT, 'M' could
still toggle CE or ECT(1) on and off to communicate covertly with
'B', because we have specified that 'E' only has one mode
regardless of what mode it says it has negotiated. We could have
specified that 'E' should have a limited functionality mode and
check for such behaviour. But we decided not to add the extra
complexity of two modes on a compliant tunnel egress merely to
cater for an historic security concern that is now considered
manageable.
10. Conclusions
This document uses previously unused combinations of inner and outer
header to augment the rules for calculating the ECN field when
decapsulating IP packets at the egress of IPsec (RFC4301) and non-
IPsec (RFC3168) tunnels. In this way it allows tunnels to propagate
an extra level of congestion severity.
This document also updates the ingress tunnelling encapsulation of
RFC3168 ECN to bring all IP in IP tunnels into line with the new
behaviour in the IPsec architecture of RFC4301, which copies rather
than resets the ECN field when creating outer headers.
The need for both these updated behaviours was triggered by the
introduction of pre-congestion notification (PCN) onto the IETF
standards track. Operators wanting to support PCN or other alternate
ECN schemes that use an extra severity level can require that their
tunnels comply with the present specification. Nonetheless, as part
of general code maintenance, any tunnel can safely be updated to
comply with this specification, because it is backward compatible
with all previous tunnelling behaviours which will continue to work
as before--just using one severity level.
The new rules propagate changes to the ECN field across tunnel end-
points that previously blocked them to restrict the bandwidth of a
potential covert channel. But limiting the channel's bandwidth to 2
bits per packet is now considered sufficient.
At the same time as removing these legacy constraints, the
opportunity has been taken to draw together diverging tunnel
specifications into a single consistent behaviour. Then any tunnel
can be deployed unilaterally, and it will support the full range of
congestion control and management schemes without any modes or
configuration. Further, any host or router can expect the ECN field
to behave in the same way, whatever type of tunnel might intervene in
the path. This new certainty could enable new uses of the ECN field
that would otherwise be confounded by ambiguity.
11. Acknowledgements
Thanks to Anil Agawaal for pointing out a case where it's safe for a Thanks to Anil Agawaal for pointing out a case where it's safe for a
tunnel decapsulator to forward a combination of headers it doesn't tunnel decapsulator to forward a combination of headers it does not
understand. Thanks to David Black for explaining a better way to understand. Thanks to David Black for explaining a better way to
think about function placement and to Louise Burness for a better way think about function placement. Also thanks to Arnaud Jacquet for
to think about multilayer transports and networks, having read the idea for Appendix C. Thanks to Michael Menth, Bruce Davie, Toby
[Patterns_Arch]. Also thanks to Arnaud Jacquet for the idea for Moncaster, Gorry Fairhurst, Sally Floyd, Alfred Hoenes, Gabriele
Appendix C. Thanks to Michael Menth, Bruce Davie, Toby Moncaster, Corliano, Ingemar Johansson and David Black for their thoughts and
Gorry Fairhurst, Sally Floyd, Alfred Hoenes and Gabriele Corliano for careful review comments.
their thoughts and careful review comments.
Bob Briscoe is partly funded by Trilogy, a research project (ICT- Bob Briscoe is partly funded by Trilogy, a research project (ICT-
216372) supported by the European Community under its Seventh 216372) supported by the European Community under its Seventh
Framework Programme. The views expressed here are those of the Framework Programme. The views expressed here are those of the
author only. author only.
11. Comments Solicited 12. Comments Solicited
Comments and questions are encouraged and very welcome. They can be Comments and questions are encouraged and very welcome. They can be
addressed to the IETF Transport Area working group mailing list addressed to the IETF Transport Area working group mailing list
<tsvwg@ietf.org>, and/or to the authors. <tsvwg@ietf.org>, and/or to the authors.
12. References 13. References
12.1. Normative References
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC 2003,
October 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black,
"Definition of the Differentiated Services Field (DS
Field) in the IPv4 and IPv6 Headers", RFC 2474,
December 1998.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 13.1. Normative References
of Explicit Congestion Notification (ECN) to IP",
RFC 3168, September 2001.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the [RFC2003] Perkins, C., "IP Encapsulation
Internet Protocol", RFC 4301, December 2005. within IP", RFC 2003, October 1996.
12.2. Informative References [RFC2119] Bradner, S., "Key words for use in
RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119,
March 1997.
[I-D.briscoe-pcn-3-in-1-encoding] [RFC3168] Ramakrishnan, K., Floyd, S., and D.
Briscoe, B., "PCN 3-State Encoding Extension in a single Black, "The Addition of Explicit
DSCP", draft-briscoe-pcn-3-in-1-encoding-00 (work in Congestion Notification (ECN) to
progress), October 2008. IP", RFC 3168, September 2001.
[I-D.charny-pcn-single-marking] [RFC4301] Kent, S. and K. Seo, "Security
Charny, A., Zhang, X., Faucheur, F., and V. Liatsos, "Pre- Architecture for the Internet
Congestion Notification Using Single Marking for Admission Protocol", RFC 4301, December 2005.
and Termination", draft-charny-pcn-single-marking-03
(work in progress), November 2007.
[I-D.ietf-pcn-architecture] 13.2. Informative References
Eardley, P., "Pre-Congestion Notification (PCN)
Architecture", draft-ietf-pcn-architecture-10 (work in
progress), March 2009.
[I-D.ietf-pcn-baseline-encoding] [I-D.ietf-pcn-3-in-1-encoding] Briscoe, B. and T. Moncaster, "PCN
Moncaster, T., Briscoe, B., and M. Menth, "Baseline 3-State Encoding Extension in a
Encoding and Transport of Pre-Congestion Information", single DSCP",
draft-ietf-pcn-baseline-encoding-02 (work in progress), draft-ietf-pcn-3-in-1-encoding-00
February 2009. (work in progress), July 2009.
[I-D.ietf-pcn-marking-behaviour] [I-D.ietf-pcn-3-state-encoding] Moncaster, T., Briscoe, B., and M.
Eardley, P., "Marking behaviour of PCN-nodes", Menth, "A PCN encoding using 2
draft-ietf-pcn-marking-behaviour-02 (work in progress), DSCPs to provide 3 or more states",
March 2009. draft-ietf-pcn-3-state-encoding-00
(work in progress), April 2009.
[I-D.ietf-pwe3-congestion-frmwk] [I-D.ietf-pcn-baseline-encoding] Moncaster, T., Briscoe, B., and M.
Bryant, S., Davie, B., Martini, L., and E. Rosen, Menth, "Baseline Encoding and
"Pseudowire Congestion Control Framework", Transport of Pre-Congestion
draft-ietf-pwe3-congestion-frmwk-01 (work in progress), Information",
May 2008. draft-ietf-pcn-baseline-encoding-04
(work in progress), May 2009.
[I-D.menth-pcn-psdm-encoding] [I-D.ietf-pcn-marking-behaviour] Eardley, P., "Metering and marking
Menth, M., Babiarz, J., Moncaster, T., and B. Briscoe, behaviour of PCN-nodes",
"PCN Encoding for Packet-Specific Dual Marking (PSDM)", draft-ietf-pcn-marking-behaviour-04
draft-menth-pcn-psdm-encoding-00 (work in progress), (work in progress), June 2009.
July 2008.
[I-D.moncaster-pcn-3-state-encoding] [I-D.ietf-pcn-psdm-encoding] Menth, M., Babiarz, J., Moncaster,
Moncaster, T., Briscoe, B., and M. Menth, "A three state T., and B. Briscoe, "PCN Encoding
extended PCN encoding scheme", for Packet-Specific Dual Marking
draft-moncaster-pcn-3-state-encoding-01 (work in (PSDM)",
progress), March 2009. draft-ietf-pcn-psdm-encoding-00
(work in progress), June 2009.
[I-D.satoh-pcn-st-marking] [I-D.ietf-pcn-sm-edge-behaviour] Charny, A., Karagiannis, G., Menth,
Satoh, D., Maeda, Y., Phanachet, O., and H. Ueno, "Single M., and T. Taylor, "PCN Boundary
PCN Threshold Marking by using PCN baseline encoding for Node Behaviour for the Single
both admission and termination controls", Marking (SM) Mode of Operation",
draft-satoh-pcn-st-marking-01 (work in progress), draft-ietf-pcn-sm-edge-behaviour-00
March 2009. (work in progress), July 2009.
[IEEE802.1au] [I-D.satoh-pcn-st-marking] Satoh, D., Maeda, Y., Phanachet,
IEEE, "IEEE Standard for Local and Metropolitan Area O., and H. Ueno, "Single PCN
Networks--Virtual Bridged Local Area Networks - Amendment Threshold Marking by using PCN
10: Congestion Notification", 2008, baseline encoding for both
<http://www.ieee802.org/1/pages/802.1au.html>. admission and termination
controls",
draft-satoh-pcn-st-marking-01 (work
in progress), March 2009.
(Work in Progress; Access Controlled link within page) [RFC2401] Kent, S. and R. Atkinson, "Security
Architecture for the Internet
Protocol", RFC 2401, November 1998.
[ITU-T.I.371] [RFC2474] Nichols, K., Blake, S., Baker, F.,
ITU-T, "Traffic Control and Congestion Control in B-ISDN", and D. Black, "Definition of the
ITU-T Rec. I.371 (03/04), March 2004. Differentiated Services Field (DS
Field) in the IPv4 and IPv6
Headers", RFC 2474, December 1998.
[PCNcharter] [RFC2481] Ramakrishnan, K. and S. Floyd, "A
IETF, "Congestion and Pre-Congestion Notification (pcn)", Proposal to add Explicit Congestion
IETF w-g charter , Feb 2007, Notification (ECN) to IP",
<http://www.ietf.org/html.charters/pcn-charter.html>. RFC 2481, January 1999.
[Patterns_Arch] [RFC2983] Black, D., "Differentiated Services
Day, J., "Patterns in Network Architecture: A Return to and Tunnels", RFC 2983,
Fundamentals", Pub: Prentice Hall ISBN-13: 9780132252423, October 2000.
Jan 2008.
[RFC1254] Mankin, A. and K. Ramakrishnan, "Gateway Congestion [RFC3540] Spring, N., Wetherall, D., and D.
Control Survey", RFC 1254, August 1991. Ely, "Robust Explicit Congestion
Notification (ECN) Signaling with
Nonces", RFC 3540, June 2003.
[RFC2205] Braden, B., Zhang, L., Berson, S., Herzog, S., and S. [RFC4306] Kaufman, C., "Internet Key Exchange
Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 (IKEv2) Protocol", RFC 4306,
Functional Specification", RFC 2205, September 1997. December 2005.
[RFC2983] Black, D., "Differentiated Services and Tunnels", [RFC4774] Floyd, S., "Specifying Alternate
RFC 2983, October 2000. Semantics for the Explicit
Congestion Notification (ECN)
Field", BCP 124, RFC 4774,
November 2006.
[RFC3426] Floyd, S., "General Architectural and Policy [RFC5129] Davie, B., Briscoe, B., and J. Tay,
Considerations", RFC 3426, November 2002. "Explicit Congestion Marking in
MPLS", RFC 5129, January 2008.
[RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit [RFC5559] Eardley, P., "Pre-Congestion
Congestion Notification (ECN) Signaling with Nonces", Notification (PCN) Architecture",
RFC 3540, June 2003. RFC 5559, June 2009.
[RFC4306] Kaufman, C., "Internet Key Exchange (IKEv2) Protocol", [VCP] Xia, Y., Subramanian, L., Stoica,
RFC 4306, December 2005. I., and S. Kalyanaraman, "One more
bit is enough", Proc. SIGCOMM'05,
ACM CCR 35(4)37--48, 2005, <http://
doi.acm.org/10.1145/
1080091.1080098>.
[RFC4423] Moskowitz, R. and P. Nikander, "Host Identity Protocol Appendix A. Early ECN Tunnelling RFCs
(HIP) Architecture", RFC 4423, May 2006.
[RFC4774] Floyd, S., "Specifying Alternate Semantics for the IP in IP tunnelling was originally defined in [RFC2003]. On
Explicit Congestion Notification (ECN) Field", BCP 124, encapsulation, the incoming header was copied to the outer and on
RFC 4774, November 2006. decapsulation the outer was simply discarded. Initially, IPsec
tunnelling [RFC2401] followed the same behaviour.
[RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion When ECN was introduced experimentally in [RFC2481], legacy (RFC2003
Marking in MPLS", RFC 5129, January 2008. or RFC2401) tunnels would have discarded any congestion markings
added to the outer header, so RFC2481 introduced rules for
calculating the outgoing header from a combination of the inner and
outer on decapsulation. RC2481 also introduced a second mode for
IPsec tunnels, which turned off ECN processing in the outer header
(Not-ECT) on encapsulation because an RFC2401 decapsulator would
discard the outer on decapsulation. For RFC2401 IPsec this had the
side-effect of completely blocking the covert channel.
[Shayman] "Using ECN to Signal Congestion Within an MPLS Domain", In RFC2481 the ECN field was defined as two separate bits. But when
2000, <http://www.ee.umd.edu/~shayman/papers.d/ ECN moved from the experimental to the standards track [RFC3168], the
draft-shayman-mpls-ecn-00.txt>. ECN field was redefined as four codepoints. This required a
different calculation of the ECN field from that used in RFC2481 on
decapsulation. RFC3168 also had two modes; a 'full functionality
mode' that restricted the covert channel as much as possible but
still allowed ECN to be used with IPsec, and another that completely
turned off ECN processing across the tunnel. This 'limited
functionality mode' both offered a way for operators to completely
block the covert channel and allowed an RFC3168 ingress to interwork
with a legacy tunnel egress (RFC2481, RFC2401 or RFC2003).
(Expired) The present specification includes a similar compatibility mode to
interwork safely with tunnels compliant with any of these three
earlier RFCs. However, unlike RFC3168, it is only a mode of the
ingress, as decapsulation behaviour is the same in either case.
Appendix A. Design Constraints Appendix B. Design Constraints
Tunnel processing of a congestion notification field has to meet Tunnel processing of a congestion notification field has to meet
congestion control and management needs without creating new congestion control and management needs without creating new
information security vulnerabilities (if information security is information security vulnerabilities (if information security is
required). This appendix documents the analysis of the tradeoffs required). This appendix documents the analysis of the tradeoffs
between these factors that led to the new encapsulation rules in between these factors that led to the new encapsulation rules in
Section 4.1. Section 4.1.
A.1. Security Constraints B.1. Security Constraints
Information security can be assured by using various end to end Information security can be assured by using various end to end
security solutions (including IPsec in transport mode [RFC4301]), but security solutions (including IPsec in transport mode [RFC4301]), but
a commonly used scenario involves the need to communicate between two a commonly used scenario involves the need to communicate between two
physically protected domains across the public Internet. In this physically protected domains across the public Internet. In this
case there are certain management advantages to using IPsec in tunnel case there are certain management advantages to using IPsec in tunnel
mode solely across the publicly accessible part of the path. The mode solely across the publicly accessible part of the path. The
path followed by a packet then crosses security 'domains'; the ones path followed by a packet then crosses security 'domains'; the ones
protected by physical or other means before and after the tunnel and protected by physical or other means before and after the tunnel and
the one protected by an IPsec tunnel across the otherwise unprotected the one protected by an IPsec tunnel across the otherwise unprotected
skipping to change at page 29, line 15 skipping to change at page 33, line 22
allowed a covert channel from 'A' to M that bypasses its encryption allowed a covert channel from 'A' to M that bypasses its encryption
of the inner header. And if 'E' copies these fields from the outer of the inner header. And if 'E' copies these fields from the outer
header to the inner, even if it validates authentication from 'I', it header to the inner, even if it validates authentication from 'I', it
will have allowed a covert channel from 'M' to 'B'. will have allowed a covert channel from 'M' to 'B'.
ECN at the IP layer is designed to carry information about congestion ECN at the IP layer is designed to carry information about congestion
from a congested resource towards downstream nodes. Typically a from a congested resource towards downstream nodes. Typically a
downstream transport might feed the information back somehow to the downstream transport might feed the information back somehow to the
point upstream of the congestion that can regulate the load on the point upstream of the congestion that can regulate the load on the
congested resource, but other actions are possible (see [RFC3168] congested resource, but other actions are possible (see [RFC3168]
S.6). In terms of the above unicast scenario, ECN is typically S.6). In terms of the above unicast scenario, ECN effectively
intended to create an information channel from 'M' to 'B' (for 'B' to intends to create an information channel (for congestion signalling)
feed back to 'A'). Therefore the goals of IPsec and ECN are mutually from 'M' to 'B' (for 'B' to feed back to 'A'). Therefore the goals
incompatible. of IPsec and ECN are mutually incompatible.
With respect to the DS or ECN fields, S.5.1.2 of RFC4301 says, With respect to the DS or ECN fields, S.5.1.2 of RFC4301 says,
"controls are provided to manage the bandwidth of this [covert] "controls are provided to manage the bandwidth of this [covert]
channel". Using the ECN processing rules of RFC4301, the channel channel". Using the ECN processing rules of RFC4301, the channel
bandwidth is two bits per datagram from 'A' to 'M' and one bit per bandwidth is two bits per datagram from 'A' to 'M' and one bit per
datagram from 'M' to 'A' (because 'E' limits the combinations of the datagram from 'M' to 'A' (because 'E' limits the combinations of the
2-bit ECN field that it will copy). In both cases the covert channel 2-bit ECN field that it will copy). In both cases the covert channel
bandwidth is further reduced by noise from any real congestion bandwidth is further reduced by noise from any real congestion
marking. RFC4301 therefore implies that these covert channels are marking. RFC4301 implies that these covert channels are sufficiently
sufficiently limited to be considered a manageable threat. However, limited to be considered a manageable threat. However, with respect
with respect to the larger (6b) DS field, the same section of RFC4301 to the larger (6b) DS field, the same section of RFC4301 says not
says not copying is the default, but a configuration option can allow copying is the default, but a configuration option can allow copying
copying "to allow a local administrator to decide whether the covert "to allow a local administrator to decide whether the covert channel
channel provided by copying these bits outweighs the benefits of provided by copying these bits outweighs the benefits of copying".
copying". Of course, an administrator considering copying of the DS Of course, an administrator considering copying of the DS field has
field has to take into account that it could be concatenated with the to take into account that it could be concatenated with the ECN field
ECN field giving an 8b per datagram covert channel. giving an 8b per datagram covert channel.
Thus, for tunnelling the 6b Diffserv field two conceptual models have For tunnelling the 6b Diffserv field two conceptual models have had
had to be defined so that administrators can trade off security to be defined so that administrators can trade off security against
against the needs of traffic conditioning [RFC2983]: the needs of traffic conditioning [RFC2983]:
The uniform model: where the DIffserv field is preserved end-to-end The uniform model: where the Diffserv field is preserved end-to-end
by copying into the outer header on encapsulation and copying from by copying into the outer header on encapsulation and copying from
the outer header on decapsulation. the outer header on decapsulation.
The pipe model: where the outer header is independent of that in the The pipe model: where the outer header is independent of that in the
inner header so it hides the Diffserv field of the inner header inner header so it hides the Diffserv field of the inner header
from any interaction with nodes along the tunnel. from any interaction with nodes along the tunnel.
However, for ECN, the new IPsec security architecture in RFC4301 only However, for ECN, the new IPsec security architecture in RFC4301 only
standardised one tunnelling model equivalent to the uniform model. standardised one tunnelling model equivalent to the uniform model.
It deemed that simplicity was more important than allowing It deemed that simplicity was more important than allowing
administrators the option of a tiny increment in security, especially administrators the option of a tiny increment in security, especially
given not copying congestion indications could seriously harm given not copying congestion indications could seriously harm
everyone's network service. everyone's network service.
A.2. Control Constraints B.2. Control Constraints
Congestion control requires that any congestion notification marked Congestion control requires that any congestion notification marked
into packets by a resource will be able to traverse a feedback loop into packets by a resource will be able to traverse a feedback loop
back to a function capable of controlling the load on that resource. back to a function capable of controlling the load on that resource.
To be precise, rather than calling this function the data source, we To be precise, rather than calling this function the data source, we
will call it the Load Regulator. This will allow us to deal with will call it the Load Regulator. This will allow us to deal with
exceptional cases where load is not regulated by the data source, but exceptional cases where load is not regulated by the data source, but
usually the two terms will be synonymous. Note the term "a function usually the two terms will be synonymous. Note the term "a function
_capable of_ controlling the load" deliberately includes a source _capable of_ controlling the load" deliberately includes a source
application that doesn't actually control the load but ought to (e.g. application that doesn't actually control the load but ought to (e.g.
skipping to change at page 30, line 33 skipping to change at page 34, line 41
Figure 6: Simple Tunnel Scenario Figure 6: Simple Tunnel Scenario
We now consider a similar tunnelling scenario to the IPsec one just We now consider a similar tunnelling scenario to the IPsec one just
described, but without the different security domains so we can just described, but without the different security domains so we can just
focus on ensuring the control loop and management monitoring can work focus on ensuring the control loop and management monitoring can work
(Figure 6). If we want resources in the tunnel to be able to (Figure 6). If we want resources in the tunnel to be able to
explicitly notify congestion and the feedback path is from 'B' to explicitly notify congestion and the feedback path is from 'B' to
'A', it will certainly be necessary for 'E' to copy any CE marking 'A', it will certainly be necessary for 'E' to copy any CE marking
from the outer header to the inner header for onward transmission to from the outer header to the inner header for onward transmission to
'B', otherwise congestion notification from resources like 'M' cannot 'B', otherwise congestion notification from resources like 'M' cannot
be fed back to the Load Regulator ('A'). But it doesn't seem be fed back to the Load Regulator ('A'). But it does not seem
necessary for 'I' to copy CE markings from the inner to the outer necessary for 'I' to copy CE markings from the inner to the outer
header. For instance, if resource 'R' is congested, it can send header. For instance, if resource 'R' is congested, it can send
congestion information to 'B' using the congestion field in the inner congestion information to 'B' using the congestion field in the inner
header without 'I' copying the congestion field into the outer header header without 'I' copying the congestion field into the outer header
and 'E' copying it back to the inner header. 'E' can still write any and 'E' copying it back to the inner header. 'E' can still write any
additional congestion marking introduced across the tunnel into the additional congestion marking introduced across the tunnel into the
congestion field of the inner header. congestion field of the inner header.
It might be useful for the tunnel egress to be able to tell whether It might be useful for the tunnel egress to be able to tell whether
congestion occurred across a tunnel or upstream of it. If outer congestion occurred across a tunnel or upstream of it. If outer
skipping to change at page 31, line 17 skipping to change at page 35, line 24
gives a simple and precise method for a tunnel egress to infer the gives a simple and precise method for a tunnel egress to infer the
congestion level introduced across a tunnel. congestion level introduced across a tunnel.
All this shows that 'E' can preserve the control loop irrespective of All this shows that 'E' can preserve the control loop irrespective of
whether 'I' copies congestion notification into the outer header or whether 'I' copies congestion notification into the outer header or
resets it. resets it.
That is the situation for existing control arrangements but, because That is the situation for existing control arrangements but, because
copying reveals more information, it would open up possibilities for copying reveals more information, it would open up possibilities for
better control system designs. For instance, Appendix E describes better control system designs. For instance, Appendix E describes
how resetting CE marking at a tunnel ingress confuses a proposed how resetting CE marking on encapsulation breaks a proposed
congestion marking scheme on the standards track. It ends up congestion marking scheme on the standards track. It ends up
removing excessive amounts of traffic unnecessarily. Whereas copying removing excessive amounts of traffic unnecessarily. Whereas copying
CE markings at ingress leads to the correct control behaviour. CE markings at ingress leads to the correct control behaviour.
A.3. Management Constraints B.3. Management Constraints
As well as control, there are also management constraints. As well as control, there are also management constraints.
Specifically, a management system may monitor congestion markings in Specifically, a management system may monitor congestion markings in
passing packets, perhaps at the border between networks as part of a passing packets, perhaps at the border between networks as part of a
service level agreement. For instance, monitors at the borders of service level agreement. For instance, monitors at the borders of
autonomous systems may need to measure how much congestion has autonomous systems may need to measure how much congestion has
accumulated since the original source, perhaps to determine between accumulated so far along the path, perhaps to determine between them
them how much of the congestion is contributed by each domain. how much of the congestion is contributed by each domain.
Therefore, when monitoring the middle of a path, it should be
possible to establish how far back in the path congestion markings
have accumulated from. In this document we term this the baseline of
congestion marking (or the Congestion Baseline), i.e. the source of
the layer that last reset (or created) the congestion notification
field. Given some tunnels cross domain borders (e.g. consider M in
Figure 6 is monitoring a border), it would therefore be desirable for
'I' to copy congestion accumulated so far into the outer headers
exposed across the tunnel.
Appendix B.2 discusses various scenarios where the Load Regulator
lies in-path, not at the source host as we would typically expect.
It concludes that a Congestion Baseline is determined by where the
Load Regulator function is, which should be identified in the
transport layer, not by addresses in network layer headers. This
applies whether the Load Regulator is at the source host or within
the path. The appendix also discusses where a Load Regulator
function should be located relative to a local tunnel encapsulation
function.
Appendix B. Relative Placement of Tunnelling and In-Path Load
Regulation
B.1. Identifiers and In-Path Load Regulators
The Load Regulator is the node to which congestion feedback should be
returned by the next downstream node with a transport layer feedback
function (typically but not always the data receiver). The Load
Regulator is often, but not always the data source. It is not always
(or even typically) the same thing as the node identified by the
source address of the outermost exposed header. In general the
addressing of the outermost encapsulation header says nothing about
the identifiers of either the upstream or the downstream transport
layer functions. As long as the transport functions know each
other's addresses, they don't have to be identified in the network
layer or in any link layer. It was only a convenience that a TCP
receiver assumed that the address of the source transport is the same
as the network layer source address of an IP packet it receives.
More generally, the return transport address for feedback could be
identified solely in the transport layer protocol. For instance, a
signalling protocol like RSVP [RFC2205] breaks up a path into
transport layer hops and informs each hop of the address of its
transport layer neighbour without any need to identify these hops in
the network layer. RSVP can be arranged so that these transport
layer hops are bigger than the underlying network layer hops. The
host identity protocol (HIP) architecture [RFC4423] also supports the
same principled separation (for mobility amongst other things), where
the transport layer sender identifies its transport address for
feedback to be sent to, using an identifier provided by a shim below
the transport layer.
Keeping to this layering principle deliberately doesn't require a
network layer packet header to reveal the origin address from where
congestion notification accumulates (its Congestion Baseline). It is
not necessary for the network and lower layers to know the address of
the Load Regulator. Only the destination transport needs to know
that. With forward congestion notification, the network and link
layers only notify congestion forwards; they aren't involved in
feeding it backwards. If they are (e.g. backward congestion
notification (BCN) in Ethernet [IEEE802.1au] or EFCI in ATM
[ITU-T.I.371]), that should be considered as a transport function
added to the lower layer, which must sort out its own addressing.
Indeed, this is one reason why ICMP source quench is now deprecated
[RFC1254]; when congestion occurs within a tunnel it is complex
(particularly in the case of IPsec tunnels) to return the ICMP
messages beyond the tunnel ingress back to the Load Regulator.
Similarly, if a management system is monitoring congestion and needs
to know the Congestion Baseline, the management system has to find
this out from the transport; in general it cannot tell solely by
looking at the network or link layer headers.
B.2. Non-Dependence of Tunnelling on In-path Load Regulation
We have said that at any point in a network, the Congestion Baseline
(where congestion notification starts from zero) should be the
previous upstream Load Regulator. We have also said that the ingress
of an IP in IP tunnel must copy congestion indications to the
encapsulating outer headers it creates. If the Load Regulator is in-
path rather than at the source, and also a tunnel ingress, these two
requirements seem to be contradictory. A tunnel ingress must not
reset incoming congestion, but a Load Regulator must be the
Congestion Baseline, implying it needs to reset incoming congestion.
In fact, the two requirements are not contradictory, because a Load
Regulator and a tunnel ingress are not the names of machines, but the
names of functions within a machine that typically occur in sequence
on a stream of packets, not at the same point. Figure 7 is borrowed
from [RFC2983] (which was making a similar point about the location
of Diffserv traffic conditioning relative to the encapsulation
function of a tunnel). An in-path Load Regulator can act on packets
either at [1 - Before] encapsulation or at [2 - Outer] after
encapsulation. Load Regulation does not ever need to be integrated
with the [Encapsulate] function (but it can be for efficiency).
Therefore we can still mandate that the [Encapsulate] function always
copies CE into the outer header.
>>-----[1 - Before]--------[Encapsulate]----[3 - Inner]---------->>
\
\
+--------[2 - Outer]------->>
Figure 7: Placement of In-Path Load Regulator Relative to Tunnel
Ingress
Then separately, if there is a Load Regulator at location [2 -
Outer], it might reset CE to ECT(0), say. Then the Congestion
Baseline for the lower layer (outer) will be [2 - Outer], while the
Congestion Baseline of the inner layer will be unchanged. But how
encapsulation works has nothing to do with whether a Load Regulator
is present or where it is.
If on the other hand a Load Regulator resets CE at [1 - Before], the
Congestion Baseline of both the inner and outer headers will be [1 -
Before]. But again, encapsulation is independent of load regulation.
B.3. Dependence of In-Path Load Regulation on Tunnelling
Although encapsulation doesn't need to depend on in-path load
regulation, the reverse is not true. The placement of an in-path
Load Regulator must be carefully considered relative to
encapsulation. Some examples are given in the following for
guidance.
In the traditional Internet architecture one tends to think of the
source host as the Load Regulator for a path. It is generally not
desirable or practical for a node part way along the path to regulate
the load. However, various reasonable proposals for in-path load
regulation have been made from time to time (e.g. fair queuing,
traffic engineering, flow admission control). The IETF has recently
chartered a working group to standardise admission control across a
part of a path using pre-congestion notification (PCN) [PCNcharter].
This is of particular relevance here because it involves congestion
notification with an in-path Load Regulator, it can involve
tunnelling and it certainly involves encapsulation more generally.
We will use the more complex scenario in Figure 8 to tease out all
the issues that arise when combining congestion notification and
tunnelling with various possible in-path load regulation schemes. In
this case 'I1' and 'E2' break up the path into three separate
congestion control loops. The feedback for these loops is shown
going right to left across the top of the figure. The 'V's are arrow
heads representing the direction of feedback, not letters. But there
are also two tunnels within the middle control loop: 'I1' to 'E1' and
'I2' to 'E2'. The two tunnels might be VPNs, perhaps over two MPLS
core networks. M is a congestion monitoring point, perhaps between
two border routers where the same tunnel continues unbroken across
the border.
______ _______________________________________ _____
/ \ / \ / \
V \ V M \ V \
A--->R--->I1===========>E1----->I2=========>==========>E2------->B
Figure 8: Complex Tunnel Scenario
The question is, should the congestion markings in the outer exposed
headers of a tunnel represent congestion only since the tunnel
ingress or over the whole upstream path from the source of the inner
header (whatever that may mean)? Or put another way, should 'I1' and
'I2' copy or reset CE markings?
Based on the design principles in Section 4.3, the answer is that the
Congestion Baseline should be the nearest upstream interface designed
to regulate traffic load--the Load Regulator. In Figure 8 'A', 'I1'
or 'E2' are all Load Regulators. We have shown the feedback loops
returning to each of these nodes so that they can regulate the load
causing the congestion notification. So the Congestion Baseline
exposed to M should be 'I1' (the Load Regulator), not 'I2'.
Therefore I1 should reset any arriving CE markings. In this case,
'I1' knows the tunnel to 'E1' is unrelated to its load regulation
function. So the load regulation function within 'I1' should be
placed at [1 - Before] tunnel encapsulation within 'I1' (using the
terminology of Figure 7). Then the Congestion Baseline all across
the networks from 'I1' to 'E2' in both inner and outer headers will
be 'I1'.
The following further examples illustrate how this answer might be
applied:
o We argued in Appendix E that resetting CE on encapsulation could
harm PCN excess rate marking, which marks excess traffic for
removal in subsequent round trips. This marking relies on not
marking packets if another node upstream has already marked them
for removal. If there were a tunnel ingress between the two which
reset CE markings, it would confuse the downstream node into
marking far too much traffic for removal. So why do we say that
'I1' should reset CE, while a tunnel ingress shouldn't? The
answer is that it is the Load Regulator function at 'I1' that is
resetting CE, not the tunnel encapsulator. The Load Regulator
needs to set itself as the Congestion Baseline, so the feedback it
gets will only be about congestion on links it can relieve itself
(by regulating the load into them). When it resets CE markings,
it knows that something else upstream will have dealt with the
congestion notifications it removes, given it is part of an end-
to-end admission control signalling loop. It therefore knows that
previous hops will be covered by other Load Regulators.
Meanwhile, the tunnel ingresses at both 'I1' and 'I2' should
follow the new rule for any tunnel ingress and copy congestion
marking into the outer tunnel header. The ingress at 'I1' will
happen to copy headers that have already been reset just
beforehand. But it doesn't need to know that.
o [Shayman] suggested feedback of ECN accumulated across an MPLS
domain could cause the ingress to trigger re-routing to mitigate
congestion. This case is more like the simple scenario of
Figure 6, with a feedback loop across the MPLS domain ('E' back to
'I'). I is a Load Regulator because re-routing around congestion
is a load regulation function. But in this case 'I' should only
reset itself as the Congestion Baseline in outer headers, as it is
not handling congestion outside its domain, so it must preserve
the end-to-end congestion feedback loop for something else to
handle (probably the data source). Therefore the Load Regulator
within 'I' should be placed at [2 - Outer] to reset CE markings
just after the tunnel ingress has copied them from arriving
headers. Again, the tunnel encapsulation function at 'I' simply
copies incoming headers, unaware that the load regulator will
subsequently reset its outer headers.
o The PWE3 working group of the IETF is considering the problem of
how and whether an aggregate edge-to-edge pseudo-wire emulation
should respond to congestion [I-D.ietf-pwe3-congestion-frmwk].
Although the study is still at the requirements stage, some
(controversial) solution proposals include in-path load regulation
at the ingress to the tunnel that could lead to tunnel
arrangements with similar complexity to that of Figure 8.
These are not contrived scenarios--they could be a lot worse. For
instance, a host may create a tunnel for IPsec which is placed inside
a tunnel for Mobile IP over a remote part of its path. And around
this all we may have MPLS labels being pushed and popped as packets
pass across different core networks. Similarly, it is possible that
subnets could be built from link technology (e.g. future Ethernet
switches) so that link headers being added and removed could involve
congestion notification in future Ethernet link headers with all the
same issues as with IP in IP tunnels.
One reason we introduced the concept of a Load Regulator was to allow
for in-path load regulation. In the traditional Internet
architecture one tends to think of a host and a Load Regulator as
synonymous, but when considering tunnelling, even the definition of a
host is too fuzzy, whereas a Load Regulator is a clearly defined
function. Similarly, the concept of innermost header is too fuzzy to
be able to (wrongly) say that the source address of the innermost
header should be the Congestion Baseline. Which is the innermost
header when multiple encapsulations may be in use? Where do we stop?
If we say the original source in the above IPsec-Mobile IP case is
the host, how do we know it isn't tunnelling an encrypted packet
stream on behalf of another host in a p2p network?
We have become used to thinking that only hosts regulate load. The In this document we define the baseline of congestion marking (or the
end to end design principle advises that this is a good idea Congestion Baseline) as the source of the layer that created (or most
[RFC3426], but it also advises that it is solely a guiding principle recently reset) the congestion notification field. When monitoring
intended to make the designer think very carefully before breaking congestion it would be desirable if the Congestion Baseline did not
it. We do have proposals where load regulation functions sit within depend on whether packets were tunnelled or not. Given some tunnels
a network path for good, if sometimes controversial, reasons, e.g. cross domain borders (e.g. consider M in Figure 6 is monitoring a
PCN edge admission control gateways [I-D.ietf-pcn-architecture] or border), it would therefore be desirable for 'I' to copy congestion
traffic engineering functions at domain borders to re-route around accumulated so far into the outer headers, so that it is exposed
congestion [Shayman]. Whether or not we want in-path load across the tunnel.
regulation, we have to work round the fact that it will not go away.
Appendix C. Contribution to Congestion across a Tunnel Appendix C. Contribution to Congestion across a Tunnel
This specification mandates that a tunnel ingress determines the ECN This specification mandates that a tunnel ingress determines the ECN
field of each new outer tunnel header by copying the arriving header. field of each new outer tunnel header by copying the arriving header.
Concern has been expressed that this will make it difficult for the Concern has been expressed that this will make it difficult for the
tunnel egress to monitor congestion introduced only along a tunnel, tunnel egress to monitor congestion introduced only along a tunnel,
which is easy if the outer ECN field is reset at a tunnel ingress which is easy if the outer ECN field is reset at a tunnel ingress
(RFC3168 full functionality mode). However, in fact copying CE marks (RFC3168 full functionality mode). However, in fact copying CE marks
at ingress will still make it easy for the egress to measure at ingress will still make it easy for the egress to measure
congestion introduced across a tunnel, as illustrated below. congestion introduced across a tunnel, as illustrated below.
Consider 100 packets measured at the egress. It measures that 30 are Consider 100 packets measured at the egress. Say it measures that 30
CE marked in the inner and outer headers and 12 have additional CE are CE marked in the inner and outer headers and 12 have additional
marks in the outer but not the inner. This means packets arriving at CE marks in the outer but not the inner. This means packets arriving
the ingress had already experienced 30% congestion. However, it does at the ingress had already experienced 30% congestion. However, it
not mean there was 12% congestion across the tunnel. The correct does not mean there was 12% congestion across the tunnel. The
calculation of congestion across the tunnel is p_t = 12/(100-30) = correct calculation of congestion across the tunnel is p_t = 12/
12/70 = 17%. This is easy for the egress to to measure. It is the (100-30) = 12/70 = 17%. This is easy for the egress to measure. It
packets with additional CE marking in the outer header (12) as a is simply the packets with additional CE marking in the outer header
proportion of packets not marked in the inner header (70). (12) as a proportion of packets not marked in the inner header (70).
Figure 9 illustrates this in a combinatorial probability diagram. Figure 7 illustrates this in a combinatorial probability diagram.
The square represents 100 packets. The 30% division along the bottom The square represents 100 packets. The 30% division along the bottom
represents marking before the ingress, and the p_t division up the represents marking before the ingress, and the p_t division up the
side represents marking along the tunnel. side represents marking introduced across the tunnel.
+-----+---------+100% ^ outer header marking
| | | |
100% +-----+---------+ The large square
| | | represents 100 packets
| 30 | | | 30 | |
| | | The large square | | | p_t = 12/(100-30)
| +---------+p_t represents 100 packets p_t + +---------+ = 12/70
| | 12 | | | 12 | = 17%
+-----+---------+0 0 +-----+---------+--->
0 30% 100% 0 30% 100% inner header marking
inner header marking
Figure 9: Tunnel Marking of Packets Already Marked at Ingress Figure 7: Tunnel Marking of Packets Already Marked at Ingress
Appendix D. Why Not Propagating ECT(1) on Decapsulation Impedes PCN Appendix D. Why Losing ECT(1) on Decapsulation Impedes PCN
Multi-level congestion notification is currently on the IETF's Congestion notification with two severity levels is currently on the
standards track agenda in the Congestion and Pre-Congestion IETF's standards track agenda in the Congestion and Pre-Congestion
Notification (PCN) working group. The PCN working group eventually Notification (PCN) working group. The PCN working group requires
requires three congestion states (not marked and two increasingly four congestion states (not PCN-enabled, not marked and two
severe levels of congestion marking) [I-D.ietf-pcn-architecture]. increasingly severe levels of congestion marking--see [RFC5559]).
The aim is for the less severe level of marking to stop admitting new The aim is for the less severe level of marking to stop admitting new
traffic and the more severe level to terminate sufficient existing traffic and the more severe level to terminate sufficient existing
flows to bring a network back to its operating point after a serious flows to bring a network back to its operating point after a link
failure. failure.
Although the ECN field gives sufficient codepoints for these three (Note on terminology: wherever this document counts four congestion
states, current ECN tunnelling RFCs prevent the PCN working group states, the PCN working group would count this as three PCN states
from using three ECN states in case any tunnel decapsulations occur plus a not-PCN-enabled state.)
within a PCN region (see Appendix A of
[I-D.ietf-pcn-baseline-encoding]). If a node in a tunnel sets the
ECN field to ECT(0) or ECT(1), this change will be discarded by a
tunnel egress compliant with RFC4301 or RFC3168. This can be seen in
Figure 2 (Section 3.2), where ECT values in the outer header are
ignored unless the inner header is the same. Effectively one ECT
codepoint is wasted; the ECT(0) and ECT(1) codepoints have to be
treated as just one codepoint when they could otherwise have been
used for their intended purpose of congestion notification.
As a consequence, the PCN w-g has initially confined itself to two Although the ECN field gives sufficient codepoints for four states,
encoding states as a baseline encoding pre-existing ECN tunnelling RFCs prevented the PCN working group from
[I-D.ietf-pcn-baseline-encoding]. And it has had to propose an using four ECN states in case any tunnel decapsulations occur within
experimental extension using extra Diffserv codepoint(s) to encode a PCN region. If a node in a tunnel changes the ECN field to ECT(0)
the extra states [I-D.moncaster-pcn-3-state-encoding], using up the or ECT(1), this change would be discarded by a tunnel egress
rapidly exhausting DSCP space while leaving ECN codepoints unused. compliant with RFC4301 or RFC3168. This can be seen in Figure 2
Another PCN encoding has been proposed that would survive tunnelling (Section 3.2), where ECT values in the outer header are ignored
without an extra DSCP [I-D.menth-pcn-psdm-encoding], but it requires unless the inner header is the same. Effectively the decapsulation
the PCN edge gateways to somehow share state so the egress can rules of RFC4301 and RFC3168 waste one ECT codepoint; they treat the
determine which marking a packet started with at the ingress. Also a ECT(0) and ECT(1) codepoints as a single codepoint.
PCN ingress node can game the system by initiating packets with
inappropriate markings. Yet another work-round to the ECN tunnelling
problem proposes a more involved marking algorithm in the forwarding
plane to encode the three congestion notification states using only
two ECN codepoints [I-D.satoh-pcn-st-marking]. Still another
proposal compromises the precision of the admission control
mechanism, but manages to work with just two encoding states and a
single marking algorithm [I-D.charny-pcn-single-marking].
Rather than require the IETF to bless any of these work-rounds, this As a consequence, the PCN w-g initially took the approach of a
specification fixes the root cause of the problem so that operators standards track baseline encoding for three states
deploying PCN can simply ask that tunnel end-points within a PCN [I-D.ietf-pcn-baseline-encoding] and a number of experimental
region should comply with this new ECN tunnelling specification. alternatives to add or avoid the fourth state. Without wishing to
disparage the ingenuity of these work-rounds, none were chosen for
the standards track because they were either somewhat wasteful,
imprecise or complicated. One uses a pair of Diffserv codepoint(s)
in place of each PCN DSCP to encode the extra state
[I-D.ietf-pcn-3-state-encoding], using up the rapidly exhausting DSCP
space while leaving an ECN codepoint unused. Another PCN encoding
has been proposed that would survive tunnelling without an extra DSCP
[I-D.ietf-pcn-psdm-encoding], but it requires the PCN edge gateways
to share state out of band so the egress edge can know which marking
a packet started with at the ingress edge. Yet another work-round to
the ECN tunnelling problem proposes a more involved marking algorithm
in forwarding elements to encode the three congestion notification
states using only two ECN codepoints [I-D.satoh-pcn-st-marking]. One
work-round takes a different approach; it compromises the precision
of the admission control mechanism in some network scenarios, but
manages to work with just three encoding states and a single marking
algorithm [I-D.ietf-pcn-sm-edge-behaviour].
Then PCN can use the trivially simple experimental 3-state ECN Rather than require the IETF to bless any of these experimental
encoding defined in [I-D.briscoe-pcn-3-in-1-encoding]. encoding work-rounds, the present specification fixes the root cause
of the problem so that operators deploying PCN can simply require
that tunnel end-points within a PCN region should comply with this
new ECN tunnelling specification. Universal compliance is feasible
for PCN, because it is intended to be deployed in a controlled
Diffserv region. Assuming tunnels within a PCN region will be
required to comply with the present specification, the PCN w-g is
progressing a trivially simple four-state ECN encoding
[I-D.ietf-pcn-3-in-1-encoding].
D.1. Alternative Ways to Introduce the New Decapsulation Rules Appendix E. Why Resetting ECN on Encapsulation Impedes PCN
There are a number of ways for the new decapsulation rules to be The PCN architecture says "...if encapsulation is done within the
introduced: PCN-domain: Any PCN-marking is copied into the outer header. Note: A
tunnel will not provide this behaviour if it complies with [RFC3168]
tunnelling in either mode, but it will if it complies with [RFC4301]
IPsec tunnelling. "
o They could be specified in the present standards track proposal The specific issue here concerns PCN excess rate marking
(preferred) or in an experimental extension; [I-D.ietf-pcn-marking-behaviour]. The purpose of excess rate marking
is to provide a bulk mechanism for interior nodes within a PCN domain
to mark traffic that is exceeding a configured threshold bit-rate,
perhaps after an unexpected event such as a reroute, a link or node
failure, or a more widespread disaster. PCN is intended for
inelastic flows, so just removing marked packets would degrade every
flow to the point of uselessness. Instead, the edge nodes around a
PCN domain terminate an equivalent amount of traffic, but at flow
granularity. As well as protecting the surviving inelastic flows,
this also protects the share of capacity set aside for elastic
traffic. But users are very sensitive to their flows being
terminated while in progress, therefore no more flows should be
terminated than absolutely necessary.
o They could be specified as a new default for all Diffserv PHBs Re-routes are a common cause of QoS degradation in IP networks.
(preferred) or as an option to be configured only for Diffserv After re-routes it is common for multiple links in a network to
PHBs requiring them (e.g. PCN). become stressed at once. Therefore, PCN excess rate marking has been
carefully designed to ensure traffic marked at one queue will not be
counted again for marking at subsequent queues (see the `Excess
traffic meter function' of [I-D.ietf-pcn-marking-behaviour]).
The argument for making this change now, rather than in a separate However, if an RFC3168 tunnel ingress intervenes, it resets the ECN
experimental extension, is to avoid the burden of an extra standard field in all the outer headers. This will cause excess traffic to be
to be compliant with and to be backwards compatible with--so we don't counted more than once, leading to many flows being removed that did
add to the already complex history of ECN tunnelling RFCs. The not need to be removed at all. This is why the an RFC3168 tunnel
argument for a separate experimental extension is that we may never ingress cannot be used in a PCN domain.
need this change (if PCN is never successfully deployed and if no-one
ever needs three ECN or PCN encoding states rather than two).
However, the change does no harm to existing mechanisms and stops
tunnels wasting of quarter of a bit (a 2-bit codepoint).
The argument for making this new decapsulation behaviour the default The original reason an RFC3168 encapsulator reset the ECN field was
for all PHBs is that it doesn't change any expected behaviour that to block a covert channel (Appendix B.1), with the overriding aim of
existing mechanisms rely on already. Also, by ending the present consistent behaviour between IPsec and non-IPsec tunnels. But later
waste of a codepoint, in the future a use of that codepoint could be RFC4301 IPsec encapsulation placed simplicity above the need to block
proposed for all PHBs, even if PCN isn't successfully deployed. the covert channel, simply copying the ECN field.
In practice, if these new decapsulation rules are specified The ECN reset in RFC3168 is no longer deemed necessary, it is
straightaway as the normative default for all PHBs, a network inconsistent with RFC4301, it is not as simple as RFC4301 and it is
operator deploying 3-state PCN would be able to request that tunnels impeding deployment of new protocols like PCN. The present
comply with the latest specification. Implementers of non-PCN specification corrects this perverse situation.
tunnels would not need to comply but, if they did, their code would
be future proofed and no harm would be done to legacy operations.
Therefore, rather than branching their code base, it would be easiest
for implementers to make all their new tunnel code comply with this
specfication, whether or not it was for PCN. But they could leave
old code untouched, unless it was for PCN.
The alternatives are worse. Implementers would otherwise have to Appendix F. Compromise on Decap with ECT(1) Inner and ECT(0) Outer
provide configurable decapsulation options and operators would have
to configure all IPsec and IP in IP tunnel endpoints for the
exceptional behaviour of certain PHBs. The rules for tunnel
endpoints to handle both the Diffserv field and the ECN field should
'just work' when handling packets with any Diffserv codepoint.
Appendix E. Why Resetting CE on Encapsulation Impedes PCN A packet with an ECT(1) inner and an ECT(0) outer should never arise
from any known IETF protocol. Without giving a reason, RFC3168 and
RFC4301 both say the outer should be ignored when decapsulating such
a packet. This appendix explains why it was decided not to change
this advice.
Regarding encapsulation, the section of the PCN architecture In summary, ECT(0) always means 'not congested' and ECT(1) may imply
[I-D.ietf-pcn-architecture] on tunnelling says that header copying the same [RFC3168] or it may imply a higher severity congestion
(RFC4301) allows PCN to work correctly. Whereas resetting CE signal [RFC4774], [I-D.ietf-pcn-3-in-1-encoding], depending on the
markings confuses PCN marking. transport in use. Whether they mean the same or not, at the ingress
the outer should have started the same as the inner and only a broken
or compromised router could have changed the outer to ECT(0).
The specific issue here concerns PCN excess rate marking The decapsulator can detect this anomaly. But the question is,
[I-D.ietf-pcn-marking-behaviour], i.e. the bulk marking of traffic should it correct the anomaly by ignoring the outer, or should it
that exceeds a configured threshold rate. One of the goals of excess reveal the anomaly to the end-to-end transport by forwarding the
rate marking is to enable the speedy removal of excess admission outer?
controlled traffic following re-routes caused by link failures or
other disasters. This maintains a share of the capacity for traffic
in lower priority classes. After failures, traffic re-routed onto
remaining links can often stress multiple links along a path.
Therefore, traffic can arrive at a link under stress with some
proportion already marked for removal by a previous link. By design,
marked traffic will be removed by the overall system in subsequent
round trips. So when the excess rate marking algorithm decides how
much traffic to mark for removal, it doesn't include traffic already
marked for removal by another node upstream (the `Excess traffic
meter function' of [I-D.ietf-pcn-marking-behaviour]).
However, if an RFC3168 tunnel ingress intervenes, it resets the ECN On balance, it was decided that the decapsulator should correct the
field in all the outer headers, hiding all the evidence of problems anomaly, but log the event and optionally raise an alarm. This is
upstream. Thus, although excess rate marking works fine with RFC4301 the safe action if ECT(1) is being used as a more severe marking than
IPsec tunnels, with RFC3168 tunnels it typically removes large ECT(0), because it passes the more severe signal to the transport.
volumes of traffic that it didn't need to remove at all. However, it is not a good idea to hide anomalies, which is why an
optional alarm is suggested. It should be noted that this anomaly
may be the result of two changes to the outer: a broken or
compromised router within the tunnel might be erasing congestion
markings introduced earlier in the same tunnel by a congested router.
In this case, the anomaly would be losing congestion signals, which
needs immediate attention.
The original reason for defining ECT(0) and ECT(1) as equivalent was
so that the data source could use the ECN nonce [RFC3540] to detect
if congestion signals were being erased. However, in this case, the
decapsulator does not need a nonce to detect any anomalies introduced
within the tunnel, because it has the inner as a record of the header
at the ingress. Therefore, it was decided that the best compromise
would be to give precedence to solving the safety issue over
revealing the anomaly, because the anomaly could at least be detected
and dealt with internally.
Superficially, the opposite case where the inner and outer carry
different ECT values, but with an ECT(1) outer and ECT(0) inner seems
to require a similar compromise. However, because that case is
reversed, no compromise is necessary; it is best to forward the outer
whether the transport expects the ECT(1) to mean a higher severity
than ECT(0) or the same severity. Forwarding the outer either
preserves a higher value (if it is higher) or it reveals an anomaly
to the transport (if the two ECT codepoints mean the same severity).
Author's Address Author's Address
Bob Briscoe Bob Briscoe
BT BT
B54/77, Adastral Park B54/77, Adastral Park
Martlesham Heath Martlesham Heath
Ipswich IP5 3RE Ipswich IP5 3RE
UK UK
Phone: +44 1473 645196 Phone: +44 1473 645196
Email: bob.briscoe@bt.com EMail: bob.briscoe@bt.com
URI: http://www.cs.ucl.ac.uk/staff/B.Briscoe/ URI: http://www.cs.ucl.ac.uk/staff/B.Briscoe/
 End of changes. 209 change blocks. 
1248 lines changed or deleted 1251 lines changed or added

This html diff was produced by rfcdiff 1.35. The latest version is available from http://tools.ietf.org/tools/rfcdiff/