draft-ietf-mptcp-architecture-03.txt   draft-ietf-mptcp-architecture-04.txt 
Internet Engineering Task Force A. Ford, Ed. Internet Engineering Task Force A. Ford, Ed.
Internet-Draft Roke Manor Research Internet-Draft Roke Manor Research
Intended status: Informational C. Raiciu Intended status: Informational C. Raiciu
Expires: June 11, 2011 M. Handley Expires: July 15, 2011 M. Handley
University College London University College London
S. Barre S. Barre
Universite catholique de Universite catholique de
Louvain Louvain
J. Iyengar J. Iyengar
Franklin and Marshall College Franklin and Marshall College
December 8, 2010 January 11, 2011
Architectural Guidelines for Multipath TCP Development Architectural Guidelines for Multipath TCP Development
draft-ietf-mptcp-architecture-03 draft-ietf-mptcp-architecture-04
Abstract Abstract
Hosts are often connected by multiple paths, but TCP restricts Hosts are often connected by multiple paths, but TCP restricts
communications to a single path per transport connection. Resource communications to a single path per transport connection. Resource
usage within the network would be more efficient were these multiple usage within the network would be more efficient were these multiple
paths able to be used concurrently. This should enhance user paths able to be used concurrently. This should enhance user
experience through improved resilience to network failure and higher experience through improved resilience to network failure and higher
throughput. throughput.
skipping to change at page 1, line 49 skipping to change at page 1, line 49
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 11, 2011. This Internet-Draft will expire on July 15, 2011.
Copyright Notice Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 3, line 20 skipping to change at page 3, line 20
1.3. Reference Scenario . . . . . . . . . . . . . . . . . . . . 5 1.3. Reference Scenario . . . . . . . . . . . . . . . . . . . . 5
2. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1. Functional Goals . . . . . . . . . . . . . . . . . . . . . 6 2.1. Functional Goals . . . . . . . . . . . . . . . . . . . . . 6
2.2. Compatibility Goals . . . . . . . . . . . . . . . . . . . 7 2.2. Compatibility Goals . . . . . . . . . . . . . . . . . . . 7
2.2.1. Application Compatibility . . . . . . . . . . . . . . 7 2.2.1. Application Compatibility . . . . . . . . . . . . . . 7
2.2.2. Network Compatibility . . . . . . . . . . . . . . . . 7 2.2.2. Network Compatibility . . . . . . . . . . . . . . . . 7
2.2.3. Compatibility with other network users . . . . . . . . 9 2.2.3. Compatibility with other network users . . . . . . . . 9
2.3. Security Goals . . . . . . . . . . . . . . . . . . . . . . 9 2.3. Security Goals . . . . . . . . . . . . . . . . . . . . . . 9
2.4. Related Protocols . . . . . . . . . . . . . . . . . . . . 9 2.4. Related Protocols . . . . . . . . . . . . . . . . . . . . 9
3. An Architectural Basis For Multipath TCP . . . . . . . . . . . 10 3. An Architectural Basis For Multipath TCP . . . . . . . . . . . 10
4. A Functional Decomposition of MPTCP . . . . . . . . . . . . . 11 4. A Functional Decomposition of MPTCP . . . . . . . . . . . . . 12
5. High-Level Design Decisions . . . . . . . . . . . . . . . . . 13 5. High-Level Design Decisions . . . . . . . . . . . . . . . . . 13
5.1. Sequence Numbering . . . . . . . . . . . . . . . . . . . . 13 5.1. Sequence Numbering . . . . . . . . . . . . . . . . . . . . 14
5.2. Reliability and Retransmissions . . . . . . . . . . . . . 14 5.2. Reliability and Retransmissions . . . . . . . . . . . . . 15
5.3. Buffers . . . . . . . . . . . . . . . . . . . . . . . . . 16 5.3. Buffers . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.4. Signalling . . . . . . . . . . . . . . . . . . . . . . . . 17 5.4. Signalling . . . . . . . . . . . . . . . . . . . . . . . . 17
5.5. Path Management . . . . . . . . . . . . . . . . . . . . . 18 5.5. Path Management . . . . . . . . . . . . . . . . . . . . . 18
5.6. Connection Identification . . . . . . . . . . . . . . . . 19 5.6. Connection Identification . . . . . . . . . . . . . . . . 19
5.7. Congestion Control . . . . . . . . . . . . . . . . . . . . 20 5.7. Congestion Control . . . . . . . . . . . . . . . . . . . . 20
5.8. Security . . . . . . . . . . . . . . . . . . . . . . . . . 20 5.8. Security . . . . . . . . . . . . . . . . . . . . . . . . . 20
6. Interactions with Applications . . . . . . . . . . . . . . . . 21 6. Interactions with Applications . . . . . . . . . . . . . . . . 21
7. Interactions with Middleboxes . . . . . . . . . . . . . . . . 21 7. Interactions with Middleboxes . . . . . . . . . . . . . . . . 22
8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 23 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 24
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 23 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
11. Security Considerations . . . . . . . . . . . . . . . . . . . 24 11. Security Considerations . . . . . . . . . . . . . . . . . . . 24
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 24 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 24
12.1. Normative References . . . . . . . . . . . . . . . . . . . 24 12.1. Normative References . . . . . . . . . . . . . . . . . . . 24
12.2. Informative References . . . . . . . . . . . . . . . . . . 24 12.2. Informative References . . . . . . . . . . . . . . . . . . 25
Appendix A. Changelog . . . . . . . . . . . . . . . . . . . . . . 25 Appendix A. Changelog . . . . . . . . . . . . . . . . . . . . . . 26
A.1. Changes since draft-ietf-mptcp-architecture-02 . . . . . . 25 A.1. Changes since draft-ietf-mptcp-architecture-03 . . . . . . 26
A.2. Changes since draft-ietf-mptcp-architecture-01 . . . . . . 26 A.2. Changes since draft-ietf-mptcp-architecture-02 . . . . . . 26
A.3. Changes since draft-ietf-mptcp-architecture-00 . . . . . . 26 A.3. Changes since draft-ietf-mptcp-architecture-01 . . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 A.4. Changes since draft-ietf-mptcp-architecture-00 . . . . . . 27
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 27
1. Introduction 1. Introduction
As the Internet evolves, demands on Internet resources are ever- As the Internet evolves, demands on Internet resources are ever-
increasing, but often these resources (in particular, bandwidth) increasing, but often these resources (in particular, bandwidth)
cannot be fully utilised due to protocol constraints both on the end- cannot be fully utilised due to protocol constraints both on the end-
systems and within the network. If these resources could instead be systems and within the network. If these resources could instead be
used concurrently, end user experience could be greatly improved. used concurrently, end user experience could be greatly improved.
Such enhancements would also reduce the necessary expenditure on Such enhancements would also reduce the necessary expenditure on
network infrastructure that would otherwise be needed to create an network infrastructure that would otherwise be needed to create an
equivalent improvement in user experience. equivalent improvement in user experience.
By the application of resource pooling[3], these available resources By the application of resource pooling [3], these available resources
can be 'pooled' such that they appear as a single logical resource to can be 'pooled' such that they appear as a single logical resource to
the user. The purpose of a multipath transport, therefore, is to the user. The purpose of a multipath transport, therefore, is to
make use of multiple available paths, through resource pooling, to make use of multiple available paths, through resource pooling, to
bring two key benefits: bring two key benefits:
o To increase the resilience of the connectivity by providing o To increase the resilience of the connectivity by providing
multiple paths, protecting end hosts from the failure of one. multiple paths, protecting end hosts from the failure of one.
o To increase the efficiency of the resource usage, and thus o To increase the efficiency of the resource usage, and thus
increase the network capacity available to end hosts. increase the network capacity available to end hosts.
Multipath TCP is a modified version of TCP[1] that implements a Multipath TCP is a modified version of TCP [1] that implements a
multipath transport and achieves these goals by pooling multiple multipath transport and achieves these goals by pooling multiple
paths within a transport connection, transparently to the paths within a transport connection, transparently to the
application. MPTCP, defined in [4], is a specific protocol that application. MPTCP, defined in [4], is a specific protocol that
instantiates the Multipath TCP concept. This document looks both at instantiates the Multipath TCP concept. This document looks both at
general architectural principles for a Multipath TCP fulfilling the general architectural principles for a Multipath TCP fulfilling the
goals described in Section 2, as well as the key design decisions goals described in Section 2, as well as the key design decisions
behind MPTCP, which are detailed in Section 5. behind MPTCP, which are detailed in Section 5.
Although multihoming and multipath functions are not new to transport Although multihoming and multipath functions are not new to transport
protocols (SCTP [5] being a notable example), MPTCP aims to gain protocols (SCTP [5] being a notable example), MPTCP aims to gain
skipping to change at page 5, line 6 skipping to change at page 5, line 6
applications). applications).
This document has three key purposes: (i) it describes goals for a This document has three key purposes: (i) it describes goals for a
multipath transport - goals that MPTCP is designed to meet; (ii) it multipath transport - goals that MPTCP is designed to meet; (ii) it
lays out an architectural basis for MPTCP's design - a discussion lays out an architectural basis for MPTCP's design - a discussion
that applies to other multipath transports as well; and (iii) it that applies to other multipath transports as well; and (iii) it
discusses and documents high-level design decisions made in MPTCP's discusses and documents high-level design decisions made in MPTCP's
development, and considers their implications. development, and considers their implications.
Companion documents to this architectural overview are those which Companion documents to this architectural overview are those which
provide details of the protocol extensions[4], congestion control provide details of the protocol extensions [4], congestion control
algorithms[6], and application-level considerations[7]. Put algorithms [6], and application-level considerations [7]. Put
together, these components specify a complete Multipath TCP design. together, these components specify a complete Multipath TCP design.
We note that specific components are replaceable in accordance with We note that specific components are replaceable in accordance with
the layer and functional decompositions discussed in this document. the layer and functional decompositions discussed in this document.
1.1. Requirements Language 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [2]. document are to be interpreted as described in RFC 2119 [2].
1.2. Terminology 1.2. Terminology
Multipath TCP: A modified version of the TCP [1] protocol that
supports the simultaneous use of multiple paths between hosts.
Path: A sequence of links between a sender and a receiver, defined Path: A sequence of links between a sender and a receiver, defined
in this context by a source and destination address pair. in this context by a source and destination address pair.
Path Identifier: Within the context of a multi-addressed multipath
TCP, a path is defined by the source and destination (address,
port) pairs (i.e. a 4-tuple).
Host: An end host either initiating or terminating a Multipath TCP Host: An end host either initiating or terminating a Multipath TCP
connection. connection.
Multipath TCP: A modified version of the TCP [1] protocol that
supports the simultaneous use of multiple paths between hosts.
MPTCP: The proposed protocol extensions specified in [4] to provide MPTCP: The proposed protocol extensions specified in [4] to provide
a Multipath TCP implementation. a Multipath TCP implementation.
Subflow: A flow of TCP packets operating over an individual path, Subflow: A flow of TCP segments operating over an individual path,
which forms part of a larger Multipath TCP connection. which forms part of a larger Multipath TCP connection.
(Multipath TCP) Connection: A set of one or more subflows combined (Multipath TCP) Connection: A set of one or more subflows combined
to provide a single Multipath TCP service to an application at a to provide a single Multipath TCP service to an application at a
host. host.
1.3. Reference Scenario 1.3. Reference Scenario
The diagram shown in Figure 1 illustrates a typical usage scenario The diagram shown in Figure 1 illustrates a typical usage scenario
for Multipath TCP. Two hosts, A and B, are communicating with each for Multipath TCP. Two hosts, A and B, are communicating with each
skipping to change at page 6, line 19 skipping to change at page 6, line 19
| A |--\______( )______/--| B | | A |--\______( )______/--| B |
| |A2 (__________) B2| | | |A2 (__________) B2| |
+------+ +------+ +------+ +------+
Figure 1: Simple Multipath TCP Usage Scenario Figure 1: Simple Multipath TCP Usage Scenario
The scenario could have any number of addresses (1 or more) on each The scenario could have any number of addresses (1 or more) on each
host, so long as the number of paths available between the two hosts host, so long as the number of paths available between the two hosts
is 2 or more (i.e. num_addr(A) * num_addr(B) > 1). The paths created is 2 or more (i.e. num_addr(A) * num_addr(B) > 1). The paths created
by these address combinations through the Internet need not be by these address combinations through the Internet need not be
entirely disjoint - shared bottlenecks will be addressed by the entirely disjoint - potential fairness issues introduced by shared
Multipath TCP congestion controller. Furthermore, the paths through bottlenecks need to be handled by the Multipath TCP congestion
the Internet may be interrupted by any number of middleboxes controller. Furthermore, the paths through the Internet often do not
including NATs and Firewalls. Finally, although the diagram refers provide a pure end-to-end service, and instead may be affected by
to the Internet, Multipath TCP may be used over any network where middleboxes such as NATs and Firewalls.
there are multiple paths that could be used concurrently.
2. Goals 2. Goals
This section outlines primary goals that Multipath TCP aims to meet. This section outlines primary goals that Multipath TCP aims to meet.
These are broadly broken down into: functional goals, which steer These are broadly broken down into: functional goals, which steer
services and features that Multipath TCP must provide; and services and features that Multipath TCP must provide; and
compatibility goals, which determine how Multipath TCP should appear compatibility goals, which determine how Multipath TCP should appear
to entities that interact with it. to entities that interact with it.
2.1. Functional Goals 2.1. Functional Goals
skipping to change at page 6, line 47 skipping to change at page 6, line 46
following two functional goals. following two functional goals.
o Improve Throughput: Multipath TCP MUST support the concurrent use o Improve Throughput: Multipath TCP MUST support the concurrent use
of multiple paths. To meet the minimum performance incentives for of multiple paths. To meet the minimum performance incentives for
deployment, a Multipath TCP connection over multiple paths SHOULD deployment, a Multipath TCP connection over multiple paths SHOULD
achieve no lesser throughput than a single TCP connection over the achieve no lesser throughput than a single TCP connection over the
best constituent path. best constituent path.
o Improve Resilience: Multipath TCP MUST support the use of multiple o Improve Resilience: Multipath TCP MUST support the use of multiple
paths interchangeably for resilience purposes, by permitting paths interchangeably for resilience purposes, by permitting
packets to be sent and re-sent on any available path. It follows segments to be sent and re-sent on any available path. It follows
that, in the worst case, the protocol MUST be no less resilient that, in the worst case, the protocol MUST be no less resilient
than regular single-path TCP. than regular single-path TCP.
As distribution of traffic among available paths and responses to As distribution of traffic among available paths and responses to
congestion are done in accordance with resource pooling congestion are done in accordance with resource pooling principles
principles[3], a secondary effect of meeting these goals is that
widespread use of Multipath TCP over the Internet should optimize [3], a secondary effect of meeting these goals is that widespread use
overall network utility by shifting load away from congested of Multipath TCP over the Internet should improve overall network
bottlenecks and by taking advantage of spare capacity wherever utility by shifting load away from congested bottlenecks and by
possible. taking advantage of spare capacity wherever possible.
Furthermore, Multipath TCP SHOULD feature automatic negotiation of Furthermore, Multipath TCP SHOULD feature automatic negotiation of
its use. A host supporting Multipath TCP that requires the other its use. A host supporting Multipath TCP that requires the other
host to do so too must be able to detect reliably whether this host host to do so too must be able to detect reliably whether this host
does in fact support the required extensions, using them if so, and does in fact support the required extensions, using them if so, and
otherwise automatically falling back to single-path TCP. otherwise automatically falling back to single-path TCP.
2.2. Compatibility Goals 2.2. Compatibility Goals
In addition to the functional goals listed above, a Multipath TCP In addition to the functional goals listed above, a Multipath TCP
skipping to change at page 7, line 36 skipping to change at page 7, line 35
Application compatibility refers to the appearance of Multipath TCP Application compatibility refers to the appearance of Multipath TCP
to the application both in terms of the API that can be used and the to the application both in terms of the API that can be used and the
expected service model that is provided. expected service model that is provided.
Multipath TCP MUST follow the same service model as TCP [1]: in- Multipath TCP MUST follow the same service model as TCP [1]: in-
order, reliable, and byte-oriented delivery. Furthermore, a order, reliable, and byte-oriented delivery. Furthermore, a
Multipath TCP connection SHOULD provide the application with no worse Multipath TCP connection SHOULD provide the application with no worse
throughput than it would expect from running a single TCP connection throughput than it would expect from running a single TCP connection
over any one of its available paths. over any one of its available paths.
A multipath-capable equivalent of TCP SHOULD retain backward A multipath-capable equivalent of TCP MUST retain some level of
compatibility with existing TCP APIs, so that existing applications backward compatibility with existing TCP APIs, so that existing
can use the newer transport merely by upgrading the operating systems applications can use the newer transport merely by upgrading the
of the end-hosts. This does not preclude the use of an advanced API operating systems of the end-hosts. This does not preclude the use
to permit multipath-aware applications to specify preferences, nor of an advanced API to permit multipath-aware applications to specify
for users to configure their systems in a different way from the preferences, nor for users to configure their systems in a different
default, for example switching on or off the automatic use of way from the default, for example switching on or off the automatic
multipath extensions. use of multipath extensions.
2.2.2. Network Compatibility 2.2.2. Network Compatibility
In the traditional Internet architecture, network devices operate at In the traditional Internet architecture, network devices operate at
the network layer and lower layers, with the layers above the network the network layer and lower layers, with the layers above the network
layer instantiated only at the end-hosts. While this architecture, layer instantiated only at the end-hosts. While this architecture,
shown in Figure 2, was initially largely adhered to, this layering no shown in Figure 2, was initially largely adhered to, this layering no
longer reflects the "ground truth" in the Internet with the longer reflects the "ground truth" in the Internet with the
proliferation of middleboxes[8]. Middleboxes routinely interpose on proliferation of middleboxes [8]. Middleboxes routinely interpose on
the transport layer; sometimes even completely terminating transport the transport layer; sometimes even completely terminating transport
connections, thus leaving the application layer as the first real connections, thus leaving the application layer as the first real
end-to-end layer, as shown in Figure 3. end-to-end layer, as shown in Figure 3.
+-------------+ +-------------+ +-------------+ +-------------+
| Application |<------------ end-to-end ------------->| Application | | Application |<------------ end-to-end ------------->| Application |
+-------------+ +-------------+ +-------------+ +-------------+
| Transport |<------------ end-to-end ------------->| Transport | | Transport |<------------ end-to-end ------------->| Transport |
+-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+
| Network |<->| Network |<->| Network |<->| Network | | Network |<->| Network |<->| Network |<->| Network |
skipping to change at page 8, line 33 skipping to change at page 8, line 31
| Transport |<------------------->| Transport |<->| Transport | | Transport |<------------------->| Transport |<->| Transport |
+-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+
| Network |<->| Network |<->| Network |<->| Network | | Network |<->| Network |<->| Network |<->| Network |
+-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+
Firewall, Firewall,
End Host Router NAT, or Proxy End Host End Host Router NAT, or Proxy End Host
Figure 3: Internet Reality Figure 3: Internet Reality
Middleboxes that interpose on the transport layer result in loss of Middleboxes that interpose on the transport layer result in loss of
"fate-sharing"[9], that is, they often hold "hard" state that, when "fate-sharing" [9], that is, they often hold "hard" state that, when
lost or corrupted, results in loss or corruption of the end-to-end lost or corrupted, results in loss or corruption of the end-to-end
transport connection. transport connection.
The network compatibility goal requires that the multipath extension The network compatibility goal requires that the multipath extension
to TCP retains compatibility with the Internet as it exists today, to TCP retains compatibility with the Internet as it exists today,
including making reasonable efforts to be able to traverse including making reasonable efforts to be able to traverse
predominant middleboxes such as firewalls, NATs, and performance predominant middleboxes such as firewalls, NATs, and performance
enhancing proxies[8]. This requirement comes from recognizing enhancing proxies [8]. This requirement comes from recognizing
middleboxes as a significant deployment bottleneck for any transport middleboxes as a significant deployment bottleneck for any transport
that is not TCP, and constrains Multipath TCP to appear as TCP does that is not TCP or UDP, and constrains Multipath TCP to appear as TCP
on the wire and to use established TCP extensions where necessary. does on the wire and to use established TCP extensions where
To ensure end-to-endness of the transport, we further require necessary. To ensure end-to-endness of the transport, we further
Multipath TCP to preserve fate-sharing without making any assumptions require Multipath TCP to preserve fate-sharing without making any
about middlebox behavior. assumptions about middlebox behavior.
A detailed analysis of middlebox behaviour and the impact on the A detailed analysis of middlebox behaviour and the impact on the
Multipath TCP architecture is presented in Section 7. In addition, Multipath TCP architecture is presented in Section 7. In addition,
network compatibility must be retained to the extent that Multipath network compatibility must be retained to the extent that Multipath
TCP MUST fall back to regular TCP if there are insurmountable TCP MUST fall back to regular TCP if there are insurmountable
incompatibilities for the multipath extension on a path. incompatibilities for the multipath extension on a path.
Middleboxes may also cause some TCP features to be able to exist on
one subflow but not another. Typically these will be at the subflow
level (such as SACK [10]) and thus do not affect the connection-level
behaviour. In the future, any proposed TCP connection-level
extensions should consider how they can co-exist with MPTCP.
The modifications to support Multipath TCP remain at the transport The modifications to support Multipath TCP remain at the transport
layer, although some knowledge of the underlying network layer is layer, although some knowledge of the underlying network layer is
required. Multipath TCP SHOULD work with IPv4 and IPv6 required. Multipath TCP SHOULD work with IPv4 and IPv6
interchangeably, i.e. one connection may operate over both IPv4 and interchangeably, i.e. one connection may operate over both IPv4 and
IPv6 networks. IPv6 networks.
2.2.3. Compatibility with other network users 2.2.3. Compatibility with other network users
As a corollary to both network and application compatibility, the As a corollary to both network and application compatibility, the
architecture must enable new Multipath TCP flows to coexist architecture must enable new Multipath TCP flows to coexist
skipping to change at page 9, line 31 skipping to change at page 9, line 35
such as with LEDBAT). The use of multiple paths MUST NOT unduly harm such as with LEDBAT). The use of multiple paths MUST NOT unduly harm
users using single-path TCP at shared bottlenecks, beyond the impact users using single-path TCP at shared bottlenecks, beyond the impact
that would occur from another single-path TCP flow. Multiple that would occur from another single-path TCP flow. Multiple
Multipath TCP flows on a shared bottleneck MUST share bandwidth Multipath TCP flows on a shared bottleneck MUST share bandwidth
between each other with similar fairness to that which occurs at a between each other with similar fairness to that which occurs at a
shared bottleneck with single-path TCP. shared bottleneck with single-path TCP.
2.3. Security Goals 2.3. Security Goals
The extension of TCP with multipath capabilities will bring with it a The extension of TCP with multipath capabilities will bring with it a
number of new threats, analysed in detail in [10]. The security goal number of new threats, analysed in detail in [11]. The security goal
for Multipath TCP is to provide a service no less secure than for Multipath TCP is to provide a service no less secure than
regular, single-path TCP. This will be achieved through a regular, single-path TCP. This will be achieved through a
combination of existing TCP security mechanisms (potentially modified combination of existing TCP security mechanisms (potentially modified
to align with the Multipath TCP extensions) and of protection against to align with the Multipath TCP extensions) and of protection against
the new multipath threats identified. The design decisions derived the new multipath threats identified. The design decisions derived
from this goal are presented in Section 5.8. from this goal are presented in Section 5.8.
2.4. Related Protocols 2.4. Related Protocols
There are several similarities between SCTP [5] and MPTCP, in that There are several similarities between SCTP [5] and MPTCP, in that
both can make use of multiple addresses at end hosts to give some both can make use of multiple addresses at end hosts to give some
multi-path capability. In SCTP, the primary use case is to support multi-path capability. In SCTP, the primary use case is to support
redundancy and mobility for multihomed hosts (i.e. a single path will redundancy and mobility for multihomed hosts (i.e. a single path will
change one of its end host addresses); the simultaneous use of change one of its end host addresses); the simultaneous use of
multiple paths is not supported . Extensions are proposed to support multiple paths is not supported . Extensions are proposed to support
simultaneous multipath transport [11], but these are yet to be simultaneous multipath transport [12], but these are yet to be
standardised. The de facto standard stream-based transport protocol standardised. By far the most widely used stream-based transport
is, however, TCP [1], and SCTP does not meet the network and protocol is, however, TCP [1], and SCTP does not meet the network and
application compatibility goals specified in Section 2.2. For application compatibility goals specified in Section 2.2. For
network compatibility, there are issues with various middleboxes network compatibility, there are issues with various middleboxes
(especially NATs) that are unaware of SCTP and consequently end up (especially NATs) that are unaware of SCTP and consequently end up
blocking it. For application compatibility, applications need to blocking it. For application compatibility, applications need to
actively choose to use SCTP, and with the deployment issues very few actively choose to use SCTP, and with the deployment issues very few
choose to do so. MPTCP's compatibility goals are in part based on choose to do so. MPTCP's compatibility goals are in part based on
these observations of SCTP's deployment issues. these observations of SCTP's deployment issues.
3. An Architectural Basis For Multipath TCP 3. An Architectural Basis For Multipath TCP
We now present one possible transport architecture that we believe We now present one possible transport architecture that we believe
can effectively support the goals for Multipath TCP. The new can effectively support the goals for Multipath TCP. The new
Internet model described here is based on ideas proposed earlier in Internet model described here is based on ideas proposed earlier in
Tng ("Transport next-generation") [12]. While by no means the only Tng ("Transport next-generation") [13]. While by no means the only
possible architecture supporting multipath transport, Tng possible architecture supporting multipath transport, Tng
incorporates many lessons learned from previous transport research incorporates many lessons learned from previous transport research
and development practice, and offers a strong starting point from and development practice, and offers a strong starting point from
which to consider the extant Internet architecture and its bearing on which to consider the extant Internet architecture and its bearing on
the design of any new Internet transports or transport extensions. the design of any new Internet transports or transport extensions.
+------------------+ +------------------+
| Application | | Application |
+------------------+ ^ Application-oriented transport +------------------+ ^ Application-oriented transport
| | | functions (Semantic Layer) | | | functions (Semantic Layer)
skipping to change at page 10, line 45 skipping to change at page 10, line 49
Tng loosely splits the transport layer into "application-oriented" Tng loosely splits the transport layer into "application-oriented"
and "network-oriented" layers, as shown in Figure 4. The and "network-oriented" layers, as shown in Figure 4. The
application-oriented "Semantic" layer implements functions driven application-oriented "Semantic" layer implements functions driven
primarily by concerns of supporting and protecting the application's primarily by concerns of supporting and protecting the application's
end-to-end communication, while the network-oriented "Flow+Endpoint" end-to-end communication, while the network-oriented "Flow+Endpoint"
layer implements functions such as endpoint identification (using layer implements functions such as endpoint identification (using
port numbers) and congestion control. These network-oriented port numbers) and congestion control. These network-oriented
functions, while traditionally located in the ostensibly "end-to-end" functions, while traditionally located in the ostensibly "end-to-end"
Transport layer, have proven in practice to be of great concern to Transport layer, have proven in practice to be of great concern to
network operators and the middleboxes they deploy in the network to network operators and the middleboxes they deploy in the network to
enforce network usage policies[13] [14] or optimize communication enforce network usage policies [14] [15] or optimize communication
performance[15]. Figure 5 shows how middleboxes interact with performance [16]. Figure 5 shows how middleboxes interact with
different layers in this decomposed model of the transport layer: the different layers in this decomposed model of the transport layer: the
application-oriented layer operates end-to-end, while the network- application-oriented layer operates end-to-end, while the network-
oriented layer operates "segment-by-segment" and can be interposed oriented layer operates "segment-by-segment" and can be interposed
upon by middleboxes. upon by middleboxes.
+-------------+ +-------------+ +-------------+ +-------------+
| Application |<------------ end-to-end ------------->| Application | | Application |<------------ end-to-end ------------->| Application |
+-------------+ +-------------+ +-------------+ +-------------+
| Semantic |<------------ end-to-end ------------->| Semantic | | Semantic |<------------ end-to-end ------------->| Semantic |
+-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ +-------------+
skipping to change at page 11, line 25 skipping to change at page 11, line 29
End Host or NAT Enhancing Proxy End Host End Host or NAT Enhancing Proxy End Host
Figure 5: Middleboxes in the new Internet model Figure 5: Middleboxes in the new Internet model
MPTCP's architectural design follows Tng's decomposition as shown in MPTCP's architectural design follows Tng's decomposition as shown in
Figure 6. MPTCP, which provides application compatibility through Figure 6. MPTCP, which provides application compatibility through
the preservation of TCP-like semantics of global ordering of the preservation of TCP-like semantics of global ordering of
application data and reliability, is an instantiation of the application data and reliability, is an instantiation of the
"application-oriented" Semantic layer; whereas the subflow TCP "application-oriented" Semantic layer; whereas the subflow TCP
component, which provides network compatibility by appearing and component, which provides network compatibility by appearing and
behaving as a TCP flow in network, is an instantiation of the behaving as a TCP flow in the network, is an instantiation of the
"network-oriented" Flow+Endpoint layer. "network-oriented" Flow+Endpoint layer.
+--------------------------+ +-------------------------------+ +--------------------------+ +-------------------------------+
| Application | | Application | | Application | | Application |
+--------------------------+ +-------------------------------+ +--------------------------+ +-------------------------------+
| Semantic | | MPTCP | | Semantic | | MPTCP |
|------------+-------------| + - - - - - - - + - - - - - - - + |------------+-------------| + - - - - - - - + - - - - - - - +
| Flow+Endpt | Flow+Endpt | | Subflow (TCP) | Subflow (TCP) | | Flow+Endpt | Flow+Endpt | | Subflow (TCP) | Subflow (TCP) |
+------------+-------------+ +---------------+---------------+ +------------+-------------+ +---------------+---------------+
| Network | Network | | IP | IP | | Network | Network | | IP | IP |
skipping to change at page 13, line 18 skipping to change at page 13, line 25
at the subflow layer. On receipt, the subflow passes its at the subflow layer. On receipt, the subflow passes its
reassembled data to the packet scheduling component for reassembled data to the packet scheduling component for
connection-level reassembly; the data sequence mapping from the connection-level reassembly; the data sequence mapping from the
sender's packet scheduling component allows re-ordering of the sender's packet scheduling component allows re-ordering of the
entire bytestream. entire bytestream.
o Congestion Control: This function coordinates congestion control o Congestion Control: This function coordinates congestion control
across the subflows. As specified, this congestion control across the subflows. As specified, this congestion control
algorithm MUST ensure that a MPTCP connection does not unfairly algorithm MUST ensure that a MPTCP connection does not unfairly
take more bandwidth than a single path TCP flow would take at a take more bandwidth than a single path TCP flow would take at a
shared bottlneck. An algorithm to support this is specified in shared bottleneck. An algorithm to support this is specified in
[6]. [6].
These functions fit together as follows. The Path Management looks These functions fit together as follows. The Path Management looks
after the discovery (and if necessary, initialisation) of multiple after the discovery (and if necessary, initialisation) of multiple
paths between two hosts. The Packet Scheduler then receives a stream paths between two hosts. The Packet Scheduler then receives a stream
of data from the application destined for the network, and undertakes of data from the application destined for the network, and undertakes
the necessary operations on it (such as segmenting the data into the necessary operations on it (such as segmenting the data into
connection-level segments, and adding a connection-level sequence connection-level segments, and adding a connection-level sequence
number) before sending it on to a subflow. The subflow then adds its number) before sending it on to a subflow. The subflow then adds its
own sequence number, acks, and passes them to network. The receiving own sequence number, ACKs, and passes them to network. The receiving
subflow re-orders data (if necessary) and passes it to the packet subflow re-orders data (if necessary) and passes it to the packet
scheduling component, which performs connection level re-ordering, scheduling component, which performs connection level re-ordering,
and sends the data stream to the application. Finally, the and sends the data stream to the application. Finally, the
congestion control component exists as part of the packet scheduling, congestion control component exists as part of the packet scheduling,
in order to schedule which packets should be sent at what rate on in order to schedule which segments should be sent at what rate on
which subflow. which subflow.
5. High-Level Design Decisions 5. High-Level Design Decisions
There is seemingly a wide range of choices when designing a multipath There is seemingly a wide range of choices when designing a multipath
extension to TCP. However, the goals as discussed earlier in this extension to TCP. However, the goals as discussed earlier in this
document constrain the possible solutions, leaving relative little document constrain the possible solutions, leaving relative little
choice in many areas. Here, we outline high-level design choices choice in many areas. Here, we outline high-level design choices
that draw from the architectural basis discussed earlier in that draw from the architectural basis discussed earlier in
Section 3, which the design of MPTCP [4] takes into account. Section 3, which the design of MPTCP [4] takes into account.
skipping to change at page 14, line 13 skipping to change at page 14, line 21
subflow-level sequence space. subflow-level sequence space.
The alternative approach would be to use a single connection level The alternative approach would be to use a single connection level
sequence number, which gets sent on multiple subflows. This has two sequence number, which gets sent on multiple subflows. This has two
problems: first, the individual subflows will appear to the network problems: first, the individual subflows will appear to the network
as TCP sessions with gaps in the sequence space; this in turn may as TCP sessions with gaps in the sequence space; this in turn may
upset certain middleboxes such as intrusion detection systems, or upset certain middleboxes such as intrusion detection systems, or
certain transparent proxies, and would thus go against the network certain transparent proxies, and would thus go against the network
compatibility goal. Second, the sender would not be able to compatibility goal. Second, the sender would not be able to
attribute packet losses or receptions to the correct path when the attribute packet losses or receptions to the correct path when the
same packet is sent on multiple paths (i.e. in the case of same segment is sent on multiple paths (i.e. in the case of
retransmissions). retransmissions).
The sender must be able to tell the receiver how to reassemble the The sender must be able to tell the receiver how to reassemble the
data, for delivery to the application. In order to achieve this, the data, for delivery to the application. In order to achieve this, the
receiver must determine how subflow-level data (carying subflow receiver must determine how subflow-level data (carrying subflow
sequence numbers) maps at the connection level. We refer to this as sequence numbers) maps at the connection level. We refer to this as
the Data Sequence Mapping. This mapping takes the form (data seq, the Data Sequence Mapping. This mapping takes the form (data seq,
subflow seq, length), i.e. for a given number of bytes (the length), subflow seq, length), i.e. for a given number of bytes (the length),
the subflow sequence space beginning at the given sequence number the subflow sequence space beginning at the given sequence number
maps to the connection-level sequence space (beginning at the given maps to the connection-level sequence space (beginning at the given
data seq number). This information could conceivably have various data seq number). This information could conceivably have various
sources. sources.
One option to signal the Data Sequence Mapping would be to use One option to signal the Data Sequence Mapping would be to use
existing fields in the TCP segment (such as subflow seqno, length) existing fields in the TCP segment (such as subflow seqno, length)
skipping to change at page 14, line 41 skipping to change at page 14, line 49
that resegment or assemble data, since there is no specified that resegment or assemble data, since there is no specified
behaviour for coalescing TCP options. If one signalled (data seqno, behaviour for coalescing TCP options. If one signalled (data seqno,
length), this would still be vulnerable to middleboxes that coalesce length), this would still be vulnerable to middleboxes that coalesce
segments and do not understand MPTCP signalling so do not correctly segments and do not understand MPTCP signalling so do not correctly
rewrite the options. rewrite the options.
Because of these potential issues, the design decision taken in the Because of these potential issues, the design decision taken in the
MPTCP protocol is that whenever a mapping for subflow data needs to MPTCP protocol is that whenever a mapping for subflow data needs to
be conveyed to the other host, all three pieces of data (data seq, be conveyed to the other host, all three pieces of data (data seq,
subflow seq, length) must be sent. To reduce the overhead, it would subflow seq, length) must be sent. To reduce the overhead, it would
be permissable for the mapping to be sent periodically and cover more be permissible for the mapping to be sent periodically and cover more
than a single segment. Further experimentation is required to than a single segment. Further experimentation is required to
determine what tradeoffs exist regarding the frequency at which determine what tradeoffs exist regarding the frequency at which
mappings should be sent. It could also be excluded entirely in the mappings should be sent. It could also be excluded entirely in the
case of a connection before more than one subflow is used, where the case of a connection before more than one subflow is used, where the
data-level and subflow-level sequence space is the same. data-level and subflow-level sequence space is the same.
5.2. Reliability and Retransmissions 5.2. Reliability and Retransmissions
MPTCP features acknowledgements at connection-level as well as MPTCP features acknowledgements at connection-level as well as
subflow-level acknowledgements, in order to provide a robust service subflow-level acknowledgements, in order to provide a robust service
to the application. to the application.
Under normal behaviour, MPTCP can use the data sequence mapping and Under normal behaviour, MPTCP can use the data sequence mapping and
subflow ACKs to decide when a connection-level segment was received. subflow ACKs to decide when a connection-level segment was received.
The transmission of TCP ACKs for a subflow are handled entirely at The transmission of TCP ACKs for a subflow are handled entirely at
the subflow level, in order to maintain TCP semantics and trigger the subflow level, in order to maintain TCP semantics and trigger
subflow-level retransmissions. This has certain implications on end- subflow-level retransmissions. This has certain implications on end-
to-end semantics. It means that once a packet is acked at the to-end semantics. It means that once a segment is ACKed at the
subflow level it cannot be discarded in the re-order buffer at the subflow level it cannot be discarded in the re-order buffer at the
connection level. Secondly, unlike in standard TCP, a receiver connection level. Secondly, unlike in standard TCP, a receiver
cannot simply drop out-of-order segments if needed (for instance, due cannot simply drop out-of-order segments if needed (for instance, due
to memory pressure). Under certain circumstances, therefore, it may to memory pressure). Under certain circumstances, therefore, it may
be desirable to drop packets after acknowledgement on the subflow but be desirable to drop segments after acknowledgement on the subflow
before delivery to the application, and this can be facilitated by a but before delivery to the application, and this can be facilitated
connection-level acknowledgement. by a connection-level acknowledgement.
Furthermore, it is possible to conceive of some cases where Furthermore, it is possible to conceive of some cases where
connection-level acknowledgements could improve robustness. Consider connection-level acknowledgements could improve robustness. Consider
a subflow traversing a transparent proxy: if the proxy acks a segment a subflow traversing a transparent proxy: if the proxy ACKs a segment
and then crashes, the sender will not retransmit the lost segment on and then crashes, the sender will not retransmit the lost segment on
another subflow, as it thinks the segment has been received. The another subflow, as it thinks the segment has been received. The
connection grinds to a halt despite having other working subflows, connection grinds to a halt despite having other working subflows,
and the sender would be unable to determine the cause of the problem. and the sender would be unable to determine the cause of the problem.
An example situation where this may occur would be mobility between An example situation where this may occur would be mobility between
wireless access points, each of which operates a transport-level wireless access points, each of which operates a transport-level
proxy. Finally, as an optimisation, it may be feasible for a proxy. Finally, as an optimisation, it may be feasible for a
connection-level acknowledgement to be transmitted over the shortest connection-level acknowledgement to be transmitted over the shortest
Round-Trip Time (RTT) path, potentially reducing send buffer Round-Trip Time (RTT) path, potentially reducing send buffer
requirements (see Section 5.3). requirements (see Section 5.3).
Therefore, to provide a fully robust multipath TCP solution, MPTCP Therefore, to provide a fully robust multipath TCP solution given the
SHOULD feature explicit connection-level acknowledgements, in above constraints, MPTCP for use on the public Internet MUST feature
addition to subflow-level acknowledgements. A connection-level explicit connection-level acknowledgements, in addition to subflow-
acknowledgement would only be required in order to signal when the level acknowledgements. A connection-level acknowledgement would
receive window moves forward; the heuristics for using such a signal only be required in order to signal when the receive window moves
are discussed in more detail in the protocol specification [4]. forward; the heuristics for using such a signal are discussed in more
detail in the protocol specification [4].
Regarding retransmissions, it MUST be possible for a packet to be Regarding retransmissions, it MUST be possible for a segments to be
retransmitted on a different subflow to that on which it was retransmitted on a different subflow to that on which it was
originally sent. This is one of MPTCP's core goals, in order to originally sent. This is one of MPTCP's core goals, in order to
maintain integrity during temporary or permanent subflow failure, and maintain integrity during temporary or permanent subflow failure, and
this is enabled by the dual sequence number space. this is enabled by the dual sequence number space.
The scheduling of retransmissions will have significant impact on The scheduling of retransmissions will have significant impact on
MPTCP user experience. The current MPTCP specification suggests that MPTCP user experience. The current MPTCP specification suggests that
data outstanding on subflows that have timed out should be data outstanding on subflows that have timed out should be
rescheduled for transmission on different subflows. This behaviour rescheduled for transmission on different subflows. This behaviour
aims to minimize disruption when a path breaks, and uses the first aims to minimize disruption when a path breaks, and uses the first
timeout as indicators. More conservative versions would be to use timeout as indicators. More conservative versions would be to use
second or third timeouts for the same packet. second or third timeouts for the same segment.
Typically, fast retransmit on an individual subflow will not trigger Typically, fast retransmit on an individual subflow will not trigger
retransmission on another subflow, although this may still be retransmission on another subflow, although this may still be
desirable in certain cases, for instance to reduce the receive buffer desirable in certain cases, for instance to reduce the receive buffer
requirements. However, in all cases with retransmissions on requirements. However, in all cases with retransmissions on
different subflows, the lost packets SHOULD still be sent on the path different subflows, the lost segments SHOULD still be sent on the
that lost them. This is currently believed to be necessary to path that lost them. This is currently believed to be necessary to
maintain subflow integrity, as per the network compatiblity goal. By maintain subflow integrity, as per the network compatibility goal.
doing this, throughput will be wasted, and it is unclear at this By doing this, some efficiency is lost, and it is unclear at this
point what the optimal retransmit strategy is. point what the optimal retransmit strategy is.
Large-scale experiments are therefore required in order to determine Large-scale experiments are therefore required in order to determine
the most appropriate retransmission strategy, and recommendations the most appropriate retransmission strategy, and recommendations
will be refined once more information is available. will be refined once more information is available.
5.3. Buffers 5.3. Buffers
To ensure in-order delivery, MPTCP must use a connection level To ensure in-order delivery, MPTCP must use a connection level
receive buffer, where segments are placed until they are in order and receive buffer, where segments are placed until they are in order and
skipping to change at page 16, line 36 skipping to change at page 16, line 46
receive buffer to 2*BDP (Bandwidth-Delay Product, i.e. BDP = BW*RTT, receive buffer to 2*BDP (Bandwidth-Delay Product, i.e. BDP = BW*RTT,
where BW = Bandwidth and RTT = Round-Trip Time). One BDP allows where BW = Bandwidth and RTT = Round-Trip Time). One BDP allows
supporting reordering of segments by the network. The other BDP supporting reordering of segments by the network. The other BDP
allows the connection to continue during fast retransmit: when a allows the connection to continue during fast retransmit: when a
segment is fast retransmitted, the receiver must be able to store segment is fast retransmitted, the receiver must be able to store
incoming data during one more RTT. incoming data during one more RTT.
For MPTCP, the story is a bit more complicated. The ultimate goal is For MPTCP, the story is a bit more complicated. The ultimate goal is
that a subflow packet loss or subflow failure should not affect the that a subflow packet loss or subflow failure should not affect the
throughput of other working subflows; the receiver should have enough throughput of other working subflows; the receiver should have enough
buffering to store all data until the missing packet is re- buffering to store all data until the missing segment is re-
transmitted and reaches the destination. transmitted and reaches the destination.
The worst case scenario would be when the subflow with the highest The worst case scenario would be when the subflow with the highest
RTT/RTO (Round-Trip Time or Retransmission TimeOut) experiences a RTT/RTO (Round-Trip Time or Retransmission TimeOut) experiences a
timeout; in that case the receiver has to buffer data from all timeout; in that case the receiver has to buffer data from all
subflows for the duration of the RTO. Thus, the smallest connection- subflows for the duration of the RTO. Thus, the smallest connection-
level receive buffer that would be needed to avoid stalling with level receive buffer that would be needed to avoid stalling with
subflow failures is sum(BW_i)*RTO_max, where BW_i = Bandwidth for subflow failures is sum(BW_i)*RTO_max, where BW_i = Bandwidth for
each subflow and RTO_max is the largest RTO across all subflows. each subflow and RTO_max is the largest RTO across all subflows.
skipping to change at page 17, line 10 skipping to change at page 17, line 21
purposes. A more sensible requirement is to avoid stalls in the purposes. A more sensible requirement is to avoid stalls in the
absence of timeouts. Therefore, the RECOMMENDED receive buffer is absence of timeouts. Therefore, the RECOMMENDED receive buffer is
2*sum(BW_i)*RTT_max, where RTT_max is the largest RTT across all 2*sum(BW_i)*RTT_max, where RTT_max is the largest RTT across all
subflows. This buffer sizing ensures subflows do not stall when fast subflows. This buffer sizing ensures subflows do not stall when fast
retransmit is triggered on any subflow. retransmit is triggered on any subflow.
The resulting buffer size should be small enough for practical use. The resulting buffer size should be small enough for practical use.
However, there may be extreme cases where fast, high throughput paths However, there may be extreme cases where fast, high throughput paths
(e.g. 100Mb/s, 10ms RTT) are used in conjunction with slow paths (e.g. 100Mb/s, 10ms RTT) are used in conjunction with slow paths
(e.g. 1Mb/s, 1000ms RTT). In that case the required receive buffer (e.g. 1Mb/s, 1000ms RTT). In that case the required receive buffer
would be 12.5MB, which is likely too big. In these cases a Multipath would be 12.5MB, which is likely too big. In extreme cases such as
TCP scheduler SHOULD use only the fast path, potentially falling back this example, it may be prudent to only use some of the fastest
to the slow path if the fast path fails. available paths for the MPTCP connection, potentially using the slow
path(s) for backup only.
Send Buffer: The RECOMMENDED send buffer is the same size as the Send Buffer: The RECOMMENDED send buffer is the same size as the
recommended receive buffer i.e., 2*sum(BW_i)*RTT_max. This is recommended receive buffer i.e., 2*sum(BW_i)*RTT_max. This is
because the sender must store locally the segments sent but because the sender must store locally the segments sent but
unacknowledged by the connection level ACK. The send buffer size unacknowledged by the connection level ACK. The send buffer size
matters particularly for hosts that maintain a large number of matters particularly for hosts that maintain a large number of
ongoing connections. If the required send buffer is too large, a ongoing connections. If the required send buffer is too large, a
host can choose to only send data on the fast subflows, using the host can choose to only send data on the fast subflows, using the
slow subflows only in cases of failure. slow subflows only in cases of failure.
skipping to change at page 18, line 38 skipping to change at page 18, line 49
seamlessly falling back to regular TCP on the loss of control data. seamlessly falling back to regular TCP on the loss of control data.
Both option and payload encoding may interfere with offloading of TCP Both option and payload encoding may interfere with offloading of TCP
processing to high speed network interface cards, such as processing to high speed network interface cards, such as
segmentation, checksumming, and reassembly. For network cards segmentation, checksumming, and reassembly. For network cards
supporting MPTCP, signalling in TCP options should simplify supporting MPTCP, signalling in TCP options should simplify
offloading due to the separate handling of MPTCP signalling and data. offloading due to the separate handling of MPTCP signalling and data.
5.5. Path Management 5.5. Path Management
Currently, the network does not expose multiple paths between hosts. Currently, the network does not expose path diversity between pairs
In the typical case, MPTCP uses multiple addresses at one or both of IP addresses. In order to achieve path diversity from today's IP
hosts to infer different paths across the network. It is expected networks, in the typical case MPTCP uses multiple addresses at one or
that these paths, whilst not necesarily entirely non-overlapping, both hosts to infer different paths across the network. It is
will be sufficiently disjoint to allow multipath to achieve improved expected that these paths, whilst not necessarily entirely non-
throughput and robustness. The use of multiple IP addresses is a overlapping, will be sufficiently disjoint to allow multipath to
simple mechanism that requires no additional features in the network. achieve improved throughput and robustness. The use of multiple IP
addresses is a simple mechanism that requires no additional features
in the network.
Multiple different (source, destination) address pairs will thus be Multiple different (source, destination) address pairs will thus be
used as path selectors in most cases. Each path will be identified used as path selectors in most cases. Each path will be identified
by a TCP 4-tuple (i.e. source address, destination address, source by a standard five-tuple (i.e. source address, destination address,
port, destination port), however, which can allow the extension of source port, destination port, protocol), however, which can allow
MPTCP to use such 4-tuples as path selectors. This will allow hosts the extension of MPTCP to use ports as well as addresses as path
to use MPTCP to load balance to different ports, for example if the selectors. This will allow hosts to use port-based load balancing
network routes different ports over different paths (which may be the with MPTCP, for example if the network routes different ports over
case with technologies such as Equal Cost MultiPath (ECMP) routing different paths (which may be the case with technologies such as
[16]). Equal Cost MultiPath (ECMP) routing [17]).
For increased chance of successfully setting up additional subflows For increased chance of successfully setting up additional subflows
(such as when one end is behind a firewall, NAT, or other restrictive (such as when one end is behind a firewall, NAT, or other restrictive
middlebox), either host SHOULD be able to add new subflows to a MPTCP middlebox), either host SHOULD be able to add new subflows to a MPTCP
connection. MPTCP MUST be able to handle paths that appear and connection. MPTCP MUST be able to handle paths that appear and
disappear during the lifetime of a connection (for example, through disappear during the lifetime of a connection (for example, through
the activation of an additional network interface). the activation of an additional network interface).
The modularity of path management will permit alternative mechanisms The path management is a separate function from the packet
to be employed if appropriate in the future. scheduling, subflow interface, and congestion control functions of
MPTCP, as documented in Section 4. As such it would be feasible to
replace this IP-address-based design with an alternative path
selection mechanism in the future, with no significant changes to the
other functional components.
5.6. Connection Identification 5.6. Connection Identification
Since a MPTCP connection may not be bound to a traditional 5-tuple Since a MPTCP connection may not be bound to a traditional 5-tuple
(source address and port, destination address and port, protocol (source address and port, destination address and port, protocol
number) for the entirety of its existence, it is desirable to provide number) for the entirety of its existence, it is desirable to provide
a new mechanism for connection identification. This will be useful a new mechanism for connection identification. This will be useful
for MPTCP-aware applications, and for the MPTCP implementation (and for MPTCP-aware applications, and for the MPTCP implementation (and
MPTCP-aware middleboxes) to have a unique identifier with which to MPTCP-aware middleboxes) to have a unique identifier with which to
associate the multiple subflows. associate the multiple subflows.
skipping to change at page 20, line 25 skipping to change at page 20, line 42
path than if it was a single path flow using only that route - this path than if it was a single path flow using only that route - this
is particularly relevant for shared bottlenecks); and balance is particularly relevant for shared bottlenecks); and balance
congestion by moving traffic away from the most congested paths. To congestion by moving traffic away from the most congested paths. To
achieve these goals, the congestion control algorithms on each achieve these goals, the congestion control algorithms on each
subflow must be coupled in some way. A proposal for a suitable subflow must be coupled in some way. A proposal for a suitable
congestion control algorithm is given in [6]. congestion control algorithm is given in [6].
5.8. Security 5.8. Security
A detailed threat analysis for Multipath TCP is presented in a A detailed threat analysis for Multipath TCP is presented in a
separate document [10]. This focuses on flooding attacks and separate document [11]. This focuses on flooding attacks and
hijacking attacks that can be launched against a Multipath TCP hijacking attacks that can be launched against a Multipath TCP
connection. connection.
The basic security goal of Multipath TCP, as introduced in The basic security goal of Multipath TCP, as introduced in
Section 2.3, can be stated as: "provide a solution that is no worse Section 2.3, can be stated as: "provide a solution that is no worse
than standard TCP". than standard TCP".
From the threat analysis, and with this goal in mind, three key From the threat analysis, and with this goal in mind, three key
security requirements can be identified. A multi-addressed Multipath security requirements can be identified. A multi-addressed Multipath
TCP SHOULD be able to: TCP SHOULD be able to:
skipping to change at page 21, line 4 skipping to change at page 21, line 21
o Provide verification that the peer can receive traffic at a new o Provide verification that the peer can receive traffic at a new
address before adding it (i.e. verify that the address belongs to address before adding it (i.e. verify that the address belongs to
the other host, to prevent flooding attacks). the other host, to prevent flooding attacks).
o Provide replay protection, i.e. ensure that a request to add/ o Provide replay protection, i.e. ensure that a request to add/
remove a subflow is 'fresh'. remove a subflow is 'fresh'.
Additional mechanisms have been deployed as part of standard TCP Additional mechanisms have been deployed as part of standard TCP
stacks to provide resistance to Denial-of-Service attacks. For stacks to provide resistance to Denial-of-Service attacks. For
example, there are various mechanisms to protect against TCP reset example, there are various mechanisms to protect against TCP reset
attacks [17], and Multipath TCP should continue to support similar attacks [18], and Multipath TCP should continue to support similar
protection. In addition, TCP SYN Cookies [18] were developed to protection. In addition, TCP SYN Cookies [19] were developed to
allow a TCP server to defer the creation of session state in the allow a TCP server to defer the creation of session state in the
SYN_RCVD state, and remain stateless until the ESTABLISHED state had SYN_RCVD state, and remain stateless until the ESTABLISHED state had
been reached. Multipath TCP should, ideally, continue to provide been reached. Multipath TCP should, ideally, continue to provide
such functionality and, at a minimum, avoid significant computational such functionality and, at a minimum, avoid significant computational
burden prior to reaching the ESTABLISHED state (of the Multipath TCP burden prior to reaching the ESTABLISHED state (of the Multipath TCP
connection as a whole). connection as a whole).
It should be noted that aspects of the Multipath TCP design space It should be noted that aspects of the Multipath TCP design space
place constraints on the security solution: place constraints on the security solution:
o The use of TCP options significantly limits the amount of o The use of TCP options significantly limits the amount of
information that can be carried in the handshake. information that can be carried in the handshake.
o The need to work through middleboxes results in the need to handle o The need to work through middleboxes results in the need to handle
mutability of packets. mutability of packets.
o The desire to support a 'break-before-make' (as well as a 'make- o The desire to support a 'break-before-make' (as well as a 'make-
before-break') approach to adding subflows implies that a host before-break') approach to adding subflows (within a limited time
cannot rely on using a pre-existing subflow to support the period) implies that a host cannot rely on using a pre-existing
addition of a new one. subflow to support the addition of a new one.
The MPTCP protocol will be designed with these security requirements The MPTCP protocol will be designed with these security requirements
in mind, and the protocol specification [4] will document how these in mind, and the protocol specification [4] will document how these
are met. are met.
6. Interactions with Applications 6. Interactions with Applications
Interactions with applications are presented in [7] - including, but Interactions with applications are presented in [7] - including, but
not limited to, performances changes that may be expected, semantic not limited to, performances changes that may be expected, semantic
changes, and new features that may be requested through an enhanced changes, and new features that may be requested through an enhanced
API. API.
TCP features the ability to send out-of-band ("Urgent") data.
Although the use of Urgent data is not recommended (see [20]), MPTCP
SHOULD still support this feature if requested by an application. An
MPTCP implementation SHOULD send the Urgent data on one subflow of
the MPTCP connection; it MAY choose this to be the best performing
subflow.
7. Interactions with Middleboxes 7. Interactions with Middleboxes
As discussed in Section 2.2, it is a goal of MPTCP to be deployable As discussed in Section 2.2, it is a goal of MPTCP to be deployable
today and thus compatible with the majority of middleboxes. This today and thus compatible with the majority of middleboxes. This
section summarises the issues that may arise with NATs, firewalls, section summarises the issues that may arise with NATs, firewalls,
proxies, intrusion detection systems, and other middleboxes that, if proxies, intrusion detection systems, and other middleboxes that, if
not considered in the protocol design, may hinder its deployment. not considered in the protocol design, may hinder its deployment.
This section is intended primarily as a description of options and This section is intended primarily as a description of options and
considerations only. Protocol-specific solutions to these issues considerations only. Protocol-specific solutions to these issues
will be given in the companion documents. will be given in the companion documents.
Multipath TCP will be deployed in a network that no longer provides Multipath TCP will be deployed in a network that no longer provides
just basic datagram delivery. A miriad of middleboxes are deployed just basic datagram delivery. A myriad of middleboxes are deployed
to optimize various perceived problems with the Internet protocols: to optimize various perceived problems with the Internet protocols:
NATs primarily address space shortage [13], Performance Enhancing NATs primarily address space shortage [14], Performance Enhancing
Proxies (PEPs) optimize TCP for different link characteristics [15], Proxies (PEPs) optimize TCP for different link characteristics [16],
firewalls [14] and intrusion detection systems try to block malicious firewalls [15] and intrusion detection systems try to block malicious
content from reaching a host, and traffic normalizers [19] ensure a content from reaching a host, and traffic normalizers [21] ensure a
consistent view of the traffic stream to Intrusion Detection Systems consistent view of the traffic stream to Intrusion Detection Systems
(IDS) and hosts. (IDS) and hosts.
All these middleboxes optimize current applications at the expense of All these middleboxes optimize current applications at the expense of
future applications. In effect, future applications will often need future applications. In effect, future applications will often need
to behave in a similar fashion to existing ones, in order to increase to behave in a similar fashion to existing ones, in order to increase
the chances of successful deployment. Further, the precise behaviour the chances of successful deployment. Further, the precise behaviour
of all these middleboxes is not clearly specified, and implementation of all these middleboxes is not clearly specified, and implementation
errors make matters worse, raising the bar for the deployment of new errors make matters worse, raising the bar for the deployment of new
technologies. technologies.
skipping to change at page 23, line 20 skipping to change at page 23, line 45
o Content-aware Firewalls: Some middleboxes may actively change data o Content-aware Firewalls: Some middleboxes may actively change data
in packets, such as re-writing URIs in HTTP traffic. in packets, such as re-writing URIs in HTTP traffic.
In addition, all classes of middleboxes may affect TCP traffic in the In addition, all classes of middleboxes may affect TCP traffic in the
following ways: following ways:
o TCP Options: some middleboxes may drop packets with unknown TCP o TCP Options: some middleboxes may drop packets with unknown TCP
options, or strip those options from the packets. options, or strip those options from the packets.
o Segmentation and Colescing: middleboxes (or even something as o Segmentation and Coalescing: middleboxes (or even something as
close to the end host as TCP Segmentation Offloading (TSO) on a close to the end host as TCP Segmentation Offloading (TSO) on a
Network Interface Card (NIC)) may change the packet boundaries Network Interface Card (NIC)) may change the packet boundaries
from those which the sender intended. It may do this by splitting from those which the sender intended. It may do this by splitting
packets, or coalescing them together. This leads to two major packets, or coalescing them together. This leads to two major
impacts: we cannot guarantee where a packet boundary will be, and impacts: we cannot guarantee where a packet boundary will be, and
we cannot say for sure what a middlebox will do with TCP options we cannot say for sure what a middlebox will do with TCP options
in these cases (they may be repeated, dropped, or sent only once). in these cases (they may be repeated, dropped, or sent only once).
8. Contributors 8. Contributors
The authors would like to acknowledge the contributions of Andrew The authors would like to acknowledge the contributions of Andrew
McDonald and Bryan Ford to this document. McDonald and Bryan Ford to this document.
The authors would also like to thank the following people for The authors would also like to thank the following people for
detailed reviews: Olivier Bonaventure, Gorry Fairhurst, Iljitsch van detailed reviews: Olivier Bonaventure, Gorry Fairhurst, Iljitsch van
Beijnum, Philip Eardley, and Michael Scharf. Beijnum, Philip Eardley, Michael Scharf, Lars Eggert, and Cullen
Jennings.
9. Acknowledgements 9. Acknowledgements
Alan Ford, Costin Raiciu, Mark Handley, and Sebastien Barre are Alan Ford, Costin Raiciu, Mark Handley, and Sebastien Barre are
supported by Trilogy (http://www.trilogy-project.org), a research supported by Trilogy (http://www.trilogy-project.org), a research
project (ICT-216372) partially funded by the European Community under project (ICT-216372) partially funded by the European Community under
its Seventh Framework Program. The views expressed here are those of its Seventh Framework Program. The views expressed here are those of
the author(s) only. The European Commission is not liable for any the author(s) only. The European Commission is not liable for any
use that may be made of the information in this document. use that may be made of the information in this document.
10. IANA Considerations 10. IANA Considerations
None. None.
11. Security Considerations 11. Security Considerations
This informational document provides an architectural overview for This informational document provides an architectural overview for
Multipath TCP and so does not, in itself, raise any security issues. Multipath TCP and so does not, in itself, raise any security issues.
A separate threat analysis [10] lists threats that can exist with a A separate threat analysis [11] lists threats that can exist with a
Multipath TCP. However, a protocol based on the architecture in this Multipath TCP. However, a protocol based on the architecture in this
document will have a number of security requirements. The high level document will have a number of security requirements. The high level
goals for such a protocol are identified in Section 2.3, whilst goals for such a protocol are identified in Section 2.3, whilst
Section 5.8 provides more detailed discussion of security Section 5.8 provides more detailed discussion of security
requirements and design decisions which are applied in the MPTCP requirements and design decisions which are applied in the MPTCP
protocol design [4]. protocol design [4].
12. References 12. References
12.1. Normative References 12.1. Normative References
skipping to change at page 24, line 42 skipping to change at page 25, line 20
<http://ccr.sigcomm.org/online/files/p47-handleyA4.pdf>. <http://ccr.sigcomm.org/online/files/p47-handleyA4.pdf>.
[4] Ford, A., Raiciu, C., and M. Handley, "TCP Extensions for [4] Ford, A., Raiciu, C., and M. Handley, "TCP Extensions for
Multipath Operation with Multiple Addresses", Multipath Operation with Multiple Addresses",
draft-ietf-mptcp-multiaddressed-02 (work in progress), draft-ietf-mptcp-multiaddressed-02 (work in progress),
October 2010. October 2010.
[5] Stewart, R., "Stream Control Transmission Protocol", RFC 4960, [5] Stewart, R., "Stream Control Transmission Protocol", RFC 4960,
September 2007. September 2007.
[6] Raiciu, C., Handley, M., and D. Wischik, "Coupled Multipath- [6] Raiciu, C., Handley, M., and D. Wischik, "Coupled Congestion
Aware Congestion Control", draft-ietf-mptcp-congestion-00 (work Control for Multipath Transport Protocols",
in progress), July 2010. draft-ietf-mptcp-congestion-01 (work in progress),
January 2011.
[7] Scharf, M. and A. Ford, "MPTCP Application Interface [7] Scharf, M. and A. Ford, "MPTCP Application Interface
Considerations", draft-ietf-mptcp-api-00 (work in progress), Considerations", draft-ietf-mptcp-api-00 (work in progress),
November 2010. November 2010.
[8] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and Issues", [8] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and Issues",
RFC 3234, February 2002. RFC 3234, February 2002.
[9] Carpenter, B., "Internet Transparency", RFC 2775, [9] Carpenter, B., "Internet Transparency", RFC 2775,
February 2000. February 2000.
[10] Bagnulo, M., "Threat Analysis for Multi-addressed/Multi-path [10] Mathis, M., Mahdavi, J., Floyd, S., and A. Romanow, "TCP
Selective Acknowledgment Options", RFC 2018, October 1996.
[11] Bagnulo, M., "Threat Analysis for Multi-addressed/Multi-path
TCP", draft-ietf-mptcp-threat-06 (work in progress), TCP", draft-ietf-mptcp-threat-06 (work in progress),
December 2010. December 2010.
[11] Becke, M., Dreibholz, T., Iyengar, J., Natarajan, P., and M. [12] Becke, M., Dreibholz, T., Iyengar, J., Natarajan, P., and M.
Tuexen, "Load Sharing for the Stream Control Transmission Tuexen, "Load Sharing for the Stream Control Transmission
Protocol (SCTP)", draft-tuexen-tsvwg-sctp-multipath-00 (work in Protocol (SCTP)", draft-tuexen-tsvwg-sctp-multipath-01 (work in
progress), July 2010. progress), December 2010.
[12] Ford, B. and J. Iyengar, "Breaking Up the Transport Logjam", [13] Ford, B. and J. Iyengar, "Breaking Up the Transport Logjam",
ACM HotNets, October 2008. ACM HotNets, October 2008.
[13] Srisuresh, P. and K. Egevang, "Traditional IP Network Address [14] Srisuresh, P. and K. Egevang, "Traditional IP Network Address
Translator (Traditional NAT)", RFC 3022, January 2001. Translator (Traditional NAT)", RFC 3022, January 2001.
[14] Freed, N., "Behavior of and Requirements for Internet [15] Freed, N., "Behavior of and Requirements for Internet
Firewalls", RFC 2979, October 2000. Firewalls", RFC 2979, October 2000.
[15] Border, J., Kojo, M., Griner, J., Montenegro, G., and Z. [16] Border, J., Kojo, M., Griner, J., Montenegro, G., and Z.
Shelby, "Performance Enhancing Proxies Intended to Mitigate Shelby, "Performance Enhancing Proxies Intended to Mitigate
Link-Related Degradations", RFC 3135, June 2001. Link-Related Degradations", RFC 3135, June 2001.
[16] Hopps, C., "Analysis of an Equal-Cost Multi-Path Algorithm", [17] Hopps, C., "Analysis of an Equal-Cost Multi-Path Algorithm",
RFC 2992, November 2000. RFC 2992, November 2000.
[17] Ramaiah, A., Stewart, R., and M. Dalal, "Improving TCP's [18] Ramaiah, A., Stewart, R., and M. Dalal, "Improving TCP's
Robustness to Blind In-Window Attacks", RFC 5961, August 2010. Robustness to Blind In-Window Attacks", RFC 5961, August 2010.
[18] Eddy, W., "TCP SYN Flooding Attacks and Common Mitigations", [19] Eddy, W., "TCP SYN Flooding Attacks and Common Mitigations",
RFC 4987, August 2007. RFC 4987, August 2007.
[19] Handley, M., Paxson, V., and C. Kreibich, "Network Intrusion [20] Gont, F. and A. Yourtchenko, "On the Implementation of the TCP
Urgent Mechanism", RFC 6093, January 2011.
[21] Handley, M., Paxson, V., and C. Kreibich, "Network Intrusion
Detection: Evasion, Traffic Normalization, and End-to-End Detection: Evasion, Traffic Normalization, and End-to-End
Protocol Semantics", Usenix Security 2001, 2001, <http:// Protocol Semantics", Usenix Security 2001, 2001, <http://
www.usenix.org/events/sec01/full_papers/handley/handley.pdf>. www.usenix.org/events/sec01/full_papers/handley/handley.pdf>.
Appendix A. Changelog Appendix A. Changelog
(For removal by the RFC Editor) (For removal by the RFC Editor)
A.1. Changes since draft-ietf-mptcp-architecture-02 A.1. Changes since draft-ietf-mptcp-architecture-03
o Responded to AD review comments.
A.2. Changes since draft-ietf-mptcp-architecture-02
o Responded to WG last call review comments. Included editorial o Responded to WG last call review comments. Included editorial
fixes, adding Section 2.4, and improving Section 5.4 and fixes, adding Section 2.4, and improving Section 5.4 and
Section 7. Section 7.
A.2. Changes since draft-ietf-mptcp-architecture-01 A.3. Changes since draft-ietf-mptcp-architecture-01
o Responded to review comments. o Responded to review comments.
o Added security sections. o Added security sections.
A.3. Changes since draft-ietf-mptcp-architecture-00 A.4. Changes since draft-ietf-mptcp-architecture-00
o Added middlebox compatibility discussion (Section 7). o Added middlebox compatibility discussion (Section 7).
o Clarified path identification (TCP 4-tuple) in Section 5.5. o Clarified path identification (TCP 4-tuple) in Section 5.5.
o Added brief scenario and diagram to Section 1.3. o Added brief scenario and diagram to Section 1.3.
Authors' Addresses Authors' Addresses
Alan Ford (editor) Alan Ford (editor)
 End of changes. 72 change blocks. 
142 lines changed or deleted 172 lines changed or added

This html diff was produced by rfcdiff 1.40. The latest version is available from http://tools.ietf.org/tools/rfcdiff/