draft-ietf-tcpm-fastopen-02.txt   draft-ietf-tcpm-fastopen-03.txt 
Internet Draft Y. Cheng Internet Draft Y. Cheng
draft-ietf-tcpm-fastopen-02.txt J. Chu draft-ietf-tcpm-fastopen-03.txt J. Chu
Intended status: Experimental S. Radhakrishnan Intended status: Experimental S. Radhakrishnan
Expiration date: April, 2013 A. Jain Expiration date: August, 2013 A. Jain
Google, Inc. Google, Inc.
Octobor 22, 2012 Feburary 25, 2013
TCP Fast Open TCP Fast Open
Status of this Memo Status of this Memo
Distribution of this memo is unlimited. Distribution of this memo is unlimited.
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
skipping to change at page 2, line 21 skipping to change at page 2, line 21
(3WHS) to complete before data can be exchanged. (3WHS) to complete before data can be exchanged.
Terminology Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
TFO refers to TCP Fast Open. Client refers to the TCP's active open TFO refers to TCP Fast Open. Client refers to the TCP's active open
side and server refers to the TCP's passive open side. side and server refers to the TCP's passive open side.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Relaxing TCP semantics on duplicated SYNs . . . . . . . . . 4
2.2. SYNs with spoofed IP addresses . . . . . . . . . . . . . . 4
3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5
4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 7
4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 7
4.1.1. TCP Options . . . . . . . . . . . . . . . . . . . . . . 7
4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 8
4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 9
4.2. Fast Open Protocol . . . . . . . . . . . . . . . . . . . . 9
4.2.1. Fast Open Cookie Request . . . . . . . . . . . . . . . 10
4.2.2. TCP Fast Open . . . . . . . . . . . . . . . . . . . . . 11
5. Reliability and Deployment Issues . . . . . . . . . . . . . . . 13
6. Security Considerations . . . . . . . . . . . . . . . . . . . . 14
6.1. Server Resource Exhaustion Attack by SYN Flood with Valid
Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . 14
6.2. Amplified Reflection Attack to Random Host . . . . . . . . 15
6.3 Attacks from behind sharing public IPs (NATs) . . . . . . . 16
7. TFO's Applicability . . . . . . . . . . . . . . . . . . . . . . 17
7.1 Duplicate data in SYNs . . . . . . . . . . . . . . . . . . . 17
7.2 Potential performance improvement . . . . . . . . . . . . . 17
7.3 Example: Web clients and servers . . . . . . . . . . . . . . 17
7.3.1 HTTP request replay . . . . . . . . . . . . . . . . . . 17
7.3.2 HTTP persistent connection . . . . . . . . . . . . . . . 18
8. Performance Experiments . . . . . . . . . . . . . . . . . . . . 18
9. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 19
9.1. T/TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
9.2. Common Defenses Against SYN Flood Attacks . . . . . . . . . 19
9.3. TCP Cookie Transaction (TCPCT) . . . . . . . . . . . . . . 20
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
11. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 20
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20
12.1. Normative References . . . . . . . . . . . . . . . . . . . 20
12.2. Informative References . . . . . . . . . . . . . . . . . . 21
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23
1. Introduction 1. Introduction
TCP Fast Open (TFO) enables data to be exchanged safely during TCP's TCP Fast Open (TFO) enables data to be exchanged safely during TCP's
connection handshake. connection handshake.
This document describes a design that enables applications to save a This document describes a design that enables applications to save a
round trip while avoiding severe security ramifications. At the core round trip while avoiding severe security ramifications. At the core
of TFO is a security cookie used by the server side to authenticate a of TFO is a security cookie used by the server side to authenticate a
client initiating a TFO connection. This document covers the details client initiating a TFO connection. This document covers the details
of exchanging data during TCP's initial handshake, the protocol for of exchanging data during TCP's initial handshake, the protocol for
TFO cookies, and potential new security vulnerabilities and their TFO cookies, and potential new security vulnerabilities and their
mitigation. It also includes discussions of deployment issues and mitigation. It also includes discussion of deployment issues and
related proposals. TFO requires extensions to the socket API but this related proposals. TFO requires extensions to the socket API but this
document does not cover that. document does not cover that.
TFO is motivated by the performance needs of today's Web TFO is motivated by the performance needs of today's Web
applications. Network latency is determined by the round-trip time applications. Network latency is largely determined by a connection's
(RTT) and the number of round trips required to transfer application round-trip time (RTT) and the number of round trips required to
data. RTT consists of propagation delay and queuing delay. Network transfer application data. RTT consists of propagation delay and
bandwidth has grown substantially over the past two decades, reducing queuing delay.
queuing delay, while propagation delay is largely constrained by the
speed of light and has remained unchanged. Therefore reducing the
number of round trips has become the most effective way to improve
the latency of Web applications [CDCM11].
Standard TCP only permits data exchange after 3WHS [RFC793], which Network bandwidth has grown substantially over the past two decades,
adds one RTT to the network latency. For short transfers (e.g., web potentially reducing queuing delay, while propagation delay is
objects) this additional RTT is a significant portion of the network largely constrained by the speed of light and has remained unchanged.
latency [THK98]. One widely deployed solution is HTTP persistent Therefore reducing the number of round trips has typically become the
connections. However, this solution is limited since hosts and middle most effective way to improve the latency of applications like the
boxes terminate idle TCP connections due to resource constraints. For Web [CDCM11].
example, the Chrome browser keeps TCP connections idle up to 5
minutes but 35% of Chrome HTTP requests are made on new TCP Current TCP only permits data exchange after 3WHS [RFC793], which
connections. We discuss HTTP persistent connections further in adds one RTT to network latency. For short transfers (e.g., web
section 7.1. objects) this additional RTT is a significant portion of overall
network latency [THK98]. One widely deployed solution is HTTP
persistent connections. However, this solution is limited since hosts
and middle boxes terminate idle TCP connections due to resource
constraints. For example, the Chrome browser keeps TCP connections
idle for up to 5 minutes but 35% of Chrome HTTP requests are made on
new TCP connections [RCCJR11]. We discuss Web applications and TFO in
detail later in section 7.
2. Data In SYN 2. Data In SYN
Allowing data in SYN packets to be delivered raises two issues
discussed in the following subsections. These issues make TFO
undesirable for certain applications. Therefore TCP implementations
MUST NOT use TFO by default and only use TFO if requested explicitly
by the application on a per service port basis. Applications need to
evaluate TFO applicability (described in Section 7) before using TFO.
2.1 Relaxing TCP semantics on duplicated SYNs
[RFC793] (section 3.4) already allows data in SYN packets but forbids [RFC793] (section 3.4) already allows data in SYN packets but forbids
the receiver to deliver the data to the application until 3WHS is the receiver from delivering the data to the application until 3WHS
completed. This is because TCP's initial handshake serves to capture is completed. This is because TCP's initial handshake serves to
1) Old or duplicate SYNs and 2)SYNs with spoofed IP addresses. capture old or duplicate SYNs.
TFO allows data to be delivered to the application before 3WHS is TFO allows data to be delivered to the application before 3WHS is
completed, thus opening itself to a possible data integrity problem completed, thus opening itself to a data integrity issue for the
caused by the problematic SYN packets above. This could cause a applications in Section 2.1 in either of the following cases:
problem in the following two examples: a) the receiver host receives
both duplicate and original SYNs before and after the host reboots,
and b) the duplicate is received after the connection created by the
original SYN has been closed. The receiver will not be protected by
the 2MSL TIMEWAIT state if the close is initiated by the sender. In
both cases, the data is replayed.
2.1. TCP Semantics and Duplicate SYNs a) the receiver host receives data in a duplicate SYN after it has
forgotten it received the original SYN (e.g. due to a reboot); b) the
duplicate is received after the connection created by the original
SYN has been closed and the close was initiated by the sender (so
the receiver will not be protected by the 2MSL TIMEWAIT state).
The proposed T/TCP protocol employs a new TCP "TAO" option and The obsoleted T/TCP protocol employs a new TCP "TAO" option and
connection count to guard against old or duplicate SYNs [RFC1644]. connection count to guard against old or duplicate SYNs [RFC1644].
The solution is complex, involving state tracking on a per remote However it is not widely used due to various vulnerabilities
peer basis, and is vulnerable to IP spoofing attacks. Moreover, it [PHRACK98].
has been shown that despite its complexity, T/TCP is still not
entirely protected. Old or duplicate SYNs may still be accepted by a
T/TCP server [PHRACK98].
Rather than trying to capture all dubious SYN packets to make TFO Rather than trying to capture all dubious SYN packets to make TFO
100% compatible with TCP semantics, we made a design decision early 100% compatible with TCP semantics, we made a design decision early
on to accept old SYN packets with data, i.e., to restrict TFO to use on to accept old SYN packets with data, i.e., to restrict TFO use to
with a class of applications that are tolerant of duplicate SYN a class of applications (Section 7) that are tolerant of duplicate
packets with data. We believe this is the right design trade-off SYN packets with data. We believe this is the right design trade-off
balancing complexity with usefulness. Applications that require balancing complexity with usefulness for certain applications.
transactional semantics already deploy specific mechanisms to
tolerate similar data replay issues in TCP today. For example, a
browser reload event may replay any HTTP request even without data in
SYN. For transactional HTTP requests applications typically include
unique identifiers in the HTTP headers. Thus, allowing data in SYN
poses little risk to existing HTTP applications.
However, we note that some applications may rely on TCP 3-way
handshake semantics. For this reason, TFO MUST be used explicitly by
applications on a per service port basis.
2.2. SYNs with spoofed IP addresses 2.2. SYNs with spoofed IP addresses
Standard TCP suffers from the SYN flood attack [RFC4987] because Standard TCP suffers from the SYN flood attack [RFC4987] because
bogus SYN packets, i.e., SYN packets with spoofed source IP addresses bogus SYN packets, i.e., SYN packets with spoofed source IP addresses
can easily fill up a listener's small queue, causing a service port can easily fill up a listener's small queue, causing a service port
to be blocked completely until timeouts. Secondary damage comes from to be blocked completely until timeouts. Secondary damage comes from
these SYN requests taking up memory space. Though this is less of an these SYN requests taking up memory space. Though this is less of an
issue today as servers typically have plenty of memory. issue today as servers typically have plenty of memory.
TFO goes one step further to allow server side TCP to process and TFO goes one step further to allow server-side TCP to process and
send up data to the application layer before 3WHS is completed. This send up data to the application layer before 3WHS is completed. This
opens up more serious new vulnerabilities. Applications serving ports opens up more serious new vulnerabilities. Applications serving ports
that have TFO enabled may waste lots of CPU and memory resources that have TFO enabled may waste lots of CPU and memory resources
processing the requests and producing the responses. If the response processing the requests and producing the responses. If the response
is much larger than the request, the attacker can mount an amplified is much larger than the request, the attacker can mount an amplified
reflection attack against victims of choice beyond the TFO server reflection attack against victims of choice beyond the TFO server
itself. itself.
Numerous mitigation techniques against the regular SYN flood attack Numerous mitigation techniques against regular SYN flood attacks
exist and have been well documented [RFC4987]. Unfortunately none are exist and have been well documented [RFC4987]. Unfortunately none are
applicable to TFO. We propose a server supplied cookie to mitigate applicable to TFO. We propose a server-supplied cookie to mitigate
most of the security issues introduced by TFO. We defer further the primary security issues introduced by TFO in Section 3. We defer
discussion of SYN flood attacks to the "Security Considerations" further discussion of SYN flood attacks to the "Security
section. Considerations" section.
3. Protocol Overview 3. Protocol Overview
The key component of TFO is the Fast Open Cookie (cookie), a message The key component of TFO is the Fast Open Cookie (cookie), a message
authentication code (MAC) tag generated by the server. The client authentication code (MAC) tag generated by the server. The client
requests a cookie in one regular TCP connection, then uses it for requests a cookie in one regular TCP connection, then uses it for
future TCP connections to exchange data during 3WHS: future TCP connections to exchange data during 3WHS: Requesting a
Requesting a Fast Open Cookie: Fast Open Cookie:
1. The client sends a SYN with a Fast Open Cookie Request option. 1. The client sends a SYN with a Fast Open Cookie Request option.
2. The server generates a cookie and sends it through the Fast Open 2. The server generates a cookie and sends it through the Fast Open
Cookie option of a SYN-ACK packet. Cookie option of a SYN-ACK packet.
3. The client caches the cookie for future TCP Fast Open connections 3. The client caches the cookie for future TCP Fast Open connections
(see below). (see below).
Performing TCP Fast Open: Performing TCP Fast Open:
skipping to change at page 7, line 18 skipping to change at page 8, line 17
Options with invalid Length values, without SYN flag set, or with ACK Options with invalid Length values, without SYN flag set, or with ACK
flag set MUST be ignored. flag set MUST be ignored.
4.1.2. Server Cookie Handling 4.1.2. Server Cookie Handling
The server is in charge of cookie generation and authentication. The The server is in charge of cookie generation and authentication. The
cookie SHOULD be a message authentication code tag with the following cookie SHOULD be a message authentication code tag with the following
properties: properties:
1. The cookie authenticates the client's (source) IP address of the 1. The cookie authenticates the client's (source) IP address of the
SYN packet. The IP address can be an IPv4 or IPv6 address. SYN packet. The IP address can be an IPv4 or IPv6 address.
2. The cookie can only be generated by the server and can not be 2. The cookie can only be generated by the server and can not be
fabricated by any other parties including the client. fabricated by any other parties including the client.
3. The generation and verification are fast relative to the rest of 3. The generation and verification are fast relative to the rest of
SYN and SYN-ACK processing. SYN and SYN-ACK processing.
4. A server may encode other information in the cookie, and accept 4. A server may encode other information in the cookie, and accept
more than one valid cookie per client at any given time. But this more than one valid cookie per client at any given time. But this
is all server implementation dependent and transparent to the is all server implementation dependent and transparent to the
client. client.
5. The cookie expires after a certain amount of time. The reason for 5. The cookie expires after a certain amount of time. The reason for
cookie expiration is detailed in the "Security Consideration" cookie expiration is detailed in the "Security Consideration"
section. This can be done by either periodically changing the section. This can be done by either periodically changing the
server key used to generate cookies or including a timestamp when server key used to generate cookies or including a timestamp when
generating the cookie. generating the cookie.
To gradually invalidate cookies over time, the server can To gradually invalidate cookies over time, the server can
implement key rotation to generate and verify cookies using implement key rotation to generate and verify cookies using
multiple keys. This approach is useful for large-scale servers to multiple keys. This approach is useful for large-scale servers to
retain Fast Open rolling key updates. We do not specify a retain Fast Open rolling key updates. We do not specify a
particular mechanism because the implementation is often server particular mechanism because the implementation is often server
specific. specific.
The server supports the cookie generation and verification The server supports the cookie generation and verification
operations: operations:
- GetCookie(IP_Address): returns a (new) cookie - GetCookie(IP_Address): returns a (new) cookie
- IsCookieValid(IP_Address, Cookie): checks if the cookie is valid, - IsCookieValid(IP_Address, Cookie): checks if the cookie is valid,
i.e., it has not expired and it authenticates the client IP address. i.e., it has not expired and it authenticates the client IP address.
Example Implementation: a simple implementation is to use AES_128 to Example Implementation: a simple implementation is to use AES_128 to
encrypt the IPv4 (with padding) or IPv6 address and truncate to 64 encrypt the IPv4 (with padding) or IPv6 address and truncate to 64
bits. The server can periodically update the key to expire the bits. The server can periodically update the key to expire the
cookies. AES encryption on recent processors is fast and takes only a cookies. AES encryption on recent processors is fast and takes only a
few hundred nanoseconds [RCCJB11]. few hundred nanoseconds [RCCJR11].
If only one valid cookie is allowed per-client and the server can If only one valid cookie is allowed per-client and the server can
regenerate the cookie independently, the best validation process is regenerate the cookie independently, the best validation process is
to simply regenerate a valid cookie and compare it against the to simply regenerate a valid cookie and compare it against the
incoming cookie. In that case if the incoming cookie fails the check, incoming cookie. In that case if the incoming cookie fails the check,
a valid cookie is readily available to be sent to the client. a valid cookie is readily available to be sent to the client.
The server MAY return a cookie request option, e.g., a null cookie, The server MAY return a cookie request option, e.g., a null cookie,
to signal the support of Fast Open without generating cookies, for to signal the support of Fast Open without generating cookies, for
probing or debugging purposes. probing or debugging purposes.
skipping to change at page 8, line 45 skipping to change at page 9, line 43
discovery. discovery.
Caching RTT allows seeding a more accurate SYN timeout than the Caching RTT allows seeding a more accurate SYN timeout than the
default value [RFC6298]. This lowers the performance penalty if the default value [RFC6298]. This lowers the performance penalty if the
network or the server drops the SYN packets with data or the cookie network or the server drops the SYN packets with data or the cookie
options (See "Reliability and Deployment Issues" section below). options (See "Reliability and Deployment Issues" section below).
The cache replacement algorithm is not specified and is left for the The cache replacement algorithm is not specified and is left for the
implementations. implementations.
Note that before TFO sees wide deployment, clients are advised to Note that before TFO sees wide deployment, clients SHOULD cache
also cache negative responses from servers in order to reduce the negative responses from servers in order to reduce the amount of
amount of futile TFO attempts. Since TFO is enabled on a per-service futile TFO attempts. Since TFO is enabled on a per-service port basis
port basis but cookies are independent of service ports, clients' but cookies are independent of service ports, clients' cache should
cache should include remote port numbers too. include remote port numbers too.
4.2. Fast Open Protocol 4.2. Fast Open Protocol
One predominant requirement of TFO is to be fully compatible with One predominant requirement of TFO is to be fully compatible with
existing TCP implementations, both on the client and the server existing TCP implementations, both on the client and the server
sides. sides.
The server keeps two variables per listening port: The server keeps two variables per listening port:
FastOpenEnabled: default is off. It MUST be turned on explicitly by FastOpenEnabled: default is off. It MUST be turned on explicitly by
skipping to change at page 9, line 34 skipping to change at page 10, line 29
The server keeps a FastOpened flag per TCB to mark if a connection The server keeps a FastOpened flag per TCB to mark if a connection
has successfully performed a TFO. has successfully performed a TFO.
4.2.1. Fast Open Cookie Request 4.2.1. Fast Open Cookie Request
Any client attempting TFO MUST first request a cookie from the server Any client attempting TFO MUST first request a cookie from the server
with the following steps: with the following steps:
1. The client sends a SYN packet with a Fast Open Cookie Request 1. The client sends a SYN packet with a Fast Open Cookie Request
option. option.
2. The server SHOULD respond with a SYN-ACK based on the procedures 2. The server SHOULD respond with a SYN-ACK based on the procedures
in the "Server Cookie Handling" section. This SYN-ACK SHOULD in the "Server Cookie Handling" section. This SYN-ACK SHOULD
contain a Fast Open Cookie option if the server currently supports contain a Fast Open Cookie option if the server currently supports
TFO for this listener port. TFO for this listener port.
3. If the SYN-ACK contains a Fast Open Cookie option, the client 3. If the SYN-ACK contains a Fast Open Cookie option, the client
replaces the cookie and other information as described in the replaces the cookie and other information as described in the
"Client Cookie Handling" section. Otherwise, if the SYN-ACK is "Client Cookie Handling" section. Otherwise, if the SYN-ACK is
first seen, i.e.,not a (spurious) retransmission, the client MAY first seen, i.e.,not a (spurious) retransmission, the client MAY
remove the server information from the cookie cache. If the SYN- remove the server information from the cookie cache. If the SYN-
ACK is a spurious retransmission without valid Fast Open Cookie ACK is a spurious retransmission without valid Fast Open Cookie
Option, the client does nothing to the cookie cache for the Option, the client does nothing to the cookie cache for the reasons
reasons below. below.
The network or servers may drop the SYN or SYN-ACK packets with the The network or servers may drop the SYN or SYN-ACK packets with the
new cookie options which causes SYN or SYN-ACK timeouts. We RECOMMEND new cookie options which causes SYN or SYN-ACK timeouts. We RECOMMEND
both the client and the server retransmit SYN and SYN-ACK without the both the client and the server retransmit SYN and SYN-ACK without the
cookie options on timeouts. This ensures the connections of cookie cookie options on timeouts. This ensures the connections of cookie
requests will go through and lowers the latency penalties (of dropped requests will go through and lowers the latency penalties (of dropped
SYN/SYN-ACK packets). The obvious downside for maximum compatibility SYN/SYN-ACK packets). The obvious downside for maximum compatibility
is that any regular SYN drop will fail the cookie (although one can is that any regular SYN drop will fail the cookie (although one can
argue the delay in the data transmission till after 3WHS is justified argue the delay in the data transmission till after 3WHS is justified
if the SYN drop is due to network congestion). Next section if the SYN drop is due to network congestion). Next section
skipping to change at page 10, line 37 skipping to change at page 11, line 32
changes relatively small in addition to [RFC793]. changes relatively small in addition to [RFC793].
Client: Sending SYN Client: Sending SYN
To open a TFO connection, the client MUST have obtained the cookie To open a TFO connection, the client MUST have obtained the cookie
from the server: from the server:
1. Send a SYN packet. 1. Send a SYN packet.
a. If the SYN packet does not have enough option space for the a. If the SYN packet does not have enough option space for the
Fast Open Cookie option, abort TFO and fall back to regular 3WHS. Fast Open Cookie option, abort TFO and fall back to regular 3WHS.
b. Otherwise, include the Fast Open Cookie option with the cookie b. Otherwise, include the Fast Open Cookie option with the cookie
of the server. Include any data up to the cached server MSS or of the server. Include any data up to the cached server MSS or
default 536 bytes. default 536 bytes.
2. Advance to SYN-SENT state and update SND.NXT to include the data 2. Advance to SYN-SENT state and update SND.NXT to include the data
accordingly. accordingly.
3. If RTT is available from the cache, seed SYN timer according to 3. If RTT is available from the cache, seed SYN timer according to
[RFC6298]. [RFC6298].
To deal with network or servers dropping SYN packets with payload or To deal with network or servers dropping SYN packets with payload or
unknown options, when the SYN timer fires, the client SHOULD unknown options, when the SYN timer fires, the client SHOULD
retransmit a SYN packet without data and Fast Open Cookie options. retransmit a SYN packet without data and Fast Open Cookie options.
Server: Receiving SYN and responding with SYN-ACK Server: Receiving SYN and responding with SYN-ACK
Upon receiving the SYN packet with Fast Open Cookie option: Upon receiving the SYN packet with Fast Open Cookie option:
1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled
is false, go to step 5. is false, go to step 5.
2. If PendingFastOpenRequests is over the system limit, go to step 5. 2. If PendingFastOpenRequests is over the system limit, go to step 5.
3. If IsCookieValid() in section 4.1.2 returns false, go to step 5. 3. If IsCookieValid() in section 4.1.2 returns false, go to step 5.
4. Buffer the data and notify the application. Set FastOpened flag 4. Buffer the data and notify the application. Set FastOpened flag
and increment PendingFastOpenRequests. and increment PendingFastOpenRequests.
5. Send the SYN-ACK packet. The packet MAY include a Fast Open 5. Send the SYN-ACK packet. The packet MAY include a Fast Open
Option. If FastOpened flag is set, the packet acknowledges the SYN Option. If FastOpened flag is set, the packet acknowledges the SYN
and data sequence. Otherwise it acknowledges only the SYN and data sequence. Otherwise it acknowledges only the SYN sequence.
sequence. The server MAY include data in the SYN-ACK packet if the The server MAY include data in the SYN-ACK packet if the response
response data is readily available. Some application may favor data is readily available. Some application may favor delaying the
delaying the SYN-ACK, allowing the application to process the SYN-ACK, allowing the application to process the request in order
request in order to produce a response, but this is left to the to produce a response, but this is left to the implementation.
implementation.
6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the
server MUST follow the congestion control [RFC5681], in particular server MUST follow the congestion control [RFC5681], in particular
the initial congestion window [RFC3390], to send more data the initial congestion window [RFC3390], to send more data packets.
packets.
Note that if SYN-ACK is lost, regular TCP reduces the initial
congestion window before sending any data. In this case TFO is
slightly more aggressive in the first data round trip even though
it does not change the congestion control.
If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK
segment with neither data nor Fast Open Cookie options for segment with neither data nor Fast Open Cookie options for
compatibility reasons. compatibility reasons.
A special case is simultaneous open where the SYN receiver is a
client in SYN-SENT state. The protocol remains the same because
[RFC793] already supports both data in SYN and simultaneous open. But
the client's socket may have data available to read before it's
connected. This document does not cover the corresponding API change.
Client: Receiving SYN-ACK Client: Receiving SYN-ACK
The client SHOULD perform the following steps upon receiving the SYN- The client SHOULD perform the following steps upon receiving the SYN-
ACK: ACK: 1. Update the cookie cache if the SYN-ACK has a Fast Open Cookie
1. Update the cookie cache if the SYN-ACK has a Fast Open Cookie Option or MSS option or both.
Option or MSS option or both.
2. Send an ACK packet. Set acknowledgment number to RCV.NXT and 2. Send an ACK packet. Set acknowledgment number to RCV.NXT and
include the data after SND.UNA if data is available. include the data after SND.UNA if data is available.
3. Advance to the ESTABLISHED state. 3. Advance to the ESTABLISHED state.
Note there is no latency penalty if the server does not acknowledge Note there is no latency penalty if the server does not acknowledge
the data in the original SYN packet. The client SHOULD retransmit any the data in the original SYN packet. The client SHOULD retransmit any
unacknowledged data in the first ACK packet in step 2. The data unacknowledged data in the first ACK packet in step 2. The data
exchange will start after the handshake like a regular TCP exchange will start after the handshake like a regular TCP
connection. connection.
If the client has timed out and retransmitted only regular SYN If the client has timed out and retransmitted only regular SYN
skipping to change at page 12, line 27 skipping to change at page 13, line 34
state. No special handling is required further. state. No special handling is required further.
5. Reliability and Deployment Issues 5. Reliability and Deployment Issues
Network or Hosts Dropping SYN packets with data or unknown options Network or Hosts Dropping SYN packets with data or unknown options
A study [MAF04] found that some middle-boxes and end-hosts may drop A study [MAF04] found that some middle-boxes and end-hosts may drop
packets with unknown TCP options incorrectly. Studies [LANGLEY06, packets with unknown TCP options incorrectly. Studies [LANGLEY06,
HNRGHT11] both found that 6% of the probed paths on the Internet drop HNRGHT11] both found that 6% of the probed paths on the Internet drop
SYN packets with data or with unknown TCP options. The TFO protocol SYN packets with data or with unknown TCP options. The TFO protocol
deals with this problem by retransmitting SYN without data or cookie deals with this problem by re-transmitting SYN without data or cookie
options and we recommend tracking these servers in the client. options and we recommend tracking these servers in the client.
Server Farms Server Farms
A common server-farm setup is to have many physical hosts behind a A common server-farm setup is to have many physical hosts behind a
load-balancer sharing the same server IP. The load-balancer forwards load-balancer sharing the same server IP. The load-balancer forwards
new TCP connections to different physical hosts based on certain new TCP connections to different physical hosts based on certain
load-balancing algorithms. For TFO to work, the physical hosts need load-balancing algorithms. For TFO to work, the physical hosts need
to share the same key and update the key at about the same time. to share the same key and update the key at about the same time.
skipping to change at page 15, line 35 skipping to change at page 17, line 5
This enables the server to issue different cookies to clients that This enables the server to issue different cookies to clients that
share the same IP address, hence can selectively discard those share the same IP address, hence can selectively discard those
misused cookies from the attacker. However the attacker can simply misused cookies from the attacker. However the attacker can simply
repeat the attack with new cookies. The server would eventually need repeat the attack with new cookies. The server would eventually need
to throttle all requests from the IP address just like the current to throttle all requests from the IP address just like the current
approach. Moreover this approach requires modifying [RFC 1323] to approach. Moreover this approach requires modifying [RFC 1323] to
send non-zero Timestamp Echo Reply in SYN, potentially cause firewall send non-zero Timestamp Echo Reply in SYN, potentially cause firewall
issues. Therefore we believe the benefit may not outweigh the issues. Therefore we believe the benefit may not outweigh the
drawbacks. drawbacks.
7. Web Performance 7. TFO's Applicability
7.1. HTTP persistent connection This section is to help applications considering TFO to evaluate
TFO's benefits and drawbacks using a Web client and server
applications as an example throughout.
7.1 Duplicate data in SYNs
It is possible, though uncommon, that using TFO the first data
written to a socket is delivered more than once to the application on
the remote host(Section 2.1). This replay potential only applies to
data in the SYN but not subsequent data exchanges. Thus applications
MUST NOT use TFO unless they can tolerate this behavior.
7.2 Potential performance improvement
TFO is designed for latency-conscious applications that are sensitive
to TCP's initial connection setup delay. For example, many
applications perform short request and response message exchanges. To
benefit from TFO, the first application data unit (e.g., an HTTP
request) needs to be no more than TCP's maximum segment size (minus
options used in SYN). Otherwise the remote server can only process
the client's application data unit once the rest of it is delivered
after the initial handshake, diminishing TFO's benefit.
To the extent possible, applications SHOULD employ long-lived
connections to best take advantage of TCP's built-in congestion
control, and to reduce the impact from TCP's connection setup
overhead. Note that when an application employs too many short-lived
connections, it may negatively impact network stability, as these
connections often exit before TCP's congestion control algorithm
takes effect. Implementations supporting a large number of short-
lived connections should employ temporal sharing of TCB data as
described in [RFC2140].
7.3 Example: Web clients and servers
We look at Web client and server applications that use HTTP and TCP
protocols and follow the guidelines above to evaluate if TFO is safe
and useful for Web.
7.3.1 HTTP request replay
We believe TFO is safe for the Web because even with standard TCP the
Web browser may replay an HTTP request to the remote Web server
multiple times. After sending an HTTP request, the browser could time
out and retry the same request on another TCP connection. This
scenario occurs far more frequently than the SYN duplication issue
presented by TFO. To ensure transactional behavior, Web sites employ
application-specific mechanisms such as including unique identifiers
in the data.
7.3.2 HTTP persistent connection
Next we evaluate if the Web can benefit from TFO given that HTTP
persistent connection support is already widely deployed.
TCP connection setup overhead has long been identified as a TCP connection setup overhead has long been identified as a
performance bottleneck for web applications [THK98]. HTTP persistent performance bottleneck for web applications [THK98]. HTTP persistent
connection was proposed to mitigate this issue and has been widely connection support was proposed to mitigate this issue and has been
deployed. However, [RCCJR11][AERG11] show that the average number of widely deployed. However, studies [RCCJR11][AERG11] show that the
transactions per connection is between 2 and 4, based on large-scale average number of transactions per connection is between 2 and 4,
measurements from both servers and clients. In these studies, the based on large-scale measurements from both servers and clients. In
servers and clients both kept the idle connections up to several these studies, the servers and clients both kept idle connections up
minutes, well into the human think time. to several minutes, well into "human think" time.
Can the utilization rate increase by keeping connections even longer? Can the utilization rate of such connections increase by keeping idle
Unfortunately, this is problematic due to middle-boxes and rapidly connections even longer? Unfortunately, such an approach is
growing mobile end hosts. One major issue is NAT. Studies problematic due to middle-boxes and the rapidly growing share of
mobile end hosts. Thus one major issue faced by persistent
connections is NAT. Studies [HNESSK10][MQXMZ11] show that the
majority of home routers and ISPs fail to meet the the 124-minute
idle timeout mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs
timeout idle connections within 30 minutes. The end hosts attempting
to use these broken connections are often forced to wait for a
lengthy TCP timeout, as they often receive no signal when middleboxes
break their connections. Thus browsers risk large performance
penalties when keeping idle connections open.
[HNESSK10][MQXMZ11] show that the majority of home routers and ISPs To circumvent this problem, some applications send frequent TCP keep-
fail to meet the the 124 minutes idle timeout mandated in [RFC5382]. alive probes. However, this technique drains power on mobile devices
In [MQXMZ11], 35% of mobile ISPs timeout idle connections within 30 [MQXMZ11]. In fact, power has become such a prominent issue in modern
minutes. NAT boxes do not possess a reliable mechanism to notify end LTE devices that mobile browsers close HTTP connections within
hosts when idle connections are removed from local tables, either due seconds or even immediately [SOUDERS11].
to resource constraints such as mapping table size, memory, or lookup
overhead, or due to the limited port number and IP address space.
Moreover, unmapped packets received by NAT boxes are often dropped
silently. (TCP RST is not required by RFC5382.) The end host
attempting to use these broken connections are often forced to wait
for a lengthy TCP timeout. Thus the browser risks large performance
penalty when keeping idle connections open. To circumvent this
problem, some applications send frequent TCP keep-alive probes.
However, this technique drains power on mobile devices [MQXMZ11]. In
fact, power has become a prominent issue in modern LTE devices that
mobile browsers close the HTTP connections within seconds or even
immediately [SOUDERS11].
Idle connections also consume more memory resources. Due to the Since TFO data duplication presents no new issues and HTTP persistent
complexity of today's web applications, the application layer often connection support has many limitations, Web applications can safely
needs orders of magnitude more memory than the TCP connection use TFO and will likely achieve performance gains. The next section
footprint. As a result, servers need to implement advanced resource presents more empirical data of the potential performance benefit.
management in order to support a large number of idle connections.
7.2 Case Study: Chrome Browser 8. Performance Experiments
[RCCJR11] studied Chrome browser performance based on 28 days of [RCCJR11] studied Chrome browser performance based on 28 days of
global statistics. Chrome browser keeps idle HTTP persistent global statistics. Chrome browser keeps idle HTTP persistent
connections up to 5 to 10 minutes. However the average number of the connections up to 5 to 10 minutes. However the average number of the
transactions per connection is only 3.3. Due to the low utilization, transactions per connection is only 3.3. Due to the low utilization,
TCP 3WHS accounts up to 25% of the HTTP transaction network latency. TCP 3WHS accounts up to 25% of the HTTP transaction network latency.
The authors tested a Linux TFO implementation with TFO enabled Chrome The authors tested a Linux TFO implementation with TFO enabled Chrome
browser on popular web sites in emulated environments such as browser on popular web sites in emulated environments such as
residential broadband and mobile networks. They showed that TFO residential broadband and mobile networks. They showed that TFO
improves page load time by 10% to 40%. More detailed on the design improves page load time by 10% to 40%. More details on the design
tradeoffs and measurement can be found at [RCCJB11]. tradeoffs and measurement can be found at [RCCJR11].
8. TFO's Applicability
TFO aims at latency conscious applications that are sensitive to
TCP's initial connection setup delay. These application protocols
often employ short-lived TCP connections, or employ long-lived
connections but are more sensitive to the connection setup delay due
to, e.g., a more strict connection fail-over requirement.
Only transaction-type applications where RTT constitutes a
significant portion of the total end-to-end latency will likely
benefit from TFO. Moreover, the client request must fit in the SYN
packet. Otherwise there may not be any saving in the total number of
round trips required to complete a transaction.
To the extent possible applications protocols SHOULD employ long-
lived connections to best take advantage of TCP's built-in congestion
control algorithm, and to reduce the impact from TCP's connection
setup overhead. E.g., for the web applications, P-HTTP will likely
help and is much easier to deploy hence should be attempted first.
TFO will likely provide further latency reduction on top of P-HTTP.
But the additional benefit will depend on how much persistency one
can get from HTTP in a given operating environment.
One alternative to short-lived TCP connection might be UDP, which is
connectionless hence doesn't inflict any connection setup delay, and
is best suited for application protocols that are transactional.
Practical deployment issues such as middle-box and/or firewall
traversal may severely limit the use of UDP based application
protocols though.
Note that when the application employs too many short-lived
connections, it may negatively impact network stability, as these
connections often exit before TCP's congestion control algorithm
kicks in. Implementations supporting large number of short-lived
connections should employ temporal sharing of TCB data as described
in [RFC2140].
More discussion on TCP Fast Open and its projected performance
benefit can be found in [RCCJB11].
9. Related Work 9. Related Work
9.1. T/TCP 9.1. T/TCP
TCP Extensions for Transactions [RFC1644] attempted to bypass the TCP Extensions for Transactions [RFC1644] attempted to bypass the
three-way handshake, among other things, hence shared the same goal three-way handshake, among other things, hence shared the same goal
but also the same set of issues as TFO. It focused most of its effort but also the same set of issues as TFO. It focused most of its effort
battling old or duplicate SYNs, but paid no attention to security battling old or duplicate SYNs, but paid no attention to security
vulnerabilities it introduced when bypassing 3WHS. Its TAO option and vulnerabilities it introduced when bypassing 3WHS. Its TAO option and
skipping to change at page 21, line 21 skipping to change at page 22, line 11
[PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue [PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue
53 artical 6. July 8, 1998. URL 53 artical 6. July 8, 1998. URL
http://www.phrack.com/issues.html?issue=53&id=6 http://www.phrack.com/issues.html?issue=53&id=6
[QWGMSS11] F. Qian, Z. Wang, A. Gerber, Z. Mao, S. Sen, O. [QWGMSS11] F. Qian, Z. Wang, A. Gerber, Z. Mao, S. Sen, O.
Spatscheck. "Profiling Resource Usage for Mobile Spatscheck. "Profiling Resource Usage for Mobile
Applications: A Cross-layer Approach", In Proceedings of Applications: A Cross-layer Approach", In Proceedings of
International Conference on Mobile Systems. April 2011. International Conference on Mobile Systems. April 2011.
[RCCJB11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A. and B. [RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A. and
Raghavan, "TCP Fast Open". In Proceedings of 7th ACM CoNEXT Raghavan, B., "TCP Fast Open". In Proceedings of 7th ACM
Conference, December 2011. CoNEXT Conference, December 2011.
[RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions [RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions
Functional Specification", RFC 1644, July 1994. Functional Specification", RFC 1644, July 1994.
[RFC2140] Touch, J., "TCP Control Block Interdependence", RFC2140, [RFC2140] Touch, J., "TCP Control Block Interdependence", RFC2140,
April 1997. April 1997.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
Mitigations", RFC 4987, August 2007. Mitigations", RFC 4987, August 2007.
skipping to change at page 22, line 6 skipping to change at page 23, line 5
[THK98] Touch, J., Heidemann, J., Obraczka, K., "Analysis of HTTP [THK98] Touch, J., Heidemann, J., Obraczka, K., "Analysis of HTTP
Performance", USC/ISI Research Report 98-463. December Performance", USC/ISI Research Report 98-463. December
1998. 1998.
[BOB12] Briscoe, B., "Some ideas building on draft-ietf-tcpm- [BOB12] Briscoe, B., "Some ideas building on draft-ietf-tcpm-
fastopen-01", tcpm list, fastopen-01", tcpm list,
http://www.ietf.org/mail-archive/web/tcpm/current/ http://www.ietf.org/mail-archive/web/tcpm/current/
msg07192.html msg07192.html
Author's Addresses Authors' Addresses
Yuchung Cheng Yuchung Cheng
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043, USA
EMail: ycheng@google.com EMail: ycheng@google.com
Jerry Chu Jerry Chu
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
 End of changes. 52 change blocks. 
198 lines changed or deleted 250 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/