draft-ietf-tcpm-fastopen-01.txt   draft-ietf-tcpm-fastopen-02.txt 
Internet Draft Y. Cheng Internet Draft Y. Cheng
draft-ietf-tcpm-fastopen-01.txt J. Chu draft-ietf-tcpm-fastopen-02.txt J. Chu
Intended status: Experimental S. Radhakrishnan Intended status: Experimental S. Radhakrishnan
Expiration date: Feburary, 2013 A. Jain Expiration date: April, 2013 A. Jain
Google, Inc. Google, Inc.
July 16, 2012 Octobor 22, 2012
TCP Fast Open TCP Fast Open
Status of this Memo Status of this Memo
Distribution of this memo is unlimited. Distribution of this memo is unlimited.
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
skipping to change at page 2, line 9 skipping to change at page 2, line 9
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Abstract Abstract
TCP Fast Open (TFO) allows data to be carried in the SYN and SYN-ACK TCP Fast Open (TFO) allows data to be carried in the SYN and SYN-ACK
packets and consumed by the receiving end during the initial packets and consumed by the receiving end during the initial
connection handshake, thus providing a saving of up to one full round connection handshake, thus saving up to one full round trip time
trip time (RTT) compared to standard TCP requiring a three-way (RTT) compared to standard TCP which requires a three-way handshake
handshake (3WHS) to complete before data can be exchanged. (3WHS) to complete before data can be exchanged.
Terminology Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
TFO refers to TCP Fast Open. Client refers to the TCP's active open TFO refers to TCP Fast Open. Client refers to the TCP's active open
side and server refers to the TCP's passive open side. side and server refers to the TCP's passive open side.
1. Introduction 1. Introduction
TCP Fast Open (TFO) enables data to be exchanged safely during TCP TCP Fast Open (TFO) enables data to be exchanged safely during TCP's
connection handshake. connection handshake.
This document describes a design that enables qualified applications This document describes a design that enables applications to save a
to attain a round trip saving while avoiding severe security round trip while avoiding severe security ramifications. At the core
ramifications. At the core of TFO is a security cookie used by the of TFO is a security cookie used by the server side to authenticate a
server side to authenticate a client initiating a TFO connection. The client initiating a TFO connection. This document covers the details
document covers the details of exchanging data during TCP's initial of exchanging data during TCP's initial handshake, the protocol for
handshake, the protocol for TFO cookies, and potential new security TFO cookies, and potential new security vulnerabilities and their
vulnerabilities and their mitigation. It also includes discussions on mitigation. It also includes discussions of deployment issues and
deployment issues and related proposals. TFO requires extensions to related proposals. TFO requires extensions to the socket API but this
the existing socket API, which will be covered in a separate document does not cover that.
document.
TFO is motivated by the performance need of today's Web applications. TFO is motivated by the performance needs of today's Web
Network latency is determined by the round-trip time (RTT) and the applications. Network latency is determined by the round-trip time
number of round trips required to transfer application data. RTT (RTT) and the number of round trips required to transfer application
consists of transmission delay and propagation delay. Network data. RTT consists of propagation delay and queuing delay. Network
bandwidth has grown substantially over the past two decades, much bandwidth has grown substantially over the past two decades, reducing
reducing the transmission delay, while propagation delay is largely queuing delay, while propagation delay is largely constrained by the
constrained by the speed of light and has remained unchanged. speed of light and has remained unchanged. Therefore reducing the
Therefore reducing the number of round trips has become the most number of round trips has become the most effective way to improve
effective way to improve the latency of Web applications [CDCM11]. the latency of Web applications [CDCM11].
Standard TCP only permits data exchange after 3WHS [RFC793], which Standard TCP only permits data exchange after 3WHS [RFC793], which
introduces one RTT delay to the network latency. For short transfers, adds one RTT to the network latency. For short transfers (e.g., web
e.g., web objects, this additional RTT becomes a significant portion objects) this additional RTT is a significant portion of the network
of the network latency [THK98]. One widely deployed solution is HTTP latency [THK98]. One widely deployed solution is HTTP persistent
persistent connections. However, this solution is limited since hosts connections. However, this solution is limited since hosts and middle
and middle boxes terminate idle TCP connections due to resource boxes terminate idle TCP connections due to resource constraints. For
constraints. E.g., the Chrome browser keeps TCP connections idle up example, the Chrome browser keeps TCP connections idle up to 5
to 5 minutes but 35% of Chrome HTTP requests are made on new TCP minutes but 35% of Chrome HTTP requests are made on new TCP
connections. More discussions on HTTP persistent connections are in connections. We discuss HTTP persistent connections further in
section 7.1. section 7.1.
2. Data In SYN 2. Data In SYN
[RFC793] (section 3.4) already allows data in SYN packets but forbids [RFC793] (section 3.4) already allows data in SYN packets but forbids
the receiver to deliver the data to the application until 3WHS is the receiver to deliver the data to the application until 3WHS is
completed. This is because TCP's initial handshake serves to capture completed. This is because TCP's initial handshake serves to capture
- Old or duplicate SYNs 1) Old or duplicate SYNs and 2)SYNs with spoofed IP addresses.
- SYNs with spoofed IP addresses
TFO allows data to be delivered to the application before 3WHS is TFO allows data to be delivered to the application before 3WHS is
completed, thus opening itself to a possible data integrity problem completed, thus opening itself to a possible data integrity problem
caused by the dubious SYN packets above. caused by the problematic SYN packets above. This could cause a
problem in the following two examples: a) the receiver host receives
both duplicate and original SYNs before and after the host reboots,
and b) the duplicate is received after the connection created by the
original SYN has been closed. The receiver will not be protected by
the 2MSL TIMEWAIT state if the close is initiated by the sender. In
both cases, the data is replayed.
2.1. TCP Semantics and Duplicate SYNs 2.1. TCP Semantics and Duplicate SYNs
A past proposal called T/TCP employs a new TCP "TAO" option and The proposed T/TCP protocol employs a new TCP "TAO" option and
connection count to guard against old or duplicate SYNs [RFC1644]. connection count to guard against old or duplicate SYNs [RFC1644].
The solution is complex, involving state tracking on per remote peer The solution is complex, involving state tracking on a per remote
basis, and is vulnerable to IP spoofing attack. Moreover, it has been peer basis, and is vulnerable to IP spoofing attacks. Moreover, it
shown that even with all the complexity, T/TCP is still not 100% has been shown that despite its complexity, T/TCP is still not
bullet proof. Old or duplicate SYNs may still slip through and get entirely protected. Old or duplicate SYNs may still be accepted by a
accepted by a T/TCP server [PHRACK98]. T/TCP server [PHRACK98].
Rather than trying to capture all the dubious SYN packets to make TFO Rather than trying to capture all dubious SYN packets to make TFO
100% compatible with TCP semantics, we've made a design decision 100% compatible with TCP semantics, we made a design decision early
early on to accept old SYN packets with data, i.e., to restrict TFO on to accept old SYN packets with data, i.e., to restrict TFO to use
for a class of applications that are tolerant of duplicate SYN with a class of applications that are tolerant of duplicate SYN
packets with data, e.g., idempotent or query type transactions. We packets with data. We believe this is the right design trade-off
believe this is the right design trade-off balancing complexity with balancing complexity with usefulness. Applications that require
usefulness. There is a large class of applications that can tolerate transactional semantics already deploy specific mechanisms to
dubious transaction requests. tolerate similar data replay issues in TCP today. For example, a
browser reload event may replay any HTTP request even without data in
SYN. For transactional HTTP requests applications typically include
unique identifiers in the HTTP headers. Thus, allowing data in SYN
poses little risk to existing HTTP applications.
For this reason, TFO MUST be disabled by default, and only enabled However, we note that some applications may rely on TCP 3-way
explicitly by applications on a per service port basis. handshake semantics. For this reason, TFO MUST be used explicitly by
applications on a per service port basis.
2.2. SYNs with spoofed IP addresses 2.2. SYNs with spoofed IP addresses
Standard TCP suffers from the SYN flood attack [RFC4987] because Standard TCP suffers from the SYN flood attack [RFC4987] because
bogus SYN packets, i.e., SYN packets with spoofed source IP addresses bogus SYN packets, i.e., SYN packets with spoofed source IP addresses
can easily fill up a listener's small queue, causing a service port can easily fill up a listener's small queue, causing a service port
to be blocked completely until timeouts. Secondary damage comes from to be blocked completely until timeouts. Secondary damage comes from
faked SYN requests taking up memory space. This is normally not an these SYN requests taking up memory space. Though this is less of an
issue today with typical servers having plenty of memory. issue today as servers typically have plenty of memory.
TFO goes one step further to allow server side TCP to process and TFO goes one step further to allow server side TCP to process and
send up data to the application layer before 3WHS is completed. This send up data to the application layer before 3WHS is completed. This
opens up much more serious new vulnerabilities. Applications serving opens up more serious new vulnerabilities. Applications serving ports
ports that have TFO enabled may waste lots of CPU and memory that have TFO enabled may waste lots of CPU and memory resources
resources processing the requests and producing the responses. If the processing the requests and producing the responses. If the response
response is much larger than the request, the attacker can mount an is much larger than the request, the attacker can mount an amplified
amplified reflection attack against victims of choice beyond the TFO reflection attack against victims of choice beyond the TFO server
server itself. itself.
Numerous mitigation techniques against the regular SYN flood attack Numerous mitigation techniques against the regular SYN flood attack
exist and have been well documented [RFC4987]. Unfortunately none are exist and have been well documented [RFC4987]. Unfortunately none are
applicable to TFO. We propose a server supplied cookie to mitigate applicable to TFO. We propose a server supplied cookie to mitigate
most of the security risks introduced by TFO. A more thorough most of the security issues introduced by TFO. We defer further
discussion on SYN flood attack against TFO is deferred to the discussion of SYN flood attacks to the "Security Considerations"
"Security Considerations" section. section.
3. Protocol Overview 3. Protocol Overview
The key component of TFO is the Fast Open Cookie (cookie), a message The key component of TFO is the Fast Open Cookie (cookie), a message
authentication code (MAC) tag generated by the server. The client authentication code (MAC) tag generated by the server. The client
requests a cookie in one regular TCP connection, then uses it for requests a cookie in one regular TCP connection, then uses it for
future TCP connections to exchange data during 3WHS: future TCP connections to exchange data during 3WHS:
Requesting a Fast Open Cookie:
Requesting Fast Open Cookie:
1. The client sends a SYN with a Fast Open Cookie Request option. 1. The client sends a SYN with a Fast Open Cookie Request option.
2. The server generates a cookie and sends it through the Fast Open 2. The server generates a cookie and sends it through the Fast Open
Cookie option of a SYN-ACK packet. Cookie option of a SYN-ACK packet.
3. The client caches the cookie for future TCP Fast Open connections 3. The client caches the cookie for future TCP Fast Open connections
(see below). (see below).
Performing TCP Fast Open: Performing TCP Fast Open:
1. The client sends a SYN with Fast Open Cookie option and data. 1. The client sends a SYN with Fast Open Cookie option and data.
2. The server validates the cookie: 2. The server validates the cookie:
a. If the cookie is valid, the server sends a SYN-ACK a. If the cookie is valid, the server sends a SYN-ACK
acknowledging both the SYN and the data. The server then delivers acknowledging both the SYN and the data. The server then
the data to the application. delivers the data to the application.
b. Otherwise, the server drops the data and sends a SYN-ACK b. Otherwise, the server drops the data and sends a SYN-ACK
acknowledging only the SYN sequence number. acknowledging only the SYN sequence number.
3. If the server accepts the data in the SYN packet, it may send the 3. If the server accepts the data in the SYN packet, it may send the
response data before the handshake finishes. The max amount is response data before the handshake finishes. The max amount is
governed by the TCP's congestion control [RFC5681]. governed by the TCP's congestion control [RFC5681].
4. The client sends an ACK acknowledging the SYN and the server data. 4. The client sends an ACK acknowledging the SYN and the server data.
If the client's data is not acknowledged, the client retransmits If the client's data is not acknowledged, the client retransmits
the data in the ACK packet. the data in the ACK packet.
5. The rest of the connection proceeds like a normal TCP connection.
The client can perform many TFO operations once it acquires a cookie 5. The rest of the connection proceeds like a normal TCP connection.
until the cookie is expired by the server. Thus TFO is useful for The client can repeat many Fast Open operations once it acquires a
applications that have temporal locality on client and server cookie (until the cookie is expired by the server). Thus TFO is
connections. useful for applications that have temporal locality on client and
server connections.
Requesting Fast Open Cookie in connection 1: Requesting Fast Open Cookie in connection 1:
TCP A (Client) TCP B(Server) TCP A (Client) TCP B(Server)
______________ _____________ ______________ _____________
CLOSED LISTEN CLOSED LISTEN
#1 SYN-SENT ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD #1 SYN-SENT ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD
#2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD #2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD
skipping to change at page 6, line 9 skipping to change at page 6, line 9
#3 ESTABLISHED <---- <ACK=x+len(DATA_A)+1,DATA_B>---- SYN-RCVD #3 ESTABLISHED <---- <ACK=x+len(DATA_A)+1,DATA_B>---- SYN-RCVD
#4 ESTABLISHED ----- <ACK=y+1>--------------------> ESTABLISHED #4 ESTABLISHED ----- <ACK=y+1>--------------------> ESTABLISHED
#5 ESTABLISHED --- <ACK=y+len(DATA_B)+1>----------> ESTABLISHED #5 ESTABLISHED --- <ACK=y+len(DATA_B)+1>----------> ESTABLISHED
4. Protocol Details 4. Protocol Details
4.1. Fast Open Cookie 4.1. Fast Open Cookie
The Fast Open Cookie is invented to mitigate new security The Fast Open Cookie is designed to mitigate new security
vulnerabilities in order to enable data exchange during handshake. vulnerabilities in order to enable data exchange during handshake.
The cookie is a message authentication code tag generated by the The cookie is a message authentication code tag generated by the
server and is opaque to the client; the client simply caches the server and is opaque to the client; the client simply caches the
cookie and passes it back on subsequent SYN packets to open new cookie and passes it back on subsequent SYN packets to open new
connections. The server can expire the cookie at any time to enhance connections. The server can expire the cookie at any time to enhance
security. security.
4.1.1. TCP Options 4.1.1. TCP Options
Fast Open Cookie Option Fast Open Cookie Option
skipping to change at page 6, line 41 skipping to change at page 6, line 41
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Kind 1 byte: constant TBD (assigned by IANA) Kind 1 byte: constant TBD (assigned by IANA)
Length 1 byte: range 6 to 18 (bytes); limited by Length 1 byte: range 6 to 18 (bytes); limited by
remaining space in the options field. remaining space in the options field.
The number MUST be even. The number MUST be even.
Cookie 4 to 16 bytes (Length - 2) Cookie 4 to 16 bytes (Length - 2)
Options with invalid Length values or without SYN flag set MUST be Options with invalid Length values or without SYN flag set MUST be
ignored. The minimum Cookie size is 4 bytes. Although the diagram ignored. The minimum Cookie size is 4 bytes. Although the diagram
shows a cookie aligned on 32-bit boundaries, that is not required. shows a cookie aligned on 32-bit boundaries, alignment is not
required.
Fast Open Cookie Request Option Fast Open Cookie Request Option
The client uses this option in the SYN packet to request a cookie The client uses this option in the SYN packet to request a cookie
from a TFO-enabled server from a TFO-enabled server
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Kind | Length | | Kind | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Kind 1 byte: same as the Fast Open Cookie option Kind 1 byte: same as the Fast Open Cookie option
Length 1 byte: constant 2. This distinguishes the option from Length 1 byte: constant 2. This distinguishes the option
the Fast Open cookie option. from the Fast Open cookie option.
Options with invalid Length values, without SYN flag set, or with ACK Options with invalid Length values, without SYN flag set, or with ACK
flag set MUST be ignored. flag set MUST be ignored.
4.1.2. Server Cookie Handling 4.1.2. Server Cookie Handling
The server is in charge of cookie generation and authentication. The The server is in charge of cookie generation and authentication. The
cookie SHOULD be a message authentication code tag with the following cookie SHOULD be a message authentication code tag with the following
properties: properties:
1. The cookie authenticates the client's (source) IP address of the 1. The cookie authenticates the client's (source) IP address of the
SYN packet. The IP address can be an IPv4 or IPv6 address. SYN packet. The IP address can be an IPv4 or IPv6 address.
2. The cookie can only be generated by the server and can not be 2. The cookie can only be generated by the server and can not be
fabricated by any other parties including the client. fabricated by any other parties including the client.
3. The cookie expires after a certain amount of time. The reason is 3. The generation and verification are fast relative to the rest of
detailed in the "Security Consideration" section. This can be SYN and SYN-ACK processing.
done by either periodically changing the server key used to
generate cookies or including a timestamp in the cookie.
4. The generation and verification are fast relative to the rest of 4. A server may encode other information in the cookie, and accept
SYN and SYN-ACK processing. more than one valid cookie per client at any given time. But this
is all server implementation dependent and transparent to the
client.
5. A server may encode other information in the cookie, and accept 5. The cookie expires after a certain amount of time. The reason for
more than one valid cookie per client at any given time. But this cookie expiration is detailed in the "Security Consideration"
is all server implementation dependent and transparent to the section. This can be done by either periodically changing the
client. server key used to generate cookies or including a timestamp when
generating the cookie.
To gradually invalidate cookies over time, the server can
implement key rotation to generate and verify cookies using
multiple keys. This approach is useful for large-scale servers to
retain Fast Open rolling key updates. We do not specify a
particular mechanism because the implementation is often server
specific.
The server supports the cookie generation and verification The server supports the cookie generation and verification
operations: operations:
- GetCookie(IP_Address): returns a (new) cookie - GetCookie(IP_Address): returns a (new) cookie
- IsCookieValid(IP_Address, Cookie): checks if the cookie is valid, - IsCookieValid(IP_Address, Cookie): checks if the cookie is valid,
i.e., it has not expired and it authenticates the client IP address. i.e., it has not expired and it authenticates the client IP address.
Example Implementation: a simple implementation is to use AES_128 to Example Implementation: a simple implementation is to use AES_128 to
encrypt the IPv4 (with padding) or IPv6 address and truncate to 64 encrypt the IPv4 (with padding) or IPv6 address and truncate to 64
bits. The server can periodically update the key to expire the bits. The server can periodically update the key to expire the
cookies. AES encryption on recent processors is fast and takes only a cookies. AES encryption on recent processors is fast and takes only a
few hundred nanoseconds [RCCJB11]. few hundred nanoseconds [RCCJB11].
Note that if only one valid cookie is allowed per-client and the If only one valid cookie is allowed per-client and the server can
server can regenerate the cookie independently, the best validation regenerate the cookie independently, the best validation process is
process may be for the server to simply regenerate a valid cookie and to simply regenerate a valid cookie and compare it against the
compare it against the incoming cookie. In that case if the incoming incoming cookie. In that case if the incoming cookie fails the check,
cookie fails the check, a valid cookie is readily available to be a valid cookie is readily available to be sent to the client.
sent to the client without additional computation.
Also note the server may want to use special cookie values, e.g., The server MAY return a cookie request option, e.g., a null cookie,
"0", for specific scenarios. For example, the server wants to notify to signal the support of Fast Open without generating cookies, for
the client the support of TFO, but chooses not to return a valid probing or debugging purposes.
cookie for security or performance reasons upon receiving a TFO
request.
4.1.3. Client Cookie Handling 4.1.3. Client Cookie Handling
The client MUST cache cookies from servers for later Fast Open The client MUST cache cookies from servers for later Fast Open
connections. For a multi-homed client, the cookies are both client connections. For a multi-homed client, the cookies are both client
and server IP dependent. Beside the cookie, we RECOMMEND that the and server IP dependent. Beside the cookie, we RECOMMEND that the
client caches the MSS and RTT to the server to enhance performance. client caches the MSS and RTT to the server to enhance performance.
The MSS advertised by the server is stored in the cache to determine The MSS advertised by the server is stored in the cache to determine
the maximum amount of data that can be supported in the SYN packet. the maximum amount of data that can be supported in the SYN packet.
skipping to change at page 9, line 27 skipping to change at page 9, line 34
The server keeps a FastOpened flag per TCB to mark if a connection The server keeps a FastOpened flag per TCB to mark if a connection
has successfully performed a TFO. has successfully performed a TFO.
4.2.1. Fast Open Cookie Request 4.2.1. Fast Open Cookie Request
Any client attempting TFO MUST first request a cookie from the server Any client attempting TFO MUST first request a cookie from the server
with the following steps: with the following steps:
1. The client sends a SYN packet with a Fast Open Cookie Request 1. The client sends a SYN packet with a Fast Open Cookie Request
option. option.
2. The server SHOULD respond with a SYN-ACK based on the procedures 2. The server SHOULD respond with a SYN-ACK based on the procedures
in the "Server Cookie Handling" section. This SYN-ACK SHOULD in the "Server Cookie Handling" section. This SYN-ACK SHOULD
contain a Fast Open Cookie option if the server currently contain a Fast Open Cookie option if the server currently supports
supports TFO for this listener port. TFO for this listener port.
3. If the SYN-ACK contains a Fast Open Cookie option, the client 3. If the SYN-ACK contains a Fast Open Cookie option, the client
replaces the cookie and other information as described in the replaces the cookie and other information as described in the
"Client Cookie Handling" section. Otherwise, if the SYN-ACK is "Client Cookie Handling" section. Otherwise, if the SYN-ACK is
first seen, i.e.,not a (spurious) retransmission, the client MAY first seen, i.e.,not a (spurious) retransmission, the client MAY
remove the server information from the cookie cache. If the SYN- remove the server information from the cookie cache. If the SYN-
ACK is a spurious retransmission without valid Fast Open Cookie ACK is a spurious retransmission without valid Fast Open Cookie
Option, the client does nothing to the cookie cache for the Option, the client does nothing to the cookie cache for the
reasons below. reasons below.
The network or servers may drop the SYN or SYN-ACK packets with the The network or servers may drop the SYN or SYN-ACK packets with the
new cookie options which causes SYN or SYN-ACK timeouts. We RECOMMEND new cookie options which causes SYN or SYN-ACK timeouts. We RECOMMEND
both the client and the server retransmit SYN and SYN-ACK without the both the client and the server retransmit SYN and SYN-ACK without the
cookie options on timeouts. This ensures the connections of cookie cookie options on timeouts. This ensures the connections of cookie
requests will go through and lowers the latency penalties (of dropped requests will go through and lowers the latency penalties (of dropped
SYN/SYN-ACK packets). The obvious downside for maximum compatibility SYN/SYN-ACK packets). The obvious downside for maximum compatibility
is that any regular SYN drop will fail the cookie (although one can is that any regular SYN drop will fail the cookie (although one can
argue the delay in the data transmission till after 3WHS is justified argue the delay in the data transmission till after 3WHS is justified
if the SYN drop is due to network congestion). Next section if the SYN drop is due to network congestion). Next section
describes a heuristic to detect such drops when the client receives describes a heuristic to detect such drops when the client receives
the SYN-ACK. the SYN-ACK.
We also RECOMMEND the client to record servers that failed to respond We also RECOMMEND the client to record servers that failed to respond
to cookie requests and only attempt another cookie request after to cookie requests and only attempt another cookie request after
certain period. certain period. An alternate proposal is to request cookie in FIN
instead since FIN-drop by incompatible middle-box does not affect
latency. However such paths are likely to drop SYN packet with data
later, and many applications close the connections with RST instead,
so the actual benefit of this approach is not clear.
4.2.2. TCP Fast Open 4.2.2. TCP Fast Open
Once the client obtains the cookie from the target server, the client Once the client obtains the cookie from the target server, the client
can perform subsequent TFO connections until the cookie is expired by can perform subsequent TFO connections until the cookie is expired by
the server. The nature of TCP sequencing makes the TFO specific the server. The nature of TCP sequencing makes the TFO specific
changes relatively small in addition to [RFC793]. changes relatively small in addition to [RFC793].
Client: Sending SYN Client: Sending SYN
To open a TFO connection, the client MUST have obtained the cookie To open a TFO connection, the client MUST have obtained the cookie
from the server: from the server:
1. Send a SYN packet. 1. Send a SYN packet.
a. If the SYN packet does not have enough option space for the a. If the SYN packet does not have enough option space for the
Fast Open Cookie option, abort TFO and fall back to regular 3WHS. Fast Open Cookie option, abort TFO and fall back to regular 3WHS.
b. Otherwise, include the Fast Open Cookie option with the cookie b. Otherwise, include the Fast Open Cookie option with the cookie
of the server. Include any data up to the cached server MSS or of the server. Include any data up to the cached server MSS or
default 536 bytes. default 536 bytes.
2. Advance to SYN-SENT state and update SND.NXT to include the data 2. Advance to SYN-SENT state and update SND.NXT to include the data
accordingly. accordingly.
3. If RTT is available from the cache, seed SYN timer according to 3. If RTT is available from the cache, seed SYN timer according to
[RFC6298]. [RFC6298].
To deal with network or servers dropping SYN packets with payload or To deal with network or servers dropping SYN packets with payload or
unknown options, when the SYN timer fires, the client SHOULD unknown options, when the SYN timer fires, the client SHOULD
retransmit a SYN packet without data and Fast Open Cookie options. retransmit a SYN packet without data and Fast Open Cookie options.
Server: Receiving SYN and responding with SYN-ACK Server: Receiving SYN and responding with SYN-ACK
Upon receiving the SYN packet with Fast Open Cookie option: Upon receiving the SYN packet with Fast Open Cookie option:
1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled
is false, go to step 5. is false, go to step 5.
2. If PendingFastOpenRequests is over the system limit, go to step 5. 2. If PendingFastOpenRequests is over the system limit, go to step 5.
3. If IsCookieValid() in section 4.1.2 returns false, go to step 5. 3. If IsCookieValid() in section 4.1.2 returns false, go to step 5.
4. Buffer the data and notify the application. Set FastOpened flag 4. Buffer the data and notify the application. Set FastOpened flag
and increment PendingFastOpenRequests. and increment PendingFastOpenRequests.
5. Send the SYN-ACK packet. The packet MAY include a Fast Open 5. Send the SYN-ACK packet. The packet MAY include a Fast Open
Option. If FastOpened flag is set, the packet acknowledges the SYN Option. If FastOpened flag is set, the packet acknowledges the SYN
and data sequence. Otherwise it acknowledges only the SYN sequence. and data sequence. Otherwise it acknowledges only the SYN
sequence. The server MAY include data in the SYN-ACK packet if the
The server MAY include data in the SYN-ACK packet if the response response data is readily available. Some application may favor
data is readily available. Some application may favor delaying the delaying the SYN-ACK, allowing the application to process the
SYN-ACK, allowing the application to process the request in order to request in order to produce a response, but this is left to the
produce a response, but this is left to the implementation. implementation.
6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the
server MAY send more data packets before the handshake completes. The server MUST follow the congestion control [RFC5681], in particular
maximum amount is ruled by the initial congestion window and the the initial congestion window [RFC3390], to send more data
receiver window [RFC3390]. packets.
If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK
segment with neither data nor Fast Open Cookie options for segment with neither data nor Fast Open Cookie options for
compatibility reasons. compatibility reasons.
Client: Receiving SYN-ACK Client: Receiving SYN-ACK
The client SHOULD perform the following steps upon receiving the SYN- The client SHOULD perform the following steps upon receiving the SYN-
ACK: ACK:
1. Update the cookie cache if the SYN-ACK has a Fast Open Cookie 1. Update the cookie cache if the SYN-ACK has a Fast Open Cookie
Option or MSS option or both. Option or MSS option or both.
2. Send an ACK packet. Set acknowledgment number to RCV.NXT and 2. Send an ACK packet. Set acknowledgment number to RCV.NXT and
include the data after SND.UNA if data is available. include the data after SND.UNA if data is available.
3. Advance to the ESTABLISHED state. 3. Advance to the ESTABLISHED state.
Note there is no latency penalty if the server does not acknowledge Note there is no latency penalty if the server does not acknowledge
the data in the original SYN packet. The client SHOULD retransmit any the data in the original SYN packet. The client SHOULD retransmit any
unacknowledged data in the first ACK packet in step 2. The data unacknowledged data in the first ACK packet in step 2. The data
exchange will start after the handshake like a regular TCP exchange will start after the handshake like a regular TCP
connection. connection.
If the client has timed out and retransmitted only regular SYN If the client has timed out and retransmitted only regular SYN
skipping to change at page 13, line 45 skipping to change at page 13, line 15
against spoofed SYN floods with invalid cookies using existing against spoofed SYN floods with invalid cookies using existing
techniques [RFC4987]. We note that generating bogus cookies is techniques [RFC4987]. We note that generating bogus cookies is
usually cheaper than validating them. But the additional cost of usually cheaper than validating them. But the additional cost of
validating the cookies, inherent to any authentication scheme, may validating the cookies, inherent to any authentication scheme, may
not be substantial compared to processing a regular SYN packet. not be substantial compared to processing a regular SYN packet.
However, the attacker may still obtain cookies from some compromised However, the attacker may still obtain cookies from some compromised
hosts, then flood spoofed SYN with data and "valid" cookies (from hosts, then flood spoofed SYN with data and "valid" cookies (from
these hosts or other vantage points). With DHCP, it's possible to these hosts or other vantage points). With DHCP, it's possible to
obtain cookies of past IP addresses without compromising any host. obtain cookies of past IP addresses without compromising any host.
Below we identify two new vulnerabilities of TFO and describe the Below we identify new vulnerabilities of TFO and describe the
countermeasures. countermeasures.
6.1. Server Resource Exhaustion Attack by SYN Flood with Valid Cookies 6.1. Server Resource Exhaustion Attack by SYN Flood with Valid Cookies
Like regular TCP handshakes, TFO is vulnerable to such an attack. But Like regular TCP handshakes, TFO is vulnerable to such an attack. But
the potential damage can be much more severe. Besides causing the potential damage can be much more severe. Besides causing
temporary disruption to service ports under attack, it may exhaust temporary disruption to service ports under attack, it may exhaust
server CPU and memory resources. server CPU and memory resources.
For this reason it is crucial for the TFO server to limit the maximum For this reason it is crucial for the TFO server to limit the maximum
skipping to change at page 15, line 43 skipping to change at page 15, line 13
bits directly. The degree of damage will be identical, but TFO- bits directly. The degree of damage will be identical, but TFO-
specific attack allows the attacker to remain anonymous and disguises specific attack allows the attacker to remain anonymous and disguises
the attack as from other servers. the attack as from other servers.
The best defense is for the server not to respond with data until The best defense is for the server not to respond with data until
handshake finishes. In this case the risk of amplification reflection handshake finishes. In this case the risk of amplification reflection
attack is completely eliminated. But the potential latency saving attack is completely eliminated. But the potential latency saving
from TFO may diminish if the server application produces responses from TFO may diminish if the server application produces responses
earlier before the handshake completes. earlier before the handshake completes.
6.3 Attacks from behind sharing public IPs (NATs)
An attacker behind NAT can easily obtain valid cookies to launch the
above attack to hurt other clients that share the path. [BOB12]
suggested that the server can extend cookie generation to include the
TCP timestamp---GetCookie(IP_Address, Timestamp)---and implement it
by encrypting the concatenation of the two values to generate the
cookie. The client stores both the cookie and its corresponding
timestamp, and echoes both in the SYN. The server then implements
IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting the IP and
timestamp data and comparing it with the cookie value.
This enables the server to issue different cookies to clients that
share the same IP address, hence can selectively discard those
misused cookies from the attacker. However the attacker can simply
repeat the attack with new cookies. The server would eventually need
to throttle all requests from the IP address just like the current
approach. Moreover this approach requires modifying [RFC 1323] to
send non-zero Timestamp Echo Reply in SYN, potentially cause firewall
issues. Therefore we believe the benefit may not outweigh the
drawbacks.
7. Web Performance 7. Web Performance
7.1. HTTP persistent connection 7.1. HTTP persistent connection
TCP connection setup overhead has long been identified as a TCP connection setup overhead has long been identified as a
performance bottleneck for web applications [THK98]. HTTP persistent performance bottleneck for web applications [THK98]. HTTP persistent
connection was proposed to mitigate this issue and has been widely connection was proposed to mitigate this issue and has been widely
deployed. However, [RCCJR11][AERG11] show that the average number of deployed. However, [RCCJR11][AERG11] show that the average number of
transactions per connection is between 2 and 4, based on large-scale transactions per connection is between 2 and 4, based on large-scale
measurements from both servers and clients. In these studies, the measurements from both servers and clients. In these studies, the
skipping to change at page 16, line 11 skipping to change at page 16, line 4
connection was proposed to mitigate this issue and has been widely connection was proposed to mitigate this issue and has been widely
deployed. However, [RCCJR11][AERG11] show that the average number of deployed. However, [RCCJR11][AERG11] show that the average number of
transactions per connection is between 2 and 4, based on large-scale transactions per connection is between 2 and 4, based on large-scale
measurements from both servers and clients. In these studies, the measurements from both servers and clients. In these studies, the
servers and clients both kept the idle connections up to several servers and clients both kept the idle connections up to several
minutes, well into the human think time. minutes, well into the human think time.
Can the utilization rate increase by keeping connections even longer? Can the utilization rate increase by keeping connections even longer?
Unfortunately, this is problematic due to middle-boxes and rapidly Unfortunately, this is problematic due to middle-boxes and rapidly
growing mobile end hosts. One major issue is NAT. Studies growing mobile end hosts. One major issue is NAT. Studies
[HNESSK10][MQXMZ11] show that the majority of home routers and ISPs [HNESSK10][MQXMZ11] show that the majority of home routers and ISPs
fail to meet the the 124 minutes idle timeout mandated in [RFC5382]. fail to meet the the 124 minutes idle timeout mandated in [RFC5382].
In [MQXMZ11], 35% of mobile ISPs timeout idle connections within 30 In [MQXMZ11], 35% of mobile ISPs timeout idle connections within 30
minutes. NAT boxes do not possess a reliable mechanism to notify minutes. NAT boxes do not possess a reliable mechanism to notify end
endhosts when idle connections are removed from local tables, either hosts when idle connections are removed from local tables, either due
due to resource constraints such as mapping table size, memory, or to resource constraints such as mapping table size, memory, or lookup
lookup overhead, or due to the limited port number and IP address overhead, or due to the limited port number and IP address space.
space. Moreover, unmapped packets received by NAT boxes are often Moreover, unmapped packets received by NAT boxes are often dropped
dropped silently. (TCP RST is not required by RFC5382.) The end host silently. (TCP RST is not required by RFC5382.) The end host
attempting to use these broken connections are often forced to wait attempting to use these broken connections are often forced to wait
for a lengthy TCP timeout. Thus the browser risks large performance for a lengthy TCP timeout. Thus the browser risks large performance
penalty when keeping idle connections open. To circumvent this penalty when keeping idle connections open. To circumvent this
problem, some applications send frequent TCP keep-alive probes. problem, some applications send frequent TCP keep-alive probes.
However, this technique drains power on mobile devices [MQXMZ11]. In However, this technique drains power on mobile devices [MQXMZ11]. In
fact, power has become a prominent issue in modern LTE devices that fact, power has become a prominent issue in modern LTE devices that
mobile browsers close the HTTP connections within seconds or even mobile browsers close the HTTP connections within seconds or even
immediately [SOUDERS11]. immediately [SOUDERS11].
Idle connections also consume more memory resources. Due to the Idle connections also consume more memory resources. Due to the
skipping to change at page 17, line 6 skipping to change at page 16, line 48
residential broadband and mobile networks. They showed that TFO residential broadband and mobile networks. They showed that TFO
improves page load time by 10% to 40%. More detailed on the design improves page load time by 10% to 40%. More detailed on the design
tradeoffs and measurement can be found at [RCCJB11]. tradeoffs and measurement can be found at [RCCJB11].
8. TFO's Applicability 8. TFO's Applicability
TFO aims at latency conscious applications that are sensitive to TFO aims at latency conscious applications that are sensitive to
TCP's initial connection setup delay. These application protocols TCP's initial connection setup delay. These application protocols
often employ short-lived TCP connections, or employ long-lived often employ short-lived TCP connections, or employ long-lived
connections but are more sensitive to the connection setup delay due connections but are more sensitive to the connection setup delay due
to, e.g., a more strict connection failover requirement. to, e.g., a more strict connection fail-over requirement.
Only transaction-type applications where RTT constitutes a Only transaction-type applications where RTT constitutes a
significant portion of the total end-to-end latency will likely significant portion of the total end-to-end latency will likely
benefit from TFO. Moreover, the client request must fit in the SYN benefit from TFO. Moreover, the client request must fit in the SYN
packet. Otherwise there may not be any saving in the total number of packet. Otherwise there may not be any saving in the total number of
round trips required to complete a transaction. round trips required to complete a transaction.
To the extent possible applications protocols SHOULD employ long- To the extent possible applications protocols SHOULD employ long-
lived connections to best take advantage of TCP's built-in congestion lived connections to best take advantage of TCP's built-in congestion
control algorithm, and to reduce the impact from TCP's connection control algorithm, and to reduce the impact from TCP's connection
setup overhead. E.g., for the web applications, P-HTTP will likely setup overhead. E.g., for the web applications, P-HTTP will likely
help and is much easier to deploy hence should be attempted first. help and is much easier to deploy hence should be attempted first.
TFO will likely provide further latency reduction on top of P-HTTP. TFO will likely provide further latency reduction on top of P-HTTP.
But the additional benefit will depend on how much persistency one But the additional benefit will depend on how much persistency one
can get from HTTP in a given operating environment. can get from HTTP in a given operating environment.
One alternative to short-lived TCP connection might be UDP, which is One alternative to short-lived TCP connection might be UDP, which is
connectionless hence doesn't inflict any connection setup delay, and connectionless hence doesn't inflict any connection setup delay, and
is best suited for application protocols that are transactional. is best suited for application protocols that are transactional.
Practical deployment issues such as middlebox and/or firewall Practical deployment issues such as middle-box and/or firewall
traversal may severely limit the use of UDP based application traversal may severely limit the use of UDP based application
protocols though. protocols though.
Note that when the application employs too many short-lived Note that when the application employs too many short-lived
connections, it may negatively impact network stability, as these connections, it may negatively impact network stability, as these
connections often exit before TCP's congestion control algorithm connections often exit before TCP's congestion control algorithm
kicks in. Implementations supporting large number of short-lived kicks in. Implementations supporting large number of short-lived
connections should employ temporal sharing of TCB data as described connections should employ temporal sharing of TCB data as described
in [RFC2140]. in [RFC2140].
skipping to change at page 19, line 9 skipping to change at page 18, line 50
as TFO. In Rapid-Restart, both the server and the client retain the as TFO. In Rapid-Restart, both the server and the client retain the
TCP control blocks after a connection is terminated in order to TCP control blocks after a connection is terminated in order to
allow/resume data exchange in next connection handshake. In contrary, allow/resume data exchange in next connection handshake. In contrary,
TFO does not require keeping both TCB on both sides and is more TFO does not require keeping both TCB on both sides and is more
scalable. scalable.
10. IANA Considerations 10. IANA Considerations
The Fast Open Cookie Option and Fast Open Cookie Request Option The Fast Open Cookie Option and Fast Open Cookie Request Option
define no new namespace. The options require IANA allocate one value define no new namespace. The options require IANA allocate one value
from the TCP option Kind namespace. from the TCP option Kind namespace. Early implementation before the
allocation SHOULD follow [EXPOPT] and use experimental option 254 and
11. Acknowledgements magic number 0xF989 (16 bits), and migrate to the new option after
the allocation according.
The authors would like to thank Tom Herbert, Rick Jones, Adam 11. Acknowledgement
Langley, Mathew Mathis, Roberto Peon, and Barath Raghavan for their We thank Rick Jones, Bob Briscoe, Adam Langley, Matt Mathis, Neal
insightful comments. Cardwell, Roberto Peon, and Tom Herbert for their feedbacks. We
especially thank Barath Raghavan for his contribution on the security
design of Fast Open.
12. References 12. References
12.1. Normative References 12.1. Normative References
[RFC793] Postel, J. "Transmission Control Protocol", RFC 793, [RFC793] Postel, J. "Transmission Control Protocol", RFC 793,
September 1981. September 1981.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, October 1989. Communication Layers", STD 3, RFC 1122, October 1989.
skipping to change at page 20, line 33 skipping to change at page 20, line 33
12.2. Informative References 12.2. Informative References
[AERG11] M. Al-Fares, K. Elmeleegy, B. Reed, and I. Gashinsky, [AERG11] M. Al-Fares, K. Elmeleegy, B. Reed, and I. Gashinsky,
"Overclocking the Yahoo! CDN for Faster Web Page Loads". In "Overclocking the Yahoo! CDN for Faster Web Page Loads". In
Proceedings of Internet Measurement Conference, November Proceedings of Internet Measurement Conference, November
2011. 2011.
[CDCM11] Chu, J., Dukkipati, N., Cheng, Y. and M. Mathis, [CDCM11] Chu, J., Dukkipati, N., Cheng, Y. and M. Mathis,
"Increasing TCP's Initial Window", Internet-Draft draft- "Increasing TCP's Initial Window", Internet-Draft draft-
ietf-tcpm- initcwnd-02.txt (work in progress), October ietf-tcpm-initcwnd-02.txt (work in progress), October 2011.
2011.
[EXPOPT] Touch, Joe, "Shared Use of Experimental TCP Options",
Internet-Draft draft-ietf-tcpm-experimental-options (work
in progress), October 2012.
[HNESSK10] S. Haetoenen, A. Nyrhinen, L. Eggert, S. Strowes, P. [HNESSK10] S. Haetoenen, A. Nyrhinen, L. Eggert, S. Strowes, P.
Sarolahti, M. Kojo., "An Experimental Study of Home Gateway Sarolahti, M. Kojo., "An Experimental Study of Home Gateway
Characteristics". In Proceedings of Internet Measurement Characteristics". In Proceedings of Internet Measurement
Conference. Octobor 2010 Conference. Octobor 2010
[HNRGHT11] M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M. [HNRGHT11] M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M.
Handley, H. Tokuda, "Is it Still Possible to Extend TCP?". Handley, H. Tokuda, "Is it Still Possible to Extend TCP?".
In Proceedings of Internet Measurement Conference. November In Proceedings of Internet Measurement Conference. November
2011. 2011.
skipping to change at page 22, line 5 skipping to change at page 21, line 49
(work in progress), July 2011. (work in progress), July 2011.
[SOUDERS11] S. Souders. "Making A Mobile Connection". [SOUDERS11] S. Souders. "Making A Mobile Connection".
http://www.stevesouders.com/blog/2011/09/21/making-a- http://www.stevesouders.com/blog/2011/09/21/making-a-
mobile-connection/ mobile-connection/
[THK98] Touch, J., Heidemann, J., Obraczka, K., "Analysis of HTTP [THK98] Touch, J., Heidemann, J., Obraczka, K., "Analysis of HTTP
Performance", USC/ISI Research Report 98-463. December Performance", USC/ISI Research Report 98-463. December
1998. 1998.
[BOB12] Briscoe, B., "Some ideas building on draft-ietf-tcpm-
fastopen-01", tcpm list,
http://www.ietf.org/mail-archive/web/tcpm/current/
msg07192.html
Author's Addresses Author's Addresses
Yuchung Cheng Yuchung Cheng
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043, USA
EMail: ycheng@google.com EMail: ycheng@google.com
Jerry Chu Jerry Chu
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043, USA
EMail: hkchu@google.com EMail: hkchu@google.com
Sivasankar Radhakrishnan Sivasankar Radhakrishnan
Google, Inc. Department of Computer Science and Engineering
1600 Amphitheatre Parkway University of California, San Diego
Mountain View, CA 94043, USA 9500 Gilman Dr
La Jolla, CA 92093-0404
EMail: sivasankar@cs.ucsd.edu EMail: sivasankar@cs.ucsd.edu
Arvind Jain Arvind Jain
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043, USA
EMail: arvind@google.com EMail: arvind@google.com
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
 End of changes. 63 change blocks. 
160 lines changed or deleted 217 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/