draft-ietf-tcpm-fastopen-03.txt   draft-ietf-tcpm-fastopen-04.txt 
Internet Draft Y. Cheng Internet Draft Y. Cheng
draft-ietf-tcpm-fastopen-03.txt J. Chu draft-ietf-tcpm-fastopen-04.txt J. Chu
Intended status: Experimental S. Radhakrishnan Intended status: Experimental S. Radhakrishnan
Expiration date: August, 2013 A. Jain Expiration date: February, 2014 A. Jain
Google, Inc. Google, Inc.
Feburary 25, 2013 July 15, 2013
TCP Fast Open TCP Fast Open
Status of this Memo Status of this Memo
Distribution of this memo is unlimited. Distribution of this memo is unlimited.
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
skipping to change at page 1, line 34 skipping to change at page 1, line 34
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire in August, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 2, line 6 skipping to change at page 2, line 4
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Abstract Abstract
TCP Fast Open (TFO) allows data to be carried in the SYN and SYN-ACK TCP Fast Open (TFO) allows data to be carried in the SYN and SYN-ACK
packets and consumed by the receiving end during the initial packets and consumed by the receiving end during the initial
connection handshake, thus saving up to one full round trip time connection handshake, thus saving up to one full round trip time
(RTT) compared to standard TCP which requires a three-way handshake (RTT) compared to the standard TCP, which requires a three-way
(3WHS) to complete before data can be exchanged. handshake (3WHS) to complete before data can be exchanged. However
TFO deviates from the standard TCP semantics in that the data in the
SYN could be replayed to an application in some rare circumstances.
Applications should not use TFO unless they can tolerate this issue,
which is detailed in the Applicability section.
Terminology Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
TFO refers to TCP Fast Open. Client refers to the TCP's active open TFO refers to TCP Fast Open. Client refers to the TCP's active open
side and server refers to the TCP's passive open side. side and server refers to the TCP's passive open side.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 Relaxing TCP semantics on duplicated SYNs . . . . . . . . . 4 2.1 Relaxing TCP Semantics on Duplicated SYNs . . . . . . . . . 4
2.2. SYNs with spoofed IP addresses . . . . . . . . . . . . . . 4 2.2. SYNs with Spoofed IP Addresses . . . . . . . . . . . . . . 4
3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5 3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5
4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 7 4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 7
4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 7 4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 7
4.1.1. TCP Options . . . . . . . . . . . . . . . . . . . . . . 7 4.1.1. TCP Options . . . . . . . . . . . . . . . . . . . . . . 7
4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 8 4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 8
4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 9 4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 9
4.2. Fast Open Protocol . . . . . . . . . . . . . . . . . . . . 9 4.2. Fast Open Protocol . . . . . . . . . . . . . . . . . . . . 9
4.2.1. Fast Open Cookie Request . . . . . . . . . . . . . . . 10 4.2.1. Fast Open Cookie Request . . . . . . . . . . . . . . . 10
4.2.2. TCP Fast Open . . . . . . . . . . . . . . . . . . . . . 11 4.2.2. TCP Fast Open . . . . . . . . . . . . . . . . . . . . . 11
5. Reliability and Deployment Issues . . . . . . . . . . . . . . . 13 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 13
6. Security Considerations . . . . . . . . . . . . . . . . . . . . 14 5.1. Resource Exhaustion Attack by SYN Flood with Valid
6.1. Server Resource Exhaustion Attack by SYN Flood with Valid Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5.1.1 Attacks from behind Sharing Public IPs (NATs) . . . . . 14
6.2. Amplified Reflection Attack to Random Host . . . . . . . . 15 5.2. Amplified Reflection Attack to Random Host . . . . . . . . 15
6.3 Attacks from behind sharing public IPs (NATs) . . . . . . . 16 6. TFO's Applicability . . . . . . . . . . . . . . . . . . . . . . 16
7. TFO's Applicability . . . . . . . . . . . . . . . . . . . . . . 17 6.1 Duplicate Data in SYNs . . . . . . . . . . . . . . . . . . . 16
7.1 Duplicate data in SYNs . . . . . . . . . . . . . . . . . . . 17 6.2 Potential Performance Improvement . . . . . . . . . . . . . 16
7.2 Potential performance improvement . . . . . . . . . . . . . 17 6.3. Example: Web Clients and Servers . . . . . . . . . . . . . 16
7.3 Example: Web clients and servers . . . . . . . . . . . . . . 17 6.3.1. HTTP Request Replay . . . . . . . . . . . . . . . . . . 16
7.3.1 HTTP request replay . . . . . . . . . . . . . . . . . . 17 6.3.2. Comparison with HTTP Persistent Connections . . . . . . 17
7.3.2 HTTP persistent connection . . . . . . . . . . . . . . . 18 7. Open Areas for Experimentation . . . . . . . . . . . . . . . . 17
8. Performance Experiments . . . . . . . . . . . . . . . . . . . . 18 7.1. Performance impact due to middle-boxes and NAT . . . . . . 18
9. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 19 7.2. Cookie-less Fast Open . . . . . . . . . . . . . . . . . . . 18
9.1. T/TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 8. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 18
9.2. Common Defenses Against SYN Flood Attacks . . . . . . . . . 19 8.1. T/TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
9.3. TCP Cookie Transaction (TCPCT) . . . . . . . . . . . . . . 20 8.2. Common Defenses Against SYN Flood Attacks . . . . . . . . . 19
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 8.3. TCP Cookie Transaction (TCPCT) . . . . . . . . . . . . . . 19
11. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 20 8.4. Speculative Connections by the Applications . . . . . . . . 19
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 19
12.1. Normative References . . . . . . . . . . . . . . . . . . . 20 10. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 20
12.2. Informative References . . . . . . . . . . . . . . . . . . 21 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 11.1. Normative References . . . . . . . . . . . . . . . . . . . 20
11.2. Informative References . . . . . . . . . . . . . . . . . . 20
Appendix A. Example Socket API Changes to support TFO . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction 1. Introduction
TCP Fast Open (TFO) enables data to be exchanged safely during TCP's TCP Fast Open (TFO) enables data to be exchanged safely during TCP's
connection handshake. connection handshake. This document describes a design that enables
applications to save a round trip while avoiding severe security
This document describes a design that enables applications to save a ramifications. At the core of TFO is a security cookie used by the
round trip while avoiding severe security ramifications. At the core server side to authenticate a client initiating a TFO connection.
of TFO is a security cookie used by the server side to authenticate a This document covers the details of exchanging data during TCP's
client initiating a TFO connection. This document covers the details initial handshake, the protocol for TFO cookies, potential new
of exchanging data during TCP's initial handshake, the protocol for security vulnerabilities and their mitigation, and the new socket
TFO cookies, and potential new security vulnerabilities and their API.
mitigation. It also includes discussion of deployment issues and
related proposals. TFO requires extensions to the socket API but this
document does not cover that.
TFO is motivated by the performance needs of today's Web TFO is motivated by the performance needs of today's Web
applications. Network latency is largely determined by a connection's applications. Current TCP only permits data exchange after 3WHS
round-trip time (RTT) and the number of round trips required to [RFC793], which adds one RTT to network latency. For short Web
transfer application data. RTT consists of propagation delay and transfers this additional RTT is a significant portion of overall
queuing delay. network latency [THK98], even when HTTP persistent connection is
widely used. For example, the Chrome browser keeps TCP connections
Network bandwidth has grown substantially over the past two decades,
potentially reducing queuing delay, while propagation delay is
largely constrained by the speed of light and has remained unchanged.
Therefore reducing the number of round trips has typically become the
most effective way to improve the latency of applications like the
Web [CDCM11].
Current TCP only permits data exchange after 3WHS [RFC793], which
adds one RTT to network latency. For short transfers (e.g., web
objects) this additional RTT is a significant portion of overall
network latency [THK98]. One widely deployed solution is HTTP
persistent connections. However, this solution is limited since hosts
and middle boxes terminate idle TCP connections due to resource
constraints. For example, the Chrome browser keeps TCP connections
idle for up to 5 minutes but 35% of Chrome HTTP requests are made on idle for up to 5 minutes but 35% of Chrome HTTP requests are made on
new TCP connections [RCCJR11]. We discuss Web applications and TFO in new TCP connections [RCCJR11]. For such Web and Web-like applications
detail later in section 7. placing data in the SYN can yield significant latency improvements.
Next we describe how we resolve the challenges that arise upon doing
so.
2. Data In SYN 2. Data In SYN
Standard TCP already allows data to be carried in SYN packets
([RFC793], section 3.4) but forbids the receiver from delivering it
to the application until 3WHS is completed. This is because TCP's
initial handshake serves to capture old or duplicate SYNs.
Allowing data in SYN packets to be delivered raises two issues Allowing data in SYN packets to be delivered raises two issues
discussed in the following subsections. These issues make TFO discussed in the following subsections. These issues make TFO
undesirable for certain applications. Therefore TCP implementations unsuitable for certain applications. Therefore TCP implementations
MUST NOT use TFO by default and only use TFO if requested explicitly MUST NOT use TFO by default, but only use TFO if requested explicitly
by the application on a per service port basis. Applications need to by the application on a per service port basis. Applications need to
evaluate TFO applicability (described in Section 7) before using TFO. evaluate TFO applicability described in Section 6 before using TFO.
2.1 Relaxing TCP semantics on duplicated SYNs
[RFC793] (section 3.4) already allows data in SYN packets but forbids 2.1 Relaxing TCP Semantics on Duplicated SYNs
the receiver from delivering the data to the application until 3WHS
is completed. This is because TCP's initial handshake serves to
capture old or duplicate SYNs.
TFO allows data to be delivered to the application before 3WHS is TFO allows data to be delivered to the application before 3WHS is
completed, thus opening itself to a data integrity issue for the completed, thus opening itself to a data integrity issue in either of
applications in Section 2.1 in either of the following cases: the two cases below:
a) the receiver host receives data in a duplicate SYN after it has a) the receiver host receives data in a duplicate SYN after it has
forgotten it received the original SYN (e.g. due to a reboot); b) the forgotten it received the original SYN (e.g. due to a reboot);
duplicate is received after the connection created by the original
SYN has been closed and the close was initiated by the sender (so
the receiver will not be protected by the 2MSL TIMEWAIT state).
The obsoleted T/TCP protocol employs a new TCP "TAO" option and b) the duplicate is received after the connection created by the
connection count to guard against old or duplicate SYNs [RFC1644]. original SYN has been closed and the close was initiated by the
However it is not widely used due to various vulnerabilities sender (so the receiver will not be protected by the 2MSL TIMEWAIT
[PHRACK98]. state).
Rather than trying to capture all dubious SYN packets to make TFO The now obsoleted T/TCP [RFC1644] attempted to address these issues.
100% compatible with TCP semantics, we made a design decision early It is not successful and not deployed due to various vulnerabilities
on to accept old SYN packets with data, i.e., to restrict TFO use to [PHRACK98]. Rather than trying to capture all dubious SYN packets to
a class of applications (Section 7) that are tolerant of duplicate make TFO 100% compatible with TCP semantics, we made a design
SYN packets with data. We believe this is the right design trade-off decision early on to accept old SYN packets with data, i.e., to
balancing complexity with usefulness for certain applications. restrict TFO use to a class of applications (Section 6) that are
tolerant of duplicate SYN packets with data. We believe this is the
right design trade-off balancing complexity with usefulness.
2.2. SYNs with spoofed IP addresses 2.2. SYNs with Spoofed IP Addresses
Standard TCP suffers from the SYN flood attack [RFC4987] because Standard TCP suffers from the SYN flood attack [RFC4987] because
bogus SYN packets, i.e., SYN packets with spoofed source IP addresses bogus SYN packets, i.e., SYN packets with spoofed source IP addresses
can easily fill up a listener's small queue, causing a service port can easily fill up a listener's small queue, causing a service port
to be blocked completely until timeouts. Secondary damage comes from to be blocked completely until timeouts. Secondary damage comes from
these SYN requests taking up memory space. Though this is less of an these SYN requests taking up memory space. Though this is less of an
issue today as servers typically have plenty of memory. issue today as servers typically have plenty of memory.
TFO goes one step further to allow server-side TCP to process and TFO goes one step further to allow server-side TCP to send up data to
send up data to the application layer before 3WHS is completed. This the application layer before 3WHS is completed. This opens up serious
opens up more serious new vulnerabilities. Applications serving ports new vulnerabilities. Applications serving ports that have TFO enabled
that have TFO enabled may waste lots of CPU and memory resources may waste lots of CPU and memory resources processing the requests
processing the requests and producing the responses. If the response and producing the responses. If the response is much larger than the
is much larger than the request, the attacker can mount an amplified request, the attacker can further mount an amplified reflection
reflection attack against victims of choice beyond the TFO server attack against victims of choice beyond the TFO server itself.
itself.
Numerous mitigation techniques against regular SYN flood attacks Numerous mitigation techniques against regular SYN flood attacks
exist and have been well documented [RFC4987]. Unfortunately none are exist and have been well documented [RFC4987]. Unfortunately none are
applicable to TFO. We propose a server-supplied cookie to mitigate applicable to TFO. We propose a server-supplied cookie to mitigate
the primary security issues introduced by TFO in Section 3. We defer these new vulnerabilities in Section 3 and evaluate the effectiveness
further discussion of SYN flood attacks to the "Security of the defense in Section 7.
Considerations" section.
3. Protocol Overview 3. Protocol Overview
The key component of TFO is the Fast Open Cookie (cookie), a message The key component of TFO is the Fast Open Cookie (cookie), a message
authentication code (MAC) tag generated by the server. The client authentication code (MAC) tag generated by the server. The client
requests a cookie in one regular TCP connection, then uses it for requests a cookie in one regular TCP connection, then uses it for
future TCP connections to exchange data during 3WHS: Requesting a future TCP connections to exchange data during 3WHS:
Fast Open Cookie:
Requesting a Fast Open Cookie:
1. The client sends a SYN with a Fast Open Cookie Request option. 1. The client sends a SYN with a Fast Open Cookie Request option.
2. The server generates a cookie and sends it through the Fast Open 2. The server generates a cookie and sends it through the Fast Open
Cookie option of a SYN-ACK packet. Cookie option of a SYN-ACK packet.
3. The client caches the cookie for future TCP Fast Open connections 3. The client caches the cookie for future TCP Fast Open connections
(see below). (see below).
Performing TCP Fast Open: Performing TCP Fast Open:
skipping to change at page 6, line 10 skipping to change at page 5, line 44
3. If the server accepts the data in the SYN packet, it may send the 3. If the server accepts the data in the SYN packet, it may send the
response data before the handshake finishes. The max amount is response data before the handshake finishes. The max amount is
governed by the TCP's congestion control [RFC5681]. governed by the TCP's congestion control [RFC5681].
4. The client sends an ACK acknowledging the SYN and the server data. 4. The client sends an ACK acknowledging the SYN and the server data.
If the client's data is not acknowledged, the client retransmits If the client's data is not acknowledged, the client retransmits
the data in the ACK packet. the data in the ACK packet.
5. The rest of the connection proceeds like a normal TCP connection. 5. The rest of the connection proceeds like a normal TCP connection.
The client can repeat many Fast Open operations once it acquires a The client can repeat many Fast Open operations once it acquires a
cookie (until the cookie is expired by the server). Thus TFO is cookie (until the cookie is expired by the server). Thus TFO is
useful for applications that have temporal locality on client and useful for applications that have temporal locality on client and
server connections. server connections.
Requesting Fast Open Cookie in connection 1: Requesting Fast Open Cookie in connection 1:
TCP A (Client) TCP B(Server) TCP A (Client) TCP B(Server)
______________ _____________ ______________ _____________
CLOSED LISTEN CLOSED LISTEN
#1 SYN-SENT ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD #1 SYN-SENT ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD
#2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD #2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD
skipping to change at page 7, line 23 skipping to change at page 7, line 23
cookie and passes it back on subsequent SYN packets to open new cookie and passes it back on subsequent SYN packets to open new
connections. The server can expire the cookie at any time to enhance connections. The server can expire the cookie at any time to enhance
security. security.
4.1.1. TCP Options 4.1.1. TCP Options
Fast Open Cookie Option Fast Open Cookie Option
The server uses this option to grant a cookie to the client in the The server uses this option to grant a cookie to the client in the
SYN-ACK packet; the client uses it to pass the cookie back to the SYN-ACK packet; the client uses it to pass the cookie back to the
server in the SYN packet. server in subsequent SYN packets.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Kind | Length | | Kind | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | | |
~ Cookie ~ ~ Cookie ~
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Kind 1 byte: constant TBD (assigned by IANA) Kind 1 byte: constant TBD (assigned by IANA)
skipping to change at page 8, line 15 skipping to change at page 8, line 13
Length 1 byte: constant 2. This distinguishes the option Length 1 byte: constant 2. This distinguishes the option
from the Fast Open cookie option. from the Fast Open cookie option.
Options with invalid Length values, without SYN flag set, or with ACK Options with invalid Length values, without SYN flag set, or with ACK
flag set MUST be ignored. flag set MUST be ignored.
4.1.2. Server Cookie Handling 4.1.2. Server Cookie Handling
The server is in charge of cookie generation and authentication. The The server is in charge of cookie generation and authentication. The
cookie SHOULD be a message authentication code tag with the following cookie SHOULD be a message authentication code tag with the following
properties: properties:
1. The cookie authenticates the client's (source) IP address of the 1. The cookie authenticates the client's (source) IP address of the
SYN packet. The IP address can be an IPv4 or IPv6 address. SYN packet. The IP address may be an IPv4 or IPv6 address.
2. The cookie can only be generated by the server and can not be 2. The cookie can only be generated by the server and can not be
fabricated by any other parties including the client. fabricated by any other parties including the client.
3. The generation and verification are fast relative to the rest of 3. The generation and verification are fast relative to the rest of
SYN and SYN-ACK processing. SYN and SYN-ACK processing.
4. A server may encode other information in the cookie, and accept 4. A server may encode other information in the cookie, and accept
more than one valid cookie per client at any given time. But this more than one valid cookie per client at any given time. But this
is all server implementation dependent and transparent to the is all server implementation dependent and transparent to the
client. client.
5. The cookie expires after a certain amount of time. The reason for 5. The cookie expires after a certain amount of time. The reason for
cookie expiration is detailed in the "Security Consideration" cookie expiration is detailed in the "Security Consideration"
section. This can be done by either periodically changing the section. This can be done by either periodically changing the
server key used to generate cookies or including a timestamp when server key used to generate cookies or including a timestamp when
generating the cookie. generating the cookie.
To gradually invalidate cookies over time, the server can To gradually invalidate cookies over time, the server can
implement key rotation to generate and verify cookies using implement key rotation to generate and verify cookies using
multiple keys. This approach is useful for large-scale servers to multiple keys. This approach is useful for large-scale servers to
retain Fast Open rolling key updates. We do not specify a retain Fast Open rolling key updates. We do not specify a
particular mechanism because the implementation is often server particular mechanism because the implementation is server
specific. specific.
The server supports the cookie generation and verification The server supports the cookie generation and verification
operations: operations:
- GetCookie(IP_Address): returns a (new) cookie - GetCookie(IP_Address): returns a (new) cookie
- IsCookieValid(IP_Address, Cookie): checks if the cookie is valid, - IsCookieValid(IP_Address, Cookie): checks if the cookie is valid,
i.e., it has not expired and it authenticates the client IP address. i.e., it has not expired and it authenticates the client IP address.
Example Implementation: a simple implementation is to use AES_128 to Example Implementation: a simple implementation is to use AES_128 to
encrypt the IPv4 (with padding) or IPv6 address and truncate to 64 encrypt the IPv4 (with padding) or IPv6 address and truncate to 64
bits. The server can periodically update the key to expire the bits. The server can periodically update the key to expire the
cookies. AES encryption on recent processors is fast and takes only a cookies. AES encryption on recent processors is fast and takes only a
few hundred nanoseconds [RCCJR11]. few hundred nanoseconds [RCCJR11].
If only one valid cookie is allowed per-client and the server can If only one valid cookie is allowed per-IP and the server can
regenerate the cookie independently, the best validation process is regenerate the cookie independently, the best validation process is
to simply regenerate a valid cookie and compare it against the to simply regenerate a valid cookie and compare it against the
incoming cookie. In that case if the incoming cookie fails the check, incoming cookie. In that case if the incoming cookie fails the check,
a valid cookie is readily available to be sent to the client. a valid cookie is readily available to be sent to the client.
The server MAY return a cookie request option, e.g., a null cookie,
to signal the support of Fast Open without generating cookies, for
probing or debugging purposes.
4.1.3. Client Cookie Handling 4.1.3. Client Cookie Handling
The client MUST cache cookies from servers for later Fast Open The client MUST cache cookies from servers for later Fast Open
connections. For a multi-homed client, the cookies are both client connections. For a multi-homed client, the cookies are both client
and server IP dependent. Beside the cookie, we RECOMMEND that the and server IP dependent. Beside the cookie, we RECOMMEND that the
client caches the MSS and RTT to the server to enhance performance. client caches the MSS and RTT to the server to enhance performance.
The MSS advertised by the server is stored in the cache to determine The MSS advertised by the server is stored in the cache to determine
the maximum amount of data that can be supported in the SYN packet. the maximum amount of data that can be supported in the SYN packet.
This information is needed because data is sent before the server This information is needed because data is sent before the server
announces its MSS in the SYN-ACK packet. Without this information, announces its MSS in the SYN-ACK packet. Without this information,
the data size in the SYN packet is limited to the default MSS of 536 the data size in the SYN packet is limited to the default MSS of 536
bytes [RFC1122]. The client SHOULD update the cache MSS value bytes [RFC1122]. The client SHOULD update the cache MSS value
whenever it discovers new MSS value, e.g., through path MTU whenever it discovers new MSS value, e.g., through path MTU
discovery. discovery.
Caching RTT allows seeding a more accurate SYN timeout than the Caching RTT allows seeding a more accurate SYN timeout than the
default value [RFC6298]. This lowers the performance penalty if the default value [RFC6298]. This lowers the performance penalty if the
network or the server drops the SYN packets with data or the cookie network or the server drops the SYN packets with data or the cookie
options (See "Reliability and Deployment Issues" section below). options.
The cache replacement algorithm is not specified and is left for the The cache replacement algorithm is not specified and is left to the
implementations. implementations.
Note that before TFO sees wide deployment, clients SHOULD cache Note that before TFO sees wide deployment, clients SHOULD cache
negative responses from servers in order to reduce the amount of negative responses from servers in order to reduce the amount of
futile TFO attempts. Since TFO is enabled on a per-service port basis futile TFO attempts. Since TFO is enabled on a per-service port basis
but cookies are independent of service ports, clients' cache should but cookies are independent of service ports, clients' cache should
include remote port numbers too. include remote port numbers too.
4.2. Fast Open Protocol 4.2. Fast Open Protocol
skipping to change at page 10, line 29 skipping to change at page 10, line 22
The server keeps a FastOpened flag per TCB to mark if a connection The server keeps a FastOpened flag per TCB to mark if a connection
has successfully performed a TFO. has successfully performed a TFO.
4.2.1. Fast Open Cookie Request 4.2.1. Fast Open Cookie Request
Any client attempting TFO MUST first request a cookie from the server Any client attempting TFO MUST first request a cookie from the server
with the following steps: with the following steps:
1. The client sends a SYN packet with a Fast Open Cookie Request 1. The client sends a SYN packet with a Fast Open Cookie Request
option. option.
2. The server SHOULD respond with a SYN-ACK based on the procedures 2. The server SHOULD respond with a SYN-ACK based on the procedures
in the "Server Cookie Handling" section. This SYN-ACK SHOULD in the "Server Cookie Handling" section. This SYN-ACK SHOULD
contain a Fast Open Cookie option if the server currently supports contain a Fast Open Cookie option if the server currently supports
TFO for this listener port. TFO for this listener port.
3. If the SYN-ACK contains a Fast Open Cookie option, the client 3. If the SYN-ACK contains a Fast Open Cookie option, the client
replaces the cookie and other information as described in the replaces the cookie and other information as described in the
"Client Cookie Handling" section. Otherwise, if the SYN-ACK is "Client Cookie Handling" section. Otherwise, if the SYN-ACK is
first seen, i.e.,not a (spurious) retransmission, the client MAY first seen, i.e.,not a (spurious) retransmission, the client MAY
remove the server information from the cookie cache. If the SYN- remove the server information from the cookie cache. If the SYN-
ACK is a spurious retransmission without valid Fast Open Cookie ACK is a spurious retransmission without valid Fast Open Cookie
Option, the client does nothing to the cookie cache for the reasons Option, the client does nothing to the cookie cache for the
below. reasons below.
The network or servers may drop the SYN or SYN-ACK packets with the The network or servers may drop the SYN or SYN-ACK packets with the
new cookie options which causes SYN or SYN-ACK timeouts. We RECOMMEND new cookie options, which will causes SYN or SYN-ACK timeouts. We
both the client and the server retransmit SYN and SYN-ACK without the RECOMMEND both the client and the server to retransmit SYN and SYN-
cookie options on timeouts. This ensures the connections of cookie ACK without the cookie options on timeouts. This ensures the
requests will go through and lowers the latency penalties (of dropped connections of cookie requests will go through and lowers the latency
SYN/SYN-ACK packets). The obvious downside for maximum compatibility penalty (of dropped SYN/SYN-ACK packets). The obvious downside for
is that any regular SYN drop will fail the cookie (although one can maximum compatibility is that any regular SYN drop will fail the
argue the delay in the data transmission till after 3WHS is justified cookie (although one can argue the delay in the data transmission
if the SYN drop is due to network congestion). Next section till after 3WHS is justified if the SYN drop is due to network
describes a heuristic to detect such drops when the client receives congestion). Next section describes a heuristic to detect such drops
the SYN-ACK. when the client receives the SYN-ACK.
We also RECOMMEND the client to record servers that failed to respond We also RECOMMEND the client to record servers that failed to respond
to cookie requests and only attempt another cookie request after to cookie requests and only attempt another cookie request after
certain period. An alternate proposal is to request cookie in FIN certain period. An alternate proposal is to request cookie in FIN
instead since FIN-drop by incompatible middle-box does not affect instead since FIN-drop by incompatible middle-box does not affect
latency. However such paths are likely to drop SYN packet with data latency. However such paths are likely to drop SYN packet with data
later, and many applications close the connections with RST instead, later, and many applications close the connections with RST instead,
so the actual benefit of this approach is not clear. so the actual benefit of this approach is not clear.
4.2.2. TCP Fast Open 4.2.2. TCP Fast Open
Once the client obtains the cookie from the target server, the client Once the client obtains the cookie from the target server, it can
can perform subsequent TFO connections until the cookie is expired by perform subsequent TFO connections until the cookie is expired by the
the server. The nature of TCP sequencing makes the TFO specific server.
changes relatively small in addition to [RFC793].
Client: Sending SYN Client: Sending SYN
To open a TFO connection, the client MUST have obtained the cookie To open a TFO connection, the client MUST have obtained a cookie from
from the server: the server:
1. Send a SYN packet. 1. Send a SYN packet.
a. If the SYN packet does not have enough option space for the a. If the SYN packet does not have enough option space for the
Fast Open Cookie option, abort TFO and fall back to regular 3WHS. Fast Open Cookie option, abort TFO and fall back to regular 3WHS.
b. Otherwise, include the Fast Open Cookie option with the cookie b. Otherwise, include the Fast Open Cookie option with the cookie
of the server. Include any data up to the cached server MSS or of the server. Include any data up to the cached server MSS or
default 536 bytes. default 536 bytes.
2. Advance to SYN-SENT state and update SND.NXT to include the data 2. Advance to SYN-SENT state and update SND.NXT to include the data
accordingly. accordingly.
3. If RTT is available from the cache, seed SYN timer according to 3. If RTT is available from the cache, seed SYN timer according to
[RFC6298]. [RFC6298].
To deal with network or servers dropping SYN packets with payload or To deal with network or servers dropping SYN packets with payload or
unknown options, when the SYN timer fires, the client SHOULD unknown options, when the SYN timer fires, the client SHOULD
retransmit a SYN packet without data and Fast Open Cookie options. retransmit a SYN packet without data and Fast Open Cookie options.
Server: Receiving SYN and responding with SYN-ACK Server: Receiving SYN and responding with SYN-ACK
Upon receiving the SYN packet with Fast Open Cookie option: Upon receiving the SYN packet with Fast Open Cookie option:
1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled
is false, go to step 5. is false, go to step 5.
2. If PendingFastOpenRequests is over the system limit, go to step 5. 2. If PendingFastOpenRequests is over the system limit, go to step 5.
3. If IsCookieValid() in section 4.1.2 returns false, go to step 5. 3. If IsCookieValid() in section 4.1.2 returns false, go to step 5.
4. Buffer the data and notify the application. Set FastOpened flag 4. Buffer the data and notify the application. Set FastOpened flag
and increment PendingFastOpenRequests. and increment PendingFastOpenRequests.
5. Send the SYN-ACK packet. The packet MAY include a Fast Open 5. Send the SYN-ACK packet. The packet MAY include a Fast Open
Option. If FastOpened flag is set, the packet acknowledges the SYN Option. If FastOpened flag is set, the packet acknowledges the SYN
and data sequence. Otherwise it acknowledges only the SYN sequence. and data sequence. Otherwise it acknowledges only the SYN
The server MAY include data in the SYN-ACK packet if the response sequence. The server MAY include data in the SYN-ACK packet if the
data is readily available. Some application may favor delaying the response data is readily available. Some application may favor
SYN-ACK, allowing the application to process the request in order delaying the SYN-ACK, allowing the application to process the
to produce a response, but this is left to the implementation. request in order to produce a response, but this is left up to the
implementation.
6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the
server MUST follow the congestion control [RFC5681], in particular server MUST follow the congestion control [RFC5681], in particular
the initial congestion window [RFC3390], to send more data packets. the initial congestion window [RFC3390], to send more data
packets.
Note that if SYN-ACK is lost, regular TCP reduces the initial Note that if SYN-ACK is lost, regular TCP reduces the initial
congestion window before sending any data. In this case TFO is congestion window before sending any data. In this case TFO is
slightly more aggressive in the first data round trip even though slightly more aggressive in the first data round trip even though
it does not change the congestion control. it does not change the congestion control.
If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK
segment with neither data nor Fast Open Cookie options for segment with neither data nor Fast Open Cookie options for
compatibility reasons. compatibility reasons.
A special case is simultaneous open where the SYN receiver is a A special case is simultaneous open where the SYN receiver is a
client in SYN-SENT state. The protocol remains the same because client in SYN-SENT state. The protocol remains the same because
[RFC793] already supports both data in SYN and simultaneous open. But [RFC793] already supports both data in SYN and simultaneous open. But
the client's socket may have data available to read before it's the client's socket may have data available to read before it's
connected. This document does not cover the corresponding API change. connected. This document does not cover the corresponding API change.
Client: Receiving SYN-ACK Client: Receiving SYN-ACK
The client SHOULD perform the following steps upon receiving the SYN- The client SHOULD perform the following steps upon receiving the SYN-
ACK: 1. Update the cookie cache if the SYN-ACK has a Fast Open Cookie ACK:
Option or MSS option or both.
1. Update the cookie cache if the SYN-ACK has a Fast Open Cookie
Option or MSS option or both.
2. Send an ACK packet. Set acknowledgment number to RCV.NXT and 2. Send an ACK packet. Set acknowledgment number to RCV.NXT and
include the data after SND.UNA if data is available. include the data after SND.UNA if data is available.
3. Advance to the ESTABLISHED state. 3. Advance to the ESTABLISHED state.
Note there is no latency penalty if the server does not acknowledge Note there is no latency penalty if the server does not acknowledge
the data in the original SYN packet. The client SHOULD retransmit any the data in the original SYN packet. The client SHOULD retransmit any
unacknowledged data in the first ACK packet in step 2. The data unacknowledged data in the first ACK packet in step 2. The data
exchange will start after the handshake like a regular TCP exchange will start after the handshake like a regular TCP
connection. connection.
If the client has timed out and retransmitted only regular SYN If the client has timed out and retransmitted only regular SYN
skipping to change at page 13, line 26 skipping to change at page 13, line 19
the initial sequence and does not carry a Fast Open cookie option, the initial sequence and does not carry a Fast Open cookie option,
presumably it is triggered by a retransmitted (regular) SYN and the presumably it is triggered by a retransmitted (regular) SYN and the
original SYN or the corresponding SYN-ACK was lost. original SYN or the corresponding SYN-ACK was lost.
Server: Receiving ACK Server: Receiving ACK
Upon receiving an ACK acknowledging the SYN sequence, the server Upon receiving an ACK acknowledging the SYN sequence, the server
decrements PendingFastOpenRequests and advances to the ESTABLISHED decrements PendingFastOpenRequests and advances to the ESTABLISHED
state. No special handling is required further. state. No special handling is required further.
5. Reliability and Deployment Issues 5. Security Considerations
Network or Hosts Dropping SYN packets with data or unknown options
A study [MAF04] found that some middle-boxes and end-hosts may drop
packets with unknown TCP options incorrectly. Studies [LANGLEY06,
HNRGHT11] both found that 6% of the probed paths on the Internet drop
SYN packets with data or with unknown TCP options. The TFO protocol
deals with this problem by re-transmitting SYN without data or cookie
options and we recommend tracking these servers in the client.
Server Farms
A common server-farm setup is to have many physical hosts behind a
load-balancer sharing the same server IP. The load-balancer forwards
new TCP connections to different physical hosts based on certain
load-balancing algorithms. For TFO to work, the physical hosts need
to share the same key and update the key at about the same time.
Network Address Translation (NAT)
The hosts behind NAT sharing same IP address will get the same cookie
to the same server. This will not prevent TFO from working. But on
some carrier-grade NAT configurations where every new TCP connection
from the same physical host uses a different public IP address, TFO
does not provide latency benefit. However, there is no performance
penalty either as described in Section "Client: Receiving SYN-ACK".
6. Security Considerations
The Fast Open cookie stops an attacker from trivially flooding The Fast Open cookie stops an attacker from trivially flooding
spoofed SYN packets with data to burn server resources or to mount an spoofed SYN packets with data to burn server resources or to mount an
amplified reflection attack on random hosts. The server can defend amplified reflection attack on random hosts. The server can defend
against spoofed SYN floods with invalid cookies using existing against spoofed SYN floods with invalid cookies using existing
techniques [RFC4987]. We note that generating bogus cookies is techniques [RFC4987]. We note that although generating bogus cookies
usually cheaper than validating them. But the additional cost of is cost-free, the cost of validating the cookies, inherent to any
validating the cookies, inherent to any authentication scheme, may authentication scheme, may not be substantial compared to processing
not be substantial compared to processing a regular SYN packet. a regular SYN packet.
5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies
However, the attacker may still obtain cookies from some compromised However, the attacker may still obtain cookies from some compromised
hosts, then flood spoofed SYN with data and "valid" cookies (from hosts, then flood spoofed SYN with data and "valid" cookies (from
these hosts or other vantage points). With DHCP, it's possible to these hosts or other vantage points). With DHCP, it's possible to
obtain cookies of past IP addresses without compromising any host. obtain cookies of past IP addresses without compromising any host.
Below we identify new vulnerabilities of TFO and describe the Below we identify new vulnerabilities of TFO and describe the
countermeasures. countermeasures.
6.1. Server Resource Exhaustion Attack by SYN Flood with Valid Cookies
Like regular TCP handshakes, TFO is vulnerable to such an attack. But Like regular TCP handshakes, TFO is vulnerable to such an attack. But
the potential damage can be much more severe. Besides causing the potential damage can be much more severe. Besides causing
temporary disruption to service ports under attack, it may exhaust temporary disruption to service ports under attack, it may exhaust
server CPU and memory resources. server CPU and memory resources. Such an attack will show up on
application server logs as a application level DoS from Bot-nets,
triggering other defenses and alerts.
For this reason it is crucial for the TFO server to limit the maximum For this reason it is crucial for the TFO server to limit the maximum
number of total pending TFO connection requests, i.e., number of total pending TFO connection requests, i.e.,
PendingFastOpenRequests. When the limit is exceeded, the server PendingFastOpenRequests. When the limit is exceeded, the server
temporarily disables TFO entirely as described in "Server Cookie temporarily disables TFO entirely as described in "Server Cookie
Handling". Then subsequent TFO requests will be downgraded to regular Handling". Then subsequent TFO requests will be downgraded to regular
connection requests, i.e., with the data dropped and only SYN connection requests, i.e., with the data dropped and only SYN
acknowledged. This allows regular SYN flood defense techniques acknowledged. This allows regular SYN flood defense techniques
[RFC4987] like SYN-cookies to kick in and prevent further service [RFC4987] like SYN-cookies to kick in and prevent further service
disruption. disruption.
There are other subtle but important differences in the vulnerability The main impact of SYN floods against the standard TCP stack is not
between TFO and regular TCP handshake. Before the SYN flood attack directly from the floods themselves costing TCP processing overhead
broke out in the late '90s, typical listener's max qlen was small, or host memory, but rather from the spoofed SYN packets filling up
enough to sustain the highest expected new connection rate and the the often small listener's queue.
average RTT for the SYN-ACK packets to be acknowledged in time. E.g.,
if a server is designed to handle at most 100 connection requests per
second, and the average RTT is 100ms, a max qlen on the order of 10
will be sufficient.
This small max qlen made it very easy for any attacker, even equipped
with just a dailup modem to the Internet, to cause major disruptions
to a web site by simply throwing a handful of "SYN bombs" at its
victim of choice. But for this attack scheme to work, the attacker
must pick a non-responsive source IP address to spoof with. Otherwise
the SYN-ACK packet will trigger TCP RST from the host whose IP
address has been spoofed, causing corresponding connection to be
removed from the server's listener queue hence defeating the attack.
In other words, the main damage of SYN bombs against the standard TCP
stack is not directly from the bombs themselves costing TCP
processing overhead or host memory, but rather from the spoofed SYN
packets filling up the often small listener's queue.
On the other hand, TFO SYN bombs can cause damage directly if On the other hand, TFO SYN floods can cause damage directly if
admitted without limit into the stack. The RST packets from the admitted without limit into the stack. The RST packets from the
spoofed host will fuel rather than defeat the SYN bombs as compared spoofed host will fuel rather than defeat the SYN floods as compared
to the non-TFO case, because the attacker can flood more SYNs with to the non-TFO case, because the attacker can flood more SYNs with
data to cost more data processing resources. For this reason, a TFO data to cost more data processing resources. For this reason, a TFO
server needs to monitor the connections in SYN-RCVD being reset in server needs to monitor the connections in SYN-RCVD being reset in
addition to imposing a reasonable max qlen. Implementations may addition to imposing a reasonable max queue length. Implementations
combine the two, e.g., by continuing to account for those connection may combine the two, e.g., by continuing to account for those
requests that have just been reset against the listener's connection requests that have just been reset against the listener's
PendingFastOpenRequests until a timeout period has passed. PendingFastOpenRequests until a timeout period has passed.
Limiting the maximum number of pending TFO connection requests does Limiting the maximum number of pending TFO connection requests does
make it easy for an attacker to overflow the queue, causing TFO to be make it easy for an attacker to overflow the queue, causing TFO to be
disabled. We argue that causing TFO to be disabled is unlikely to be disabled. We argue that causing TFO to be disabled is unlikely to be
of interest to attackers because the service will remain intact of interest to attackers because the service will remain intact
without TFO hence there is hardly any real damage. without TFO hence there is hardly any real damage.
6.2. Amplified Reflection Attack to Random Host 5.1.1 Attacks from behind Sharing Public IPs (NATs)
An attacker behind NAT can easily obtain valid cookies to launch the
above attack to hurt other clients that share the path. [BRISCOE12]
suggested that the server can extend cookie generation to include the
TCP timestamp---GetCookie(IP_Address, Timestamp)---and implement it
by encrypting the concatenation of the two values to generate the
cookie. The client stores both the cookie and its corresponding
timestamp, and echoes both in the SYN. The server then implements
IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting the IP and
timestamp data and comparing it with the cookie value.
This enables the server to issue different cookies to clients that
share the same IP address, hence can selectively discard those
misused cookies from the attacker. However the attacker can simply
repeat the attack with new cookies. The server would eventually need
to throttle all requests from the IP address just like the current
approach. Moreover this approach requires modifying [RFC 1323] to
send non-zero Timestamp Echo Reply in SYN, potentially cause firewall
issues. Therefore we believe the benefit does not outweigh the
drawbacks.
5.2. Amplified Reflection Attack to Random Host
Limiting PendingFastOpenRequests with a system limit can be done Limiting PendingFastOpenRequests with a system limit can be done
without Fast Open Cookies and would protect the server from resource without Fast Open Cookies and would protect the server from resource
exhaustion. It would also limit how much damage an attacker can cause exhaustion. It would also limit how much damage an attacker can cause
through an amplified reflection attack from that server. However, it through an amplified reflection attack from that server. However, it
would still be vulnerable to an amplified reflection attack from a would still be vulnerable to an amplified reflection attack from a
large number of servers. An attacker can easily cause damage by large number of servers. An attacker can easily cause damage by
tricking many servers to respond with data packets at once to any tricking many servers to respond with data packets at once to any
spoofed victim IP address of choice. spoofed victim IP address of choice.
With the use of Fast Open Cookies, the attacker would first have to With the use of Fast Open Cookies, the attacker would first have to
steal a valid cookie from its target victim. This likely requires the steal a valid cookie from its target victim. This likely requires the
attacker to compromise the victim host or network first. attacker to compromise the victim host or network first.
The attacker here has little interest in mounting an attack on the The attacker here has little interest in mounting an attack on the
victim host that has already been compromised. But she may be victim host that has already been compromised. But it may be
motivated to disrupt the victim's network. Since a stolen cookie is motivated to disrupt the victim's network. Since a stolen cookie is
only valid for a single server, she has to steal valid cookies from a only valid for a single server, it has to steal valid cookies from a
large number of servers and use them before they expire to cause large number of servers and use them before they expire to cause
sufficient damage without triggering the defense in the previous sufficient damage without triggering the defense.
section.
One can argue that if the attacker has compromised the target network One can argue that if the attacker has compromised the target network
or hosts, she could perform a similar but simpler attack by injecting or hosts, it could perform a similar but simpler attack by injecting
bits directly. The degree of damage will be identical, but TFO- bits directly. The degree of damage will be identical, but TFO-
specific attack allows the attacker to remain anonymous and disguises specific attack allows the attacker to remain anonymous and disguises
the attack as from other servers. the attack as from other servers.
The best defense is for the server not to respond with data until The best defense is for the server not to respond with data until
handshake finishes. In this case the risk of amplification reflection handshake finishes. In this case the risk of amplification reflection
attack is completely eliminated. But the potential latency saving attack is completely eliminated. But the potential latency saving
from TFO may diminish if the server application produces responses from TFO may diminish if the server application produces responses
earlier before the handshake completes. earlier before the handshake completes.
6.3 Attacks from behind sharing public IPs (NATs) 6. TFO's Applicability
An attacker behind NAT can easily obtain valid cookies to launch the
above attack to hurt other clients that share the path. [BOB12]
suggested that the server can extend cookie generation to include the
TCP timestamp---GetCookie(IP_Address, Timestamp)---and implement it
by encrypting the concatenation of the two values to generate the
cookie. The client stores both the cookie and its corresponding
timestamp, and echoes both in the SYN. The server then implements
IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting the IP and
timestamp data and comparing it with the cookie value.
This enables the server to issue different cookies to clients that
share the same IP address, hence can selectively discard those
misused cookies from the attacker. However the attacker can simply
repeat the attack with new cookies. The server would eventually need
to throttle all requests from the IP address just like the current
approach. Moreover this approach requires modifying [RFC 1323] to
send non-zero Timestamp Echo Reply in SYN, potentially cause firewall
issues. Therefore we believe the benefit may not outweigh the
drawbacks.
7. TFO's Applicability
This section is to help applications considering TFO to evaluate This section is to help applications considering TFO to evaluate
TFO's benefits and drawbacks using a Web client and server TFO's benefits and drawbacks using the Web client and server
applications as an example throughout. applications as an example throughout. A proposed socket API change
is in the Appendix.
7.1 Duplicate data in SYNs 6.1 Duplicate Data in SYNs
It is possible, though uncommon, that using TFO the first data It is possible, though uncommon, that using TFO results in the first
written to a socket is delivered more than once to the application on data written to a socket to be delivered more than once to the
the remote host(Section 2.1). This replay potential only applies to application on the remote host(Section 2.1). This replay potential
data in the SYN but not subsequent data exchanges. Thus applications only applies to data in the SYN but not subsequent data exchanges.
MUST NOT use TFO unless they can tolerate this behavior. The client MUST NOT use TFO to send data in the SYN, and the server
MUST NOT accept data in the SYN if it cannot handle receiving the
same SYN data more than once, due to reasons described before.
7.2 Potential performance improvement 6.2 Potential Performance Improvement
TFO is designed for latency-conscious applications that are sensitive TFO is designed for latency-conscious applications that are sensitive
to TCP's initial connection setup delay. For example, many to TCP's initial connection setup delay. To benefit from TFO, the
applications perform short request and response message exchanges. To first application data unit (e.g., an HTTP request) needs to be no
benefit from TFO, the first application data unit (e.g., an HTTP more than TCP's maximum segment size (minus options used in SYN).
request) needs to be no more than TCP's maximum segment size (minus Otherwise the remote server can only process the client's application
options used in SYN). Otherwise the remote server can only process data unit once the rest of it is delivered after the initial
the client's application data unit once the rest of it is delivered handshake, diminishing TFO's benefit.
after the initial handshake, diminishing TFO's benefit.
To the extent possible, applications SHOULD employ long-lived To the extent possible, applications SHOULD reuse the connection to
connections to best take advantage of TCP's built-in congestion take advantage of TCP's built-in congestion control and reduce
control, and to reduce the impact from TCP's connection setup connection setup overhead. An application employs too many short-
overhead. Note that when an application employs too many short-lived lived connections will negatively impact network stability, as these
connections, it may negatively impact network stability, as these
connections often exit before TCP's congestion control algorithm connections often exit before TCP's congestion control algorithm
takes effect. Implementations supporting a large number of short- takes effect.
lived connections should employ temporal sharing of TCB data as
described in [RFC2140].
7.3 Example: Web clients and servers 6.3. Example: Web Clients and Servers
We look at Web client and server applications that use HTTP and TCP 6.3.1. HTTP Request Replay
protocols and follow the guidelines above to evaluate if TFO is safe
and useful for Web.
7.3.1 HTTP request replay While TFO is motivated by Web applications, the browser should not
use TFO to send requests in SYNs if those requests cannot tolerate
replays. One example is POST requests without application-layer
transaction protection (e.g., a unique identifier in the request
header).
We believe TFO is safe for the Web because even with standard TCP the TFO is particularly useful for GET requests. Even though not all GET
Web browser may replay an HTTP request to the remote Web server requests are idempotent, GETs are frequently replayed today across
multiple times. After sending an HTTP request, the browser could time striped TCP connections. After a server receives an HTTP request but
out and retry the same request on another TCP connection. This before the ACKs of the requests reach the browser, the browser may
scenario occurs far more frequently than the SYN duplication issue timeout and retry the same request on another (possibly new) TCP
presented by TFO. To ensure transactional behavior, Web sites employ connection. This differs from a TFO replay only in that the replay is
application-specific mechanisms such as including unique identifiers initiated by the browser, not by the TCP stack.
in the data.
7.3.2 HTTP persistent connection Finally, TFO is safe and useful for HTTPS requests because it saves
the first SSL handshake RTT and the HTTP request is sent after the
connection establishes.
Next we evaluate if the Web can benefit from TFO given that HTTP 6.3.2. Comparison with HTTP Persistent Connections
persistent connection support is already widely deployed.
TCP connection setup overhead has long been identified as a Is TFO useful given the wide deployment of HTTP persistent
performance bottleneck for web applications [THK98]. HTTP persistent connections? The short answer is yes. Studies [RCCJR11][AERG11] show
connection support was proposed to mitigate this issue and has been that the average number of transactions per connection is between 2
widely deployed. However, studies [RCCJR11][AERG11] show that the and 4, based on large-scale measurements from both servers and
average number of transactions per connection is between 2 and 4, clients. In these studies, the servers and clients both kept idle
based on large-scale measurements from both servers and clients. In connections up to several minutes, well into "human think" time.
these studies, the servers and clients both kept idle connections up
to several minutes, well into "human think" time.
Can the utilization rate of such connections increase by keeping idle Keeping connections open and idle even longer risks a greater
connections even longer? Unfortunately, such an approach is performance penalty. [HNESSK10][MQXMZ11] show that the majority of
problematic due to middle-boxes and the rapidly growing share of home routers and ISPs fail to meet the the 124-minute idle timeout
mobile end hosts. Thus one major issue faced by persistent mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs silently
connections is NAT. Studies [HNESSK10][MQXMZ11] show that the timeout idle connections within 30 minutes. End hosts, unaware of
majority of home routers and ISPs fail to meet the the 124-minute silent middle-box timeouts, suffer multi-minute TCP timeouts upon
idle timeout mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs using those long-idle connections.
timeout idle connections within 30 minutes. The end hosts attempting
to use these broken connections are often forced to wait for a
lengthy TCP timeout, as they often receive no signal when middleboxes
break their connections. Thus browsers risk large performance
penalties when keeping idle connections open.
To circumvent this problem, some applications send frequent TCP keep- To circumvent this problem, some applications send frequent TCP keep-
alive probes. However, this technique drains power on mobile devices alive probes. However, this technique drains power on mobile devices
[MQXMZ11]. In fact, power has become such a prominent issue in modern [MQXMZ11]. In fact, power has become such a prominent issue in modern
LTE devices that mobile browsers close HTTP connections within LTE devices that mobile browsers close HTTP connections within
seconds or even immediately [SOUDERS11]. seconds or even immediately [SOUDERS11].
Since TFO data duplication presents no new issues and HTTP persistent [RCCJR11] studied Chrome browser performance based on 28 days of
connection support has many limitations, Web applications can safely global statistics. The Chrome browser keeps idle HTTP persistent
use TFO and will likely achieve performance gains. The next section connections for 5 to 10 minutes. However the average number of the
presents more empirical data of the potential performance benefit. transactions per connection is only 3.3 and TCP 3WHS accounts for up
to 25% of the HTTP transaction network latency. The authors tested a
Linux TFO implementation with TFO enabled Chrome browser on popular
web sites in emulated environments such as residential broadband and
mobile networks. They showed that TFO improves page load time by 10%
to 40%.
8. Performance Experiments 7. Open Areas for Experimentation
[RCCJR11] studied Chrome browser performance based on 28 days of We now outline some areas that need experimentation in the Internet
global statistics. Chrome browser keeps idle HTTP persistent and under different network scenarios. These experiments should help
connections up to 5 to 10 minutes. However the average number of the the community evaluate Fast Open benefits and risks towards further
transactions per connection is only 3.3. Due to the low utilization, standardization and implementation of Fast Open and its related
TCP 3WHS accounts up to 25% of the HTTP transaction network latency. protocols.
The authors tested a Linux TFO implementation with TFO enabled Chrome
browser on popular web sites in emulated environments such as
residential broadband and mobile networks. They showed that TFO
improves page load time by 10% to 40%. More details on the design
tradeoffs and measurement can be found at [RCCJR11].
9. Related Work 7.1. Performance impact due to middle-boxes and NAT
9.1. T/TCP [MAF04] found that some middle-boxes and end-hosts may drop packets
with unknown TCP options. Studies [LANGLEY06, HNRGHT11] both found
that 6% of the probed paths on the Internet drop SYN packets with
data or with unknown TCP options. The TFO protocol deals with this
problem by falling back to regular TCP handshake and re-transmitting
SYN without data or cookie options after the initial SYN timeout.
Moreover the implementation is recommended to negatively cache such
incidents to avoid recurring timeouts. Further study is required to
evaluate the performance impact of these malicious drop behaviors.
Another interesting study is the (loss of) TFO performance benefit
behind certain carrier-grade NAT. Typically hosts behind a NAT
sharing the same IP address will get the same cookie for the same
server. This will not prevent TFO from working. But on some carrier-
grade NAT configurations where every new TCP connection from the same
physical host uses a different public IP address, TFO does not
provide latency benefits. However, there is no performance penalty
either, as described in Section "Client: Receiving SYN-ACK".
7.2. Cookie-less Fast Open
The cookie mechanism mitigates resource exhaustion and amplification
attacks. However cookies are not necessary if the server has
application-level protection or is immune to these attacks. For
example a Web server that only replies with a simple HTTP redirect
response that fits in the SYN-ACK packet may not care about resource
exhaustion. Such an application can safely disable TFO cookie checks.
Disabling cookies simplifies both the client and the server, as the
client no longer needs to request a cookie and the server no longer
needs to check or generate cookies. Disabling cookies also
potentially simplifies configuration, as the server no longer needs a
key. It may be preferable to enable SYN cookies and disable TFO
[RFC4987] when a server is overloaded by a large-scale Bot-net
attack.
Careful experimentation is necessary to evaluate if cookie-less TFO
is practical. The implementation can provide an experimental feature
to allow zero length, or null, cookies as opposed to the minimum 4
bytes cookies. Thus the server may return a null cookie and the
client will send data in SYN with it subsequently. If the server
believes it's under a DoS attack through other defense mechanisms, it
can switch to regular Fast Open for listener sockets.
8. Related Work
8.1. T/TCP
TCP Extensions for Transactions [RFC1644] attempted to bypass the TCP Extensions for Transactions [RFC1644] attempted to bypass the
three-way handshake, among other things, hence shared the same goal three-way handshake, among other things, hence shared the same goal
but also the same set of issues as TFO. It focused most of its effort but also the same set of issues as TFO. It focused most of its effort
battling old or duplicate SYNs, but paid no attention to security battling old or duplicate SYNs, but paid no attention to security
vulnerabilities it introduced when bypassing 3WHS. Its TAO option and vulnerabilities it introduced when bypassing 3WHS [PHRACK98].
connection count, besides adding complexity, require the server to
keep state per remote host, while still leaving it wide open for
attacks. It is trivial for an attacker to fake a CC value that will
pass the TAO test. Unfortunately, in the end its scheme is still not
100% bullet proof as pointed out by [PHRACK98].
As stated earlier, we take a practical approach to focus TFO on the As stated earlier, we take a practical approach to focus TFO on the
security aspect, while allowing old, duplicate SYN packets with data security aspect, while allowing old, duplicate SYN packets with data
after recognizing that 100% TCP semantics is likely infeasible. We after recognizing that 100% TCP semantics is likely infeasible. We
believe this approach strikes the right tradeoff, and makes TFO much believe this approach strikes the right tradeoff, and makes TFO much
simpler and more appealing to TCP implementers and users. simpler and more appealing to TCP implementers and users.
9.2. Common Defenses Against SYN Flood Attacks 8.2. Common Defenses Against SYN Flood Attacks
TFO is still vulnerable to SYN flood attacks just like normal TCP
handshakes, but the damage may be much worse, thus deserves a careful
thought.
There have been plenty of studies on how to mitigate attacks from [RFC4987] studies on mitigating attacks from regular SYN flood, i.e.,
regular SYN flood, i.e., SYN without data [RFC4987]. But from the SYN without data . But from the stateless SYN-cookies to the stateful
stateless SYN-cookies to the stateful SYN Cache, none can preserve SYN Cache, none can preserve data sent with SYN safely while still
data sent with SYN safely while still providing an effective defense. providing an effective defense.
The best defense may be to simply disable TFO when a host is The best defense may be to simply disable TFO when a host is
suspected to be under a SYN flood attack, e.g., the SYN backlog is suspected to be under a SYN flood attack, e.g., the SYN backlog is
filled. Once TFO is disabled, normal SYN flood defenses can be filled. Once TFO is disabled, normal SYN flood defenses can be
applied. The "Security Consideration" section contains a thorough applied. The "Security Consideration" section contains a thorough
discussion on this topic. discussion on this topic.
9.3. TCP Cookie Transaction (TCPCT) 8.3. TCP Cookie Transaction (TCPCT)
TCPCT [RFC6013] eliminates server state during initial handshake and TCPCT [RFC6013] eliminates server state during initial handshake and
defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK
packets to carry data. However, TCPCT and TFO are designed for packets to carry data. But the server can only send up to MSS bytes
different goals and they are not compatible. of data during the handshake instead of the initial congestion window
unlike TFO. Therefore applications like Web may not receive the
The TCPCT server does not keep any connection state during the latency benefit as TFO.
handshake, therefore the server application needs to consume the data
in SYN and (immediately) produce the data in SYN-ACK before sending
SYN-ACK. Otherwise the application's response has to wait until
handshake completes. In contrary, TFO allows server to respond data
during handshake. Therefore for many request-response style
applications, TCPCT may not achieve same latency benefit as TFO.
Rapid-Restart [SIMPSON11] is based on TCPCT and shares similar goal 8.4. Speculative Connections by the Applications
as TFO. In Rapid-Restart, both the server and the client retain the
TCP control blocks after a connection is terminated in order to
allow/resume data exchange in next connection handshake. In contrary,
TFO does not require keeping both TCB on both sides and is more
scalable.
10. IANA Considerations Some Web browsers maintain a history of the domains for frequently
visited web pages. The browsers then speculatively pre-open TCP
connections to these domains before the user initiates any requests
for them [BELSHE11]. The downside of this approach is that it wastes
server and network resources by initiating and maintaining idle
connections; It is also subject to the NAT timeout issues described
in Section 6.3.2. TFO offers similar performance improvement without
the added overhead.
9. IANA Considerations
The Fast Open Cookie Option and Fast Open Cookie Request Option The Fast Open Cookie Option and Fast Open Cookie Request Option
define no new namespace. The options require IANA allocate one value define no new namespace. The options require IANA to allocate one
from the TCP option Kind namespace. Early implementation before the value from the TCP option Kind namespace. Early implementation before
allocation SHOULD follow [EXPOPT] and use experimental option 254 and the IANA allocation SHOULD follow [EXPOPT] and use experimental
magic number 0xF989 (16 bits), and migrate to the new option after option 254 and magic number 0xF989 (16 bits), then migrate to the new
the allocation according. option after the allocation accordingly.
11. Acknowledgement 10. Acknowledgement
We thank Rick Jones, Bob Briscoe, Adam Langley, Matt Mathis, Neal We thank Rick Jones, Bob Briscoe, Adam Langley, Matt Mathis, Neal
Cardwell, Roberto Peon, and Tom Herbert for their feedbacks. We Cardwell, Roberto Peon, William Chan, Eric Dumazet, and Tom Herbert
especially thank Barath Raghavan for his contribution on the security for their feedbacks. We especially thank Barath Raghavan for his
design of Fast Open. contribution on the security design of Fast Open and proofreading
this draft numerous times.
12. References 11. References
12.1. Normative References 11.1. Normative References
[RFC793] Postel, J. "Transmission Control Protocol", RFC 793, [RFC793] Postel, J. "Transmission Control Protocol", RFC 793,
September 1981. September 1981.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, October 1989. Communication Layers", STD 3, RFC 1122, October 1989.
[RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh, [RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh,
P., "NAT Behavioral Requirements for TCP", RFC 5382 P., "NAT Behavioral Requirements for TCP", RFC 5382
[RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion [RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion
Control", RFC 5681, September 2009. Control", RFC 5681, September 2009.
[RFC6298] Paxson, V., Allman, M., Chu, J. and M. Sargent, "Computing [RFC6298] Paxson, V., Allman, M., Chu, J. and M. Sargent, "Computing
TCP's Retransmission Timer", RFC 6298, June 2011. TCP's Retransmission Timer", RFC 6298, June 2011.
12.2. Informative References [RFC6928] Chu, J., Dukkipati, N., Cheng, Y. and M. Mathis,
"Increasing TCP's Initial Window", RFC 6928, April 2013.
11.2. Informative References
[AERG11] M. Al-Fares, K. Elmeleegy, B. Reed, and I. Gashinsky, [AERG11] M. Al-Fares, K. Elmeleegy, B. Reed, and I. Gashinsky,
"Overclocking the Yahoo! CDN for Faster Web Page Loads". In "Overclocking the Yahoo! CDN for Faster Web Page Loads". In
Proceedings of Internet Measurement Conference, November Proceedings of Internet Measurement Conference, November
2011. 2011.
[CDCM11] Chu, J., Dukkipati, N., Cheng, Y. and M. Mathis,
"Increasing TCP's Initial Window", Internet-Draft draft-
ietf-tcpm-initcwnd-02.txt (work in progress), October 2011.
[EXPOPT] Touch, Joe, "Shared Use of Experimental TCP Options", [EXPOPT] Touch, Joe, "Shared Use of Experimental TCP Options",
Internet-Draft draft-ietf-tcpm-experimental-options (work Internet-Draft draft-ietf-tcpm-experimental-options (work
in progress), October 2012. in progress), June 2013.
[HNESSK10] S. Haetoenen, A. Nyrhinen, L. Eggert, S. Strowes, P. [HNESSK10] S. Haetoenen, A. Nyrhinen, L. Eggert, S. Strowes, P.
Sarolahti, M. Kojo., "An Experimental Study of Home Gateway Sarolahti, M. Kojo., "An Experimental Study of Home Gateway
Characteristics". In Proceedings of Internet Measurement Characteristics". In Proceedings of Internet Measurement
Conference. Octobor 2010 Conference. Octobor 2010
[HNRGHT11] M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M. [HNRGHT11] M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M.
Handley, H. Tokuda, "Is it Still Possible to Extend TCP?". Handley, H. Tokuda, "Is it Still Possible to Extend TCP?".
In Proceedings of Internet Measurement Conference. November In Proceedings of Internet Measurement Conference. November
2011. 2011.
[LANGLEY06] Langley, A, "Probing the viability of TCP extensions", [LANGLEY06] Langley, A, "Probing the viability of TCP extensions",
skipping to change at page 22, line 18 skipping to change at page 21, line 42
Applications: A Cross-layer Approach", In Proceedings of Applications: A Cross-layer Approach", In Proceedings of
International Conference on Mobile Systems. April 2011. International Conference on Mobile Systems. April 2011.
[RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A. and [RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A. and
Raghavan, B., "TCP Fast Open". In Proceedings of 7th ACM Raghavan, B., "TCP Fast Open". In Proceedings of 7th ACM
CoNEXT Conference, December 2011. CoNEXT Conference, December 2011.
[RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions [RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions
Functional Specification", RFC 1644, July 1994. Functional Specification", RFC 1644, July 1994.
[RFC2140] Touch, J., "TCP Control Block Interdependence", RFC2140,
April 1997.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
Mitigations", RFC 4987, August 2007. Mitigations", RFC 4987, August 2007.
[RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013, [RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013,
January 2011. January 2011.
[SIMPSON11] Simpson, W., "Tcp cookie transactions (tcpct) rapid
restart", Internet draft draft-simpson-tcpct-rr-02.txt
(work in progress), July 2011.
[SOUDERS11] S. Souders. "Making A Mobile Connection". [SOUDERS11] S. Souders. "Making A Mobile Connection".
http://www.stevesouders.com/blog/2011/09/21/making-a- http://www.stevesouders.com/blog/2011/09/21/making-a-
mobile-connection/ mobile-connection/
[THK98] Touch, J., Heidemann, J., Obraczka, K., "Analysis of HTTP [THK98] Touch, J., Heidemann, J., Obraczka, K., "Analysis of HTTP
Performance", USC/ISI Research Report 98-463. December Performance", USC/ISI Research Report 98-463. December
1998. 1998.
[BOB12] Briscoe, B., "Some ideas building on draft-ietf-tcpm- [BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm-
fastopen-01", tcpm list, fastopen-01", tcpm list,
http://www.ietf.org/mail-archive/web/tcpm/current/ http://www.ietf.org/mail-archive/web/tcpm/current/
msg07192.html January 16, 2014msg07192.html
Authors' Addresses [BELSHE12] Belshe, M., "The era of browser preconnect.",
http://www.belshe.com/2011/02/10/
the-era-of-browser-preconnect/
Yuchung Cheng Appendix A. Example Socket API Changes to support TFO
Google, Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043, USA
EMail: ycheng@google.com
Jerry Chu The design rationale is to minimize changes to the socket API and
Google, Inc. hence applications, in order to reduce the deployment hurdle. The
1600 Amphitheatre Parkway following changes have been implemented in Linux 3.7 or later
Mountain View, CA 94043, USA kernels.
EMail: hkchu@google.com
Sivasankar Radhakrishnan A.1 MSG_FASTOPEN flag for sendto() or sendmsg()
Department of Computer Science and Engineering
University of California, San Diego
9500 Gilman Dr
La Jolla, CA 92093-0404
EMail: sivasankar@cs.ucsd.edu
Arvind Jain MSG_FASTOPEN marks the attempt to send data in SYN like a combination
Google, Inc. of connect() and sendto(), by performing an implicit connect()
1600 Amphitheatre Parkway operation. It blocks until the handshake has completed and the data
Mountain View, CA 94043, USA is buffered.
EMail: arvind@google.com
For non-blocking socket it returns the number of bytes buffered and
sent in the SYN packet. If the cookie is not available locally, it
returns -1 with errno EINPROGRESS, and sends a SYN with TFO cookie
request automatically. The caller needs to write the data again when
the socket is connected.
It returns the same errno as connect() if the handshake fails.
A.2 TCP_FASTOPEN setsockopt() Socket Option
The option enables Fast Open on the listener socket. The option value
specifies the PendingFastOpenRequests threshold, i.e., the maximum
length of pending SYNs with data payload. Once enabled, the TCP
implementation will respond with TFO cookies per request.
Previously accept() returns only after a socket is connected. But for
a Fast Open connection, accept() returns upon receiving a SYN with a
valid Fast Open cookie and data, and the data is available to be read
through, e.g., recvmsg(), read().
Authors' Addresses
Yuchung Cheng Google, Inc. 1600 Amphitheatre Parkway Mountain View,
CA 94043, USA EMail: ycheng@google.com
Jerry Chu Google, Inc. 1600 Amphitheatre Parkway Mountain View, CA
94043, USA EMail: hkchu@google.com
Sivasankar Radhakrishnan Department of Computer Science and
Engineering University of California, San Diego 9500 Gilman Dr La
Jolla, CA 92093-0404 EMail: sivasankar@cs.ucsd.edu
Arvind Jain Google, Inc. 1600 Amphitheatre Parkway Mountain View, CA
94043, USA EMail: arvind@google.com
 End of changes. 114 change blocks. 
429 lines changed or deleted 384 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/