draft-ietf-tcpm-fastopen-07.txt   draft-ietf-tcpm-fastopen-08.txt 
Internet Draft Y. Cheng Internet Draft Y. Cheng
draft-ietf-tcpm-fastopen-07.txt J. Chu draft-ietf-tcpm-fastopen-08.txt J. Chu
Intended status: Experimental S. Radhakrishnan Intended status: Experimental S. Radhakrishnan
Expiration date: August, 2014 A. Jain Expiration date: August, 2014 A. Jain
Google, Inc. Google, Inc.
February 14, 2014 March 11, 2014
TCP Fast Open TCP Fast Open
Abstract
This document describes an experimental TCP mechanism TCP Fast Open
(TFO). TFO allows data to be carried in the SYN and SYN-ACK packets
and consumed by the receiving end during the initial connection
handshake, thus saving up to one full round trip time (RTT) compared
to the standard TCP, which requires a three-way handshake (3WHS) to
complete before data can be exchanged. However TFO deviates from the
standard TCP semantics since the data in the SYN could be replayed to
an application in some rare circumstances. Applications should not
use TFO unless they can tolerate this issue detailed in the
Applicability section.
Status of this Memo Status of this Memo
Distribution of this memo is unlimited. Distribution of this memo is unlimited.
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts. groups may also distribute working documents as Internet-Drafts.
skipping to change at page 1, line 49 skipping to change at page 2, line 15
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Abstract
This document describes an experimental TCP mechanism TCP Fast Open
(TFO). TFO allows data to be carried in the SYN and SYN-ACK packets
and consumed by the receiving end during the initial connection
handshake, thus saving up to one full round trip time (RTT) compared
to the standard TCP, which requires a three-way handshake (3WHS) to
complete before data can be exchanged. However TFO deviates from the
standard TCP semantics the data in the SYN could be replayed to an
application in some rare circumstances. Applications should not use
TFO unless they can tolerate this issue detailed in the Applicability
section.
Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
TFO refers to TCP Fast Open. Client refers to the TCP's active open
side and server refers to the TCP's passive open side.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Relaxing TCP Semantics on Duplicated SYNs . . . . . . . . . 4 2.1 Relaxing TCP Semantics on Duplicated SYNs . . . . . . . . . 4
2.2. SYNs with Spoofed IP Addresses . . . . . . . . . . . . . . 4 2.2. SYNs with Spoofed IP Addresses . . . . . . . . . . . . . . 4
3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5 3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5
4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 7 4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 7
4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 7 4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 7
4.1.1. TCP Options . . . . . . . . . . . . . . . . . . . . . . 7 4.1.1. TCP Options . . . . . . . . . . . . . . . . . . . . . . 7
4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 8 4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 8
4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 9 4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 9
4.1.3.1 Client Caching Negative Responses . . . . . . . . . 9 4.1.3.1 Client Caching Negative Responses . . . . . . . . . 9
skipping to change at page 4, line 5 skipping to change at page 3, line 39
handshake (3WHS)[RFC793], which adds one RTT to network latency. For handshake (3WHS)[RFC793], which adds one RTT to network latency. For
short Web transfers this additional RTT is a significant portion of short Web transfers this additional RTT is a significant portion of
overall network latency, even when HTTP persistent connection is overall network latency, even when HTTP persistent connection is
widely used. For example, the Chrome browser keeps TCP connections widely used. For example, the Chrome browser keeps TCP connections
idle for up to 5 minutes but 35% of Chrome HTTP requests are made on idle for up to 5 minutes but 35% of Chrome HTTP requests are made on
new TCP connections [RCCJR11]. For such Web and Web-like applications new TCP connections [RCCJR11]. For such Web and Web-like applications
placing data in the SYN can yield significant latency improvements. placing data in the SYN can yield significant latency improvements.
Next we describe how we resolve the challenges that arise upon doing Next we describe how we resolve the challenges that arise upon doing
so. so.
1.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
TFO refers to TCP Fast Open. Client refers to the TCP's active open
side and server refers to the TCP's passive open side.
2. Data In SYN 2. Data In SYN
Standard TCP already allows data to be carried in SYN packets Standard TCP already allows data to be carried in SYN packets
([RFC793], section 3.4) but forbids the receiver from delivering it ([RFC793], section 3.4) but forbids the receiver from delivering it
to the application until 3WHS is completed. This is because TCP's to the application until 3WHS is completed. This is because TCP's
initial handshake serves to capture old or duplicate SYNs. initial handshake serves to capture old or duplicate SYNs.
To enable applications exchange data in TCP handshake, TFO removes To enable applications exchange data in TCP handshake, TFO removes
the constraint and allows data in SYN packets to be delivered to the constraint and allows data in SYN packets to be delivered to the
application. This change of TCP semantic raises two issues discussed application. This change of TCP semantic raises two issues discussed
in the following subsections, making TFO unsuitable for certain in the following subsections, making TFO unsuitable for certain
applications. applications.
Therefore TCP implementations MUST NOT use TFO by default, but only Therefore TCP implementations MUST NOT use TFO by default, but only
use TFO if requested explicitly by the application on a per service use TFO if requested explicitly by the application on a per service
port basis. Applications need to evaluate TFO applicability described port basis. Applications need to evaluate TFO applicability described
in Section 6 before using TFO. in Section 6 before using TFO.
2.1 Relaxing TCP Semantics on Duplicated SYNs 2.1 Relaxing TCP Semantics on Duplicated SYNs
TFO allows data to be delivered to application before the 3WHS is TFO allows data to be delivered to the application before the 3WHS
completed, thus opening itself to a data integrity issue in either of is completed, thus opening itself to a data integrity issue in either
the two cases below: of the two cases below:
a) the receiver host receives data in a duplicate SYN after it has a) the receiver host receives data in a duplicate SYN after it has
forgotten it received the original SYN (e.g. due to a reboot); forgotten it received the original SYN (e.g. due to a reboot);
b) the duplicate is received after the connection created by the b) the duplicate is received after the connection created by the
original SYN has been closed and the close was initiated by the original SYN has been closed and the close was initiated by the
sender (so the receiver will not be protected by the 2MSL TIMEWAIT sender (so the receiver will not be protected by the 2MSL TIMEWAIT
state). state).
The now obsoleted T/TCP [RFC1644] attempted to address these issues. The now obsoleted T/TCP [RFC1644] attempted to address these issues.
skipping to change at page 5, line 49 skipping to change at page 5, line 49
2. The server validates the cookie: 2. The server validates the cookie:
a. If the cookie is valid, the server sends a SYN-ACK a. If the cookie is valid, the server sends a SYN-ACK
acknowledging both the SYN and the data. The server then acknowledging both the SYN and the data. The server then
delivers the data to the application. delivers the data to the application.
b. Otherwise, the server drops the data and sends a SYN-ACK b. Otherwise, the server drops the data and sends a SYN-ACK
acknowledging only the SYN sequence number. acknowledging only the SYN sequence number.
3. If the server accepts the data in the SYN packet, it may send the 3. If the server accepts the data in the SYN packet, it may send the
response data before the handshake finishes. The max amount is response data before the handshake finishes. The maximum amount is
governed by the TCP's congestion control [RFC5681]. governed by the TCP's congestion control [RFC5681].
4. The client sends an ACK acknowledging the SYN and the server data. 4. The client sends an ACK acknowledging the SYN and the server data.
If the client's data is not acknowledged, the client retransmits If the client's data is not acknowledged, the client retransmits
the data in the ACK packet. the data in the ACK packet.
5. The rest of the connection proceeds like a normal TCP connection. 5. The rest of the connection proceeds like a normal TCP connection.
The client can repeat many Fast Open operations once it acquires a The client can repeat many Fast Open operations once it acquires a
cookie (until the cookie is expired by the server). Thus TFO is cookie (until the cookie is expired by the server). Thus TFO is
skipping to change at page 9, line 24 skipping to change at page 9, line 24
The client MUST cache cookies from servers for later Fast Open The client MUST cache cookies from servers for later Fast Open
connections. For a multi-homed client, the cookies are both client connections. For a multi-homed client, the cookies are both client
and server IP dependent. Beside the cookie we RECOMMEND that the and server IP dependent. Beside the cookie we RECOMMEND that the
client caches the MSS to the server to enhance performance. client caches the MSS to the server to enhance performance.
The MSS advertised by the server is stored in the cache to determine The MSS advertised by the server is stored in the cache to determine
the maximum amount of data that can be supported in the SYN packet. the maximum amount of data that can be supported in the SYN packet.
This information is needed because data is sent before the server This information is needed because data is sent before the server
announces its MSS in the SYN-ACK packet. Without this information, announces its MSS in the SYN-ACK packet. Without this information,
the data size in the SYN packet is limited to the default MSS of 536 the data size in the SYN packet is limited to the default MSS of 536
bytes [RFC1122]. In particular it's known IPv4 receiver advertised bytes for IPv4 [RFC1122] and 1240 bytes for IPv6 [RFC2460]. In
MSS less than 536 bytes would result in transmission of an unexpected particular it's known an IPv4 receiver advertised MSS less than 536
large segment. If the cache MSS is larger than the typical 1460 bytes would result in transmission of an unexpected large segment. If
bytes, the extra large data in SYN segment may have issues that the cache MSS is larger than the typical 1460 bytes, the extra large
offsets the performance benefit of Fast Open. For examples, the data in SYN segment may have issues that offsets the performance
super-size SYN may trigger IP fragmentation and may confuse firewall benefit of Fast Open. For examples, the super-size SYN may trigger IP
or middle-boxes, causing SYN retransmission and other side-effects. fragmentation and may confuse firewall or middle-boxes, causing SYN
Therefore the client MAY limit the cached MSS to 1460 bytes. retransmission and other side-effects. Therefore the client MAY limit
the cached MSS to 1460 bytes.
4.1.3.1 Client Caching Negative Responses 4.1.3.1 Client Caching Negative Responses
The client MUST cache negative responses from the server in order to The client MUST cache negative responses from the server in order to
avoid potential connection failures. Negative responses include avoid potential connection failures. Negative responses include
server not acknowledging the data in SYN, ICMP error messages, and server not acknowledging the data in SYN, ICMP error messages, and
most importantly no response (SYN/ACK) from the server at all, i.e., most importantly no response (SYN/ACK) from the server at all, i.e.,
connection timeout. The last case is likely due to incompatible connection timeout. The last case is likely due to incompatible
middle-boxes or firewall blocking the connection completely after it middle-boxes or firewall blocking the connection completely after it
sees data in SYN. If the client does not react to these negative sees data in SYN. If the client does not react to these negative
responses and continue to retry Fast Open, the client may never be responses and continue to retry Fast Open, the client may never be
able to connect to the specific server. able to connect to the specific server.
For any negative responses, the client SHOULD disable Fast Open on For any negative responses, the client SHOULD disable Fast Open on
the specific path and port, at least temporarily. Since TFO is the specific path (the source and destination IP addresses and ports)
enabled on a per-service port basis but cookies are independent of at least temporarily. Since TFO is enabled on a per-service port
service ports, clients' cache should include remote port numbers too. basis but cookies are independent of service ports, the client's
cache should include remote port numbers too.
4.2. Fast Open Protocol 4.2. Fast Open Protocol
One predominant requirement of TFO is to be fully compatible with One predominant requirement of TFO is to be fully compatible with
existing TCP implementations, both on the client and the server existing TCP implementations, both on the client and the server
sides. sides.
The server keeps two variables per listening port: The server keeps two variables per listening port:
FastOpenEnabled: default is off. It MUST be turned on explicitly by FastOpenEnabled: default is off. It MUST be turned on explicitly by
the application. When this flag is off, the server does not perform the application. When this flag is off, the server does not perform
any TFO related operations and MUST ignore all cookie options. any TFO related operations and MUST ignore all cookie options.
PendingFastOpenRequests: tracks number of TFO connections in SYN-RCVD PendingFastOpenRequests: tracks number of TFO connections in SYN-RCVD
state. If this variable goes over a preset system limit, the server state. If this variable goes over a preset system limit, the server
MUST disable TFO for all new connection requests until MUST disable TFO for all new connection requests until
PendingFastOpenRequests drops below the system limit. This variable PendingFastOpenRequests drops below the system limit. This variable
is used for defending some vulnerabilities discussed in the "Security is used for defending some vulnerabilities discussed in the "Security
Considerations" section. Considerations" section.
The server keeps a FastOpened flag per TCB to mark if a connection The server keeps a FastOpened flag per connection to mark if a
has successfully performed a TFO. connection has successfully performed a TFO.
4.2.1. Fast Open Cookie Request 4.2.1. Fast Open Cookie Request
Any client attempting TFO MUST first request a cookie from the server Any client attempting TFO MUST first request a cookie from the server
with the following steps: with the following steps:
1. The client sends a SYN packet with a Fast Open Cookie Request 1. The client sends a SYN packet with a Fast Open Cookie Request
option. option.
2. The server SHOULD respond with a SYN-ACK based on the procedures 2. The server SHOULD respond with a SYN-ACK based on the procedures
in the "Server Cookie Handling" section. This SYN-ACK SHOULD in the "Server Cookie Handling" section. This SYN-ACK SHOULD
contain a Fast Open Cookie option if the server currently supports contain a Fast Open Cookie option if the server currently supports
TFO for this listener port. TFO for this listener port.
3. If the SYN-ACK contains a Fast Open Cookie option, the client 3. If the SYN-ACK contains a Fast Open Cookie option, the client
replaces the cookie and other information as described in the replaces the cookie and other information as described in the
"Client Cookie Handling" section. Otherwise, if the SYN-ACK is "Client Cookie Handling" section. Otherwise, if the SYN-ACK is
first seen, i.e.,not a (spurious) retransmission, the client MAY first seen, i.e., not a (spurious) retransmission, the client MAY
remove the server information from the cookie cache. If the SYN- remove the server information from the cookie cache. If the SYN-
ACK is a spurious retransmission without valid Fast Open Cookie ACK is a spurious retransmission without valid Fast Open Cookie
Option, the client does nothing to the cookie cache for the Option, the client does nothing to the cookie cache for the
reasons below. reasons below.
The network or servers may drop the SYN or SYN-ACK packets with the The network or servers may drop the SYN or SYN-ACK packets with the
new cookie options, which will cause SYN or SYN-ACK timeouts. We new cookie options, which will cause SYN or SYN-ACK timeouts. We
RECOMMEND both the client and the server to retransmit SYN and SYN- RECOMMEND both the client and the server to retransmit SYN and SYN-
ACK without the cookie options on timeouts. This ensures the ACK without the cookie options on timeouts. This ensures the
connections of cookie requests will go through and lowers the latency connections of cookie requests will go through and lowers the latency
skipping to change at page 11, line 16 skipping to change at page 11, line 16
maximum compatibility is that any regular SYN drop will fail the maximum compatibility is that any regular SYN drop will fail the
cookie (although one can argue the delay in the data transmission cookie (although one can argue the delay in the data transmission
till after 3WHS is justified if the SYN drop is due to network till after 3WHS is justified if the SYN drop is due to network
congestion). Next section describes a heuristic to detect such drops congestion). Next section describes a heuristic to detect such drops
when the client receives the SYN-ACK. when the client receives the SYN-ACK.
We also RECOMMEND the client to record the set of servers that failed We also RECOMMEND the client to record the set of servers that failed
to respond to cookie requests and only attempt another cookie request to respond to cookie requests and only attempt another cookie request
after certain period. after certain period.
An alternate proposal is to request cookie in FIN instead since FIN- An alternate proposal is to request a TFO cookie in the FIN instead,
drop by incompatible middle-box does not affect latency. However such since FIN-drop by incompatible middle-boxes does not affect latency.
paths are likely to drop SYN packet with data later, and many However paths that block SYN cookies may be more likely to drop a
applications close the connections with RST instead, so the actual later SYN packet with data, and many applications close a connection
benefit of this approach is not clear. with RST instead anyway.
Although cookie-in-FIN may not improve robustness, it would give
clients using a single connection a latency advantage over clients
opening multiple parallel connections. If experiments with TFO find
that it leads to increased connection-sharding, cookie-in-FIN may
prove to be a useful alternative.
4.2.2. TCP Fast Open 4.2.2. TCP Fast Open
Once the client obtains the cookie from the target server, it can Once the client obtains the cookie from the target server, it can
perform subsequent TFO connections until the cookie is expired by the perform subsequent TFO connections until the cookie is expired by the
server. server.
Client: Sending SYN Client: Sending SYN
To open a TFO connection, the client MUST have obtained a cookie from To open a TFO connection, the client MUST have obtained a cookie from
skipping to change at page 11, line 45 skipping to change at page 12, line 5
a. If the SYN packet does not have enough option space for the a. If the SYN packet does not have enough option space for the
Fast Open Cookie option, abort TFO and fall back to regular 3WHS. Fast Open Cookie option, abort TFO and fall back to regular 3WHS.
b. Otherwise, include the Fast Open Cookie option with the cookie b. Otherwise, include the Fast Open Cookie option with the cookie
of the server. Include any data up to the cached server MSS or of the server. Include any data up to the cached server MSS or
default 536 bytes. default 536 bytes.
2. Advance to SYN-SENT state and update SND.NXT to include the data 2. Advance to SYN-SENT state and update SND.NXT to include the data
accordingly. accordingly.
3. If RTT is available from the cache, seed SYN timer according to
[RFC6298].
To deal with network or servers dropping SYN packets with payload or To deal with network or servers dropping SYN packets with payload or
unknown options, when the SYN timer fires, the client SHOULD unknown options, when the SYN timer fires, the client SHOULD
retransmit a SYN packet without data and Fast Open Cookie options. retransmit a SYN packet without data and Fast Open Cookie options.
Server: Receiving SYN and responding with SYN-ACK Server: Receiving SYN and responding with SYN-ACK
Upon receiving the SYN packet with Fast Open Cookie option: Upon receiving the SYN packet with Fast Open Cookie option:
1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled
is false, go to step 5. is false, go to step 5.
skipping to change at page 12, line 29 skipping to change at page 12, line 33
5. Send the SYN-ACK packet. The packet MAY include a Fast Open 5. Send the SYN-ACK packet. The packet MAY include a Fast Open
Option. If FastOpened flag is set, the packet acknowledges the SYN Option. If FastOpened flag is set, the packet acknowledges the SYN
and data sequence. Otherwise it acknowledges only the SYN and data sequence. Otherwise it acknowledges only the SYN
sequence. The server MAY include data in the SYN-ACK packet if the sequence. The server MAY include data in the SYN-ACK packet if the
response data is readily available. Some application may favor response data is readily available. Some application may favor
delaying the SYN-ACK, allowing the application to process the delaying the SYN-ACK, allowing the application to process the
request in order to produce a response, but this is left up to the request in order to produce a response, but this is left up to the
implementation. implementation.
6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the
server MUST follow the congestion control [RFC5681] to set the server MUST follow [RFC5681] (based on [RFC3390]) to set the
initial congestion window [RFC3390], to send more data packets. initial congestion window for sending more data packets.
If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK
segment with neither data nor Fast Open Cookie options for segment with neither data nor Fast Open Cookie options for
compatibility reasons. compatibility reasons.
A special case is simultaneous open where the SYN receiver is a A special case is simultaneous open where the SYN receiver is a
client in SYN-SENT state. The protocol remains the same because client in SYN-SENT state. The protocol remains the same because
[RFC793] already supports both data in SYN and simultaneous open. But [RFC793] already supports both data in SYN and simultaneous open. But
the client's socket may have data available to read before it's the client's socket may have data available to read before it's
connected. This document does not cover the corresponding API change. connected. This document does not cover the corresponding API change.
skipping to change at page 13, line 35 skipping to change at page 13, line 39
5. Security Considerations 5. Security Considerations
The Fast Open cookie stops an attacker from trivially flooding The Fast Open cookie stops an attacker from trivially flooding
spoofed SYN packets with data to burn server resources or to mount an spoofed SYN packets with data to burn server resources or to mount an
amplified reflection attack on random hosts. The server can defend amplified reflection attack on random hosts. The server can defend
against spoofed SYN floods with invalid cookies using existing against spoofed SYN floods with invalid cookies using existing
techniques [RFC4987]. We note that although generating bogus cookies techniques [RFC4987]. We note that although generating bogus cookies
is cost-free, the cost of validating the cookies, inherent to any is cost-free, the cost of validating the cookies, inherent to any
authentication scheme, may be substantial compared to processing a authentication scheme, may be substantial compared to processing a
regular SYN packet. regular SYN packet. We describe these new vulnerabilities of TFO and
the countermeasures in detail below.
5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies 5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies
An attacker may still obtain cookies from some compromised hosts, An attacker may still obtain cookies from some compromised hosts,
then flood spoofed SYN with data and "valid" cookies (from these then flood spoofed SYN with data and "valid" cookies (from these
hosts or other vantage points). With DHCP, it's possible to obtain hosts or other vantage points). Like regular TCP handshakes, TFO is
cookies of past IP addresses without compromising any host. Below we vulnerable to such an attack. But the potential damage can be much
identify new vulnerabilities of TFO and describe the countermeasures. more severe. Besides causing temporary disruption to service ports
under attack, it may exhaust server CPU and memory resources. Such an
Like regular TCP handshakes, TFO is vulnerable to such an attack. But attack will show up on application server logs as a application level
the potential damage can be much more severe. Besides causing DoS from Bot-nets, triggering other defenses and alerts.
temporary disruption to service ports under attack, it may exhaust
server CPU and memory resources. Such an attack will show up on
application server logs as a application level DoS from Bot-nets,
triggering other defenses and alerts.
To protect the server it is important to limit the maximum number of To protect the server it is important to limit the maximum number of
total pending TFO connection requests, i.e., PendingFastOpenRequests total pending TFO connection requests, i.e., PendingFastOpenRequests
(Section 4.2). When the limit is exceeded, the server temporarily (Section 4.2). When the limit is exceeded, the server temporarily
disables TFO entirely as described in "Server Cookie Handling". Then disables TFO entirely as described in "Server Cookie Handling". Then
subsequent TFO requests will be downgraded to regular connection subsequent TFO requests will be downgraded to regular connection
requests, i.e., with the data dropped and only SYN acknowledged. This requests, i.e., with the data dropped and only SYN acknowledged. This
allows regular SYN flood defense techniques [RFC4987] like SYN- allows regular SYN flood defense techniques [RFC4987] like SYN-
cookies to kick in and prevent further service disruption. cookies to kick in and prevent further service disruption.
skipping to change at page 14, line 51 skipping to change at page 15, line 4
generate the cookie. The client stores both the cookie and its generate the cookie. The client stores both the cookie and its
corresponding timestamp, and echoes both in the SYN. The server then corresponding timestamp, and echoes both in the SYN. The server then
implements IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting implements IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting
the IP and timestamp data and comparing it with the cookie value. the IP and timestamp data and comparing it with the cookie value.
This enables the server to issue different cookies to clients that This enables the server to issue different cookies to clients that
share the same IP address, hence can selectively discard those share the same IP address, hence can selectively discard those
misused cookies from the attacker. However the attacker can simply misused cookies from the attacker. However the attacker can simply
repeat the attack with new cookies. The server would eventually need repeat the attack with new cookies. The server would eventually need
to throttle all requests from the IP address just like the current to throttle all requests from the IP address just like the current
approach. Moreover this approach requires modifying [RFC 1323] to approach. Moreover this approach requires modifying [RFC1323] to send
send non-zero Timestamp Echo Reply in SYN, potentially cause firewall non-zero Timestamp Echo Reply in SYN, potentially cause firewall
issues. Therefore we believe the benefit does not outweigh the issues. Therefore we believe the benefit does not outweigh the
drawbacks. drawbacks.
5.2. Amplified Reflection Attack to Random Host 5.2. Amplified Reflection Attack to Random Host
Limiting PendingFastOpenRequests with a system limit can be done Limiting PendingFastOpenRequests with a system limit can be done
without Fast Open Cookies and would protect the server from resource without Fast Open Cookies and would protect the server from resource
exhaustion. It would also limit how much damage an attacker can cause exhaustion. It would also limit how much damage an attacker can cause
through an amplified reflection attack from that server. However, it through an amplified reflection attack from that server. However, it
would still be vulnerable to an amplified reflection attack from a would still be vulnerable to an amplified reflection attack from a
large number of servers. An attacker can easily cause damage by large number of servers. An attacker can easily cause damage by
tricking many servers to respond with data packets at once to any tricking many servers to respond with data packets at once to any
spoofed victim IP address of choice. spoofed victim IP address of choice.
With the use of Fast Open Cookies, the attacker would first have to With the use of Fast Open Cookies, the attacker would first have to
steal a valid cookie from its target victim. This likely requires the steal a valid cookie from its target victim. This likely requires the
attacker to compromise the victim host or network first. attacker to compromise the victim host or network first. But in some
case it may be relatively easy.
The attacker here has little interest in mounting an attack on the The attacker here has little interest in mounting an attack on the
victim host that has already been compromised. But it may be victim host that has already been compromised. But it may be
motivated to disrupt the victim's network. Since a stolen cookie is motivated to disrupt the victim's network. Since a stolen cookie is
only valid for a single server, it has to steal valid cookies from a only valid for a single server, it has to steal valid cookies from a
large number of servers and use them before they expire to cause large number of servers and use them before they expire to cause
sufficient damage without triggering the defense. sufficient damage without triggering the defense.
One can argue that if the attacker has compromised the target network One can argue that if the attacker has compromised the target network
or hosts, it could perform a similar but simpler attack by injecting or hosts, it could perform a similar but simpler attack by injecting
bits directly. The degree of damage will be identical, but TFO- bits directly. The degree of damage will be identical, but TFO-
specific attack allows the attacker to remain anonymous and disguises specific attack allows the attacker to remain anonymous and disguises
the attack as from other servers. the attack as from other servers.
For example with DHCP an attacker can obtain cookies when he (or the
host he has compromised) owns a particular IP address by performing
regular Fast Open to servers supporting TFO and collect valid
cookies. The attacker then actively or passively releases his IP
address. When the IP address is re-assigned to a victim, the attacker
now owning a different IP address, floods spoofed Fast Open requests
to perform an amplified reflection attack on the victim.
The best defense is for the server not to respond with data until The best defense is for the server not to respond with data until
handshake finishes. In this case the risk of amplification reflection handshake finishes. In this case the risk of amplification reflection
attack is completely eliminated. But the potential latency saving attack is completely eliminated. But the potential latency saving
from TFO may diminish if the server application produces responses from TFO may diminish if the server application produces responses
earlier before the handshake completes. earlier before the handshake completes.
6. TFO's Applicability 6. TFO's Applicability
This section is to help applications considering TFO to evaluate This section is to help applications considering TFO to evaluate
TFO's benefits and drawbacks using the Web client and server TFO's benefits and drawbacks using the Web client and server
applications as an example throughout. Applications here refer applications as an example throughout. Applications here refer
specifically to the process that writes data into the socket, i.e., a specifically to the process that writes data into the socket, i.e., a
java script process that sends data to the server. A proposed socket JavaScript process that sends data to the server. A proposed socket
API change is in the Appendix. API change is in the Appendix.
6.1 Duplicate Data in SYNs 6.1 Duplicate Data in SYNs
It is possible that using TFO results in the first data written to a It is possible that using TFO results in the first data written to a
socket to be delivered more than once to the application on the socket to be delivered more than once to the application on the
remote host(Section 2.1). This replay potential only applies to data remote host (Section 2.1). This replay potential only applies to data
in the SYN but not subsequent data exchanges. in the SYN but not subsequent data exchanges.
Empirically [JIDKT07] showed the packet duplication on a Tier-1 Empirically [JIDKT07] showed the packet duplication on a Tier-1
network is rare. Since the replay only happens specifically when the network is rare. Since the replay only happens specifically when the
SYN data packet is duplicated and also the duplicate arrives after SYN data packet is duplicated and also the duplicate arrives after
the receiver has cleared the original SYN's connection state, the the receiver has cleared the original SYN's connection state, the
replay is thought to be uncommon in practice. Neverthless a client replay is thought to be uncommon in practice. Neverthless a client
that cannot handle receiving the same SYN data more than once MUST that cannot handle receiving the same SYN data more than once MUST
NOT enable TFO to send data in a SYN. Similarly a server that cannot NOT enable TFO to send data in a SYN. Similarly a server that cannot
accept receiving the same SYN data more than once MUST NOT enable TFO accept receiving the same SYN data more than once MUST NOT enable TFO
skipping to change at page 17, line 36 skipping to change at page 17, line 36
Some Web browsers maintain a history of the domains for frequently Some Web browsers maintain a history of the domains for frequently
visited web pages. The browsers then speculatively pre-open TCP visited web pages. The browsers then speculatively pre-open TCP
connections to these domains before the user initiates any requests connections to these domains before the user initiates any requests
for them [BELSHE11]. While this technique also saves the handshake for them [BELSHE11]. While this technique also saves the handshake
latency, it wastes server and network resources by initiating and latency, it wastes server and network resources by initiating and
maintaining idle connections. maintaining idle connections.
6.3.3. HTTP over TLS (HTTPS) 6.3.3. HTTP over TLS (HTTPS)
For TLS over TCP, it is safe and useful to include TLS CLIENT_HELLO For TLS over TCP, it is safe and useful to include TLS CLIENT_HELLO
in the SYN packet to save 1-RTT in TLS handshake. There is no concern in the SYN packet to save one RTT in TLS handshake. There is no
about violating idem-potency. In particular it can be used alone with concern about violating idem-potency. In particular it can be used
the speculative connection above. alone with the speculative connection above.
6.3.4. Comparison with HTTP Persistent Connections 6.3.4. Comparison with HTTP Persistent Connections
Is TFO useful given the wide deployment of HTTP persistent Is TFO useful given the wide deployment of HTTP persistent
connections? The short answer is yes. Studies [RCCJR11][AERG11] show connections? The short answer is yes. Studies [RCCJR11][AERG11] show
that the average number of transactions per connection is between 2 that the average number of transactions per connection is between 2
and 4, based on large-scale measurements from both servers and and 4, based on large-scale measurements from both servers and
clients. In these studies, the servers and clients both kept idle clients. In these studies, the servers and clients both kept idle
connections up to several minutes, well into "human think" time. connections up to several minutes, well into "human think" time.
skipping to change at page 19, line 15 skipping to change at page 19, line 15
7.2. Cookie-less Fast Open 7.2. Cookie-less Fast Open
The cookie mechanism mitigates resource exhaustion and amplification The cookie mechanism mitigates resource exhaustion and amplification
attacks. However cookies are not necessary if the server has attacks. However cookies are not necessary if the server has
application-level protection or is immune to these attacks. For application-level protection or is immune to these attacks. For
example a Web server that only replies with a simple HTTP redirect example a Web server that only replies with a simple HTTP redirect
response that fits in the SYN-ACK packet may not care about resource response that fits in the SYN-ACK packet may not care about resource
exhaustion. For such an application, the server could decide to exhaustion. For such an application, the server could decide to
disable TFO cookie checks. disable TFO cookie checks.
Disabling cookies simplifies both the client and the server, as the Disabling cookies (i.e., no Fast Open TCP options in SYN and SYN/ACK)
client no longer needs to request a cookie and the server no longer simplifies both the client and the server, as the client no longer
needs to check or generate cookies. Disabling cookies also needs to request a cookie and the server no longer needs to check or
potentially simplifies configuration, as the server no longer needs a generate cookies. Disabling cookies also potentially simplifies
key. It may be preferable to enable SYN cookies and disable TFO configuration, as the server no longer needs a key. It may be
[RFC4987] when a server is overloaded by a large-scale Bot-net preferable to enable SYN cookies and disable TFO [RFC4987] when a
attack. server is overloaded by a large-scale Bot-net attack.
Careful experimentation is necessary to evaluate if cookie-less TFO Careful experimentation is necessary to evaluate if cookie-less TFO
is practical. The implementation can provide an experimental feature is practical. The implementation can provide an experimental feature
to allow zero length, or null, cookies as opposed to the minimum 4 to allow zero length, or null, cookies as opposed to the minimum 4
bytes cookies. Thus the server may return a null cookie and the bytes cookies. Thus the server may return a null cookie and the
client will send data in SYN with it subsequently. If the server client will send data in SYN with it subsequently. If the server
believes it's under a DoS attack through other defense mechanisms, it believes it's under a DoS attack through other defense mechanisms, it
can switch to regular Fast Open for listener sockets. can switch to regular Fast Open for listener sockets.
7.3 Impact on congestion control 7.3 Impact on congestion control
skipping to change at page 19, line 46 skipping to change at page 19, line 46
[RFC5681]. However in TFO the server may have already sent up to an [RFC5681]. However in TFO the server may have already sent up to an
initial window of data. initial window of data.
If the server serves mostly short connections then the losses of SYN- If the server serves mostly short connections then the losses of SYN-
ACKs are not as effective as regular TCP on reducing the congestion ACKs are not as effective as regular TCP on reducing the congestion
window. This could result in an unstable network condition. The window. This could result in an unstable network condition. The
connections that experience losses may attempt again and add more connections that experience losses may attempt again and add more
load under congestion. A potential solution is to temporarily disable load under congestion. A potential solution is to temporarily disable
Fast Open if the server observes many SYN-ACK or data losses during Fast Open if the server observes many SYN-ACK or data losses during
the handshake across connections. Further experimentation regarding the handshake across connections. Further experimentation regarding
the congestion control impact are useful. the congestion control impact will be useful.
8. Related Work 8. Related Work
8.1. T/TCP 8.1. T/TCP
TCP Extensions for Transactions [RFC1644] attempted to bypass the TCP Extensions for Transactions [RFC1644] attempted to bypass the
three-way handshake, among other things, hence shared the same goal three-way handshake, among other things, hence shared the same goal
but also the same set of issues as TFO. It focused most of its effort but also the same set of issues as TFO. It focused most of its effort
battling old or duplicate SYNs, but paid no attention to security battling old or duplicate SYNs, but paid no attention to security
vulnerabilities it introduced when bypassing 3WHS [PHRACK98]. vulnerabilities it introduced when bypassing 3WHS [PHRACK98].
As stated earlier, we take a practical approach to focus TFO on the As stated earlier, we take a practical approach to focus TFO on the
security aspect, while allowing old, duplicate SYN packets with data security aspect, while allowing old, duplicate SYN packets with data
after recognizing that 100% TCP semantics is likely infeasible. We after recognizing that 100% TCP semantics is likely infeasible. We
believe this approach strikes the right tradeoff, and makes TFO much believe this approach strikes the right tradeoff, and makes TFO much
simpler and more appealing to TCP implementers and users. simpler and more appealing to TCP implementers and users.
8.2. Common Defenses Against SYN Flood Attacks 8.2. Common Defenses Against SYN Flood Attacks
[RFC4987] studies on mitigating attacks from regular SYN flood, i.e., [RFC4987] studies on mitigating attacks from regular SYN flood, i.e.,
SYN without data . But from the stateless SYN-cookies to the stateful SYN without data. But from the stateless SYN-cookies to the stateful
SYN Cache, none can preserve data sent with SYN safely while still SYN Cache, none can preserve data sent with SYN safely while still
providing an effective defense. providing an effective defense.
The best defense may be to simply disable TFO when a host is The best defense may be to simply disable TFO when a host is
suspected to be under a SYN flood attack, e.g., the SYN backlog is suspected to be under a SYN flood attack, e.g., the SYN backlog is
filled. Once TFO is disabled, normal SYN flood defenses can be filled. Once TFO is disabled, normal SYN flood defenses can be
applied. The "Security Consideration" section contains a thorough applied. The "Security Consideration" section contains a thorough
discussion on this topic. discussion on this topic.
8.3. TCP Cookie Transaction (TCPCT) 8.3. TCP Cookie Transaction (TCPCT)
TCPCT [RFC6013] eliminates server state during initial handshake and TCPCT [RFC6013] eliminates server state during initial handshake and
defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK
packets to carry data. But the server can only send up to MSS bytes packets to carry data. But the server can only send up to MSS bytes
of data during the handshake instead of the initial congestion window of data during the handshake instead of the initial congestion window
unlike TFO. Therefore applications like Web may not receive the unlike TFO. Therefore the latency of applications such as Web may be
latency benefit as TFO. worse than with TFO.
9. IANA Considerations 9. IANA Considerations
The Fast Open Cookie Option and Fast Open Cookie Request Option The Fast Open Cookie Option and Fast Open Cookie Request Option
define no new namespace. The options require IANA to allocate one define no new namespace. The options require IANA to allocate one
value from the TCP option Kind namespace. Early implementation before value from the TCP option Kind namespace. Early implementation before
the IANA allocation SHOULD follow [RFC6994] and use experimental the IANA allocation SHOULD follow [RFC6994] and use experimental
option 254 and magic number 0xF989 (16 bits), then migrate to the new option 254 and magic number 0xF989 (16 bits), then migrate to the new
option after the allocation accordingly. option after the allocation accordingly.
skipping to change at page 21, line 23 skipping to change at page 21, line 23
11. References 11. References
11.1. Normative References 11.1. Normative References
[RFC793] Postel, J. "Transmission Control Protocol", RFC 793, [RFC793] Postel, J. "Transmission Control Protocol", RFC 793,
September 1981. September 1981.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, October 1989. Communication Layers", STD 3, RFC 1122, October 1989.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh, [RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh,
P., "NAT Behavioral Requirements for TCP", RFC 5382 P., "NAT Behavioral Requirements for TCP", RFC 5382
[RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion [RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion
Control", RFC 5681, September 2009. Control", RFC 5681, September 2009
[RFC6298] Paxson, V., Allman, M., Chu, J. and M. Sargent, "Computing
TCP's Retransmission Timer", RFC 6298, June 2011.
[RFC6928] Chu, J., Dukkipati, N., Cheng, Y. and M. Mathis,
"Increasing TCP's Initial Window", RFC 6928, April 2013.
[RFC6994] Touch, Joe, "Shared Use of Experimental TCP Options", [RFC6994] Touch, Joe, "Shared Use of Experimental TCP Options",
RFC6994, August 2013. RFC6994, August 2013.
[RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's [RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's
Initial Window", RFC 3390, October 2002. Initial Window", RFC 3390, October 2002.
11.2. Informative References 11.2. Informative References
[AERG11] M. Al-Fares, K. Elmeleegy, B. Reed, and I. Gashinsky, [AERG11] Al-Fares, M., Elmeleegy, K., Reed, B., Gashinsky, I.,
"Overclocking the Yahoo! CDN for Faster Web Page Loads". In "Overclocking the Yahoo! CDN for Faster Web Page Loads".
Proceedings of Internet Measurement Conference, November In Proceedings of Internet Measurement Conference,
2011. November 2011.
[HNESSK10] S. Haetoenen, A. Nyrhinen, L. Eggert, S. Strowes, P. [HNESSK10] Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S.,
Sarolahti, M. Kojo., "An Experimental Study of Home Gateway Sarolahti, P., Kojo., M., "An Experimental Study of Home
Characteristics". In Proceedings of Internet Measurement Gateway Characteristics". In Proceedings of Internet
Conference. Octobor 2010 Measurement Conference. October 2010
[HNRGHT11] M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M. [HNRGHT11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A.,
Handley, H. Tokuda, "Is it Still Possible to Extend TCP?". Handley, M., Tokuda, H., "Is it Still Possible to
In Proceedings of Internet Measurement Conference. November Extend TCP?". In Proceedings of Internet Measurement
2011. Conference. November 2011.
[LANGLEY06] Langley, A, "Probing the viability of TCP extensions", [LANGLEY06] Langley, A, "Probing the viability of TCP extensions",
URL http://www.imperialviolet.org/binary/ecntest.pdf URL http://www.imperialviolet.org/binary/ecntest.pdf
[MAF04] Medina, A., Allman, M., and S. Floyd, "Measuring [MAF04] Medina, A., Allman, M., and S. Floyd, "Measuring
Interactions Between Transport Protocols and Middleboxes", Interactions Between Transport Protocols and
In Proceedings of Internet Measurement Conference, October Middleboxes". In Proceedings of Internet Measurement
2004. Conference, October 2004.
[MQXMZ11] Z. Mao, Z. Qian, Q. Xu, Z. Mao, M. Zhang. "An Untold Story [MQXMZ11] Wang, Z., Qian, Z., Xu, Q., Mao, Z., Zhang, M.,
of Middleboxes in Cellular Networks", In Proceedings of "An Untold Story of Middleboxes in Cellular Networks".
SIGCOMM. August 2011. In Proceedings of SIGCOMM. August 2011.
[PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue [PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue
53 artical 6. July 8, 1998. URL 53 artical 6. July 8, 1998. URL
http://www.phrack.com/issues.html?issue=53&id=6 http://www.phrack.com/issues.html?issue=53&id=6
[QWGMSS11] F. Qian, Z. Wang, A. Gerber, Z. Mao, S. Sen, O. [RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A.,
Spatscheck. "Profiling Resource Usage for Mobile Raghavan, B., "TCP Fast Open". In Proceedings of 7th
Applications: A Cross-layer Approach", In Proceedings of ACM CoNEXT Conference, December 2011.
International Conference on Mobile Systems. April 2011.
[RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A. and [RFC1323] Jacobson, V., Braden, R., Borman, D., "TCP Extensions for
Raghavan, B., "TCP Fast Open". In Proceedings of 7th ACM High Performance", RFC 1323, May 1992.
CoNEXT Conference, December 2011.
[RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions [RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions
Functional Specification", RFC 1644, July 1994. Functional Specification", RFC 1644, July 1994.
[RFC2460] Deering, S., Hinden, R., "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, December 1998.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
Mitigations", RFC 4987, August 2007. Mitigations", RFC 4987, August 2007.
[RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013, [RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013,
January 2011. January 2011.
[SOUDERS11] S. Souders. "Making A Mobile Connection". [SOUDERS11] Souders, S., "Making A Mobile Connection".
http://www.stevesouders.com/blog/2011/09/21/making-a- http://www.stevesouders.com/blog/2011/09/21/making-a-
mobile-connection/ mobile-connection/
[BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm- [BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm-
fastopen-01", tcpm list, fastopen-01", tcpm list,
http://www.ietf.org/mail-archive/web/tcpm/current/ http://www.ietf.org/mail-archive/web/tcpm/
January 16, 2014msg07192.html current/msg07192.html
[BELSHE12] Belshe, M., "The era of browser preconnect.", [BELSHE11] Belshe, M., "The era of browser preconnect.",
http://www.belshe.com/2011/02/10/ http://www.belshe.com/2011/02/10/
the-era-of-browser-preconnect/ the-era-of-browser-preconnect/
[JIDKT07] Jaiswal, S., Iannaccone, G., Diot, C., Kurose, J., and [JIDKT07] Jaiswal, S., Iannaccone, G., Diot, C., Kurose, J.,
Towsley, D. "Measurement and classification of Towsley, D., "Measurement and classification of
out-of-sequence packets in a tier-1 IP backbone.", out-of-sequence packets in a tier-1 IP backbone.".
IEEE/ACM Transactions on Networking (TON), 15(1), 54-66. IEEE/ACM Transactions on Networking (TON), 15(1), 54-66.
Appendix A. Example Socket API Changes to support TFO Appendix A. Example Socket API Changes to support TFO
A.1 Active Open A.1 Active Open
The active open side involves changing or replacing the connect() The active open side involves changing or replacing the connect()
call, which does not take a user data buffer argument. We recommend call, which does not take a user data buffer argument. We recommend
replacing connect() call to minimize API changes and hence replacing connect() call to minimize API changes and hence
applications to reduce the deployment hurdle. applications to reduce the deployment hurdle.
 End of changes. 42 change blocks. 
120 lines changed or deleted 131 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/