--- 1/draft-ietf-grow-anycast-01.txt 2006-02-04 17:26:22.000000000 +0100 +++ 2/draft-ietf-grow-anycast-02.txt 2006-02-04 17:26:22.000000000 +0100 @@ -1,19 +1,19 @@ Network Working Group J. Abley Internet-Draft ISC -Expires: January 19, 2006 K. Lindqvist +Expires: April 24, 2006 K. Lindqvist Netnod Internet Exchange - July 18, 2005 + October 21, 2005 Operation of Anycast Services - draft-ietf-grow-anycast-01 + draft-ietf-grow-anycast-02 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that @@ -24,21 +24,21 @@ and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on January 19, 2006. + This Internet-Draft will expire on April 24, 2006. Copyright Notice Copyright (C) The Internet Society (2005). Abstract As the Internet has grown, and as systems and networked services within enterprises have become more pervasive, many services with high availability requirements have emerged. These requirements have @@ -59,47 +59,47 @@ 4. Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.1 Protocol Suitability . . . . . . . . . . . . . . . . . . . 7 4.2 Node Placement . . . . . . . . . . . . . . . . . . . . . . 7 4.3 Routing Systems . . . . . . . . . . . . . . . . . . . . . 8 4.3.1 Anycast within an IGP . . . . . . . . . . . . . . . . 8 4.3.2 Anycast within the Global Internet . . . . . . . . . . 9 4.4 Routing Considerations . . . . . . . . . . . . . . . . . . 9 4.4.1 Signalling Service Availability . . . . . . . . . . . 9 4.4.2 Covering Prefix . . . . . . . . . . . . . . . . . . . 10 4.4.3 Equal-Cost Paths . . . . . . . . . . . . . . . . . . . 10 - 4.4.4 Route Dampening . . . . . . . . . . . . . . . . . . . 11 - 4.4.5 Reverse Path Forwarding Checks . . . . . . . . . . . . 12 - 4.4.6 Propagation Scope . . . . . . . . . . . . . . . . . . 12 - 4.4.7 Other Peoples' Networks . . . . . . . . . . . . . . . 13 + 4.4.4 Route Dampening . . . . . . . . . . . . . . . . . . . 12 + 4.4.5 Reverse Path Forwarding Checks . . . . . . . . . . . . 13 + 4.4.6 Propagation Scope . . . . . . . . . . . . . . . . . . 13 + 4.4.7 Other Peoples' Networks . . . . . . . . . . . . . . . 14 4.4.8 Aggregation Risks . . . . . . . . . . . . . . . . . . 14 - 4.5 Addressing Considerations . . . . . . . . . . . . . . . . 14 + 4.5 Addressing Considerations . . . . . . . . . . . . . . . . 15 4.6 Data Synchronisation . . . . . . . . . . . . . . . . . . . 15 - 4.7 Node Autonomy . . . . . . . . . . . . . . . . . . . . . . 15 + 4.7 Node Autonomy . . . . . . . . . . . . . . . . . . . . . . 16 4.8 Multi-Service Nodes . . . . . . . . . . . . . . . . . . . 16 - 4.8.1 Multiple Covering Prefixes . . . . . . . . . . . . . . 16 - 4.8.2 Pessimistic Withdrawal . . . . . . . . . . . . . . . . 16 + 4.8.1 Multiple Covering Prefixes . . . . . . . . . . . . . . 17 + 4.8.2 Pessimistic Withdrawal . . . . . . . . . . . . . . . . 17 4.8.3 Intra-Node Interior Connectivity . . . . . . . . . . . 17 - 5. Service Management . . . . . . . . . . . . . . . . . . . . . . 18 - 5.1 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . 18 - 6. Security Considerations . . . . . . . . . . . . . . . . . . . 19 - 6.1 Denial-of-Service Attack Mitigation . . . . . . . . . . . 19 - 6.2 Service Compromise . . . . . . . . . . . . . . . . . . . . 19 - 6.3 Service Hijacking . . . . . . . . . . . . . . . . . . . . 19 - 7. Protocol Considerations . . . . . . . . . . . . . . . . . . . 20 - 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 - 9. Acknowlegements . . . . . . . . . . . . . . . . . . . . . . . 22 - 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 - 10.1 Normative References . . . . . . . . . . . . . . . . . . . 23 - 10.2 Informative References . . . . . . . . . . . . . . . . . . 23 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 25 - A. Change History . . . . . . . . . . . . . . . . . . . . . . . . 26 - Intellectual Property and Copyright Statements . . . . . . . . 27 + 5. Service Management . . . . . . . . . . . . . . . . . . . . . . 19 + 5.1 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . 19 + 6. Security Considerations . . . . . . . . . . . . . . . . . . . 20 + 6.1 Denial-of-Service Attack Mitigation . . . . . . . . . . . 20 + 6.2 Service Compromise . . . . . . . . . . . . . . . . . . . . 20 + 6.3 Service Hijacking . . . . . . . . . . . . . . . . . . . . 20 + 7. Protocol Considerations . . . . . . . . . . . . . . . . . . . 21 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 + 9. Acknowlegements . . . . . . . . . . . . . . . . . . . . . . . 23 + 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 + 10.1 Normative References . . . . . . . . . . . . . . . . . . . 24 + 10.2 Informative References . . . . . . . . . . . . . . . . . . 24 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 26 + A. Change History . . . . . . . . . . . . . . . . . . . . . . . . 27 + Intellectual Property and Copyright Statements . . . . . . . . 28 1. Introduction To distribute a service using anycast, the service is first associated with a stable set of IP addresses, and reachability to those addresses is advertised in a routing system from multiple, independent service nodes. Various techniques for anycast deployment of services are discussed in [RFC1546], [ISC-TN-2003-1] and [ISC-TN- 2004-1]. @@ -432,37 +432,71 @@ 4.4.3 Equal-Cost Paths Some routing systems support equal-cost paths to the same destination. Where multiple, equal-cost paths exist and lead to different anycast nodes, there is a risk that different request packets associated with a single transaction might be delivered to more than one node. Services provided over TCP [RFC0793] necessarily involve transactions with multiple request packets, due to the TCP setup handshake. + For services which are distributed across the global Internet using + BGP, equal-cost paths are normally not a consideration: BGP's exit + selection algorithm usually selects a single, consistent exit for a + single destination regardless of whether multiple candidate paths + exist. Implementations of BGP exist that support multi-path exit + selection, however. + Equal cost paths are commonly supported in IGPs. Multi-node selection for a single transaction can be avoided in most cases by careful consideration of IGP link metrics, or by applying equal-cost multi-path (ECMP) selection algorithms which cause a single node to be selected for a single multi-packet transaction. For an example of the use of hash-based ECMP selection in anycast service distribution, see [ISC-TN-2004-1]. - For services which are distributed across the global Internet using - BGP, equal-cost paths are normally not a consideration: BGP's exit - selection algorithm usually selects a single, consistent exit for a - single destination regardless of whether multiple candidate paths - exist. Implementations of BGP exist that support multi-path exit - selection, however, and corner cases where dual selected exits route - to different nodes are possible. Analysis of the likely incidence of - such corner cases for particular distributions of Anycast Nodes are - recommended for services which involve multi-packet transactions. + Other ECMP selection algorithms are commonly available, including + those in which packets from the same flow are not guaranteed to be + routed towards the same destination. ECMP algorithms which select a + route on a per-packet basis rather than per-flow are commonly + referred to as performing "Per Packet Load Balancing" (PPLB). + + With respect to anycast service distribution, some uses of PPLB may + cause different packets from a single multi-packet transaction sent + by a client to be delivered to different anycast nodes, effectively + making the anycast service unavailable. Whether this affects + specific anycast services will depend on how and where anycast nodes + are deployed within the routing system, and on where the PPLB is + being performed: + + 1. PPLB across multiple, parallel links between the same pair of + routers should cause no node selection problems; + + 2. PPLB across diverse paths within a single autonomous system (AS), + where the paths converge to a single exit as they leave the AS, + should cause no node selection problems; + + 3. PPLB across links to different neighbour ASes where where the + neighbour ASes have selected different nodes for a particular + anycast destination will, in general, cause request packets to be + distributed across multiple anycast nodes. This will have the + effect that the anycast service is unavailable to clients + downstream of the router performing PPLB. + + The uses of PPLB which have the potential to interact badly with + anycast service distribution can also cause persistent packet + reordering. A network path that persistently reorders segments will + degrade the performance of traffic carried by TCP [Allman2000]. TCP, + according to several documented measurements, accounts for the bulk + of traffic carried on the Internet ([McCreary2000], [Fomenkov2004]). + Consequently, in many cases it is reasonable to consider networks + making such use of PPLB to be pathological. 4.4.4 Route Dampening Frequent advertisements and withdrawals of individual prefixes in BGP are known as flaps. Rapid flapping can lead to CPU exhaustion on routers quite remote from the source of the instability, and for this reason rapid route oscillations are frequently "dampened", as described in [RFC2439]. A dampened path will be suppressed by routers for an interval which @@ -623,23 +656,25 @@ For an IPv4-numbered service deployed across the Internet, for example, an address might be chosen from a block where the minimum RIR allocation size is 24 bits, and reachability to that address might be provided by originating the covering 24-bit prefix. For an IPv4-numbered service deployed within a private network, a locally-unused [RFC1918] address might be chosen, and rechability to that address might be signalled using a (32-bit) host route. For IPv6-numbered services, Anycast Addresses are not scoped - differently from unicast addresses [RFC3513]. As such the guidelines - presented for IPv4 with respect to address suitability follow for - IPv6. + differently from unicast addresses. As such the guidelines presented + for IPv4 with respect to address suitability follow for IPv6. Note + that historical prohibitions on anycast distribution of services over + IPv6 have been removed from the IPv6 addressing specification in + [I-D.ietf-ipv6-addr-arch-v4]. 4.6 Data Synchronisation Although some services have been deployed in localised form (such that clients from particular regions are presented with regionally- relevant content) many services have the property that responses to client requests should be consistent, regardless of where the request originates. For a service distributed using anycast, that implies that different Anycast Nodes must operate in a consistent manner and, where that consistent behaviour is based on a data set, that the data @@ -827,68 +862,87 @@ This document does not impose any protocol considerations. 8. IANA Considerations This document requests no action from IANA. 9. Acknowlegements The authors gratefully acknowledge the contributions from various participants of the grow working group, and in particular Geoff - Huston, Pekka Savola, Danny McPherson and Ben Black. + Huston, Pekka Savola, Danny McPherson, Ben Black and Alan Barrett. This work was supported by the US National Science Foundation (research grant SCI-0427144) and DNS-OARC. 10. References 10.1 Normative References + [I-D.ietf-ipv6-addr-arch-v4] + Hinden, R. and S. Deering, "IP Version 6 Addressing + Architecture", draft-ietf-ipv6-addr-arch-v4-04 (work in + progress), May 2005. + [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. [RFC1771] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995. [RFC1918] Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and E. Lear, "Address Allocation for Private Internets", BCP 5, RFC 1918, February 1996. [RFC1997] Chandrasekeran, R., Traina, P., and T. Li, "BGP Communities Attribute", RFC 1997, August 1996. [RFC2439] Villamizar, C., Chandra, R., and R. Govindan, "BGP Route Flap Damping", RFC 2439, November 1998. [RFC2827] Ferguson, P. and D. Senie, "Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing", BCP 38, RFC 2827, May 2000. - [RFC3513] Hinden, R. and S. Deering, "Internet Protocol Version 6 - (IPv6) Addressing Architecture", RFC 3513, April 2003. - [RFC3704] Baker, F. and P. Savola, "Ingress Filtering for Multihomed Networks", BCP 84, RFC 3704, March 2004. 10.2 Informative References + [Allman2000] + Allman, M. and E. Blanton, "On Making TCP More Robust to + Packet Reordering", January 2000, + . + + [Fomenkov2004] + Fomenkov, M., Keys, K., Moore, D., and k. claffy, + "Longitudinal Study of Internet Traffic from 1999-2003", + January 2004, . + [ISC-TN-2003-1] Abley, J., "Hierarchical Anycast for Global Service Distribution", March 2003, . [ISC-TN-2004-1] Abley, J., "A Software Approach to Distributing Requests for DNS Service using GNU Zebra, ISC BIND 9 and FreeBSD", March 2004, . + [McCreary2000] + McCreary, S. and k. claffy, "Trends in Wide Area IP + Traffic Patterns: A View from Ames Internet Exchange", + September 2000, . + [RFC1546] Partridge, C., Mendez, T., and W. Milliken, "Host Anycasting Service", RFC 1546, November 1993. [RFC2267] Ferguson, P. and D. Senie, "Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing", RFC 2267, January 1998. [RFC3765] Huston, G., "NOPEER Community for Border Gateway Protocol (BGP) Route Scope Control", RFC 3765, April 2004. @@ -926,25 +980,30 @@ This section should be removed before publication. draft-kurtis-anycast-bcp-00: Initial draft. Discussed at IETF 61 in the grow meeting and adopted as a working group document shortly afterwards. draft-ietf-grow-anycast-00: Missing and empty sections completed; some structural reorganisation; general wordsmithing. Document discussed at IETF 62. - draft-ietd-grow-anycast-01: This appendix added; acknowledgements - section added; commentary on [RFC3513] prohibition of anycast on + draft-ietf-grow-anycast-01: This appendix added; acknowledgements + section added; commentary on RFC3513 prohibition of anycast on hosts removed; minor sentence re-casting and related jiggery- pokery. This revision published for discussion at IETF 63. + draft-ietf-grow-anycast-02: Normative reference to [I-D.ietf-ipv6- + addr-arch-v4] added (in the RFC editor's queue at the time of + writing; reference should be updated to an RFC number when + available). Added commentary on per-packet load balancing. + Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.