--- 1/draft-ietf-opsec-blackhole-urpf-00.txt 2009-03-06 14:12:11.000000000 +0100 +++ 2/draft-ietf-opsec-blackhole-urpf-01.txt 2009-03-06 14:12:11.000000000 +0100 @@ -1,20 +1,20 @@ Opsec Working Group W. Kumari Internet Draft Google - D. McPherson + D. McPherson Category: Informational Arbor Networks -Expires: July 19, 2009 - January 19, 2009 +Expires: September 5, 2009 + March 5, 2009 Remote Triggered Black Hole filtering with uRPF - + Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. @@ -23,240 +23,309 @@ and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on July 13, 2009. + This Internet-Draft will expire on September 5, 2009. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Abstract Remote Triggered Black Hole (RTBH) filtering is a popular and effective technique for the mitigation of denial-of-service attacks. This document expands upon destination-based RTBH filtering by - outlining a method to enable filtering by source address as well. It - also defines a standard BGP community for black hole prefixes to - simplify associated semantics. + outlining a method to enable filtering by source address as well. Table of Contents 1. Introduction ....................................................2 2. Background ......................................................2 3. Destination address RTBH filtering ..............................3 3.1. Overview ...................................................3 3.2. Detail .....................................................3 4. Source address RTBH filtering ...................................4 5. Security Considerations .........................................6 6. IANA Considerations .............................................6 7. Acknowledgments .................................................7 8. References ......................................................7 A. Cisco Router Configuration Sample................................8 B. Juniper Configuration Sample....................................10 + C. A brief history of RTBH.........................................12 1. Introduction This document expands upon the technique outlined in "Configuring BGP - to Block Denial-of-Service Attacks" [RFC3882] to present a method - that allows filtering by source address(es). It also defines a - standard BGP community to signal that Remote Triggered Black Hole - (RTBH) filtering should occur for a network. + to Block Denial-of-Service Attacks" [RFC3882] to demonstrate a method + that allows for filtering by source address(es). 1.2 Terminology The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in BCP 14, RFC 2119 [RFC2119]. 2. Background Network operators have developed a variety of techniques for - mitigating these types of attacks. While the different techniques - have varying strengths and weaknesses from an implementation - perspective, the selection of which method to use for each type of - attack involves evaluating tradeoffs. + mitigating denial of service attacks. While different techniques have + varying strengths and weaknesses, from an implementation perspective + the selection of which method to use for each type of attack involves + evaluating the tradeoffs associated with each method. A common DoS attack directed against a customer of a service provider - involves generating more attack traffic destined for the target than - will fit down the links from the service provider to the victim - (customer). This traffic "starves out" legitimate traffic and often - results in collateral damage or negative effects to other customers - or the network infrastructure as well. Rather than having all of - their network affected the attack, the customer may ask their service - provider to filter traffic destined to the target destination IP - address(es), or the service provider may determine that this is - necessary themselves, in order to preserve network availability. + involves generating a greater volume of attack traffic destined for + the target than will fit down the links from the service provider(s) + to the victim (customer). This traffic "starves out" legitimate + traffic and often results in collateral damage or negative effects to + other customers or the network infrastructure as well. Rather than + having all destinations on their network be affected the attack, the + customer may ask their service provider to filter traffic destined to + the target destination IP address(es), or the service provider may + determine that this is necessary themselves, in order to preserve + network availability. One method that the service provider can use to implement this filtering is to deploy access control lists on the edge of their network. While this technique provides a large amount of flexibility in the filtering, it runs into scalability issues, both in terms of the number of entries in the filter and the packet rate. Most routers are able to forward traffic at a much higher rate than they are able to filter, and are able to hold many more forwarding table entries and routes than filter entries. RTBH leverages the forwarding performance of modern routers to filter both more entries - and at a higher rate than access control lists would allow. + and at a higher rate than access control lists would otherwise allow. - However, with destination-based RTBH filtering, the impact is that - the attack is complete. That is, with destination-based RTBH - filtering you inject a discard route into the forwarding table for - the prefix in question. All packets towards that destination, attack - traffic AND legitimate traffic, are then dropped by the participating + However, with destination-based RTBH filtering, the impact of the + attack on the target is complete. That is, destination-based RTBH + filtering injects a discard route into the forwarding table for the + target prefix. All packets towards that destination, attack traffic + AND legitimate traffic, are then dropped by the participating routers, thereby taking the target completely offline. The benefit is that collateral damage to other systems or network availability at - the customer location or in the ISP network is limited, but the - negative impact to the target itself is arguably increased. + the customer location or in the ISP network is limited, but negative + impact to the target itself is arguably increased. By coupling unicast reverse path forwarding (RPF) [RFC3704] techniques with RTBH, BGP can be used to distribute discard routes that are based not on destination or target addresses, but based on - source addresses. + source addresses of unwanted traffic. 3. Destination address RTBH filtering 3.1. Overview A "discard" route is installed on each edge router in the network with the destination set to be the discard (or null) interface. In - order to use RTBH filtering for an IP address (or network) a BGP + order to use RTBH filtering for a single IP address (or prefix) a BGP route for the address to be filtered is announced, with the next-hop set as the "discard" route. This causes traffic to the announced - network to be forwarded to the discard interface and so it does not - transit the network and waste resources or trigger collateral damage - or negative impact to other resources along the path towards the + network prefix to be forwarded to the discard interface so that it + does not transit the network wasting resources or triggering + collateral damage to other resources along the path towards the target. - While this does "complete" the attack in that the attacked - address(es) are made unreachable, it minimizes collateral damage. It - may also be possible to move the host / service on the attacked IP - address(es) to another address and keep the service up, for example - by updating associated DNS resource records. + While this does "complete" the attack in that the target address(es) + are made unreachable, collateral damage in minimized. It may also be + possible to move the host or service on the target IP address(es) to + another address and keep the service up, for example by updating + associated DNS resource records. 3.2. Detail Steps to configure destination based RTBH filtering: - 1: An address is chosen to become the "discard address". This is - often chosen from 192.0.2.0/24 (TEST-NET [RFC3330]), or from - RFC 1918 [RFC1918] space. + 1. Select your Discard Address schema. An address is chosen to become + the "discard address". This is often chosen from 192.0.2.0/24 (TEST- + NET [RFC3330]), or from RFC 1918 [RFC1918] space. Test-Net is popular + as a network which an operator will know never show up operationally + on the network. It also allows an operator to configure multiple + static routes, one for each incident: - 2: A route for the "discard address" is installed on the routers - that form the edge/perimeter of the network, in all routers - in the network, or some subset (e.g., peering, but not customer, - etc.), with the destination of the route being the "discard" or - "null" interface. This route is called the "discard route". + 192.0.2.1 = Incident #1 + 192.0.2.2 = Incident #2 + 192.0.2.3 = Incident #3 + 192.0.2.4 = Incident #4 + 192.0.2.5 = Incident #5 - 3: A BGP policy is configured on all routers that have the discard - route so that routes announced with the community - [ TBD1 ] will have their next hop set to the discard address. - The BGP policy should be made restrictive so that only BGP - routes covering a defined number of hosts addresses will - be accepted. That is, typically, only specific /32s are - necessary, - unless shorter prefix blocks are required. When filtering based - on - shorter prefixes, extreme caution should be used as to avoid - collateral damage to other hosts that reside within those - address blocks. + Customer #1 who has a DDOS attack can be pointed to discard route + 192.0.2.1. Customer #2 can be pointed to discard route 192.0.2.2. If + capable, the router can then count the drops for each, providing some + level of telemetry on the volume of drops as well status of an + ongoing attack. A consistent address schema facilitate operations. - 4: When RTBH filtering is desired for a specific address, that - address is announced from a central router (or route server), - tagged with the community [ TBD1 ]. The receiving routers check - the BGP policy, setting the next-hop to be the discard route, - which resolves to the discard interface. + 2. Set the Discard Route(s) on each router. A route for the "discard + address" is installed on the routers that form the edge/perimeter of + the network, in all routers in the network, or some subset (e.g., + peering, but not customer, etc.). The destination of the route is set + to the "discard" or "null" interface. This route is called the + "discard route". Implementation experience demonstrates the value of + configuring each ingress router with a capability for dropping + traffic via RTBH. - 5: Traffic entering the network will now be forwarded to the - discard interface on all edge routers and so will be dropped at - the edge of the network, saving resources. + 3. Configure the RTBH BGP Policy on each router. A BGP policy is + configured on all routers that have the discard route so that routes + announced with a chosen community will have their next hop set to the + discard address. The BGP policy should be made restrictive so that + only BGP routes covering a defined number of hosts addresses will be + accepted. That is, typically, only specific /32s are necessary. + Shorter prefix blocks may also be required or desirable, for example + if larger numbers of attack targets are located within a single + prefix, or the employment of this mechanism is to drop traffic bound + for specific networks. When filtering based on shorter prefixes, + extreme caution should be used as to avoid collateral damage to other + hosts that reside within those address blocks. Full implementations + will have multiple communities, with each community used for + different parts of a provider's network and for different security + incidents. - This technique can be expanded by having multiple discard addresses, - routes and communities to allow for monitoring of the discarded - traffic volume on devices that support multiple discard interfaces. + 4. Configure the Safety Egress Prefix Filters (Murphy Prefix + Filters). There is always a chance that the triggering BGP Update + could leak from the network. This has happened [ Youtube/Pakistan + incident ]. Egress prefix filters are recommended. These egress + prefix filter be many things, but experience has demonstrated that + the following works: - The technique can also be expanded by relaxing the AS path rule to - allow customers of a service provider to enable RTBH filtering - without interacting with the service provider. If this is configured, - an operator MUST only accept announcements for prefixes from the - customer that the customer is authorized to advertise, to prevent the - customer accidentally (or intentionally) black-holing space that is - not theirs. + 4.1 Deny all prefixes /30 through /32 from leaving your network. If + your triggering BGP update is only /32s, then this egress prefix + filter will add a safe measure in case the "no-export" community does + not work. - A common policy for this type of setup would be to accept from a - customer their authorized aggregate block and then permit any more - specific of the authorized prefix only if the blackhole communities - are equal or similar to attached, append NO_EXPORT, NO_ADVERTISE. + 4.2 Deny all communities used for triggering RTBH. This is also a + "safety" measure in case the "no-export" community does not work. + + 5: Configure the Trigger Router. Set the trigger router, workstation, + or other device so that adding and removing the triggers can be easy + and quickly added and removed. The BGP Update should have the no- + export community as a mandatory attribute. A egress prefix filter or + policy which prevent RTBHing prefixes in the /8 to /24 ranges is also + recommended as a safety tool. The trigger router can be set up as a + iBGP route reflector client which does not receive any prefixes from + its BGP peer. This allows a low cost router/workstation to be used as + the trigger router. + + 6. When RTBH filtering is desired for a specific address, that + address is announced from a trigger router (or route server), tagged + with the chosen "RTBH" community and with the community "no-export," + and passed via iBGP. The receiving routers check the BGP policy, set + the next-hop to be the discard route, restling in a fib entry pointed + to the discard interface. + + 7. Traffic entering the network will now be forwarded to the discard + interface on all edge routers and will therefore be dropped at the + edge of the network, saving resources. + + 7.1 Multiple Discard Address for Incident Granularity. This technique + can be expanded by having multiple discard addresses, routes and + communities to allow for monitoring of the discarded traffic volume + on devices that support multiple discard interfaces. As mentioned + earlier, each router can have a discard address schema that allows + for multiple incident - making it easier to monitor the life-cycle of + the incident. + + 7.2 Multiple "Trigger Communities" for Network Wide Granularity. The + network can be sectioned into multiple communities, providing the + operator with an ability to drop in discreete parts of their network. + For example, the network can be divided into the following + communities: + + XXX:600 RTBH on all router + XXX:601 RTBH on only peering router + XXX:602 RTBH on only customers who peer BGP + XXX:603 RTBH on Datacenters (to see if the data center is the + source of attack) + XXX:604 RTBH on all customers (to see how many customers are + BOTed) + + Some diligent thinking is required to develop a community schema + which provides flexibility while reflecting topological + considerations. + + 7.3 "Customer Triggered" RTBH. The technique can also be expanded by + relaxing the AS path rule to allow customers of a service provider to + enable RTBH filtering without interacting with the service provider's + trigger routers. If this is configured, an operator MUST only accept + announcements for prefixes from the customer that the customer is + authorized to advertise, in order to prevent the customer from + accidentally (or intentionally) black-holing space that is not theirs + to advertise.. + + A common policy for this type of setup would first permit any more + specific of an authorized prefix only if the blackhole communities is + attached, append NO_EXPORT, NO_ADVERTISE, or similar communities, and + then also accept from the customer the original aggregate prefix that + will be advertised as standard policy permits. Extreme caution should be used in order to avoid leaking any more specifics beyond the local routing domain, unless policy explicitly aims at doing just that. 4. Source address RTBH filtering. - In many instances the denial-of-service attacks are being sourced - from botnets and are being configured to "follow DNS" (the attacking - machines are instructed to attack www.example.com, and re-resolve - this periodically. Historically the attacks were aimed simply at an - IP address and so renumbering www.example.com to a new address was an - effective mitigation). This makes a technique that allows black- - holing based upon source address desirable. + In many instances denial-of-service attacks sourced from are being + configured to "follow DNS" (the attacking machines are instructed to + attack www.example.com, and re-resolve this periodically. + Historically the attacks were aimed simply at an IP address and so + renumbering www.example.com to a new address was an effective + mitigation). This makes employing technique that allows black-holing + to be based on source address desirable. By combining traditional RTBH filtering with unicast Reverse Path Forwarding (uRPF) a network operator can filter based upon the source address. uRPF performs a route lookup of the source address of the packet and checks to see if the ingress interface of the packet is a valid egress interface for the packet source address (strict mode) or if any route to the source address of the packet exists (loose mode). If the check fails, the packet is typically dropped. In loose mode some vendors also verify that the destination route does not point to a discard interface - this allows source based RTBH filtering to be deployed in networks that cannot implement strict (or feasible path) mode uRPF. By enabling the uRPF feature on interfaces at pre-determined points - of their network and announcing the source address(es) of attack - traffic, a network operator can effectively drop the attack traffic - at specified devices (ideally ingress edge) of their network based on - source addresses. + in their network and announcing the source address(es) of attack + traffic, a network operator can effectively drop the identified + attack traffic at specified devices (ideally ingress edge) of their + network based on source address. - While administrators may choose to drop any prefixes they wish, it is - recommended when employing source-based RTBH inter-domain that - explicit policy be defined that enables peers to only announce + While administrators may choose to drop traffic from any prefix they + wish, it is recommended when employing source-based RTBH inter-domain + that explicit policy be defined that enables peers to only announce source-based RTBH routes for prefixes which they originate. 4.1 Steps to deploy RTBH with uRPF for source filtering. The same steps that are required to implement destination address RTBH filtering are taken with the additional step of enabling unicast reverse path forwarding on predetermined interfaces. When a source address (or network) needs to be blocked, that address (or network) is announced using BGP tagged with a community. This will cause the route to be installed with a next hop of the discard interface, causing the uRPF check to fail. The destination based RTBH filtering - community ([ TBD1 ]) should not be used for source based RTBH - filtering, and the routes tagged with the selected community should - be carefully filtered. + community should not be used for source based RTBH filtering, and the + routes tagged with the selected community should be carefully + filtered. The BGP policy will need to be relaxed to accept announcements tagged with this community to be accepted even though they contain addresses not controlled by the network announcing them. These announcements must NOT be propagated outside the local AS and should carry the NO_EXPORT community. As a matter of policy, operators SHOULD NOT accept source-based RTBH announcements from their peers or customers, they should only be installed by local or attack management systems within their @@ -272,40 +341,33 @@ - Are not propagated outside the AS (NO_EXPORT). - Are not accepted from outside the AS (except from customers). - Except where source based filtering is deployed, that the network contained in the announcement falls within the address ranges controlled by the announcing AS (i.e.: for customers that the address falls within their space). 6. IANA Considerations - This document requests registration of a regular Type, non-transitive - BGP Extended Communities Attribute [RFC4360] from the First Come, - First Served range to be named "Remote Triggered Black Hole Filter". - - This community will provide a standard method to signal a provider - that RTBH filtering should occur for a destination and will eliminate - the need for customers to track different communities for each - provider. + No action required. 7. Acknowledgments - I would like to thank Joe Abley, Rodnet Dunn, Alfred Hoenes, Donald + I would like to thank Joe Abley, Rodney Dunn, Alfred Hoenes, Donald Smith, Joel Jaeggli and Steve Williams for their assistance, feedback and not laughing *too* much at the quality of the initial drafts. I would also like to thank all of the regular contributors to the OpSec Working Group and Google for 20% time :-) - The authors would also like to thank Barry Greene for his efforts in - getting this implemented and Chris Morrow for publicizing the + The authors would also like to thank Steven L Johnson and Barry Greene + for getting this implemented and Chris Morrow for publicizing the technique in multiple talks. 8. References 8.1. Normative References [RFC1918] Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G., and E. Lear, "Address Allocation for Private Internets", BCP 5, RFC 1918, February 1996. @@ -323,20 +385,26 @@ [RFC3882] Turk, D., "Configuring BGP to Block Denial-of-Service Attacks", RFC 3882, September 2004. 8.2. Informative References [2223BIS] Reynolds, J. and R. Braden, "Instructions to Request for Comments (RFC) Authors", draft-rfc-editor- rfc2223bis-08.txt, August 2004. + [Greene2001] Greene Barry Raveendran and Jarvis Neil "Unicast Reverse + Path Forwarding (uRPF) Enhancements for the ISP-ISP Edge", + [ftp://ftp- + eng.cisco.com/cons/isp/documents/uRPF_Enhancement.pdf], + 2001. + Appendix A: Cisco Router Configuration Sample This section provides a partial configuration for configuring RTBH on a Cisco router. This is not a complete configuration and should be customized before being used. Announcing router: ! The discard route ip route 192.0.2.1 255.255.255.255 Null0 ! @@ -480,20 +548,67 @@ unit 201 { vlan-id 201; family inet { rpf-check; address 10.11.12.1/24; } } } } +Appendix C: A Brief History of RTBH + + Understanding the history and motivation behind the development of a + technique often helps with understanding how to best utilize the + technique. In this spirit we present a history of Unicast RPF and + RTBH. + + This section provided by Barry Raveendran Greene: + + Unicast RPF Loose Check (uRPF Loose Check) was created by Neil Jarvis + and Barry Greene to be used with dRTBH as a rapid reaction tool to + DDOS Attacks. The requirements for this rapid reaction tool was based + on post mortem conversation after the Feb 2000 attacks on several big + content hosting companies. The summary of the requirement became the + "Exodus Requirement" which stated: + + "We need a tool to drop packets based on source IP address that can + be pushed out to over 60 routers within 60 seconds, be longer than a + thousand lines, be modified on the fly, and work in all your + platforms filtering at line rate." + + A variety of options were looked at to meet this requirement, from + reviving COPS, to pushing out ACLs with BGP, creating a new protocol. + In 2000, the quickest way to meet the "Exodus requirement" was to + marry two functions. First, modify Unicast RPF so that the interface + check was no longer required and to make sure that a "null" or + discard route would drop the packet (i.e. loose check). Second, the + technique where BGP is used to trigger a distributed drop is dusted + off and documented. Combining these two techniques was deemed to + fast way to put a distributed capability to drop packets out into the + industry. + + To clarify and restate, uRPF Loose Check was created as one part of a + rapid reaction tool to DDOS attacks that "drop packets based on + source IP address that can be pushed out to over 60 routers with in + 60 seconds, be longer than a thousand lines, be modified on the fly, + and work in all your platforms filtering at line rate." The secondary + benefits of using uRPF Loose Check for other functions is a secondary + benefit - not the primary goal for its creation. + + To facilitate the adoption to the industry, uRPF Loose Check was not + patent. It was publicly published and disclosed in "Unicast Reverse + Path Forwarding (uRPF) Enhancements for the ISP-ISP Edge "Remote + Triggering Black Hole Filtering," by Barry Raveendran Greene and Neil + Jarvis [ftp://ftp- + eng.cisco.com/cons/isp/documents/uRPF_Enhancement.pdf]. + Authors' Addresses Warren Kumari Google 1600 Amphitheatre Parkway Mountain View, CA 94043 Email: warren@kumari.net Danny McPherson Arbor Networks, Inc.