Network Working Group
   INTERNET-DRAFT
   Expires in: December 2003 April 2004
                                       		Scott Poretsky
                                                   Avici Systems
                                                Quarry Technologies

						   		Shankar Rao
						   		Qwest Communications

						   		Ray Piatt
						   		Cable and Wireless

                                                   June

                                                October 2003

             Terminology for Benchmarking
		Core Router Software Accelerated Life Testing

              	<draft-ietf-bmwg-acc-bench-term-00.txt> Stress Benchmarking

              <draft-ietf-bmwg-acc-bench-term-01.txt>

   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   ABSTRACT
   This terminology document provides the terms to be used for
   benchmarking a router software or switch under accelerated stress Accelerated Stress conditions.
   A framework
   Terminology is defined to configure routing protocols, security
   policies, traffic forwarding, and management.  Conditions to
   produce instability and accelerate operational conditions are
   also defined.  Benchmarks provided for evaluating a router subjected to performing this benchmark using
   the accelerated life test are introduced.  The DUT configuration Contol Plane, Data Plane, Management Plane, and accelerated stress conditions emulate those Security
   Plane of Internet
   Core routers. the Device Under Test.  Terms are defined for
   Configuration, Startup Conditions, Instability Conditions, Failure
   Conditions, and Benchmarks.

   Table of Contents
     1. Introduction ............................................... 2
           	Router Software Accelerated Life Testing 3
     2. Existing definitions ....................................... 3
     3. Term definitions............................................ 3
      3.1 Configuration Modules..................................... Benchmark Planes.......................................... 3
 	3.1.1 Control Plane Configuration Module.................... Plane........................................... 4
 	3.1.2 Data Plane Configuration Module....................... Plane.............................................. 5
 	3.1.3 Management Configuration Module....................... 6 Plane........................................ 5
	3.1.4 Security Configuration Module......................... 7 Plane.......................................... 6
	3.2 Configuration Sets...................................... 7 Sets........................................ 6
 	3.2.1 Control Plane Configuration Sets...................... 7
	3.2.1.1 Routing Protocol Configuration Set.................. Set......................... 7
       	3.2.1.2 Multicast Protocol Configuration Set................ 8
       	3.2.1.3 MPLS Protocol Configuration Set..................... 9
 	3.2.2 Data Plane Configuration Set.......................... 9 Set............................ 7
      3.2.2.1 Traffic Profile....................................... 8
 	3.2.3 Management Plane Configuration Sets.........................10
	3.2.3.1 User Access Configuration Set.......................10
   	3.2.3.2 SNMP Configuration Set..............................10
   	3.2.3.3 Logging/Debug Configuration Set.....................11
	3.2.3.4 Packet Statistics Collector Configuration...........11 Set...................... 8
	3.2.4 Security Plane Configuration Set............................11
      3.2.4.1 ACL Configuration Set.................................12 Set........................ 9
	3.3 Instability Conditions..................................12 Startup Conditions........................................10
 	3.3.1 Interface Shutdown Cycling Rate.......................12 Control Plane Startup Conditions........................10
	3.3.2 BGP Route Flap Rate...................................13
      	3.3.3 IGP Route Flap Rate...................................13
      	3.3.4 Route Convergence Due to Better Next-Hop..............14
	3.3.5 LSP Reroute Rate......................................14
	3.3.6 Intended Test Duration................................15 Security Plane Startup Conditions.......................11
	3.4 Evaluation Benchmarks...................................15 Instability Conditions....................................11
 	3.4.1 Run-Time without Error................................15
	3.4.2 Run-Time without Control Plane Error..................16
	3.4.3 Run-Time without Instability Conditions....................12
 	3.4.2 Data Plane Error.....................16
	3.4.4 Run-Time without Instability Conditions.......................12
 	3.4.3 Management Plane Error...............17
	3.4.5 Run-Time without Instability Conditions.................13
	3.4.4 Security Plane Error.................17 Instability Conditions...................13
	3.5 Failure Conditions........................................14
	3.5.1 Unexpected Packet Loss..................................14
	3.5.2 Unexpected Session Loss.................................14
	3.5.3 Misrouted Packets.......................................15
	3.5.4 Access Denial...........................................15
	3.5.5 Errored Management Value................................16
	3.5.6 Security Plane Failure..................................16
	3.6 Benchmarks................................................17
	3.6.1 Success Threshold.......................................17
	3.6.2 Accelerated-Life Test Duration..........................17
     4. Security Considerations.....................................18
     5. References..................................................18
     6. Author's Address............................................18
     7. Full Copyright Statement....................................19
     Appendix 1 - White Box Benchmarks..............................19 Benchmarks..............................20
   1. Introduction

   Router testing benchmarks have consistently been made in a
   monolithic fashion in which a single protocol or behavior is
   measured in an isolated environment.  It is important to know the
   limits for a router/switch's (hereby referred to as Router) behavior
   for each protocol, however this does not produce a reliable benchmark
   of the router's behavior in a deployed network.

   Routers in an operational network are simultaneously configured with
   multiple protocols and security policies while forwarding traffic and
   being managed.  To accurately benchmark a router for deployment it is
   necessary to test that router in operational conditions by
   simultaneously configuring the and scaling network protocols and security
   policies, sourcing forwarding traffic, and managing the router. device.  It is helpful
           	Router Software Accelerated Life Testing
   to accelerate these network operational conditions so that the
   router under test can be benchmarked with faster test duration.
   Testing a router in accelerated network conditions is known as
   Accelerated Life Stress Testing.

   This document defines the Benchmark Planes used for the Accelerated
   Life Benchmarking.  These are the Control Plane, Data Plane,
   Management Plane, and Security Plane.  For each plane, the
   Configuration Set, Startup Conditions, and Instability Conditions.
   Terms are also defined for observing Failure Conditions and reporting
   Benchmarks.  White Box benchmarks are defined provided in Appendix 1.
   Accelerated Life Testing of Routers provides the following benefits:
   1. Evaluation of multiple protocols enabled simultaneously as
      configured in deployed networks
   2. Evaluation of System and Software Stability
   3. Evaluation of Manageability under stressful conditions
   4. Identification of Software Coding bugs such as:
	a. Memory Leaks
	b. Suboptimal CPU Utilization
	c. Coding Logic

   These evaluations are externally observable to the DUT as control
   plane or data plane errors.

   Configuration and operational conditions vary for Core, Peering,
   Aggregate, and Edge Routers.  This document focuses on defining
   the terminology for Accelerated Life Testing of Core Routers.
   Terminology for configuring network conditions, accelerating
   operational parameters, accelerating network instability, and
   evaluating results are provided. 1 for
   additional DUT behavior measurements.

   2.  Existing definitions

   RFC 1242 "Benchmarking Terminology for Network Interconnect
   Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching
   Devices" should be consulted before attempting to make use of this
   document.

   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

3. Term definitions

   3.1 Configuration Modules Benchmark Planes

      Definition:
	The features features, conditions, and protocols enabled behavior for the Accelerated Life
	Test and conditions created during the test. Stress
	Benchmarking.

      Discussion:
	Features and Protocols
	There are divided into four configuration
	modules as shown in Figure 1.  These are the Benchmark Planes: Control Plane, Data Plane,
Management Plane, and Security Plane.

           Router Software Accelerated Life Testing

	Definitions for each of these modules are provided Plane as shown in Figure 1. The
	Benchmark Planes define the Configuration, Startup Conditions,
	Instability Conditions, and Failure Conditions used for the
	following sections. test.

	 ___________		 ___________
	|  Control  |		| Management|
	|   Plane   |___     ___|   Plane   |
	|   Module   	      |   |   |	|   Module   	      |
	 -----------    |   |	 -----------
		          \/  \/		       ___________
		      ___________	      	| Security  |
		     |	     |<-----------|   Plane	|
		     |    DUT    |	      |   Module           |
		|--->|	     |<---|        -----------
	 	|     -----------     |
		|		          |
		|     ___________     |
		|    |   Data    |    |
		|--->|   Plane   |<---|
		     |   Module   	     |
	 	      -----------

Figure 1.  Configuration Modules  Router Accelerated Stress Benchmarking Planes

      Measurement units: N/A

      Issues: None

      See Also:
	Control Plane Configuration Module
 	Data Plane Configuration Module
 	Management Configuration Module Plane
	Security Configuration Module Plane

   3.1.1 Control Plane Configuration Module
      Definition:
	The Description of the control protocols enabled for
	the Accelerated Life Test. Stress Benchmarking.

      Discussion:
	The Control Plane Configuration Module is divided into three
	Configuration Sets as shown in Figure 2.  These are Routing
	Protocol, Multicast, defines the Configuration, Startup
	Conditions, Instability Conditions, and Failure
	Conditions of the control protocols used for the test.
	Control Plane protocols may include routing protocols,
	multicast protocols, and MPLS. MPLS protocols.  These can be
	enabled or disabled for a benchmark test.  For example, a network operator
	that operates a network with BGP and an IGP without Multicast
	and MPLS protocols can execute a benchmark test with the
	Routing Protocol

      Measurement units: N/A

      Issues: None

      See Also:
	Benchmark Planes
	Control Plane Configuration Set enabled and
	Control Plane Startup Conditions
	Control Plane Instability Conditions
	Control Plane Failure Conditions
   3.1.2 Data Plane

      Definition:
	The data traffic profile used for the Multicast Accelerated Stress
	Benchmarking.

      Discussion:
	The Data Plane defines the Configuration, Startup
	Conditions, Instability Conditions, and MPLS Configuration Sets disabled.  Configuration Sets are
	further explained in Failure
	Conditions of the data traffic used for the test.
	The Data Plane includes the next section. traffic and interface
	profile.

      Measurement units: Enabled/Disabled

      Issues:None
           Router Software Accelerated Life Testing

	 ____________ 		 ____________		 ____________
	|  Routing   |	 	|  Multicast |		|    MPLS    |
	|  Protocol  |___	|  Protocol  | 	      __|  Protocol  |
	| Config Set |  |  	| Config Set |       |	| Config Set |
	 ------------   |   	 ------------        |	 ------------
		        |		|	     |
			|		|	     |
			|  	        \/	     |
	    		|	  ___________	     |
			|	 |	     | 	     |
			|------->|    DUT    |<------|
				 |	     |
	 		 	  -----------
	Figure 2.  Control Plane Configuration Module Units:
        N/A

      See Also:
	Routing Protocol Configuration Set
       	Multicast Protocol Configuration Set
      	MPLS Protocol Configuration Set

   3.1.2
	Benchmark Planes
	Data Plane Configuration Module

      Definition:
	  The data traffic and interfaces enabled for the Accelerated
	  Life Test.

      Discussion:
	Accelerated Life Testing includes protocols, as defined by the
	Control Set
	Data Plane Configuration Module, and data forwarding, as
	defined by the Startup Conditions
	Data Plane Configuration Module.  As shown in
	Figure 3, the Instability Conditions
	Data Plane Configuration Module has a
	corresponding Traffic Profile Configuration Set.

	 ___________
	|  Traffic  |
	|  Profile  |___
	| Config Set|   |
	 -----------    |
		        \/
		    ____________
		   |		|
		   |    DUT	|
		   |		|
	 	    ------------

      Measurement Units:
        Enabled/Disabled

      See Also:
	Traffic Profile Configuration Set
  	System Configuration Set
           Router Software Accelerated Life Testing Failure Conditions

   3.1.3 Management Configuration Module Plane

      Definition:
  	The router management Management features enabled and tools used for the
	Accelerated Life Test. Stress Benchmarking.

      Discussion:
	A key component of the Accelerated Life Test Stress Benchmarking is the
	Management Configuration Module Plane to assess manageability of the router
	under stress.  The Management Configuration
	Module can be divided into User Access Configuration Set,
	SNMP Configuration Set, Logging/Debug Configuration Set, Plane defines the Configuration,
	Startup Conditions, Instability Conditions, and Failure
	Conditions of the management protocols and features used for
	the test.  The Management Plane includes SNMP, Logging/Debug,
	and Packet Statistics Collector Collection.

      Measurement units:
        N/A

      Issues:
	None

      See Also:
	Benchmark Planes
	Management Plane Configuration Set as
	shown in Figure 4.

   	 ____________ 	  ____________	      ____________
	|    User    |	 |  	      |	     |  Logging/  |
	|   Access   |	 |    SNMP    |    __|   Debug    |
	| Config Set |   | Config Set |   |  | Config Set |
	 ------------     ------------    |   ------------
	      |		 	|	  |
	      |		 	|	  |
	      |                \/	  |
	      |		___________	  |
	      |	       |	   | 	  |
	      |------->|    DUT	   |<-----|
		       |	   |
	 		-----------
			     |
			     |
			    \/
			___________
		       |   Packet  |
		       | Statistics|
		       | Collector |
		       | Config Set|
	 	        -----------

	Figure 4.
	Management Configuration Module

      Measurement units:
        Enabled/Disabled

      Issues:
	None

      See Also:
	User Access Configuration Set
   	SNMP Configuration Set
   	Logging/Debug Configuration Set
	Packet Statistics Collector Configuration Set
           Router Software Accelerated Life Testing Plane Startup Conditions
	Management Plane Instability Conditions
	Management Plane Failure Conditions
   3.1.4 Security Configuration Module Plane

      Definition:
	The Security features enabled for used during the Accelerated Life Test. Stress
	Benchmarking.

      Discussion:
   	Accelerated Life Testing of Core Router can include
	configuration
   	The Control Plane defines the Configuration, Startup
	Conditions, Instability Conditions, and Failure
	Conditions of  Security features. the security features and protocols used for
	the test.  The Security
	Configuration Module Plane includes the ACL Configuration
	Set, ACLs, Firewall,
	Secure Protocol Configuration Set, Protocols, and User Login
	Configuration Set as shown in Figure 5.

	 ____________ 	     ____________	  ____________
	|            |	    |   Secure	 |	 |    User    |
	|    ACL     |	    |  Protocol  |     __|    Login   |
	| Config Set |      | Config Set |    |	 | Config Set |
	 ------------        ------------     |	  ------------
	     |			|	      |
	     |			|	      |
	     |  	        \/	      |
	     |		 ___________	      |
	     |		|	    | 	      |
	     |------->	|    DUT    |<--------|
			|	    |
	 		 -----------
	Figure 5.  Security Configuration Module Login.

      Measurement units:
        Enabled/Disabled
        N/A

      Issues:
	None

      See Also:
  	ACL Configuration Set
  	Secure Protocol Configuration Set
  	Password Login Configuration Set

   3.2 Configuration Sets

   3.2.1 Control
	Benchmark Planes
	Security Plane Configuration Sets
   3.2.1.1 Routing Protocol Configuration Set

	Definition:
  	   Table of Routing Protocols configuration for Accelerated
  	   Life Test.

      	Discussion:
	   The Routing Protocol Configuration Set is one of the
	   Control
	Security Plane Startup Conditions
	Security Plane Instability Conditions
	Security Plane Failure Conditions

   3.2 Configuration Sets BGP, MBGP, ISIS, OSPF are
 	   the routing protocols that can be enabled or disabled in
	   any combination.

      Definition:
	The specific protocol parameters features and scaling limits used to
           Router Software Accelerated Life Testing

	   establish during the test conditions Accelerated Stress
	Benchmarking.

      Discussion:
	There are also provided in the table.

      	Measurement units:
	   PARAMETER				UNITS
  	   BGP					Enabled/Disabled
	   Number of EBGP Peers			Peers
	   Number of IBGP Peers			Peers
	   Number of BGP Route Instances	Routes
	   Number of BGP Installed Routes	Routes

 	   MBGP					Enabled/Disabled
	   Number of MBGP Route Instances	Routes
	   Number of MBGP Installed Routes	Routes

	   ISIS					Enabled/Disabled
	   ISIS-TE				Enabled/Disabled
	   Number of ISIS Adjacencies		Adjacencies
	   Number of ISIS Routes		Routes
	   Number of Nodes per Area		Nodes

   	   OSPF					Enabled/Disabled
	   OSPF-TE				Enabled/Disabled
	   Number of OSPF Adjacencies		Adjacencies
	   Number of OSPF Routes		Routes
	   Number of Nodes per Area		Nodes

      Issues:
	  None

      See Also: four Configuration Sets: Control Plane Configuration Module
	  Multicast Protocol Configuration Set
	  MPLS Protocol Configuration Set

   3.2.1.2 Multicast Protocol Configuration Set

      Definition:
	Table of Multicast Protocols configuration for Accelerated
  	Life Test.

      Discussion:
  	   The Multicast Protocol Set,
	Data Plane Configuration Set is one of the
	   Control Set, Management Plane Configuration Sets PIM-SM Set,
	and MSDP are
 	   the routing protocols that can be enabled or disabled in
	   any combination.  The specific protocol parameters used to
	   establish the test conditions are also provided in the table. Security Plane Configuration Set.

      Measurement units:
	   PARAMETER				UNITS
	   PIM-SM				Enabled/Disabled
	   RP					Enabled/Disabled
	   Number of Multicast Groups		Groups
           Router Software Accelerated Life Testing

	   MSDP					Enabled/Disabled N/A

      Issues: None

      See Also:
	Control Plane Configuration Module
	  Routing Protocol Set
 	Data Plane Configuration Set
	  MPLS Protocol
 	Management Plane Configuration Set

   3.2.1.3 MPLS Protocol
	Security Plane Configuration Set

3.2.1 Control Plane Configuration Set
      Definition:
	Table of MPLS Protocols configuration
	The routing protocols and scaling values used for the Accelerated
	Life Test. Benchmarking.

      Discussion:
	The MPLS Protocol Configuration Set is one of the
	Control Plane Configuration Sets MPLS-TE (RSVP-TE, ISIS-TE, OSPF-TE) Set is shown in Figure 2 and LDP
	are specifies
	the routing Routing Protocols, Multicast, and MPLS configuration.  Specific
	protocols that can be enabled or disabled in any
	combination.  The specific protocol parameters used to establish
	the test conditions are also provided in the table. for a benchmark test.

      Measurement units:
	   PARAMETER				UNITS
	   MPLS-TE
	   Number of Ingress Tunnels		Tunnels
	   Number of Mid-Point Tunnels		Tunnels
	   Number of Egress Tunnels		Tunnels

	   LDP
	   Number of Sessions			Sessions
	   Number of FECs			FECs None

      Issues: None

      See Also:

	 ____________ 		 ____________		 ____________
	|  Routing   |	 	|  Multicast |		|    MPLS    |
	|  Protocols |___		|  Protocols | 	    __|  Protocols |
	| 	       |   |  	| 	       |        |	| 	       |
	 ------------    |   	 ------------         |	 ------------
			     |		 |	          |
			     |		 |	          |
			     |  	       \/	          |
	    		     |	   ___________	    |
			     |	  |	        | 	    |
			     |------->|    DUT    |<------|
				 	  |	        |
	 		 	         -----------
	Figure 2.  Control Plane Configuration Module
	  Routing Protocol

      See Also:
	Data Plane Configuration Set
	  Multicast Protocol
      Management Configuration Set
      Security Configuration Set

   3.2.2 Data Plane Configuration Sets

   3.2.2.1 Set

      Definition:
	The data traffic profile enabled for the Accelerated Stress
	Benchmarking.

      Discussion:
	Data Plane Configuration Set

      Definition:
	Table of Forwarded includes the Traffic configuration Profile and
	interfaces used for the Accelerated Life
	Test.

      Discussion: Stress Benchmarking.

      Measurement units:

           Router Software Units:
        N/A

      Issues:
	None

      See Also:
	Traffic Profile
   3.2.2.1 Traffic Profile
      Definition
	The characteristics of the Offered Load to the DUT used for
	the Accelerated Life Testing

	PARAMETER				UNITS Stress Benchmarking.

      Discussion
	The Traffic Forwarding			Enabled/Disabled Profile specifies the number of packet size(s), packet
	rate per interface, number of flows, and encapsulation used for the 	offered load to the DUT.

      Measurement Units:
      Traffic Profile is reported as follows:

	Parameter			Units
	---------			------
	Packet Size(s)			bytes
	Packet Size				Byte Size Distribution Rate(interface)		array of packets per second
	Number of Flows				Flows
	Offered Load				bps (or pps) Aggregate
	Number			number
	Encapsulation(flow)		array of Interfaces			Interfaces encapsulation type

      Issues:
	None

      See Also:
	Data Plane Configuration Module Set

   3.2.3 Management Configuration Sets
   3.2.3.1 User Access Plane Configuration Set

      Definition:
	Table of User Access methods during
  	The router management features enabled for the
	Accelerated Life Stress Test.

      Discussion:
	A key component of the Accelerated Stress Test is the Management 	Configuration Set to assess manageability of the router under
	stress.  The Management Configuration Set defines the management 	configuration of the DUT.  Features that are part of the
	Management Configuration Set include SNMP, Logging/Debug, and
	Statistics Collection, and services such as FTP, as shown in
Figure 3.

      Measurement units:
	PARAMETER				UNITS
	Telnet Rate				Sessions/Hour
	FTP Rate				Sessions/Hour
	Concurrent Telnet Sessions		Sessions
	Concurrent FTP Session			Sessions
	SSH					Enabled/Disabled
	RADIUS					Enabled/Disabled
	TACACS					Enabled/Disabled
      N/A

      Issues:
	None

      See Also:
	Management
	Control Plane Configuration Module
	SNMP Set
	Data Plane Configuration Set
	Logging/Debug
      Security Plane Configuration Set
   	 	 	  ____________	    ____________
			 |  	        |	   |  Logging/  |
			 |    SNMP    |    __|   Debug    |
		   	 | 	        |   |  |  	    |
	 	        ------------    |   ------------
	      		 |	      |
	      		 |	      |
	       	       \/	      |
	      	  ___________	|
	      	 |	       | 	|
	      	 |    DUT	 |<---|
		       |	       |
	 		  -----------
			      |
			      |
			      \/
			  ___________
		       |   Packet Statistics  |
		       | Statistics|
		       | Collector |
		       | 	       |
	 	        -----------

		Figure 3.  Management Plane Configuration Set

   3.2.3.2 SNMP

   3.2.4 Security Plane Configuration Set

      Definition:
	Table of SNMP Configuration
	Security features and scaling enabled for the Accelerated Life Stress
	Test.

      Discussion:
   	The Security Plane Configuration Set includes the configuration
	and scaling of ACLs, Firewall, IPsec, and User Access, as shown
	in Figure 4.

	 ____________ 	   ____________	      ____________
	|            |	  |   Secure   |	     |    User    |
	|ACL/Firewall|      |  Protocol  |     __|   Access   |
	| 	       |      | 		   |    |  |            |
	 ------------        ------------     |	------------
	     |			|	        |
	     |			|	        |
	     |  	        	\/	        |
	     |	     ___________	        |
	     |	    |	          | 	  |
	     |------->  |    DUT    |<--------|
			    |	          |
	 		     -----------
		Figure 4.  Security Configuration Module
      Measurement units:
	SNMP GET Rate				SNMP Gets/minute
        N/A

      Issues:
	None
           Router Software Accelerated Life Testing

      See Also:
	Management Configuration Module
	User Access Configuration Set
	Logging/Debug
  	ACL Configuration Set
	Packet Statistics Collector
  	Secure Protocol Configuration Set

   3.2.3.3 Logging/Debug
  	Password Login Configuration Set

   3.3 Startup Conditions

      Definition:
	Table
	Test conditions that occur at the start of Logging and Debug configuration for the Accelerated
	Life Test. Benchmark to establish conditions for the remainder of
	the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions are defined for the Control
	Plane and Security Plane.

      Measurement units:
	Logging					Enabled/Disabled
	Protocol Debug				Enabled/Disabled
	N/A

      Issues:
	None

      See Also:
	Control Plane Startup Conditions
	Data Plane Startup Conditions
	Management Configuration Module
	User Access Configuration Set
	SNMP Configuration Set
	Packet Statistics Collector Configuration Set

   3.2.3.4 Packet Statistics Collector Configuration Set Plane Startup Conditions
	Security Plane Startup Conditions

   3.3.1 Control Plane Startup Conditions

      Definition:
	Table
	Control Plane conditions that occur at the start
	of Packet Statistics Collection for the Accelerated Life Test. Stress Benchmarking to establish conditions
	for the remainder of the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions for the Control Plane include
	session establishment rate, number of sessions established
	and number of routes learned.

      Measurement units:
	Packet Statistics Collector		Enabled/Disabled
	Sampling Rate				X:1 packets
	N/A
      Issues:
	None

      See Also:
	Management Configuration Module
	User Access Configuration Set
	SNMP Configuration Set
	Logging/Debug Configuration Set

   3.2.4
	Startup Conditions
	Security Configuration Sets
           Router Software Accelerated Life Testing

  3.2.4.1 ACL Plane Startup Conditions
	Control Plane Configuration Set

   3.3.2 Security Plane Startup Conditions

      Definition:
	Table
	Security Plane conditions that occur at the start
	of ACL configuration for the Accelerated Life Test Stress Benchmarking to establish conditions
	for the remainder of the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions for the Security Plane include
	session establishment rate, number of sessions established
	and number of policies learned, and number of user access
	sessions opened.

      Measurement units:
	Access-Control Lists			Enabled/Disabled
	Number of ACLs				ACLs
	ACL Depth				ACEs
	N/A

      Issues:
	None

      See Also:
	Startup Conditions
	Data Plane Startup Conditions
	Management Plane Startup Conditions
	Security Configuration Module

   3.3 Plane Startup Conditions

   3.4 Instability Conditions

      Definition:
	Test conditions that occur during the Accelerated Life Test
	that are typical of Stress
	Benchmark to produce instability in an operational network.
	These conditions occur at a much greater rate during the
	test.

      Discussion:
	Configuration Modules and Configuration Sets establish stress the
	configuration of DUT.

      Discussion:
	Instability Conditions are experienced by the DUT for after the Accelerated Life Test.
	Startup Conditions have completed.  Instability Conditions are events that
	occur during the
	Accelerated Life Test to stress for the router. Control Plane, Data Plane, Management Plane,
	and Security Plane.

      Measurement units:
	N/A

      Issues:
	None
     See Also:
	Interface Shutdown Cycling Rate
	BGP Route Flap Rate
	IGP Route Flap Rate
	Route Convergence Due to Better Next-Hop
	LSP Reroute Rate
	Target Run Time

   3.3.1 Interface Shutdown Rate
   	Control Plane Instability Conditions
   	Data Plane Instability Conditions
 	Management Plane Instability Conditions
	Security Plane Instability Conditions

   3.4.1 Control Plane Instability Conditions

      Definition:
	The rate at which physical interfaces are shutdown on
	Control Plane conditions that occur during the DUT.

      Discussion:

           Router Software Accelerated Life Testing
	This Stress
	Benchmark to produce instability condition is an administrative shutdown or remote
	loss of carrier for every interface of and stress the DUT.  This produces great
      instability on

      Discussion:
	Control Plane Instability Conditions are experienced by the DUT due to continuous protocol
	after the Startup Conditions have completed.  Control Plane
	Instability Conditions experienced by the DUT include session loss
	loss, route withdrawal, and
	convergence. route cost changes.

      Measurement units:
	number of interface shutdowns per minute
	N/A

      Issues:
	None

      See Also:
   	Instability Conditions
	Route Flap Rate
	Target Run Time

   3.3.2 BGP Route Flap Rate
   	Data Plane Instability Conditions
 	Management Plane Instability Conditions
	Security Plane Instability Conditions

   3.4.2 Data Plane Instability Conditions
     Definition:
	The rate at which BGP routes installed in
	Data Plane conditions that occur during the route table are
	flapped.

      Discussion:
	This Accelerated Stress
	Benchmark to produce instability condition defines the rate of repeated
	installation, withdrawal,and installation of routes installed in and stress the route table. This produces great instability on DUT.

      Discussion:
	Data Plane Instability Conditions are experienced by the DUT due
	to continuous changes to
	after the forwarding table. Startup Conditions have completed.  Data Plane
	Instability Conditions experienced by the DUT include interface
	shutdown, link loss, and overloaded links.

      Measurement units:
	routes flapped per second
	N/A

      Issues:
	None

      See Also:
   	Instability Conditions
	Interface Shutdown Cycling Rate
	Target Run Time

  3.3.3 IGP Route Flap Rate
   	Control Plane Instability Conditions
 	Management Plane Instability Conditions
	Security Plane Instability Conditions
   3.4.3 Management Plane Instability Conditions

     Definition:
	The rate at which IGP routes installed in
	Management Plane conditions that occur during the route table are
	flapped.

      Discussion:
	This Accelerated
	Life Benchmark to produce instability condition defines the rate of repeated
	installation, withdrawal,and installation of routes installed in and stress the route table. This produces great instability on DUT.

      Discussion:
	Management Plane Instability Conditions are experienced by the DUT due
	to continuous changes to
	after the forwarding table.

           Router Software Accelerated Life Testing Startup Conditions have completed.  Management Plane
	Instability Conditions experienced by the DUT include repeated
	FTP of large files.

      Measurement units:
	routes flapped per second
	N/A

      Issues:
	None

      See Also:
   	Instability Conditions
	Interface Shutdown Cycling Rate
	Target Run Time

  3.3.4 Route Convergence Due to Better Next-Hop
 	Control Plane Instability Conditions
   	Data Plane Instability Conditions
	Security Plane Instability Conditions

   3.4.4 Security Plane Instability Conditions

     Definition:
	The rate at which routes in
	Security Plane conditions that occur during the FIB are replaced by a route
	instance in Accelerated
	Life Benchmark to produce instability and stress the RIB with a different next-hop becoming more
	preferred. DUT.

      Discussion:
	The route changes
	Security Plane Instability Conditions are due to attribute and TLV changes. RIB
	contains routes matching those in experienced by the FIB.  Convergence due
	to better path can occur for BGP DUT
	after the Startup Conditions have completed.  Security Plane
	Instability Conditions experienced by the DUT include session
	loss and IGP routes. policy changes.

      Measurement units:
	more preferred routes per second
	N/A

      Issues:
	None

      See Also:
   	Instability Conditions
	Interface Shutdown Cycling Rate
	Target Run Time

   3.3.5 LSP Reroute Rate
 	Control Plane Instability Conditions
   	Data Plane Instability Conditions
	Management Plane Instability Conditions
   3.5 Failure Conditions

   3.5.1 Unexpected Session Loss

      Definition:
	The rate at which established LSPs experience a failure condition
	and are forced
	Session Loss due to reroute. Control Plane or Data Plane failure during the
	Accelerated Stress Benchmarking.

      Discussion:
	This instability condition defines
	Session loss will be observed due to Convergence Events produced
	by the rate of repeated LSP
	reroutes. Instability Conditions.  This produces great instability on is not considered a failure
	during the DUT Accelerated Stress Benchmarking. It is expected behavior.
	Unexpected packet loss occurs due to
	continuous tunnel path calculations and changes to the
	forwarding table.  Rerouting can be performed with Headend
	Reroute, Standby LSP, Data Plane or Fast Reroute. Control Plane
	failure.

      Measurement units:
	routes flapped per second
	sessions

      Issues:
	None
           Router Software Accelerated Life Testing

      See Also:
	Instability Conditions
	Interface Shutdown Cycling Rate
	Target Run Time

   3.3.6 Intended Test Duration
	Unexpected Packet Loss
	Misrouted Packets
	Access Denial
	Errored Management Values
	Security Plane Failure

   3.5.2 Unexpected Packet Loss
      Definition:
	The maximum targetted run time for
	Packet Loss due to Control Plane or Data Plane failure during the
	Accelerated Life Test. Stress Benchmarking.

      Discussion:
	This is an instability condition because the protocol sessions
	and scaling values must be maintained for this specified time with
	interfaceshutdowns and route flapping.  The test may
	Packet loss will be halted
	upon reaching observed due to Convergence Events produced
	by the Instability Conditions.  This is not considered a failure
	during the Intended Test Duration. Accelerated Stress Benchmarking. It is expected behavior.
	Unexpected packet loss occurs due to Data Plane or Control Plane
	failure.

      Measurement units:
	Hours
	packets

      Issues:
	None

      See Also:
	Instability Conditions
	Interface Shutdown Cycling Rate
	Route Flap Rate

   3.4 Evaluation Benchmarks

   3.4.1 Run-Time without Error
	Unexpected Session Loss
	Misrouted Packets
	Access Denial
	Errored Management Values
	Security Plane Failure
   3.5.3 Misrouted Packets

      Definition:
	The run-time for the Accelerated Life Test without the DUT
	exhibiting an error in
	Packets routed to the control plane, data plane, management
	plane, incorrect next-hop or security plane.

      Discussion:
	For a successful test, LSP due to Control
	Plane failure during the Run-Time Without Error Accelerated Stress Benchmarking.

      Discussion:
	Packets will equal the
	Intended Test Duration.  In the event of any failure, the Run-Time
	Without Error is less than the Intended Test Duration be rerouted to new next-hops and will equal LSPs due to Convergence
	Events produced by the maximum of Instability Conditions.  This is not considered
	a failure during the Run-Time without Accelerated Stress Benchmarking. It is expected
	behavior.  Unexpected Misrouted packets occur due to Control Plane Error, Run-Time
	without Data Plane Error, Run-Time without Management Plane Error,
	and Run-Time without Security Plane Error.
	failure.

      Measurement units:
	Hours and Minutes
	packets

      Issues:
	None

      See Also:

           Router Software Accelerated Life Testing

	Intended Test Duration
	Run-Time without Control Plane Error
	Run-Time without Data Plane Error
	Run-Time without
	Unexpected Packet Loss
	Unexpected Session Loss
	Access Denial
	Errored Management Plane Error
	Run-Time without Values
	Security Plane Error

   3.4.2 Run-Time without Control Plane Error Failure

   3.5.4 Access Denial

      Definition:
	The run-time for the Accelerated Life Test without
	Inability to management the DUT
	exhibiting an error in due to the control plane inability
	to Routing, Multicast,
	and MPLS Protocol. gain remote access to the DUT.

      Discussion:
	For
	Handling of Instability Conditions may cause a successful test, the Run-Time Without Control Plane Error
	will equal failure
	of the Run-Time without Error.  In DUT externally observable as the event of a Control
	Plane failure, inability to remotely manage
	the Run-Time Without Control Plane Error is less
	than DUT.  Whether using SNMP, Telnet, SSH, or FTP, failure to manage
	the Run-Time without Error. device constitues a failure.

      Measurement units:
	Hours and Minutes
	Failed session

      Issues:
	None

      See Also:
	Intended Test Duration
	Run-Time without Error
	Run-Time without Data Plane Error
	Run-Time without
	Unexpected Packet Loss
	Unexpected Session Loss
	Misrouted Packets
	Errored Management Value
	Security Plane Error
	Run-Time without Failure
   3.5.5 Errored Management Value

      Definition:
	Incorrect SNMP object value or statistic during the Accelerated
	Life Benchmark.

      Discussion:
	Management of a router relies upon accurate reporting of object
	values and statistics.  The Accelerated Stress Benchmarkinging validates
	that the DUT is able to report correct values when experiencing
	Instability Conditions.

      Measurement units:
	Name of incorrect object or statistic

      Issues:
	None

      See Also:
	Unexpected Packet Loss
	Unexpected Session Loss
	Misrouted Packets
	Access Denial
	Security Plane Error

   3.4.3 Run-Time without Data Failure
	Success Threshold

   3.5.6 Security Plane Error Failure
      Definition:
	The run-time
	Violation of Security Policy for the Accelerated Life Test without packets directed to the DUT
	exhibiting an error in the Data Plane for traffic forwarding
	or DUT interfaces.

      Discussion:
	For a successful test, routed through the Run-Time Without Data Plane Error
	will equal DUT.

      Discussion:
	Security Policy is configured to permit and deny access to packets
	destined for the Run-Time without Error.  In DUT and packets to be routed through the event DUT.
	A violation of this security policy is when either a Data
	Plane failure, the Run-Time Without Data Plane Error packet to
	be denied is less
	than the Run-Time without Error. permitted or a packet to be permitted is denied.

      Measurement units:
	Hours and Minutes
	packets

      Issues:
	None

      See Also:

           Router Software Accelerated Life Testing

	Intended Test Duration
	Run-Time without Error
	Run-Time without Control Plane Error
	Run-Time without Management Plane Error
	Run-Time without Security Plane Error

   3.4.4 Run-Time without
	Unexpected Packet Loss
	Unexpected Session Loss
	Misrouted Packets
	Access Denial
	Errored Management Plane Error Value
   3.6 Benchmarks

   3.6.1 Success Threshold

     Definition:
	The run-time intended number of hours for the Accelerated Life Test without the DUT
	exhibiting an error in the Management Plane to User Access,
	SNMP, or Logging/Debug.

      Discussion:
	For a successful test, the Run-Time Without Management Plane
	Error will equal the Run-Time operate without Error.  In failure
	during the event of Accelerated Stress Benchmarking.

      Discussion:
	Success Threshold is a Management Plane failure, user determined value of time.  It is
	necessary to define the Run-Time Without Management
	Plane Error Success Threshold because it is less than possible
	for the Run-Time without Error. DUT to experience zero failures for given Configuration
	Sets, Startup Conditions, and Instability Conditions

      Measurement units:
	Hours and Minutes

      Issues:
	None

      See Also:
	Intended
	Accelerated-Life Test Duration

   3.6.2 Accelerated-Life Test Duration
	Run-Time without Error
	Run-Time without Control Plane Error
	Run-Time without Data Plane Error
	Run-Time without Security Plane Error

   3.4.5 Run-Time without Security Plane Error

      Definition:
	The run-time for the Accelerated Life Stress Test without the DUT
	exhibiting an error in the Security Plane to ACLs. control plane, data plane, management
	plane, or security plane.

      Discussion:
	For a successful test, the Run-Time Without Security Plane
	Error Test Duration will equal the Run-Time without Error.
	Success Threshold.  In the event of a Security Plane any failure, the  Run-Time Without Control
	Plane Error Test Duration
	is less than the Run-Time without Error. Success Threshold.  Reporting of the Test
	Duration MUST inlcude the Failure Condition from those defined
	in this document.

      Measurement units:
	Hours and Minutes

      Issues:
	None

      See Also:
	Intended Test Duration
           Router Software Accelerated Life Testing

	Run-Time without Error
	Run-Time without Control Plane Error
	Run-Time without Data Plane Error
	Run-Time without Management Plane Error
	Success Threshold
   4. Security Considerations
        Documents of this type do not directly effect the security of
        the Internet or of corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.

   5. References

      [1]   Bradner, S., Editor, "Benchmarking Terminology for Network
            Interconnection Devices", RFC 1242, July 1991.

      [2]   Mandeville, R., "Benchmarking Terminology for LAN Switching
            Devices", RFC 2285, June 1998.

      [3]   Bradner, S. and McQuaid, J., "Benchmarking Methodology for
	    Network Interconnect Devices", RFC 2544, March 1999.

      [4]   "Core Router Evaluation for Higher Availability", Scott
	    Poretsky, NANOG 25, June 8, 2002, Toronto, CA.

      [5]   "Router Stress Testing to Validate Readiness for Network
	    Deployment", Scott Poretsky, IEEE CQR 2003.

   6. Author's Address

     	Scott Poretsky
        Avici Systems
        101 Billerica Avenue
        N. Billerica,
   	Quarry Technologies
  	8 New England Executive Park
   	Burlington, MA 01862 01803
    	USA
    	Phone: + 1 978 964 2287 781 395 5090
   	EMail: sporetsky@avici.com sporetsky@quarrytech.com

	Shankar Rao
	950 17th Street
	Suite 1900
	Qwest Communications
	Denver, CO 80210
	USA
	Phone: + 1 303 437 6643
	Email: srao@qwest.net

	Ray Piatt
	Cable and Wireless
	11700 Plaza America Drive
	Reston, VA 20190
	USA
           Router Software Accelerated Life Testing
	Phone: + 1 703 292 2113
	Email: rpiatt@cw.net

7.  Full Copyright Statement

        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
        copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.

   Appendix 1.  White Box Benchmarking Terminology
   Appendix 1.1   Minimum Available Memory

      Definition:
	Minimum DUT Available Memory during the duration of the
	Accelerated Life Stress Test.

      Discussion:
	It is necessary to monitor DUT memory to measure this
	benchmark.

      Measurement units:
	bytes

      Issues:
	None

      See Also:
	Maximum CPU Utilization
           Router Software Accelerated Life Testing

   Appendix 1.2 Maximum CPU Utilization

      Definition:
	Maximum DUT CPU utilization during the duration of the
	Accelerated Life Stress Test.

      Discussion:
	It is necessary to monitor DUT CPU Utilization to measure
	this benchmark.

      Measurement units:
	%

      Issues:
	None

      See Also:
	Minimum Available Memory