Network Working Group
   INTERNET-DRAFT
   Expires in: April August 2004
                                       		Scott Poretsky
                                                Quarry Technologies

						Shankar Rao
						Qwest Communications

						   		Ray Piatt
						   		Cable and Wireless

                                                October 2003

                                                February 2004

             Terminology for Accelerated Stress Benchmarking

              <draft-ietf-bmwg-acc-bench-term-01.txt>

              <draft-ietf-bmwg-acc-bench-term-02.txt>

   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   ABSTRACT
   This terminology document provides the terms to be used for
   benchmarking a router or switch under Accelerated Stress conditions. Terminology is provided for performing this benchmark using
   the Contol Plane, Data Plane, Management Plane, and Security
   Plane Stress
   Benchmarking of networking devices.  The three phases of the Device Under Test.  Terms Stress
   Test: Startup, Instability and Recovery are defined for
   Configuration, Startup Conditions, Instability Conditions, Failure
   Conditions, along with the
   benchmarks and configuration terms associated with the each phase.
   Also defined are the Benchmark Planes fundamental to stress testing
   configuration, setup and measurement.  The terminology is to be
   used with the companion framework and Benchmarks. methodology documents.

   Table of Contents
     1. Introduction ............................................... 3
     2. Existing definitions ....................................... 3
     3. Term definitions............................................ 3
      3.1 Benchmark Planes.......................................... General Terms............................................. 3
	3.1.1 Control Plane........................................... 4 Benchmark Planes...................................... 3
        3.1.2 Data Plane.............................................. 5 Configuration Sets.................................... 4
        3.1.3 Management Plane........................................ 5 Startup Conditions.................................... 4
        3.1.4 Security Plane.......................................... Instability Conditions................................ 5
        3.1.5 Aggregate Forwarding Rate............................. 6
	3.2 Configuration Sets........................................
   	3.1.6 Controlled Session Loss............................... 6
   	3.1.7 Uncontrolled Session Loss............................. 6
      3.2 Benchmark Planes.......................................... 7
        3.2.1 Control Plane Configuration Set......................... Plane......................................... 7
        3.2.2 Data Plane Configuration Set............................ Plane............................................ 7
      3.2.2.1 Traffic Profile....................................... 8
        3.2.3 Management Plane Configuration Set...................... Plane...................................... 8
        3.2.4 Security Plane Configuration Set........................ 9 Plane........................................ 8
      3.3 Startup Conditions........................................10 Startup................................................... 9
        3.3.1 Startup Phase......................................... 9
	3.3.2 Benchmarks............................................10
   	  3.3.2.1 Stable Aggregate Forwarding Rate..................10
   	  3.3.2.2 Stable Session Count..............................11
        3.3.3 Control Plane.........................................11
          3.3.3.1 Control Plane Configuration Set...................11
          3.3.3.2 Control Plane Startup Conditions........................10
	3.3.2 Conditions..................12
   	3.3.4 Data Plane............................................12
          3.3.4.1 Data Plane Configuration Set......................12
  	  3.3.4.2 Traffic Profile...................................13
  	3.3.5 Management Plane......................................13
  	  3.3.5.1 Management Plane Configuration Set................13
        3.3.6 Security Plane........................................14
          3.3.6.1 Security Plane Configuration Set..................14
   	  3.3.6.2 Security Plane Startup Conditions.......................11 Conditions.................15
      3.4 Instability Conditions....................................11 Instability...............................................15
        3.4.1 Instability Phase.....................................15
        3.4.2 Benchmarks............................................16
          3.4.2.1 Unstable Aggregate Forwarding Rate................16
          3.4.2.2 Degraded Aggregate Forwarding Rate................16
          3.4.2.3 Average Degraded Aggregate Forwarding Rate........17
   	  3.4.2.4 Unstable Uncontrolled Sessions Lost...............17
        3.4.3 Instability Conditions................................18
          3.4.3.1 Control Plane Instability Conditions....................12
 	3.4.2 Conditions..............18
   	  3.4.3.2 Data Plane Instability Conditions.......................12
 	3.4.3 Conditions.................18
          3.4.3.3 Management Plane Instability Conditions.................13
	3.4.4 Conditions...........19
          3.4.3.4 Security Plane Instability Conditions...................13 Conditions.............19
      3.5 Failure Conditions........................................14 Recovery..................................................20
        3.5.1 Unexpected Packet Loss..................................14 Recovery Phase........................................20
        3.5.2 Unexpected Session Loss.................................14
	3.5.3 Misrouted Packets.......................................15
	3.5.4 Access Denial...........................................15
	3.5.5 Errored Management Value................................16
	3.5.6 Security Plane Failure..................................16
	3.6 Benchmarks................................................17
	3.6.1 Success Threshold.......................................17
	3.6.2 Accelerated-Life Test Duration..........................17 Benchmarks............................................20
          3.5.2.1 Recovered Aggregate Forwarding Rate...............20
          3.5.2.2 Recovery Time.....................................21
          3.5.2.3 Recovered Uncontrolled Sessions Lost..............21
     4. Security Considerations.....................................18 Considerations.....................................22
     5. References..................................................18 References..................................................22
     6. Author's Address............................................18 Address............................................22
     7. Full Copyright Statement....................................19 Statement....................................23
     Appendix 1 - White Box Benchmarks..............................20 Benchmarks..............................23
   1. Introduction

   Routers in an operational network are simultaneously configured with
   multiple protocols and security policies while forwarding traffic and
   being managed.  To accurately benchmark a router for deployment it is
   necessary to test that router in operational conditions by
   simultaneously configuring and scaling network protocols and security
   policies, forwarding traffic, and managing the device.  It is helpful
   to accelerate these network operational conditions so that the
   router under test can be benchmarked with faster test duration.
   Testing a router in accelerated network conditions is known as
   Accelerated Stress Testing.

   This document defines provides the Benchmark Planes used Terminology for performing Stress
   Benchmarking of networking devices.  The three phases of the Accelerated
   Life Benchmarking. Stress
   Test: Startup, Instability and Recovery are defined along with the
   benchmark and configuration terms associated with the each phase.
   Benchmarks for stress testing are defined using the Aggregate
   Forwarding Rate and control plane Session Count during each phase
   of the test.  Also defined are the Benchmark Planes fundamental to
   stress testing configuration, setup and measurement.  These are
   the Control Plane, Data Plane, Management Plane, Plane and Security Plane. Plane
   For each plane, the Configuration Set, Startup Conditions, and
   Instability Conditions.
   Terms are also defined for observing Failure Conditions and reporting
   Benchmarks. are defined.  White Box benchmarks are
   provided in Appendix 1 for additional DUT behavior measurements.
   The terminology is to be used with the companion framework
   document [6] and methodology document [7].

   2.  Existing definitions

   RFC 1242 "Benchmarking Terminology for Network Interconnect
   Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching
   Devices" should be consulted before attempting to make use of this
   document.

   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

   3. Term definitions
   3.1 General Terms
   3.1.1 Benchmark Planes

      Definition:
	The features, conditions, and behavior for the Accelerated Stress
	Benchmarking.

      Discussion:
	There are four Benchmark Planes: Control Plane, Data Plane,
	Management Plane, and Security Plane as shown in Figure 1. The
	Benchmark Planes define the Configuration, Startup Conditions,
	Instability Conditions, and Failure Conditions used for the test.

	 ___________		 ___________
	|  Control  |		| Management|
	|   Plane   |___     ___|   Plane   |
	|   	    |   |   |	|   	    |
	 -----------    |   |	 -----------
		       \/  \/		       ___________
		      ___________	      | Security  |
		     |	     	 |<-----------|   Plane	  |
		     |    DUT    |	      |           |
		|--->|	     	 |<---|        -----------
	 	|     -----------     |
		|		      |
		|     ___________     |
		|    |   Data    |    |
		|--->|   Plane   |<---|
		     |   	 |
	 	      -----------

	Figure 1.  Router Accelerated Stress Benchmarking Planes

      Measurement units:
	N/A

      Issues:
	None

      See Also:
	Control Plane
 	Data Plane
 	Management Plane
	Security Plane

   3.1.1 Control Plane

    3.1.2 Configuration Sets

      Definition:
	The Description of the control protocols enabled for features and scaling limits used during the Accelerated Stress
	Benchmarking.

      Discussion:
	The
	There are four Configuration Sets: Control Plane defines the Configuration, Startup
	Conditions, Instability Conditions, and Failure
	Conditions of the control protocols used for the test.
	Control Configuration Set,
	Data Plane protocols may include routing protocols,
	multicast protocols, Configuration Set, Management Plane Configuration Set,
	and MPLS protocols.  These can be
	enabled or disabled for a benchmark test. Security Plane Configuration Set.

      Measurement units:
	N/A
      Issues:
	None

      See Also:
	Benchmark Planes
	Control Plane Configuration Set
	Control
 	Data Plane Startup Conditions
	Control Configuration Set
 	Management Plane Instability Conditions
	Control Configuration Set
	Security Plane Failure Configuration Set

   3.1.3 Startup Conditions
   3.1.2 Data Plane

      Definition:
	The data traffic profile used for
	Test conditions that occur at the start of the Accelerated Stress
	Benchmarking.

      Discussion:
	The Data Plane defines
	Life Benchmark to establish conditions for the Configuration, remainder of
	the test.

      Discussion:
	Startup
	Conditions, Instability Conditions, and Failure Conditions of may cause stress on the data traffic used DUT and produce
	failure.  Startup Conditions are defined for the test.
	The Data Control
	Plane includes the traffic and interface
	profile. Security Plane.

      Measurement Units: units:
	N/A

      Issues:
	None

      See Also:
	Benchmark Planes
	Data Plane Configuration Set
	Data
	Control Plane Startup Conditions
	Data Plane Instability Startup Conditions
	Data
	Management Plane Failure Startup Conditions

   3.1.3 Management
	Security Plane Startup Conditions

   3.1.4 Instability Conditions

      Definition:
  	The Management features and tools used for the
	Accelerated Stress Benchmarking.

      Discussion:
	A key component of
	Test conditions that occur during the Accelerated Stress Benchmarking is
	Benchmark to produce instability and stress the
	Management Plane DUT.

      Discussion:
	Instability Conditions are applied to assess manageability of the router
	under stress.  The Management Plane defines DUT after the Configuration,
	Startup Conditions, Conditions have completed.  Instability Conditions, and Failure Conditions of the management protocols and features used
	occur for the test.  The Control Plane, Data Plane, Management Plane includes SNMP, Logging/Debug, Plane,
	and Statistics Collection. Security Plane.

      Measurement units:
	N/A

      Issues:
	None
     See Also:
	Benchmark Planes
	Management Plane Configuration Set
	Management
   	Control Plane Startup Instability Conditions
	Management
   	Data Plane Instability Conditions
 	Management Plane Failure Instability Conditions
   3.1.4
	Security Plane Instability Conditions

   3.1.5 Aggregate Forwarding Rate

     Definition:
	The Security features used
	Sum of forwarding rates for all interfaces on the
	DUT during the Startup Phase.

      Discussion:
	Each interface of the DUT forwards traffic at some
	measured rate.  The Aggregate Forwarding Rate is the
	sum of forwarding rates for all interfaces on the DUT.

      Measurement units:
	pps

      Issues:
	None

      See Also:
   	Startup Phase

   3.1.6 Controlled Session Loss

     Definition:
	Control Plane sessions that are intentionally brought
	down during the Stress test.

      Discussion:
	The test equipment is able to control protocol
	session state with the DUT.

      Measurement units:
	None

      Issues:
	None

      See Also:
	Uncontrolled Session Loss

   3.1.7 Uncontrolled Session Loss

     Definition:
	Control Plane sessions that are in the down state
	but were not intentionally brought down during the
	Stress test.

      Discussion:
	The test equipment is able to control protocol
	session state with the DUT.  The test equipment
	is also to monitor for sessions lost with the
	DUT which the test equipment itself did not
	intentionally bring down.

      Measurement units:
	N/A

      Issues:
	None

      See Also:
	Controlled Session Loss

   3.2 Benchmark Planes

   3.2.1 Control Plane
      Definition:
	The Description of the control protocols enabled for
	the Accelerated Stress Benchmarking.

      Discussion:
	The Control Plane defines the Configuration, Startup
	Conditions, Instability Conditions, and Failure
	Conditions of the security features and control protocols used for the test.  The Security
	Control Plane includes the ACLs, Firewall,
	Secure Protocols, protocols may include routing protocols,
	multicast protocols, and User Login. MPLS protocols.  These can be
	enabled or disabled for a benchmark test.

      Measurement units:
	N/A

      Issues:
	None

      See Also:
	Benchmark Planes
	Security
	Control Plane Configuration Set
	Security
	Control Plane Startup Conditions
	Security
	Control Plane Instability Conditions
	Security
	Control Plane Failure Conditions

   3.2 Configuration Sets

  3.2.2 Data Plane

      Definition:
	The features and scaling limits data traffic profile used during for the Accelerated Stress
	Benchmarking.

      Discussion:
	There are four Configuration Sets: Control Plane Configuration Set,
	The Data Plane Configuration Set, Management Plane Configuration Set, defines the Configuration, Startup
	Conditions, Instability Conditions, and Security Failure
	Conditions of the data traffic used for the test.
	The Data Plane Configuration Set. includes the traffic and interface
	profile.

      Measurement units: Units:
        N/A

      Issues: None

      See Also:
	Control
	Benchmark Planes
	Data Plane Configuration Set
	Data Plane Configuration Set
 	Management Startup Conditions
	Data Plane Configuration Set
	Security Instability Conditions
	Data Plane Failure Conditions

   3.2.3 Management Plane

      Definition:
  	The Management features and tools used for the
	Accelerated Stress Benchmarking.

      Discussion:
	A key component of the Accelerated Stress Benchmarking is the
	Management Plane to assess manageability of the router
	under stress.  The Management Plane defines the Configuration,
	Startup Conditions, Instability Conditions, and Failure
	Conditions of the management protocols and features used for
	the test.  The Management Plane includes SNMP, Logging/Debug,
	and Statistics Collection.

      Measurement units:
        N/A

      Issues:
	None

      See Also:
	Benchmark Planes
	Management Plane Configuration Set

3.2.1
	Management Plane Startup Conditions
	Management Plane Instability Conditions
	Management Plane Failure Conditions

   3.2.4 Security Plane

      Definition:
	The Security features used during the Accelerated Stress
	Benchmarking.

      Discussion:
   	The Control Plane defines the Configuration, Startup
	Conditions, Instability Conditions, and Failure
	Conditions of the security features and protocols used for
	the test.  The Security Plane includes the ACLs, Firewall,
	Secure Protocols, and User Login.

      Measurement units:
        N/A

      Issues:
	None

      See Also:
	Benchmark Planes
	Security Plane Configuration Set
	Security Plane Startup Conditions
	Security Plane Instability Conditions
	Security Plane Failure Conditions

   3.3 Startup

   3.3.1 Startup Phase

     Definition
	The portion of the benchmarking test in which the
	Startup Conditions are generated with the DUT.  This
	begins with the attempt to establish the first session
	and ends when the last Control Plane session is
	established.

     Discussion:
	The Startup Phase is the first Phase of the
	benchmarking test preceding the Instability Phase
	and Recovery Phase.  It is specified by the
	Configuration Sets and Startup Conditions for
	each Benchmark Plane.  The Startup Phase ends
	and Instability Phase may begin when the
	Configuration Sets are achieved with the DUT.

     Measurement Units:
	None

     Issues:
	The 'last control plane session is established' may not
	be a sufficient indicator that steady-state is achieved
	and Instability Conditions can be applied to begin the
	Instability Phase.

     See Also:
	Benchmark Plane
	Configuration Sets
	Startup Conditions
	Instability Phase
	Recovery Phase

   3.3.2 Benchmarks

   3.3.2.1 Stable Aggregate Forwarding Rate

     Definition:
	Average rate of traffic forwarded by the DUT during the
	Startup Phase.

      Discussion:
	Stable Aggregate Forwarding Rate is the calculated
	average the Aggregate Forwarding Rates measured during
	the Startup Phase.  It is recommended that the
	Aggregate Forwarding Rate is measured at one-second
	intervals until the Startup Phase ends.

      Measurement units:
	pps

      Issues:
	The act of the DUT establishing the Startup Conditions
	could influence the forwarding rate in certain
	implementations so that this "baseline" for the
	remainder of the test is lowered.  The alternative is
	to change the definition of Startup Aggregate
	Forwarding Rate so that it measured during the Startup
	Phase, but after Startup Conditions are achieved.
	The disadvantage of this definition would be that it
	loses measurement of any impact establishing Startup
	Conditions would have on forwarding rate. When
	comparing the Startup Aggregate Forwarding Rate benchmark
	of two devices it is preferred to know the impact
	establishing Startup Conditions has on Forwarding Rate.
	The definition was therefore selected so that Stable
	Aggregate Forwarding Rate is calculated from measurement
	samples throughout the entire Startup Phase.

      See Also:
   	Instability Conditions
	Aggregate Forwarding Rate
	Stable Aggregate Forwarding Rate
   3.3.2.2 Stable Session Count

     Definition:
	Total number of control plane sessions/adjacencies
	established and maintained by the DUT prior to
	Instability Conditions being initiated.

      Discussion:
	This measurement should be made after the Control
	Plane Startup Conditions are applied to the DUT.

      Measurement units:
	sessions

      Issues:
	None

      See Also:
   	Instability Conditions

   3.3.3 Control Plane

   3.3.3.1 Control Plane Configuration Set
      Definition:
	The routing protocols and scaling values used for the Accelerated
	Life Benchmarking.

      Discussion:
	Control Plane Configuration Set is shown in Figure 2 and specifies
	the Routing Protocols, Multicast, and MPLS configuration.  Specific
	protocols can be enabled or disabled for a benchmark test.

      Measurement units: None
	N/A

      Issues:
	None

	 ____________ 		 ____________		 ____________
	|  Routing   |	 	|  Multicast |		|    MPLS    |
	|  Protocols |___	|  Protocols | 	      __|  Protocols |
	| 	     |   |  	| 	     |        |	| 	     |
	 ------------    |   	 ------------         |	 ------------
			 |		|	      |
			 |		|	      |
			 |  	       \/	      |
	    		 |	   ___________	      |
			 |	  |	      |	      |
			 |------->|    DUT    |<------|
				 	  |
				``|	      |
	 		 	   -----------
	Figure 2.  Control Plane Configuration Module
      See Also:
	Data Plane Configuration Set
      Management Configuration Set
      Security Configuration Set

   3.2.2

   3.3.3.2 Control Plane Startup Conditions

      Definition:
	Control Plane conditions that occur at the start
	of the Accelerated Stress Benchmarking to establish conditions
	for the remainder of the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions for the Control Plane include
	session establishment rate, number of sessions established
	and number of routes learned.

      Measurement units:
	N/A

      Issues:
	None

      See Also:
	Startup Conditions
	Security Plane Startup Conditions
	Control Plane Configuration Set

   3.3.4 Data Plane

   3.3.4.1 Data Plane Configuration Set

      Definition:
	The data traffic profile enabled for the Accelerated Stress
	Benchmarking.

      Discussion:
	Data Plane Configuration Set includes the Traffic Profile and
	interfaces used for the Accelerated Stress Benchmarking.

      Measurement Units:
        N/A

      Issues:
	None

      See Also:
	Traffic Profile
   3.2.2.1
 3.3.4.2 Traffic Profile
       Definition
	The characteristics of the Offered Load to the DUT used for
	the Accelerated Stress Benchmarking.

      Discussion
	The Traffic Profile specifies the number of packet size(s), packet
	rate per interface, number of flows, and encapsulation used for the
 	offered load to the DUT.

      Measurement Units:
      Traffic Profile is reported as follows:

	Parameter			Units
	---------			------
	Packet Size(s)			bytes
	Packet Rate(interface)		array of packets per second
	Number of Flows			number
	Encapsulation(flow)		array of encapsulation type

      Issues:
	None

      See Also:
	Data Plane Configuration Set

   3.2.3

  3.3.5 Management Plane
  3.3.5.1 Management Plane Configuration Set

      Definition:
  	The router management features enabled for the
	Accelerated Stress Test.

      Discussion:
	A key component of the Accelerated Stress Test is the Management
	Configuration Set to assess manageability of the router under
	stress.  The Management Configuration Set defines the management
	configuration of the DUT.  Features that are part of the
	Management Configuration Set include SNMP, Logging/Debug, and
	Statistics Collection, and services such as FTP, as shown in
	Figure 3.

      Measurement units:
      N/A

      Issues:
	None

      See Also:
	Control Plane Configuration Set
	Data Plane Configuration Set
      Security Plane Configuration Set
   	 	 	  ____________	      ____________
			 |  	      |	     |  Logging/  |
			 |    SNMP    |    __|   Debug    |
		   	 | 	      |   |  |      	  |
	 	          ------------    |   ------------
	      		 	|	  |
	      		 	|	  |
	       	       		\/	  |
	      	  	  ___________	  |
	      		 |	     | 	  |
	      	 	 |    DUT    |<---|
		         |	     |
	 		  -----------
			      |
			      |
			      \/
			___________
		       |   Packet  |
		       | Statistics|
		       | Collector |
		       | 	   |
	 	        -----------

		Figure 3.  Management Plane Configuration Set

   3.2.4

   3.3.6 Security Plane

   3.3.6.1 Security Plane Configuration Set

      Definition:
	Security features and scaling enabled for the Accelerated Stress
	Test.

      Discussion:
   	The Security Plane Configuration Set includes the configuration
	and scaling of ACLs, Firewall, IPsec, and User Access, as shown
	in Figure 4.

	 ____________ 	   ____________	        ____________
	|            |	  |   Secure   |       |    User    |
	|ACL/Firewall|    |  Protocol  |     __|   Access   |
	| 	     |    | 	       |    |  |            |
	 ------------      ------------     |	------------
	     |			|	    |
	     |			|	    |
	     |  	        \/	    |
	     |	       ___________	    |
	     |	    |	          | 	  |
	     |------->  |    DUT    |<--------|
			    |	          |
	 		     -----------
		Figure 4.  Security Configuration Module
      Measurement units:
        N/A

      Issues:
	None

      See Also:
  	ACL Configuration Set
  	Secure Protocol Configuration Set
  	Password Login Configuration Set

   3.3 Startup Conditions

      Definition:
	Test conditions that occur at the start of the Accelerated
	Life Benchmark to establish conditions for the remainder of
	the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions are defined for the Control
	Plane and Security Plane.

      Measurement units:
	N/A

      Issues:
	None

      See Also:
	Control Plane Startup Conditions
	Data Plane Startup Conditions
	Management Plane Startup Conditions
	Security Plane Startup Conditions

   3.3.1 Control Plane Startup Conditions

      Definition:
	Control Plane conditions that occur at the start
	of the Accelerated Stress Benchmarking to establish conditions
	for the remainder of the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions for the Control Plane include
	session establishment rate, number of sessions established
	and number of routes learned.
	     |	      |	    	  | 	    |
	     |------->|    DUT    |<--------|
		      |	          |
	 	       -----------
		Figure 4.  Security Configuration Module
      Measurement units:
        N/A

      Issues:
	None

      See Also:
	Startup Conditions
	Security Plane Startup Conditions
	Control Plane
  	ACL Configuration Set

   3.3.2
  	Secure Protocol Configuration Set
  	Password Login Configuration Set

   3.3.6.2 Security Plane Startup Conditions

      Definition:
	Security Plane conditions that occur at the start
	of the Accelerated Stress Benchmarking to establish conditions
	for the remainder of the test.

      Discussion:
	Startup Conditions may cause stress on the DUT and produce
	failure.  Startup Conditions for the Security Plane include
	session establishment rate, number of sessions established
	and number of policies learned, and number of user access
	sessions opened.

      Measurement units:
	N/A

      Issues:
	None

      See Also:
	Startup Conditions
	Data Plane Startup Conditions
	Management Plane Startup Conditions
	Security Plane Startup Conditions

   3.4 Instability Conditions

   3.4.1 Instability Phase

     Definition:
	Test conditions that occur during
	The portion of the Accelerated Stress
	Benchmark benchmarking test in which the
	Instability Conditions are offered to produce instability and stress the DUT.

     Discussion:
	The Instability Conditions are experienced by Phase is the DUT after middle Phase of
	of the
	Startup Conditions have completed.  Instability Conditions
	occur for benchmarking test following the Control Plane, Data Plane, Management Plane, Startup
	Phase and Security Plane. preceding the Recovery Phase.

     Measurement units:
	N/A Units:
	None
     Issues:
	None

     See Also:
   	Control Plane Instability Conditions
   	Data Plane Instability Conditions
 	Management Plane Instability Conditions
	Security Plane Instability Conditions

   3.4.1 Control Plane
	Instability Conditions
	Startup Phase
	Recovery Phase

   3.4.2 Benchmarks

   3.4.2.1 Unstable Aggregate Forwarding Rate

     Definition:
	Control Plane conditions that occur during
	Rate of taffic forwarded by the Accelerated Stress
	Benchmark to produce instability and stress DUT during the DUT.

      Discussion:
	Control Plane
	Instability Conditions are experienced by Phase.

      Discussion:
	Unstable Aggregated Forwarding Rate is an instantaneous
	measurement of the DUT
	after Aggregate Forwarding Rate during the Startup Conditions have completed.  Control Plane
	Instability Conditions experienced by Phase.  It is recommended that the DUT include session
	loss, route withdrawal, and route cost changes.
	Unstable Aggregate Forwarding Rate is measured at
	one-second intervals.

      Measurement units:
	N/A
	pps

      Issues:
	None

      See Also:
   	Instability Conditions
   	Data Plane Instability Conditions
 	Management Plane Instability Conditions
	Security Plane Instability Conditions

   3.4.2 Data Plane Instability Conditions
	Aggregate Forwarding Rate

   3.4.2.2 Degraded Aggregate Forwarding Rate

      Definition:
	Data Plane conditions that occur
	The reduction in Aggregate Forwarding Rate during the Accelerated Stress
	Benchmark to produce instability and stress the DUT.

      Discussion:
	Data Plane
	Instability Conditions are experienced Phase.

      Discussion:
	The Degraded Aggregate Forwarding Rate is calculated
	for each measurement of the Unstable Aggregate
	Forwarding Rate.  The Degraded Aggregate Forwarding
	Rate is calculated by subtracting each measurement
	of the DUT
	after Unstable Aggregate Forwarding Rate from the Startup Conditions have completed.  Data Plane
	Stable Aggregate Forwarding Rate, such that

	Degraded Forwarding Rate =
	Stable Aggregate Forwarding Rate -
		Unstable Aggregate Forwarding Rate

	Ideally, the Degraded Aggregate Forwarding Rate is zero.

      Measurement Units:
	pps

      Issues:
	None

      See Also:
	Instability Conditions experienced by the DUT include interface
	shutdown, link loss, and overloaded links. Phase
	Unstable Aggregate Forwarding Rate

   3.4.2.3 Average Degraded Aggregate Forwarding Rate

      Definition
	DUT Benchmark that is the calculated average of the
	obtained Degraded Forwarding Rates.

      Discussion:

      Measurement units:
	N/A Units:
	pps

      Issues:
	None

      See Also:
   	Instability Conditions
   	Control Plane Instability Conditions
 	Management Plane Instability Conditions
	Security Plane Instability Conditions
   3.4.3 Management Plane Instability Conditions
	Degraded Aggregate Forwarding Rate

   3.4.2.4 Unstable Uncontrolled Sessions Lost

     Definition:
	Management
	Control Plane conditions sessions that occur are in the down state
	but were not intentionally brought down during the Accelerated
	Life Benchmark
	Instability Phase.

      Discussion:
	The test equipment is able to produce instability and stress control protocol
	session state with the DUT.

      Discussion:
	Management Plane Instability Conditions are experienced by  The test equipment
	is also to monitor for sessions lost with the
	DUT
	after the Startup Conditions have completed.  Management Plane
	Instability Conditions experienced by which the DUT include repeated
	FTP of large files. test equipment itself did not
	intentionally bring down.

      Measurement units:
	N/A
	sessions

      Issues:
	None

      See Also:
	Controlled Session Loss
	Uncontrolled Session Loss
   3.4.3 Instability Conditions

   3.4.3.1 Control Plane Instability Conditions
   	Data Plane Instability Conditions
	Security Plane Instability Conditions

   3.4.4 Security Plane Instability Conditions

      Definition:
	Security
	Control Plane conditions that occur during the Accelerated
	Life Stress
	Benchmark to produce instability and stress the DUT.

      Discussion:
	Security
	Control Plane Instability Conditions are experienced by the DUT
	after the Startup Conditions have completed.  Security  Control Plane
	Instability Conditions experienced by the DUT include session
	loss
	loss, route withdrawal, and policy route cost changes.

      Measurement units:
	N/A

      Issues:
	None

      See Also:
   	Instability Conditions
 	Control Plane Instability Conditions
   	Data Plane Instability Conditions
 	Management Plane Instability Conditions
   3.5 Failure Conditions

   3.5.1 Unexpected Session Loss

      Definition:
	Session Loss due to Control Plane or Data Plane failure during the
	Accelerated Stress Benchmarking.

      Discussion:
	Session loss will be observed due to Convergence Events produced
	by the Instability Conditions.  This is not considered a failure
	during the Accelerated Stress Benchmarking. It is expected behavior.
	Unexpected packet loss occurs due to Data Plane or Control Plane
	failure.

      Measurement units:
	sessions

      Issues:
	None

      See Also:
	Unexpected Packet Loss
	Misrouted Packets
	Access Denial
	Errored Management Values
	Security Plane Failure

   3.5.2 Unexpected Packet Loss
      Definition:
	Packet Loss due to Control Plane or Data Plane failure during the
	Accelerated Stress Benchmarking.

      Discussion:
	Packet loss will be observed due to Convergence Events produced
	by the Instability Conditions.  This is not considered a failure
	during the Accelerated Stress Benchmarking. It is expected behavior.
	Unexpected packet loss occurs due to Data Plane or Control Plane
	failure.

      Measurement units:
	packets

      Issues:
	None

      See Also:
	Unexpected Session Loss
	Misrouted Packets
	Access Denial
	Errored Management Values Instability Conditions
	Security Plane Failure
   3.5.3 Misrouted Packets Instability Conditions

   3.4.3.2 Data Plane Instability Conditions

     Definition:
	Packets routed to the incorrect next-hop or LSP due to Control
	Data Plane failure conditions that occur during the Accelerated Stress Benchmarking.

      Discussion:
	Packets will be rerouted
	Benchmark to new next-hops produce instability and LSPs due to Convergence
	Events produced by stress the DUT.

      Discussion:
	Data Plane Instability Conditions.  This is not considered
	a failure during Conditions are experienced by the Accelerated Stress Benchmarking. It is expected
	behavior.  Unexpected Misrouted packets occur due to Control DUT
	after the Startup Conditions have completed.  Data Plane
	failure.
	Instability Conditions experienced by the DUT include interface
	shutdown, link loss, and overloaded links.

      Measurement units:
	packets
	N/A

      Issues:
	None

      See Also:
	Unexpected Packet Loss
	Unexpected Session Loss
	Access Denial
	Errored
   	Instability Conditions
   	Control Plane Instability Conditions
 	Management Values Plane Instability Conditions
	Security Plane Failure

   3.5.4 Access Denial Instability Conditions
   3.4.3.3 Management Plane Instability Conditions

     Definition:
	Inability to management the DUT due to
	Management Plane conditions that occur during the inability
	to gain remote access Accelerated
	Life Benchmark to produce instability and stress the DUT.

      Discussion:
	Handling of
	Management Plane Instability Conditions may cause a Control Plane failure
	of are experienced by the DUT externally observable as the inability to remotely manage
	after the DUT.  Whether using SNMP, Telnet, SSH, or FTP, failure to manage Startup Conditions have completed.  Management Plane
	Instability Conditions experienced by the device constitues a failure. DUT include repeated
	FTP of large files.

      Measurement units:
	Failed session
	N/A

      Issues:
	None

      See Also:
	Unexpected Packet Loss
	Unexpected Session Loss
	Misrouted Packets
	Errored Management Value
   	Instability Conditions
 	Control Plane Instability Conditions
   	Data Plane Instability Conditions
	Security Plane Failure
   3.5.5 Errored Management Value Instability Conditions

   3.4.3.4 Security Plane Instability Conditions

     Definition:
	Incorrect SNMP object value or statistic
	Security Plane conditions that occur during the Accelerated
	Life Benchmark.

      Discussion:
	Management of a router relies upon accurate reporting of object
	values Benchmark to produce instability and statistics.  The Accelerated Stress Benchmarkinging validates
	that stress the DUT.

      Discussion:
	Security Plane Instability Conditions are experienced by the DUT is able to report correct values when experiencing
	after the Startup Conditions have completed.  Security Plane
	Instability Conditions. Conditions experienced by the DUT include session
	loss and uninitiated policy changes.

      Measurement units:
	Name of incorrect object or statistic
	N/A

      Issues:
	None

      See Also:
	Unexpected Packet Loss
	Unexpected Session Loss
	Misrouted Packets
	Access Denial
	Security
   	Instability Conditions
 	Control Plane Instability Conditions
   	Data Plane Failure
	Success Threshold

   3.5.6 Security Instability Conditions
	Management Plane Failure Instability Conditions
   3.5 Recovery

   3.5.1 Recovery Phase

     Definition:
	Violation
	The portion of Security Policy for packets directed to the DUT
	or routed through benchmarking test in which the DUT.

      Discussion:
	Security Policy is configured to permit and deny access to packets
	destined for
	Startup Conditions are generated with the DUT and packets DUT, but
	the Instability Conditions are no longer offered to be routed through
	the DUT.
	A violation of this security policy is when either a packet to
	be denied

     Discussion:
	The Recovery Phase is permitted or a packet to the final Phase of the
	benchmarking test following the Startup Phase and
	Instability Phase. Startup Conditions must not be permitted is denied.
	Restarted.

     Measurement units:
	packets Units:
	None

     Issues:
	None

     See Also:
	Unexpected Packet Loss
	Unexpected Session Loss
	Misrouted Packets
	Access Denial
	Errored Management Value
   3.6
	Startup Conditions
	Startup Phase
	Instability Conditions
	Instability Phase

   3.5.2 Benchmarks

   3.6.1 Success Threshold

     Definition:
	The intended number

   3.5.2.1 Recovered Aggregate Forwarding Rate

      Definition
	Rate of hours for traffic forwarded by the DUT to operate without failure during the Accelerated Stress Benchmarking. Recovery
	Phase.

      Discussion:
	Success Threshold
	Recovered Aggregate Forwarding Rate is a user determined value an instantaneous
	measurement of time. the Aggregate Forwarding Rate during the
	Recovery Phase.  It is
	necessary to define recommended that the Success Threshold because it Recovered
	Aggregate Forwarding Rate is possible
	for measured at one-second
	intervals.  Ideally, each measurement of the Recovered
	Aggregate Forwarding Rate equals the Stable Aggregate
	Forwarding Rate because the DUT to experience zero failures for given Configuration
	Sets, Startup Conditions, and Instability Conditions
	do not exist in both the Startup and Recovery Phases.

      Measurement units:
	Hours Units:
	pps

      Issues:
	None
      See Also:
	Accelerated-Life Test Duration

   3.6.2 Accelerated-Life Test Duration

      Definition:
	Aggregate Forwarding Rate
	Recovery Phase
	Recovered Aggregate Forwarding Rate
	Startup Phase
	Stable Aggregate Forwarding Rate

   3.5.2.2 Recovery Time

      Definition
	The run-time amount of time for the Accelerated Stress Test without the DUT
	exhibiting an error in the control plane, data plane, management
	plane, or security plane.

      Discussion:
	For a successful test, the Test Duration will Recovered Aggregate Forwarding
	Rate to become equal to the Stable Aggregate Forwarding Rate.

      Discussion
	Recovery Time is measured beginning at the
	Success Threshold.  In instant the event
	Instability Phase ends until the Recovered Aggregate
	Forwarding Rate equals the Stable Aggregate Forwarding
	Rate for a minimum of any failure, 180 consecutive seconds.

      Measurement Units:
	seconds

      Issues:
	None

      See Also:
	Recovered Aggregate Forwarding  Rate
	Stable Aggregate Forwarding Rate

   3.5.2.3 Recovered Uncontrolled Control Plane Sessions Lost

     Definition:
	Control Plane sessions that are in the Test Duration down state
	but were not intentionally brought down during the
	Recovery Phase.

      Discussion:
	The test equipment is less than able to control protocol
	session state with the Success Threshold.  Reporting of DUT.  The test equipment
	is also to monitor for sessions lost with the Test
	Duration MUST inlcude
	DUT which the Failure Condition from those defined
	in this document. test equipment itself did not
	intentionally bring down.

      Measurement units:
	Hours
	sessions

      Issues:
	None

      See Also:
	Success Threshold
	Controlled Session Loss
	Uncontrolled Session Loss
   4. Security Considerations
        Documents of this type do not directly effect the security of
        the Internet or of corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.

   5. References

      [1]   Bradner, S., Editor, "Benchmarking Terminology for Network
            Interconnection Devices", RFC 1242, July 1991.

      [2]   Mandeville, R., "Benchmarking Terminology for LAN Switching
            Devices", RFC 2285, June 1998.

      [3]   Bradner, S. and McQuaid, J., "Benchmarking Methodology for
	    Network Interconnect Devices", RFC 2544, March 1999.

      [4]   "Core Router Evaluation for Higher Availability", Scott
	    Poretsky, NANOG 25, June 8, 2002, Toronto, CA.

      [5]   "Router Stress Testing to Validate Readiness for Network
	    Deployment", Scott Poretsky, IEEE CQR 2003.

      [6]   Poretsky, S. and Rao, S., "Framework for Accelerated
	    Stress Benchmarking", draft-ietf-bmwg-acc-bench-frame-01,
	    work in progress, February 2004.

      [7]   Poretsky, S. and Rao, S., "Methodology for Accelerated
	    Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-00,
	    work in progress, February 2004.

   6. Author's Address

     	Scott Poretsky
   	Quarry Technologies
  	8 New England Executive Park
   	Burlington, MA 01803
    	USA
    	Phone: + 1 781 395 5090
   	EMail: sporetsky@quarrytech.com

	Shankar Rao
	950 17th Street
	Suite 1900
	Qwest Communications
	Denver, CO 80210
	USA
	Phone: + 1 303 437 6643
	Email: srao@qwest.net

	Ray Piatt
	Cable and Wireless
	11700 Plaza America Drive
	Reston, VA 20190
	USA
	Phone: + 1 703 292 2113
	Email: rpiatt@cw.net shankar.rao@qwest.com

7.  Full Copyright Statement

        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
        copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.

   Appendix 1.  White Box Benchmarking Terminology
   Appendix 1.1

   Minimum Available Memory

      Definition:
	Minimum DUT Available Memory during the duration of the
	Accelerated Stress Test.

      Discussion:
	It is necessary to monitor DUT memory to measure this
	benchmark.

      Measurement units:
	bytes

      Issues:
	None

      See Also:
	Maximum CPU Utilization

   Appendix 1.2
   Maximum CPU Utilization

      Definition:
	Maximum DUT CPU utilization during the duration of the
	Accelerated Stress Test.

      Discussion:
	It is necessary to monitor DUT CPU Utilization to measure
	this benchmark.

      Measurement units:
	%

      Issues:
	None

      See Also:
	Minimum Available Memory