draft-ietf-bmwg-acc-bench-term-06.txt   draft-ietf-bmwg-acc-bench-term-07.txt 
Network Working Group Network Working Group
INTERNET-DRAFT INTERNET-DRAFT
Expires in: January 2006 Expires in: April 2006
Scott Poretsky Scott Poretsky
Reef Point Systems Reef Point Systems
Shankar Rao Shankar Rao
Qwest Communications Qwest Communications
July 2005 October 2005
Terminology for Accelerated Stress Benchmarking Terminology for Accelerated Stress Benchmarking
<draft-ietf-bmwg-acc-bench-term-06.txt> <draft-ietf-bmwg-acc-bench-term-07.txt>
Intellectual Property Rights (IPR) statement: Intellectual Property Rights (IPR) statement:
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as
Drafts. Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six months
months and may be updated, replaced, or obsoleted by other and may be updated, replaced, or obsoleted by other documents at any
documents at any time. It is inappropriate to use Internet-Drafts time. It is inappropriate to use Internet-Drafts as reference
as reference material or to cite them other than as "work in material or to cite them other than as "work in progress."
progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Copyright Notice
Copyright (C) The Internet Society (2005). All Rights Reserved.
ABSTRACT ABSTRACT
This document provides the Terminology for performing Stress This document provides the Terminology for performing Stress
Benchmarking of networking devices. The three phases of the Stress Benchmarking of networking devices. The three phases of the Stress
Test: Startup, Instability and Recovery are defined along with the Test: Startup, Instability and Recovery are defined along with the
benchmarks and configuration terms associated with the each phase. benchmarks and configuration terms associated with the each phase.
Also defined are the Benchmark Planes fundamental to stress testing Also defined are the Benchmark Planes fundamental to stress testing
configuration, setup and measurement. The terminology is to be configuration, setup and measurement. The terminology is to be
used with the companion framework and methodology documents. used with the companion framework and methodology documents.
Table of Contents Table of Contents
1. Introduction ............................................... 3 1. Introduction ............................................... 3
2. Existing definitions ....................................... 3 2. Existing definitions ....................................... 3
3. Term definitions............................................ 4 3. Term definitions............................................ 4
3.1 General Terms............................................. 4 3.1 General Terms............................................. 4
Stress Benchmarking
3.1.1 Benchmark Planes...................................... 4 3.1.1 Benchmark Planes...................................... 4
3.1.2 Configuration Sets.................................... 5 3.1.2 Configuration Sets.................................... 5
3.1.3 Startup Conditions.................................... 5 3.1.3 Startup Conditions.................................... 5
3.1.4 Instability Conditions................................ 6 3.1.4 Instability Conditions................................ 6
3.1.5 Aggregate Forwarding Rate............................. 6 3.1.5 Aggregate Forwarding Rate............................. 6
3.1.6 Controlled Session Loss............................... 7 3.1.6 Controlled Session Loss............................... 7
3.1.7 Uncontrolled Session Loss............................. 7 3.1.7 Uncontrolled Session Loss............................. 7
3.2 Benchmark Planes.......................................... 8 3.2 Benchmark Planes.......................................... 8
3.2.1 Control Plane......................................... 8 3.2.1 Control Plane......................................... 8
3.2.2 Data Plane............................................ 8 3.2.2 Data Plane............................................ 8
skipping to change at page 3, line 4 skipping to change at page 3, line 4
3.4.3.4 Security Plane Instability Conditions.............20 3.4.3.4 Security Plane Instability Conditions.............20
3.5 Recovery..................................................21 3.5 Recovery..................................................21
3.5.1 Recovery Phase........................................21 3.5.1 Recovery Phase........................................21
3.5.2 Benchmarks............................................21 3.5.2 Benchmarks............................................21
3.5.2.1 Recovered Aggregate Forwarding Rate...............21 3.5.2.1 Recovered Aggregate Forwarding Rate...............21
3.5.2.2 Recovered Latency.................................22 3.5.2.2 Recovered Latency.................................22
3.5.2.3 Recovery Time.....................................22 3.5.2.3 Recovery Time.....................................22
3.5.2.4 Recovered Uncontrolled Sessions Lost..............23 3.5.2.4 Recovered Uncontrolled Sessions Lost..............23
3.5.2.5 Variability Benchmarks............................23 3.5.2.5 Variability Benchmarks............................23
4. IANA Considerations.........................................24 4. IANA Considerations.........................................24
Stress Benchmarking
5. Security Considerations.....................................24 5. Security Considerations.....................................24
6. References..................................................24 6. References..................................................24
7. Author's Address............................................25 7. Author's Address............................................25
Appendix 1 - White Box Benchmarks..............................25 Appendix 1 - White Box Benchmarks..............................25
1. Introduction 1. Introduction
Routers in an operational network are simultaneously configured with Routers in an operational network are simultaneously configured with
multiple protocols and security policies while forwarding traffic and multiple protocols and security policies while forwarding traffic and
being managed. To accurately benchmark a router for deployment it is being managed. To accurately benchmark a router for deployment it is
skipping to change at page 4, line 5 skipping to change at page 4, line 5
of reference. of reference.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[5]. RFC 2119 defines the use of these key words to help make the [5]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track document uses these keywords, this document is not a standards track
document. document.
Stress Benchmarking
Table 1. Phase Sequence and Benchmarks Table 1. Phase Sequence and Benchmarks
III. Recovery Phase II. Instability Phase I. Startup Phase III. Recovery Phase II. Instability Phase I. Startup Phase
<-----------------<---<-------------------<----<--------------< <-----------------<---<-------------------<----<--------------<
Remove Instability Achieve Configuration Apply Startup Remove Instability Achieve Configuration Apply Startup
Conditions Set Conditions Conditions Set Conditions
Benchmark: Benchmark: Benchmark: Benchmark: Benchmark: Benchmark:
Recovered Aggregate Unstable Aggregate Stable Aggregate Recovered Aggregate Unstable Aggregate Stable Aggregate
Forwarding Rate Forwarding Rate Forwarding Rate Forwarding Rate Forwarding Rate Forwarding Rate
skipping to change at page 5, line 4 skipping to change at page 5, line 4
N/A N/A
Issues: Issues:
None None
See Also: See Also:
Control Plane Control Plane
Data Plane Data Plane
Management Plane Management Plane
Security Plane Security Plane
Stress Benchmarking
___________ ___________ ___________ ___________
| Control | | Management| | Control | | Management|
| Plane |___ ___| Plane | | Plane |___ ___| Plane |
| | | | | | | | | | | |
----------- | | ----------- ----------- | | -----------
\/ \/ ___________ \/ \/ ___________
___________ | Security | ___________ | Security |
| |<-----------| Plane | | |<-----------| Plane |
| DUT | | | | DUT | | |
|--->| |<---| ----------- |--->| |<---| -----------
skipping to change at page 6, line 5 skipping to change at page 6, line 5
Management Plane Configuration Set Management Plane Configuration Set
Security Plane Configuration Set Security Plane Configuration Set
3.1.3 Startup Conditions 3.1.3 Startup Conditions
Definition: Definition:
Test conditions that occur at the start of the Accelerated Test conditions that occur at the start of the Accelerated
Life Benchmark to establish conditions for the remainder of Life Benchmark to establish conditions for the remainder of
the test. the test.
Stress Benchmarking
Discussion: Discussion:
Startup Conditions may cause stress on the DUT and produce Startup Conditions may cause stress on the DUT and produce
failure. Startup Conditions are defined for the Control failure. Startup Conditions are defined for the Control
Plane and Security Plane. Plane and Security Plane.
Measurement units: Measurement units:
N/A N/A
Issues: Issues:
None None
skipping to change at page 7, line 5 skipping to change at page 7, line 5
Definition: Definition:
Sum of forwarding rates for all interfaces on the Sum of forwarding rates for all interfaces on the
DUT during the Startup Phase. DUT during the Startup Phase.
Discussion: Discussion:
Each interface of the DUT forwards traffic at some Each interface of the DUT forwards traffic at some
measured rate. The Aggregate Forwarding Rate is the measured rate. The Aggregate Forwarding Rate is the
sum of forwarding rates for all interfaces on the DUT. sum of forwarding rates for all interfaces on the DUT.
Stress Benchmarking
Measurement units: Measurement units:
pps pps
Issues: Issues:
None None
See Also: See Also:
Startup Phase Startup Phase
3.1.6 Controlled Session Loss 3.1.6 Controlled Session Loss
skipping to change at page 8, line 4 skipping to change at page 8, line 4
intentionally bring down. intentionally bring down.
Measurement units: Measurement units:
N/A N/A
Issues: Issues:
None None
See Also: See Also:
Controlled Session Loss Controlled Session Loss
Stress Benchmarking
3.2 Benchmark Planes 3.2 Benchmark Planes
3.2.1 Control Plane 3.2.1 Control Plane
Definition: Definition:
The Description of the control protocols enabled for The Description of the control protocols enabled for
the Accelerated Stress Benchmarking. the Accelerated Stress Benchmarking.
Discussion: Discussion:
The Control Plane defines the Configuration, Startup The Control Plane defines the Configuration, Startup
Conditions, and Instability Conditions of the control Conditions, and Instability Conditions of the control
skipping to change at page 9, line 5 skipping to change at page 9, line 5
Data Plane Configuration Set Data Plane Configuration Set
Data Plane Startup Conditions Data Plane Startup Conditions
Data Plane Instability Conditions Data Plane Instability Conditions
3.2.3 Management Plane 3.2.3 Management Plane
Definition: Definition:
The Management features and tools used for the The Management features and tools used for the
Accelerated Stress Benchmarking. Accelerated Stress Benchmarking.
Stress Benchmarking
Discussion: Discussion:
A key component of the Accelerated Stress Benchmarking is the A key component of the Accelerated Stress Benchmarking is the
Management Plane to assess manageability of the router Management Plane to assess manageability of the router
under stress. The Management Plane defines the Configuration, under stress. The Management Plane defines the Configuration,
Startup Conditions, and Instability Conditions of the Startup Conditions, and Instability Conditions of the
management protocols and features. The Management Plane management protocols and features. The Management Plane
includes SNMP, Logging/Debug, Statistics Collection, and includes SNMP, Logging/Debug, Statistics Collection, and
management configuration sessions such as telnet, SSH, and management configuration sessions such as telnet, SSH, and
serial console. SNMP Gets SHOULD be performed continuously. serial console. SNMP Gets SHOULD be performed continuously.
Management configuration sessions should be open Management configuration sessions should be open
skipping to change at page 10, line 4 skipping to change at page 10, line 4
Measurement units: Measurement units:
N/A N/A
Issues: None Issues: None
See Also: See Also:
Benchmark Planes Benchmark Planes
Security Plane Configuration Set Security Plane Configuration Set
Security Plane Startup Conditions Security Plane Startup Conditions
Security Plane Instability Conditions Security Plane Instability Conditions
Stress Benchmarking
3.3 Startup 3.3 Startup
3.3.1 Startup Phase 3.3.1 Startup Phase
Definition Definition
The portion of the benchmarking test in which the The portion of the benchmarking test in which the
Startup Conditions are generated with the DUT. This Startup Conditions are generated with the DUT. This
begins with the attempt to establish the first session begins with the attempt to establish the first session
and ends when the last Control Plane session is and ends when the last Control Plane session is
established. established.
skipping to change at page 11, line 4 skipping to change at page 11, line 4
Average rate of traffic forwarded by the DUT during the Average rate of traffic forwarded by the DUT during the
Startup Phase. Startup Phase.
Discussion: Discussion:
Stable Aggregate Forwarding Rate is the calculated Stable Aggregate Forwarding Rate is the calculated
average of the Aggregate Forwarding Rates measured during average of the Aggregate Forwarding Rates measured during
the Startup Phase. the Startup Phase.
Measurement units: Measurement units:
pps pps
Stress Benchmarking
Issues: Issues:
The act of the DUT establishing the Startup Conditions The act of the DUT establishing the Startup Conditions
could influence the forwarding rate in certain could influence the forwarding rate in certain
implementations so that this "baseline" for the implementations so that this "baseline" for the
remainder of the test is lowered. The alternative is remainder of the test is lowered. The alternative is
to change the definition of Stable Aggregate to change the definition of Stable Aggregate
Forwarding Rate so that it measured during the Startup Forwarding Rate so that it measured during the Startup
Phase, but after Startup Conditions are achieved. Phase, but after Startup Conditions are achieved.
The disadvantage of this definition would be that it The disadvantage of this definition would be that it
loses measurement of any impact that establishing loses measurement of any impact that establishing
skipping to change at page 12, line 5 skipping to change at page 12, line 5
Definition: Definition:
Total number of control plane sessions/adjacencies Total number of control plane sessions/adjacencies
established and maintained by the DUT during the Startup established and maintained by the DUT during the Startup
Phase and prior to Instability Conditions being initiated. Phase and prior to Instability Conditions being initiated.
Discussion: Discussion:
This measurement should be made after the Control This measurement should be made after the Control
Plane Startup Conditions are applied to the DUT. Plane Startup Conditions are applied to the DUT.
Stress Benchmarking
Measurement units: Measurement units:
sessions sessions
Issues: Issues:
None None
See Also: See Also:
Startup Phase Startup Phase
3.3.3 Control Plane 3.3.3 Control Plane
skipping to change at page 13, line 4 skipping to change at page 13, line 4
------------ | ------------ | ------------ ------------ | ------------ | ------------
| | | | | |
| | | | | |
| \/ | | \/ |
| ___________ | | ___________ |
| | | | | | | |
|------->| DUT |<------| |------->| DUT |<------|
``| | ``| |
----------- -----------
Figure 2. Control Plane Configuration Module Figure 2. Control Plane Configuration Module
Stress Benchmarking
3.3.3.2 Control Plane Startup Conditions 3.3.3.2 Control Plane Startup Conditions
Definition: Definition:
Control Plane conditions that occur at the start Control Plane conditions that occur at the start
of the Accelerated Stress Benchmarking to establish conditions of the Accelerated Stress Benchmarking to establish conditions
for the remainder of the test. for the remainder of the test.
Discussion: Discussion:
Startup Conditions may cause stress on the DUT and produce Startup Conditions may cause stress on the DUT and produce
failure. Startup Conditions for the Control Plane include failure. Startup Conditions for the Control Plane include
skipping to change at page 14, line 5 skipping to change at page 14, line 5
3.3.4.2 Traffic Profile 3.3.4.2 Traffic Profile
Definition Definition
The characteristics of the Offered Load to the DUT used for The characteristics of the Offered Load to the DUT used for
the Accelerated Stress Benchmarking. the Accelerated Stress Benchmarking.
Discussion Discussion
The Traffic Profile specifies the number of packet size(s), packet The Traffic Profile specifies the number of packet size(s), packet
rate per interface, number of flows, and encapsulation used for the rate per interface, number of flows, and encapsulation used for the
offered load to the DUT. offered load to the DUT.
Stress Benchmarking
Measurement Units: Measurement Units:
Traffic Profile is reported as follows: Traffic Profile is reported as follows:
Parameter Units Parameter Units
--------- ------ --------- ------
Packet Size(s) bytes Packet Size(s) bytes
Packet Rate(interface) array of packets per second Packet Rate(interface) array of packets per second
Number of Flows number Number of Flows number
Encapsulation(flow) array of encapsulation type Encapsulation(flow) array of encapsulation type
skipping to change at page 15, line 4 skipping to change at page 15, line 4
Measurement units: Measurement units:
N/A N/A
Issues: Issues:
None None
See Also: See Also:
Control Plane Configuration Set Control Plane Configuration Set
Data Plane Configuration Set Data Plane Configuration Set
Security Plane Configuration Set Security Plane Configuration Set
Stress Benchmarking
____________ ____________ ____________ ____________
| | | Logging/ | | | | Logging/ |
| SNMP | __| Debug | | SNMP | __| Debug |
| | | | | | | | | |
------------ | ------------ ------------ | ------------
| | | |
| | | |
\/ | \/ |
___________ | ___________ |
| | | | | |
skipping to change at page 16, line 4 skipping to change at page 16, line 4
------------ ------------ | ------------ ------------ ------------ | ------------
| | | | | |
| | | | | |
| \/ | | \/ |
| ___________ | | ___________ |
| | | | | | | |
|------->| DUT |<--------| |------->| DUT |<--------|
| | | |
----------- -----------
Figure 4. Security Configuration Module Figure 4. Security Configuration Module
Stress Benchmarking
Measurement units: Measurement units:
N/A N/A
Issues: Issues:
None None
See Also: See Also:
ACL Configuration Set ACL Configuration Set
Secure Protocol Configuration Set Secure Protocol Configuration Set
Password Login Configuration Set Password Login Configuration Set
skipping to change at page 17, line 4 skipping to change at page 17, line 4
The portion of the benchmarking test in which the The portion of the benchmarking test in which the
Instability Conditions are offered to the DUT. Instability Conditions are offered to the DUT.
Discussion: Discussion:
The Instability Phase is the middle Phase of The Instability Phase is the middle Phase of
of the benchmarking test following the Startup of the benchmarking test following the Startup
Phase and preceding the Recovery Phase. Phase and preceding the Recovery Phase.
Measurement Units: Measurement Units:
None None
Stress Benchmarking
Issues: Issues:
None None
See Also: See Also:
Instability Conditions Instability Conditions
Startup Phase Startup Phase
Recovery Phase Recovery Phase
3.4.2 Benchmarks 3.4.2 Benchmarks
3.4.2.1 Unstable Aggregate Forwarding Rate 3.4.2.1 Unstable Aggregate Forwarding Rate
skipping to change at page 18, line 4 skipping to change at page 18, line 4
Stable Aggregate Forwarding Rate, such that Stable Aggregate Forwarding Rate, such that
Degraded Forwarding Rate = Degraded Forwarding Rate =
Stable Aggregate Forwarding Rate - Stable Aggregate Forwarding Rate -
Unstable Aggregate Forwarding Rate Unstable Aggregate Forwarding Rate
Ideally, the Degraded Aggregate Forwarding Rate is zero. Ideally, the Degraded Aggregate Forwarding Rate is zero.
Measurement Units: Measurement Units:
pps pps
Stress Benchmarking
Issues: Issues:
None None
See Also: See Also:
Instability Phase Instability Phase
Unstable Aggregate Forwarding Rate Unstable Aggregate Forwarding Rate
3.4.2.3 Average Degraded Aggregate Forwarding Rate 3.4.2.3 Average Degraded Aggregate Forwarding Rate
Definition Definition
skipping to change at page 19, line 4 skipping to change at page 19, line 4
Measurement units: Measurement units:
seconds seconds
Issues: Issues:
None None
See Also: See Also:
Instability Phase Instability Phase
Stable Latency Stable Latency
Stress Benchmarking
3.4.2.5 Unstable Uncontrolled Sessions Lost 3.4.2.5 Unstable Uncontrolled Sessions Lost
Definition: Definition:
Control Plane sessions that are in the down state Control Plane sessions that are in the down state
but were not intentionally brought down during the but were not intentionally brought down during the
Instability Phase. Instability Phase.
Discussion: Discussion:
The test equipment is able to control protocol The test equipment is able to control protocol
session state with the DUT. The test equipment session state with the DUT. The test equipment
skipping to change at page 20, line 4 skipping to change at page 20, line 4
N/A N/A
Issues: Issues:
None None
See Also: See Also:
Instability Conditions Instability Conditions
Data Plane Instability Conditions Data Plane Instability Conditions
Management Plane Instability Conditions Management Plane Instability Conditions
Security Plane Instability Conditions Security Plane Instability Conditions
Stress Benchmarking
3.4.3.2 Data Plane Instability Conditions 3.4.3.2 Data Plane Instability Conditions
Definition: Definition:
Data Plane conditions that occur during the Accelerated Stress Data Plane conditions that occur during the Accelerated Stress
Benchmark to produce instability and stress the DUT. Benchmark to produce instability and stress the DUT.
Discussion: Discussion:
Data Plane Instability Conditions are experienced by the DUT Data Plane Instability Conditions are experienced by the DUT
after the Startup Conditions have completed. Data Plane after the Startup Conditions have completed. Data Plane
Instability Conditions experienced by the DUT include interface Instability Conditions experienced by the DUT include interface
shutdown, link loss, and overloaded links. shutdown, link loss, and overloaded links.
skipping to change at page 21, line 5 skipping to change at page 21, line 5
Control Plane Instability Conditions Control Plane Instability Conditions
Data Plane Instability Conditions Data Plane Instability Conditions
Security Plane Instability Conditions Security Plane Instability Conditions
3.4.3.4 Security Plane Instability Conditions 3.4.3.4 Security Plane Instability Conditions
Definition: Definition:
Security Plane conditions that occur during the Accelerated Security Plane conditions that occur during the Accelerated
Life Benchmark to produce instability and stress the DUT. Life Benchmark to produce instability and stress the DUT.
Stress Benchmarking
Discussion: Discussion:
Security Plane Instability Conditions are experienced by the DUT Security Plane Instability Conditions are experienced by the DUT
after the Startup Conditions have completed. Security Plane after the Startup Conditions have completed. Security Plane
Instability Conditions experienced by the DUT include session Instability Conditions experienced by the DUT include session
loss and uninitiated policy changes. loss and uninitiated policy changes.
Measurement units: Measurement units:
N/A N/A
Issues: Issues:
skipping to change at page 22, line 5 skipping to change at page 22, line 5
Instability Conditions Instability Conditions
Instability Phase Instability Phase
3.5.2 Benchmarks 3.5.2 Benchmarks
3.5.2.1 Recovered Aggregate Forwarding Rate 3.5.2.1 Recovered Aggregate Forwarding Rate
Definition Definition
Rate of traffic forwarded by the DUT during the Recovery Rate of traffic forwarded by the DUT during the Recovery
Phase. Phase.
Stress Benchmarking
Discussion: Discussion:
Recovered Aggregate Forwarding Rate is an instantaneous Recovered Aggregate Forwarding Rate is an instantaneous
measurement of the Aggregate Forwarding Rate during the measurement of the Aggregate Forwarding Rate during the
Recovery Phase. Ideally, each measurement of the Recovered Recovery Phase. Ideally, each measurement of the Recovered
Aggregate Forwarding Rate equals the Stable Aggregate Aggregate Forwarding Rate equals the Stable Aggregate
Forwarding Rate because the Instability Conditions Forwarding Rate because the Instability Conditions
do not exist in both the Startup and Recovery Phases. do not exist in both the Startup and Recovery Phases.
Measurement Units: Measurement Units:
pps pps
skipping to change at page 23, line 5 skipping to change at page 23, line 5
See Also: See Also:
Recovery Phase Recovery Phase
Stable Latency Stable Latency
3.5.2.3 Recovery Time 3.5.2.3 Recovery Time
Definition Definition
The amount of time for the Recovered Aggregate Forwarding The amount of time for the Recovered Aggregate Forwarding
Rate to become equal to the Stable Aggregate Forwarding Rate. Rate to become equal to the Stable Aggregate Forwarding Rate.
Stress Benchmarking
Discussion Discussion
Recovery Time is measured beginning at the instant the Recovery Time is measured beginning at the instant the
Instability Phase ends until the Recovered Aggregate Instability Phase ends until the Recovered Aggregate
Forwarding Rate equals the Stable Aggregate Forwarding Forwarding Rate equals the Stable Aggregate Forwarding
Rate for a minimum duration of 180 consecutive seconds. Rate for a minimum duration of 180 consecutive seconds.
Measurement Units: Measurement Units:
seconds seconds
Issues: Issues:
skipping to change at page 24, line 5 skipping to change at page 24, line 5
The difference between the measured Benchmarks of the The difference between the measured Benchmarks of the
same DUT over multiple iterations. same DUT over multiple iterations.
Discussion: Discussion:
Ideally, the measured benchmarks should be the same for multiple Ideally, the measured benchmarks should be the same for multiple
iterations with the same DUT. Configuration Sets and Instability iterations with the same DUT. Configuration Sets and Instability
Conditions SHOULD be held constant for this benchmark. Whether the Conditions SHOULD be held constant for this benchmark. Whether the
DUT can exhibit such predictable and repeatable behavior is an DUT can exhibit such predictable and repeatable behavior is an
important benchmark in itself. important benchmark in itself.
Stress Benchmarking
Measurement units: Measurement units:
As applicable to each Benchmark. The results are to be As applicable to each Benchmark. The results are to be
presented in a table format for successive Iterations. presented in a table format for successive Iterations.
Ideally, the differences should be zero. Ideally, the differences should be zero.
Issues: Issues:
None None
See Also: See Also:
Startup Period Startup Period
skipping to change at page 24, line 39 skipping to change at page 24, line 41
[1] Bradner, S., Editor, "Benchmarking Terminology for Network [1] Bradner, S., Editor, "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, October 1991. Interconnection Devices", RFC 1242, October 1991.
[2] Mandeville, R., "Benchmarking Terminology for LAN Switching [2] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, June 1998. Devices", RFC 2285, June 1998.
[3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[4] Poretsky, S. and Rao, S., "Methodology for Accelerated [4] Poretsky, S. and Rao, S., "Methodology for Accelerated
Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-03, Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-04,
work in progress, July 2005. work in progress, October 2005.
[5] Bradner, S., "Key words for use in RFCs to Indicate Requirement [5] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", RFC 2119, March 1997. Levels", RFC 2119, March 1997.
6.2 Informative References 6.2 Informative References
[RFC3871] RFC 3871 "Operational Security Requirements for Large [RFC3871] RFC 3871 "Operational Security Requirements for Large
Internet Service Provider (ISP) IP Network Infrastructure. Internet Service Provider (ISP) IP Network Infrastructure.
G. Jones, Ed.. IETF, September 2004. G. Jones, Ed.. IETF, September 2004.
[NANOG25] "Core Router Evaluation for Higher Availability", Scott [NANOG25] "Core Router Evaluation for Higher Availability", Scott
Poretsky, NANOG 25, June 8, 2002, Toronto, CA. Poretsky, NANOG 25, June 8, 2002, Toronto, CA.
[IEEECQR] "Router Stress Testing to Validate Readiness for Network [IEEECQR] "Router Stress Testing to Validate Readiness for Network
Deployment", Scott Poretsky, IEEE CQR 2003. Deployment", Scott Poretsky, IEEE CQR 2003.
Stress Benchmarking
7. Author's Address 7. Author's Address
Reef Point Systems Reef Point Systems
8 New England Executive Park 8 New England Executive Park
Burlington, MA 01803 Burlington, MA 01803
USA USA
Phone: + 1 781 395 5090 Phone: + 1 781 395 5090
EMail: sporetsky@reefpoint.com EMail: sporetsky@reefpoint.com
Shankar Rao Shankar Rao
skipping to change at page 26, line 4 skipping to change at page 26, line 4
Discussion: Discussion:
It is necessary to monitor DUT CPU Utilization to measure It is necessary to monitor DUT CPU Utilization to measure
this benchmark. this benchmark.
Measurement units: % Measurement units: %
Issues: None Issues: None
See Also: See Also:
Minimum Available Memory Minimum Available Memory
Stress Benchmarking
Intellectual Property Statement Full Copyright Statement
The IETF takes no position regarding the validity or scope of any Intel- Copyright (C) The Internet Society (2005).
lectual Property Rights or other rights that might be claimed to pertain
to the implementation or use of the technology described in this docu- This document is subject to the rights, licenses and restrictions
ment or the extent to which any license under such rights might or might contained in BCP 78, and except as set forth therein, the authors
not be available; nor does it represent that it has made any independent retain all their rights.
effort to identify any such rights. Information on the procedures with
respect to rights in RFC documents can be found in BCP 78 and BCP 79. This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an attempt assurances of licenses to be made available, or the result of an
made to obtain a general license or permission for the use of such attempt made to obtain a general license or permission for the use of
proprietary rights by implementers or users of this specification can be such proprietary rights by implementers or users of this
obtained from the IETF on-line IPR repository at specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr. http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary rights copyrights, patents or patent applications, or other proprietary
that may cover technology that may be required to implement this stan- rights that may cover technology that may be required to implement
dard. Please address the information to the IETF at ietf-ipr@ietf.org. this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
Disclaimer of Warranty
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR
IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMA-
TION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement Acknowledgement
Copyright (C) The Internet Society (2005). This document is subject to Funding for the RFC Editor function is currently provided by the
the rights, licenses and restrictions contained in BCP 78, and except as Internet Society.
set forth therein, the authors retain all their rights.
 End of changes. 40 change blocks. 
41 lines changed or deleted 98 lines changed or added

This html diff was produced by rfcdiff 1.27, available from http://www.levkowetz.com/ietf/tools/rfcdiff/