draft-ietf-bmwg-acc-bench-term-10.txt   draft-ietf-bmwg-acc-bench-term-11.txt 
Network Working Group Network Working Group
INTERNET-DRAFT INTERNET-DRAFT
Expires in: September 2007
Intended Status: Informational
Scott Poretsky Scott Poretsky
Reef Point Systems Reef Point Systems
Shankar Rao Shankar Rao
Qwest Communications Qwest Communications
October 2006
Terminology for Accelerated Stress Benchmarking Terminology for Accelerated Stress Benchmarking
<draft-ietf-bmwg-acc-bench-term-10.txt> <draft-ietf-bmwg-acc-bench-term-11.txt>
Intellectual Property Rights (IPR) statement: Intellectual Property Rights (IPR) statement:
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Status of this Memo Status of this Memo
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
skipping to change at page 1, line 40 skipping to change at page 1, line 39
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2006). Copyright (C) The IETF Trust (2007).
ABSTRACT ABSTRACT
This document provides the Terminology for performing Accelerated This document provides the Terminology for performing Accelerated
Stress Benchmarking of networking devices. The three phases of Stress Benchmarking of networking devices. The three phases of
the Stress Test: Startup, Instability and Recovery are defined the Stress Test: Startup, Instability and Recovery are defined
along with the benchmarks and configuration terms associated with along with the benchmarks and configuration terms associated with
the each phase. Also defined are the Benchmark Planes fundamental the each phase. Also defined are the Benchmark Planes fundamental
to stress testing configuration, setup and measurement. The to stress testing configuration, setup and measurement. The
terminology is to be used with the companion framework and terminology is to be used with the companion framework and
methodology documents. methodology documents.
Table of Contents Table of Contents
1. Introduction ............................................... 3 1. Introduction ............................................... 3
2. Existing definitions ....................................... 3 2. Existing definitions ....................................... 3
3. Term definitions............................................ 4 3. Term definitions............................................ 4
3.1 General Terms............................................. 4
Stress Benchmarking Stress Benchmarking
3.1 General Terms............................................. 4
3.1.1 Benchmark Planes...................................... 4 3.1.1 Benchmark Planes...................................... 4
3.1.2 Configuration Sets.................................... 5 3.1.2 Configuration Sets.................................... 5
3.1.3 Startup Conditions.................................... 5 3.1.3 Startup Conditions.................................... 5
3.1.4 Instability Conditions................................ 6 3.1.4 Instability Conditions................................ 6
3.1.5 Aggregate Forwarding Rate............................. 6 3.1.5 Aggregate Forwarding Rate............................. 6
3.1.6 Controlled Session Loss............................... 7 3.1.6 Controlled Session Loss............................... 7
3.1.7 Uncontrolled Session Loss............................. 7 3.1.7 Uncontrolled Session Loss............................. 7
3.2 Benchmark Planes.......................................... 8 3.2 Benchmark Planes.......................................... 8
3.2.1 Control Plane......................................... 8 3.2.1 Control Plane......................................... 8
3.2.2 Data Plane............................................ 8 3.2.2 Data Plane............................................ 8
skipping to change at page 3, line 45 skipping to change at page 3, line 45
for each Benchmark Plane during each Phase. Benchmarks can be for each Benchmark Plane during each Phase. Benchmarks can be
compared across multiple planes for the same DUT or at the same compared across multiple planes for the same DUT or at the same
plane for 2 or more DUTS. Benchmarks of internal DUT characteristics plane for 2 or more DUTS. Benchmarks of internal DUT characteristics
such as memory and CPU utilization (also known as White Box such as memory and CPU utilization (also known as White Box
benchmarks) are described in Appendix 1, to allow additional benchmarks) are described in Appendix 1, to allow additional
characterization of DUT behavior. The terminology is to be used with characterization of DUT behavior. The terminology is to be used with
the companion methodology document [4]. The sequence of phases, the companion methodology document [4]. The sequence of phases,
actions, and benchmarks are shown in Table 1. actions, and benchmarks are shown in Table 1.
2. Existing definitions 2. Existing definitions
RFC 1242 "Benchmarking Terminology for Network Interconnect RFC 1242 [1] and RFC 2285 [2] should be consulted before
Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching attempting to make use of this document. For the sake of clarity
Devices" should be consulted before attempting to make use of this and continuity this RFC adopts the template for definitions set
document. For the sake of clarity and continuity this RFC adopts out in Section 2 of RFC 1242. Definitions are indexed and grouped
the template for definitions set out in Section 2 of RFC 1242. together in sections for ease of reference.
Definitions are indexed and grouped together in sections for ease
of reference.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[5]. RFC 2119 defines the use of these key words to help make the [5]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this intent of standards track documents as clear as possible. While this
document uses these keywords, it is not a standards track document. document uses these keywords, it is not a standards track document.
Stress Benchmarking Stress Benchmarking
skipping to change at page 11, line 12 skipping to change at page 11, line 12
Measurement units: Measurement units:
pps pps
Stress Benchmarking Stress Benchmarking
Issues: Issues:
The act of the DUT establishing the Startup Conditions The act of the DUT establishing the Startup Conditions
could influence the forwarding rate in certain could influence the forwarding rate in certain
implementations so that this "baseline" for the implementations so that this "baseline" for the
remainder of the test is lowered. The alternative is remainder of the test is lowered. The alternative is
to change the definition of Stable Aggregate to change the definition of Stable Aggregate
Forwarding Rate so that it measured during the Startup Forwarding Rate so that it is measured during the Startup
Phase, but after Startup Conditions are achieved. Phase, but after Startup Conditions are achieved.
The disadvantage of this definition would be that it The disadvantage of this definition would be that it
loses measurement of any impact that establishing loses measurement of any impact that establishing
Startup Conditions would have on forwarding rate. When Startup Conditions would have on forwarding rate. When
comparing the Startup Aggregate Forwarding Rate benchmark comparing the Startup Aggregate Forwarding Rate benchmark
of two devices it is preferred to know the impact of two devices it is preferred to know the impact
establishing Startup Conditions has on Forwarding Rate. establishing Startup Conditions has on Forwarding Rate.
The definition was therefore selected so that Stable The definition was therefore selected so that Stable
Aggregate Forwarding Rate is calculated from measurement Aggregate Forwarding Rate is calculated from measurement
samples throughout the entire Startup Phase. samples throughout the entire Startup Phase.
skipping to change at page 24, line 24 skipping to change at page 24, line 24
See Also: See Also:
Startup Period Startup Period
Instability Period Instability Period
Recovery Period Recovery Period
4. IANA Considerations 4. IANA Considerations
This document requires no IANA considerations. This document requires no IANA considerations.
5. Security Considerations 5. Security Considerations
Documents of this type do not directly effect the security of Documents of this type do not directly affect the security of
the Internet or of corporate networks as long as benchmarking the Internet or of corporate networks as long as benchmarking
is not performed on devices or systems connected to operating is not performed on devices or systems connected to operating
networks. networks.
6. References 6. References
6.1 Normative References 6.1 Normative References
[1] Bradner, S., Editor, "Benchmarking Terminology for Network [1] Bradner, S., Editor, "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, March 1991. Interconnection Devices", RFC 1242, March 1991.
[2] Mandeville, R., "Benchmarking Terminology for LAN Switching [2] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, June 1998. Devices", RFC 2285, June 1998.
[3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[4] Poretsky, S. and Rao, S., "Methodology Guidelines for [4] Poretsky, S. and Rao, S., "Methodology Guidelines for
Accelerated Stress Benchmarking", Accelerated Stress Benchmarking",
draft-ietf-bmwg-acc-bench-meth-06, work in progress, draft-ietf-bmwg-acc-bench-meth-07, work in progress,
October 2006. March 2007.
[5] Bradner, S., "Key words for use in RFCs to Indicate [5] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, March 1997. Requirement Levels", RFC 2119, March 1997.
6.2 Informative References 6.2 Informative References
[RFC3871] RFC 3871 "Operational Security Requirements for Large [RFC3871] Jones, G., "Operational Security Requirements for Large
Internet Service Provider (ISP) IP Network Infrastructure. Internet Service Provider (ISP) IP Network Infrastructure.",
G. Jones, Ed.. IETF, September 2004. IETF RFC 3871 , September 2004.
[NANOG25] Poretsky, S., "Core Router Evaluation for Higher [NANOG25] Poretsky, S., "Core Router Evaluation for Higher
Availability", NANOG 25, June 8, 2002, Toronto, CA. Availability", NANOG 25, June 8, 2002, Toronto, CA.
[IEEECQR] Poretsky, S., "Router Stress Testing to Validate [IEEECQR] Poretsky, S., "Router Stress Testing to Validate
Readiness for Network Deployment", IEEE CQR 2003. Readiness for Network Deployment", IEEE CQR 2003.
Stress Benchmarking Stress Benchmarking
7. Author's Address 7. Author's Address
skipping to change at page 26, line 8 skipping to change at page 26, line 8
Measurement units: % Measurement units: %
Issues: None Issues: None
See Also: See Also:
Minimum Available Memory Minimum Available Memory
Stress Benchmarking Stress Benchmarking
Full Copyright Statement Full Copyright Statement
Copyright (C) The Internet Society (2006). Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors contained in BCP 78, and except as set forth therein, the authors
retain all their rights. retain all their rights.
This document and the information contained herein are provided on an This document and the information contained herein are provided
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Intellectual Property Intellectual Property
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be on the procedures with respect to rights in RFC documents can be
 End of changes. 13 change blocks. 
28 lines changed or deleted 26 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/