draft-ietf-bmwg-dcbench-terminology-10.txt   draft-ietf-bmwg-dcbench-terminology-11.txt 
Internet Engineering Task Force L. Avramov Internet Engineering Task Force L. Avramov
INTERNET-DRAFT, Intended status: Informational Google INTERNET-DRAFT, Intended status: Informational Google
Expires: November 26,2017 J. Rapp Expires: December 17,2017 J. Rapp
June 9, 2017 VMware June 15, 2017 VMware
Data Center Benchmarking Terminology Data Center Benchmarking Terminology
draft-ietf-bmwg-dcbench-terminology-10 draft-ietf-bmwg-dcbench-terminology-11
Abstract Abstract
The purpose of this informational document is to establish definitions The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center. benchmarking network switches and routers in the data center. The
terminologies are not only data center specific and can be seen as
updates to [RFC1242], [RFC2432], [RFC2544], [RFC2889] and [RFC3918],
without the intent to cover the all use cases outside of data center.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the provisions This Internet-Draft is submitted in full conformance with the provisions
of BCP 78 and BCP 79. of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF). Note that other groups may also distribute working Force (IETF). Note that other groups may also distribute working
documents as Internet-Drafts. The list of current Internet-Drafts is at documents as Internet-Drafts. The list of current Internet-Drafts is at
http://datatracker.ietf.org/drafts/current. http://datatracker.ietf.org/drafts/current.
skipping to change at page 2, line 9 skipping to change at page 2, line 13
Relating to IETF Documents (http://trustee.ietf.org/license-info) in Relating to IETF Documents (http://trustee.ietf.org/license-info) in
effect on the date of publication of this document. Please review these effect on the date of publication of this document. Please review these
documents carefully, as they describe your rights and restrictions with documents carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this document respect to this document. Code Components extracted from this document
must include Simplified BSD License text as described in Section 4.e of must include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4
1.2. Definition format . . . . . . . . . . . . . . . . . . . . . 4 1.2. Definition format . . . . . . . . . . . . . . . . . . . . . 4
2. Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 6 2.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 6
3 Jitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 Jitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 7 3.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 7
skipping to change at page 3, line 14 skipping to change at page 3, line 19
Traffic patterns in the data center are not uniform and are Traffic patterns in the data center are not uniform and are
constantly changing. They are dictated by the nature and variety of constantly changing. They are dictated by the nature and variety of
applications utilized in the data center. It can be largely east-west applications utilized in the data center. It can be largely east-west
traffic flows in one data center and north-south in another, while traffic flows in one data center and north-south in another, while
some may combine both. Traffic patterns can be bursty in nature and some may combine both. Traffic patterns can be bursty in nature and
contain many-to-one, many-to-many, or one-to-many flows. Each flow contain many-to-one, many-to-many, or one-to-many flows. Each flow
may also be small and latency sensitive or large and throughput may also be small and latency sensitive or large and throughput
sensitive while containing a mix of UDP and TCP traffic. One or more sensitive while containing a mix of UDP and TCP traffic. One or more
of these may coexist in a single cluster and flow through a single of these may coexist in a single cluster and flow through a single
network device simultaneously. Benchmarking of network devices have network device simultaneously. Benchmarking of network devices have
long used [RFC1242], [RFC2432], [RFC2544], [2] and [3]. These long used [RFC1242], [RFC2432], [RFC2544], [RFC2889] and [RFC3918].
benchmarks have largely been focused around various latency These benchmarks have largely been focused around various latency
attributes and max throughput of the Device Under Test being attributes and max throughput of the Device Under Test being
benchmarked. These standards are good at measuring theoretical max benchmarked. These standards are good at measuring theoretical max
throughput, forwarding rates and latency under testing conditions, throughput, forwarding rates and latency under testing conditions,
but they do not represent real traffic patterns that may affect these but they do not represent real traffic patterns that may affect these
networking devices. The data center networking devices covered are networking devices. The data center networking devices covered are
switches and routers. switches and routers.
The following document defines a set of definitions, metrics and The following document defines a set of definitions, metrics and
terminologies including congestion scenarios, switch buffer analysis terminologies including congestion scenarios, switch buffer analysis
and redefines basic definitions in order to represent a wide mix of and redefines basic definitions in order to represent a wide mix of
skipping to change at page 10, line 44 skipping to change at page 10, line 44
6.1 Buffer 6.1 Buffer
6.1.1 Definition 6.1.1 Definition
Buffer Size: The term buffer size represents the total amount of Buffer Size: The term buffer size represents the total amount of
frame buffering memory available on a DUT. This size is expressed in frame buffering memory available on a DUT. This size is expressed in
B (bytes); KB (kilobytes), MB (megabytes) or GB (gigabyte). When the B (bytes); KB (kilobytes), MB (megabytes) or GB (gigabyte). When the
buffer size is expressed it SHOULD be defined by a size metric stated buffer size is expressed it SHOULD be defined by a size metric stated
above. When the buffer size is expressed, an indication of the frame above. When the buffer size is expressed, an indication of the frame
MTU used for that measurement is also necessary as well as the cos or MTU used for that measurement is also necessary as well as the cos
dscp value set; as often times the buffers are carved by quality of (class of service) or dscp (differentiated services code point) value
service implementation. Please refer to the buffer efficiency section set; as often times the buffers are carved by quality of service
for further details. implementation. Please refer to the buffer efficiency section for
further details.
Example: Buffer Size of DUT when sending 1518 bytes frames is 18 Mb. Example: Buffer Size of DUT when sending 1518 bytes frames is 18 Mb.
Port Buffer Size: The port buffer size is the amount of buffer for a Port Buffer Size: The port buffer size is the amount of buffer for a
single ingress port, egress port or combination of ingress and egress single ingress port, egress port or combination of ingress and egress
buffering location for a single port. The reason for mentioning the buffering location for a single port. The reason for mentioning the
three locations for the port buffer is because the DUT buffering three locations for the port buffer is because the DUT buffering
scheme can be unknown or untested, and so knowing the buffer location scheme can be unknown or untested, and so knowing the buffer location
helps clarify the buffer architecture and consequently the total helps clarify the buffer architecture and consequently the total
buffer size. The Port Buffer Size is an informational value that MAY buffer size. The Port Buffer Size is an informational value that MAY
skipping to change at page 15, line 51 skipping to change at page 15, line 51
9. IANA Considerations 9. IANA Considerations
NO IANA Action is requested at this time. NO IANA Action is requested at this time.
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC1242] Bradner, S. "Benchmarking Terminology for Network [RFC1242] Bradner, S. "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, July 1991. Interconnection Devices", RFC 1242, July 1991, <http://www.rfc-
editor.org/info/rfc1242>
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999,
<http://www.rfc-editor.org/info/rfc2554>
10.2. Informative References 10.2. Informative References
[1] Avramov L. and Rapp J., "Data Center Benchmarking Methodology", [1] Avramov L. and Rapp J., "Data Center Benchmarking Methodology",
April 2017. April 2017.
[2] Mandeville R. and Perser J., "Benchmarking Methodology for [RFC2889] Mandeville R. and Perser J., "Benchmarking
LAN Switching Devices", RFC 2889, August 2000. Methodology for LAN Switching Devices", RFC 2889, August 2000,
<http://www.rfc-editor.org/info/rfc2889>
[3] Stopp D. and Hickman B., "Methodology for IP Multicast [RFC3918] Stopp D. and Hickman B., "Methodology for IP Multicast
Benchmarking", RFC 3918, October 2004. Benchmarking", RFC 3918, October 2004, <http://www.rfc-
editor.org/info/rfc3918>
[4] Yanpei Chen, Rean Griffith, Junda Liu, Randy H. Katz, Anthony D. [4] Yanpei Chen, Rean Griffith, Junda Liu, Randy H. Katz, Anthony D.
Joseph, "Understanding TCP Incast Throughput Collapse in Joseph, "Understanding TCP Incast Throughput Collapse in
Datacenter Networks, Datacenter Networks,
"http://yanpeichen.com/professional/usenixLoginIncastReady.pdf" "http://yanpeichen.com/professional/usenixLoginIncastReady.pdf"
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119,
March 1997, <http://www.rfc-editor.org/info/rfc2119> March 1997, <http://www.rfc-editor.org/info/rfc2119>
skipping to change at page 16, line 41 skipping to change at page 16, line 44
10.3. Acknowledgments 10.3. Acknowledgments
The authors would like to thank Alfred Morton, Scott Bradner, The authors would like to thank Alfred Morton, Scott Bradner,
Ian Cox, Tim Stevenson for their reviews and feedback. Ian Cox, Tim Stevenson for their reviews and feedback.
Authors' Addresses Authors' Addresses
Lucien Avramov Lucien Avramov
Google Google
170 West Tasman drive 1600 Amphitheatre Parkway
Mountain View, CA 94043 Mountain View, CA 94043
United States United States
Email: lucienav@google.com Email: lucienav@google.com
Jacob Rapp Jacob Rapp
VMware VMware
3401 Hillview Ave 3401 Hillview Ave
Palo Alto, CA 94304 Palo Alto, CA 94304
United States United States
Phone: +1 650 857 3367 Phone: +1 650 857 3367
Email: jrapp@vmware.com Email: jrapp@vmware.com
 End of changes. 12 change blocks. 
19 lines changed or deleted 26 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/