draft-ietf-bmwg-dcbench-terminology-17.txt   draft-ietf-bmwg-dcbench-terminology-18.txt 
Internet Engineering Task Force L. Avramov Internet Engineering Task Force L. Avramov
INTERNET-DRAFT, Intended status: Informational Google INTERNET-DRAFT, Intended status: Informational Google
Expires: December 23,2017 J. Rapp Expires: December 23,2017 J. Rapp
June 21, 2017 VMware June 21, 2017 VMware
Data Center Benchmarking Terminology Data Center Benchmarking Terminology
draft-ietf-bmwg-dcbench-terminology-17 draft-ietf-bmwg-dcbench-terminology-18
Abstract Abstract
The purpose of this informational document is to establish definitions The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to performance well as it is to introduce new terminologies applicable to performance
evaluations of data center network equipment. This document establishes evaluations of data center network equipment. This document establishes
the important concepts for benchmarking network switches and routers in the important concepts for benchmarking network switches and routers in
the data center and, is a pre-requisite to the test methodology the data center and, is a pre-requisite to the test methodology
publication [1]. Many of these terms and methods may be applicable to publication [draft-ietf-bmwg-dcbench-methodology]. Many of these terms
network equipment beyond this publication's scope as the technologies and methods may be applicable to network equipment beyond this
originally applied in the data center are deployed elsewhere. publication's scope as the technologies originally applied in the data
center are deployed elsewhere.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the provisions This Internet-Draft is submitted in full conformance with the provisions
of BCP 78 and BCP 79. of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF). Note that other groups may also distribute working Force (IETF). Note that other groups may also distribute working
documents as Internet-Drafts. The list of current Internet-Drafts is at documents as Internet-Drafts. The list of current Internet-Drafts is at
http://datatracker.ietf.org/drafts/current. http://datatracker.ietf.org/drafts/current.
skipping to change at page 2, line 44 skipping to change at page 2, line 45
6.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 12 6.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 12
6.1.3 Measurement Units . . . . . . . . . . . . . . . . . . . 12 6.1.3 Measurement Units . . . . . . . . . . . . . . . . . . . 12
6.2 Incast . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6.2 Incast . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 13 6.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 13
6.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 14 6.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 14
6.2.3 Measurement Units . . . . . . . . . . . . . . . . . . . 14 6.2.3 Measurement Units . . . . . . . . . . . . . . . . . . . 14
7 Application Throughput: Data Center Goodput . . . . . . . . . . 14 7 Application Throughput: Data Center Goodput . . . . . . . . . . 14
7.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 14 7.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 14
7.2. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 14 7.2. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 14
7.3. Measurement Units . . . . . . . . . . . . . . . . . . . . . 15 7.3. Measurement Units . . . . . . . . . . . . . . . . . . . . . 15
8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 8. Security Considerations . . . . . . . . . . . . . . . . . . . 16
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16
10.1. Normative References . . . . . . . . . . . . . . . . . . 16 10.1. Normative References . . . . . . . . . . . . . . . . . . 16
10.2. Informative References . . . . . . . . . . . . . . . . . 16 10.2. Informative References . . . . . . . . . . . . . . . . . 16
10.3. Acknowledgments . . . . . . . . . . . . . . . . . . . . . 17 10.3. Acknowledgments . . . . . . . . . . . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction 1. Introduction
Traffic patterns in the data center are not uniform and are Traffic patterns in the data center are not uniform and are
constantly changing. They are dictated by the nature and variety of constantly changing. They are dictated by the nature and variety of
applications utilized in the data center. It can be largely east-west applications utilized in the data center. It can be largely east-west
traffic flows (server to server inside the data center) in one data traffic flows (server to server inside the data center) in one data
center and north-south (outside of the data center to server) in center and north-south (outside of the data center to server) in
another, while some may combine both. Traffic patterns can be bursty another, while some may combine both. Traffic patterns can be bursty
skipping to change at page 3, line 45 skipping to change at page 3, line 47
-Low latency (in the microsecond or nanosecond range) -Low latency (in the microsecond or nanosecond range)
-Low amount of buffer (in the MB range per networking device) -Low amount of buffer (in the MB range per networking device)
-Layer 2 and Layer 3 forwarding capability (Layer 3 not mandatory) -Layer 2 and Layer 3 forwarding capability (Layer 3 not mandatory)
The following document defines a set of definitions, metrics and The following document defines a set of definitions, metrics and
terminologies including congestion scenarios, switch buffer analysis terminologies including congestion scenarios, switch buffer analysis
and redefines basic definitions in order to represent a wide mix of and redefines basic definitions in order to represent a wide mix of
traffic conditions. The test methodologies are defined in [1]. traffic conditions. The test methodologies are defined in [draft-
ietf-bmwg-dcbench-methodology].
1.1. Requirements Language 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
1.2. Definition format 1.2. Definition format
Term to be defined. (e.g., Latency) Term to be defined. (e.g., Latency)
skipping to change at page 6, line 35 skipping to change at page 6, line 35
2.3 Measurement Units 2.3 Measurement Units
The measuring methods to use for benchmarking purposes are as The measuring methods to use for benchmarking purposes are as
follows: follows:
1) FILO MUST be used as a measuring method, as this will include the 1) FILO MUST be used as a measuring method, as this will include the
latency of the packet; and today the application commonly needs to latency of the packet; and today the application commonly needs to
read the whole packet to process the information and take an action. read the whole packet to process the information and take an action.
2) FIFO MAY be used for certain applications able to proceed the data 2) FIFO MAY be used for certain applications able to proceed the data
as the first bits arrive (FPGA for example) as the first bits arrive, as for example for a Field-Programmable
Gate Array (FPGA)
3) LIFO MUST NOT be used, because it subtracts the latency of the 3) LIFO MUST NOT be used, because it subtracts the latency of the
packet; unlike all the other methods. packet; unlike all the other methods.
3 Jitter 3 Jitter
3.1 Definition 3.1 Definition
Jitter in the data center context is synonymous with the common term Jitter in the data center context is synonymous with the common term
Delay variation. It is derived from multiple measurements of one-way Delay variation. It is derived from multiple measurements of one-way
skipping to change at page 7, line 49 skipping to change at page 8, line 4
-Type of device used to generate traffic / measure traffic -Type of device used to generate traffic / measure traffic
-Type of line cards used on the traffic generator -Type of line cards used on the traffic generator
-Type of transceivers on traffic generator -Type of transceivers on traffic generator
-Type of transceivers on DUT -Type of transceivers on DUT
-Type of cables -Type of cables
-Length of cables -Length of cables
-Software name, and version of traffic generator and DUT -Software name, and version of traffic generator and DUT
-List of enabled features on DUT MAY be provided and is recommended -List of enabled features on DUT MAY be provided and is recommended
(especially the control plane protocols such as LLDP, Spanning-Tree (especially the control plane protocols such as Link Layer Discovery
etc.). A comprehensive configuration file MAY be provided to this Protocol, Spanning-Tree etc.). A comprehensive configuration file MAY
effect. be provided to this effect.
4.2 Discussion 4.2 Discussion
Physical layer calibration is part of the end to end latency, which Physical layer calibration is part of the end to end latency, which
should be taken into acknowledgment while evaluating the DUT. Small should be taken into acknowledgment while evaluating the DUT. Small
variations of the physical components of the test may impact the variations of the physical components of the test may impact the
latency being measured, therefore they MUST be described when latency being measured, therefore they MUST be described when
presenting results. presenting results.
4.3 Measurement Units 4.3 Measurement Units
skipping to change at page 11, line 13 skipping to change at page 11, line 14
measurement. measurement.
6 Buffering 6 Buffering
6.1 Buffer 6.1 Buffer
6.1.1 Definition 6.1.1 Definition
Buffer Size: The term buffer size represents the total amount of Buffer Size: The term buffer size represents the total amount of
frame buffering memory available on a DUT. This size is expressed in frame buffering memory available on a DUT. This size is expressed in
B (bytes); KB (kilobytes), MB (megabytes) or GB (gigabyte). When the B (byte); KB (kilobyte), MB (megabyte) or GB (gigabyte). When the
buffer size is expressed it SHOULD be defined by a size metric stated buffer size is expressed it SHOULD be defined by a size metric stated
above. When the buffer size is expressed, an indication of the frame above. When the buffer size is expressed, an indication of the frame
MTU used for that measurement is also necessary as well as the cos MTU used for that measurement is also necessary as well as the cos
(class of service) or dscp (differentiated services code point) value (class of service) or dscp (differentiated services code point) value
set; as often times the buffers are carved by quality of service set; as often times the buffers are carved by quality of service
implementation. Please refer to the buffer efficiency section for implementation. Please refer to the buffer efficiency section for
further details. further details.
Example: Buffer Size of DUT when sending 1518 byte frames is 18 MB. Example: Buffer Size of DUT when sending 1518 byte frames is 18 MB.
skipping to change at page 13, line 17 skipping to change at page 13, line 22
-The intensity of microburst MAY be mentioned when a microburst test -The intensity of microburst MAY be mentioned when a microburst test
is performed is performed
-The cos or dscp value set during the test SHOULD be provided -The cos or dscp value set during the test SHOULD be provided
6.2 Incast 6.2 Incast
6.2.1 Definition 6.2.1 Definition
The term Incast, very commonly utilized in the data center, refers to The term Incast, very commonly utilized in the data center, refers to
the traffic pattern of many-to-one or many-to-many conversations. It the traffic pattern of many-to-one or many-to-many traffic patterns.
measures the number of ingress and egress ports and the level of It measures the number of ingress and egress ports and the level of
synchronization attributed, as defined in this section. Typically in synchronization attributed, as defined in this section. Typically in
the data center it would refer to many different ingress server ports the data center it would refer to many different ingress server ports
(many), sending traffic to a common uplink (one), or multiple uplinks (many), sending traffic to a common uplink (many-to-one), or multiple
(many). This pattern is generalized for any network as many incoming uplinks (many-to-many). This pattern is generalized for any network
ports sending traffic to one or few uplinks. It can also be found in as many incoming ports sending traffic to one or few uplinks.
many-to-many traffic patterns.
Synchronous arrival time: When two, or more, frames of respective Synchronous arrival time: When two, or more, frames of respective
sizes L1 and L2 arrive at their respective one or multiple ingress sizes L1 and L2 arrive at their respective one or multiple ingress
ports, and there is an overlap of the arrival time for any of the ports, and there is an overlap of the arrival time for any of the
bits on the Device Under Test (DUT), then the frames L1 and L2 have a bits on the Device Under Test (DUT), then the frames L1 and L2 have a
synchronous arrival times. This is called incast. synchronous arrival times. This is called Incast regardless of in
many-to-one (simpler form) or, many-to-many.
Asynchronous arrival time: Any condition not defined by synchronous Asynchronous arrival time: Any condition not defined by synchronous
arrival time. arrival time.
Percentage of synchronization: This defines the level of overlap Percentage of synchronization: This defines the level of overlap
[amount of bits] between the frames L1,L2..Ln. [amount of bits] between the frames L1,L2..Ln.
Example: Two 64 bytes frames, of length L1 and L2, arrive to ingress Example: Two 64 bytes frames, of length L1 and L2, arrive to ingress
port 1 and port 2 of the DUT. There is an overlap of 6.4 bytes port 1 and port 2 of the DUT. There is an overlap of 6.4 bytes
between the two where L1 and L2 were at the same time on the between the two where L1 and L2 were at the same time on the
skipping to change at page 16, line 29 skipping to change at page 16, line 33
networks. networks.
9. IANA Considerations 9. IANA Considerations
NO IANA Action is requested at this time. NO IANA Action is requested at this time.
10. References 10. References
10.1. Normative References 10.1. Normative References
[1] Avramov L. and Rapp J., "Data Center Benchmarking Methodology", [draft-ietf-bmwg-dcbench-methodology] Avramov L. and Rapp J., "Data
April 2017. Center Benchmarking Methodology", RFC "draft-ietf-bmwg-dcbench-
methodology", DATE (to be updated once published)
[RFC1242] Bradner, S. "Benchmarking Terminology for Network [RFC1242] Bradner, S. "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, July 1991, <http://www.rfc- Interconnection Devices", RFC 1242, July 1991, <http://www.rfc-
editor.org/info/rfc1242> editor.org/info/rfc1242>
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999, Network Interconnect Devices", RFC 2544, March 1999,
<http://www.rfc-editor.org/info/rfc2554> <http://www.rfc-editor.org/info/rfc2544>
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119,
March 1997, <http://www.rfc-editor.org/info/rfc2119> March 1997, <http://www.rfc-editor.org/info/rfc2119>
[RFC5841] , Hay, R., "TCP Option to Denote Packet Mood", BCP
14, RFC 5841, April 2010, <http://www.rfc-
editor.org/info/rfc5841>
10.2. Informative References 10.2. Informative References
[RFC2889] Mandeville R. and Perser J., "Benchmarking [RFC2889] Mandeville R. and Perser J., "Benchmarking
Methodology for LAN Switching Devices", RFC 2889, August 2000, Methodology for LAN Switching Devices", RFC 2889, August 2000,
<http://www.rfc-editor.org/info/rfc2889> <http://www.rfc-editor.org/info/rfc2889>
[RFC3918] Stopp D. and Hickman B., "Methodology for IP Multicast [RFC3918] Stopp D. and Hickman B., "Methodology for IP Multicast
Benchmarking", RFC 3918, October 2004, <http://www.rfc- Benchmarking", RFC 3918, October 2004, <http://www.rfc-
editor.org/info/rfc3918> editor.org/info/rfc3918>
 End of changes. 16 change blocks. 
26 lines changed or deleted 27 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/