draft-ietf-bmwg-mcast-02.txt   draft-ietf-bmwg-mcast-03.txt 
Network Working Group K. Dubray Network Working Group K. Dubray
INTERNET-DRAFT Bay Networks INTERNET-DRAFT Bay Networks
Expiration Date: January 1998 July 1997 Expiration Date: September 1998 March 1998
Terminology for IP Multicast Benchmarking Terminology for IP Multicast Benchmarking
<draft-ietf-bmwg-mcast-02.txt> <draft-ietf-bmwg-mcast-03.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts. working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
skipping to change at page 3, line 4 skipping to change at page 3, line 8
3. Table of Defined Terms 3. Table of Defined Terms
3.1 General Nomenclature 3.1 General Nomenclature
3.1.1 Traffic Class. 3.1.1 Traffic Class.
3.1.2 Group Class. 3.1.2 Group Class.
3.1.3 Service Class. 3.1.3 Service Class.
3.2 Forwarding and Throughput 3.2 Forwarding and Throughput
3.2.1 Mixed Class Throughput (MCT). 3.2.1 Mixed Class Throughput (MCT).
3.2.2 Scaled Group Forwarding Matrix (SGFM). 3.2.2 Scaled Group Forwarding Matrix (SGFM).
3.2.3 Aggregated Multicast Throughput (AMT) 3.2.3 Aggregated Multicast Throughput (AMT)
3.2.4 Translational Throughput (TT) 3.2.4 Encapsulation Throughput (ET)
3.2.5 Decapsulation Throughput (DT)
3.2.6 Re-encapsulation Throughput (RET)
3.3 Fairness 3.3 Forwarding Latency
3.3.1 Multicast Latency
3.3.2 Min/Max Multicast Latency
3.4 Forwarding Latency 3.4 Overhead
3.4.1 Multicast Latency 3.4.1 Group Join Delay.
3.4.2 Min/Max Multicast Latency 3.4.2 Group Leave Delay.
3.5 Overhead 3.5 Capacity
3.5.1 Group Join Delay. 3.5.1 Multicast Group Capacity.
3.5.2 Group Leave Delay.
3.6 Capacity 3.6 Interaction
3.6.1 Multicast Group Capacity. 3.6.1 Burdened Response
3.6.2 Forwarding Burdened Multicast Latency
3.6.3 Forwarding Burdened Join Delay
3.6.4 Forwarding Burdened Multicast Group Capacity.
3.1 General Nomenclature 3.1 General Nomenclature
This section will present general terminology to be used in This section will present general terminology to be used in
this and other documents. this and other documents.
3.1.1 Traffic Class. 3.1.1 Traffic Class.
Definition: Definition:
An equivalence class of packets comprising one or more data An equivalence class of packets comprising one or more data
streams. streams.
skipping to change at page 4, line 31 skipping to change at page 4, line 42
This section presents terminology relating to the characterization of This section presents terminology relating to the characterization of
the packet forwarding ability of a DUT/SUT in a multicast environment. the packet forwarding ability of a DUT/SUT in a multicast environment.
Some metrics extend the concept of throughput presented in RFC 1242. Some metrics extend the concept of throughput presented in RFC 1242.
3.2.1 Mixed Class Throughput (MCT). 3.2.1 Mixed Class Throughput (MCT).
Definition: Definition:
The maximum rate at which none of the offered frames, comprised The maximum rate at which none of the offered frames, comprised
from a unicast Class and a multicast Class, to be forwarded are from a unicast Class and a multicast Class, to be forwarded are
dropped by the device. dropped by the device across a fixed number of ports.
Discussion: Discussion:
Often times, throughput is collected on a homogenous traffic Often times, throughput is collected on a homogenous traffic
type - though the packets' destinations may vary, the packets type - though the packets' destinations may vary, the packets
follow the same packet forwarding path through the DUT. follow the same packet forwarding path through the DUT.
Based on the RFC 1242 definition for throughput, the Mixed Based on the RFC 1242 definition for throughput, the Mixed
Class Throughput benchmark attempts to characterize the DUT's Class Throughput benchmark attempts to characterize the DUT's
ability to process both unicast and multicast frames in the ability to process both unicast and multicast frames in the
same aggregated traffic stream. same aggregated traffic stream.
skipping to change at page 6, line 5 skipping to change at page 6, line 5
DUT/SUT's ability to handle traffic as a function of the DUT/SUT's ability to handle traffic as a function of the
multicast destination ports it is required to support. multicast destination ports it is required to support.
Measurement units: Measurement units:
The ordered pair (N,t) where, The ordered pair (N,t) where,
N = the number of destination ports of the multicast group. N = the number of destination ports of the multicast group.
t = the throughput, in frames per second, relative to the t = the throughput, in frames per second, relative to the
source stream. source stream.
3.2.4 Translational Throughput (TT) 3.2.4 Encapsulation Throughput (ET)
Definition: Definition:
The maximum rate at which none of the frames offered in an The maximum rate at which frames offered a DUT are encapsulated
transitional format to the SUT are dropped in the process of and correctly forwarded by the DUT without loss.
converting those frames to their appropriate, final format and
subsequent correct delivery.
Discussion: Discussion:
A popular technique in presenting frames to devices that may A popular technique in presenting a frame to a device that may
not support a protocol feature is to encapsulate, or tunnel, not support a protocol feature is to encapsulate, or tunnel,
the packet containing the unsupported feature in a format that the packet containing the unsupported feature in a format that
is supported by that device. This benchmark attempts to is supported by that device.
characterize the overhead behavior associated with that
transitional process. More specifically, encapsulation refers to the act of taking a
frame or part of a frame and embedding it as a payload of another
frame. This benchmark attempts to characterize the overhead behavior
associated with that translational process.
Consideration may need to be given with respect to the impact Consideration may need to be given with respect to the impact
of different frame formats on usable bandwidth. of different frame formats on usable bandwidth.
Measurement units: Measurement units:
Frames per second. Frames per second.
3.3 Fairness. 3.2.5 Decapsulation Throughput (DT)
Definition: Definition:
The ability of a SUT to fulfill the requirements of a Traffic The maximum rate at which frames offered a DUT are decapsulated
Class without compromising the requirements, if any, of other and correctly forwarded by the DUT without loss.
Classes.
Discussion: Discussion:
A popular technique in presenting a frame to a device that may
not support a protocol feature is to encapsulate, or tunnel,
the packet containing the unsupported feature in a format that
is supported by that device. At some point, the frame may be required
to be returned its orginal format from its encapsulation wrapper for
use by the frame's next destination.
More specifically, decapsulation refers to the act of taking a
frame or part of a frame embedded as a payload of another frame and
returning it to the payload's appropriate format. This benchmark
attempts to characterize the overhead behavior associated with that
translational process.
Consideration may need to be given with respect to the impact
of different frame formats on usable bandwidth.
Measurement units: Measurement units:
Not applicable. Frames per second.
3.4 Forwarding Latency. 3.2.6 Re-encapsulation Throughput (RET)
Definition:
The maximum rate at which frames of one encapsulated format offered a DUT
are converted to another encapsulated format and correctly forwarded
by the DUT without loss.
Discussion:
A popular technique in presenting a frame to a device that may
not support a protocol feature is to encapsulate, or tunnel,
the packet containing the unsupported feature in a format that
is supported by that device. At some point, the frame may be required
to be converted from one encapsulation format to another encapsulation
format.
More specifically, re-encapsulation refers to the act of taking an
encapsulated payload of one format and replacing it with another
encapsulated format - all the while preserving the original payload's
contents. This benchmark attempts to characterize the overhead
behavior associated with that translational process.
Consideration may need to be given with respect to the impact
of different frame formats on usable bandwidth.
Measurement units:
Frames per second.
3.3 Forwarding Latency.
This section presents terminology relating to the characterization of This section presents terminology relating to the characterization of
the forwarding latency of a DUT/SUT in a multicast environment. the forwarding latency of a DUT/SUT in a multicast environment.
It extends the concept of latency presented in RFC 1242. It extends the concept of latency presented in RFC 1242.
3.4.1 Multicast Latency. 3.3.1 Multicast Latency.
Definition: Definition:
The set of individual latencies from a single input port on The set of individual latencies from a single input port on
the DUT or SUT to all tested ports belonging to the destination the DUT or SUT to all tested ports belonging to the destination
multicast group. multicast group.
Discussion: Discussion:
This benchmark is based on the RFC 1242 definition of latency. This benchmark is based on the RFC 1242 definition of latency.
While it is useful to collect latency between a pair of source While it is useful to collect latency between a pair of source
and destination multicast ports, it may be insightful to collect and destination multicast ports, it may be insightful to collect
the same type of measurements across a range of ports supporting the same type of measurements across a range of ports supporting
that Group Class. that Group Class.
A variety of statistical exercises can be applied to the set of A variety of statistical exercises can be applied to the set of
latencies measurements. latencies measurements.
Dubray, K. Expires September 1998 [Page 7]^L
Measurement units: Measurement units:
Time units with enough precision to reflect measurement. Time units with enough precision to reflect measurement.
3.4.2 Min/Max Multicast Latency. 3.3.2 Min/Max Multicast Latency.
Definition: Definition:
The difference between the maximum latency measurement and the The difference between the maximum latency measurement and the
minimum latency measurement from the set of latencies produced by minimum latency measurement from the set of latencies produced by
the Multicast Latency benchmark. the Multicast Latency benchmark.
Discussion: Discussion:
This statistic may yield some insight into how a particular This statistic may yield some insight into how a particular
implementation handles its multicast traffic. This may be useful implementation handles its multicast traffic. This may be useful
to users of multicast synchronization types of applications. to users of multicast synchronization types of applications.
Measurement units: Measurement units:
Time units with enough precision to reflect measurement. Time units with enough precision to reflect measurement.
3.5 Overhead 3.4 Overhead
This section presents terminology relating to the characterization of This section presents terminology relating to the characterization of
the overhead delays associated with explicit operations found in the overhead delays associated with explicit operations found in
multicast environments. multicast environments.
3.5.1 Group Join Delay. 3.4.1 Group Join Delay.
Definition: Definition:
The time duration it takes a DUT/SUT to start forwarding multicast The time duration it takes a DUT/SUT to start forwarding multicast
packets from the time a successful IGMP group membership report has packets from the time a successful IGMP group membership report has
been issued to the DUT/SUT. been issued to the DUT/SUT.
Discussion: Discussion:
Many different factors can contribute to different results, such as Many factors can contribute to different results, such as
the number or type of multicast-related protocols configured the number or type of multicast-related protocols configured
on the system under test. on the system under test. Other factors are physical topology and
"tree" configuration.
Because of the number of variables that could impact this metric,
the metric may be a better characterization tool for a device or
system rather than a basis for comparisons with other devices.
A consideration for the related methodology: possible need to A consideration for the related methodology: possible need to
differentiate a specifically-forwarded multicast frame from those differentiate a specifically-forwarded multicast frame from those
sprayed by protocols implementing a flooding tactic to solicit prune sprayed by protocols implementing a flooding tactic to solicit prune
feedback. feedback.
Measurement units: Measurement units:
Microseconds. Microseconds.
3.5.2 Group Leave Delay. 3.4.2 Group Leave Delay.
Definition: Definition:
The time duration it takes a DUT/SUT to cease forwarding multicast The time duration it takes a DUT/SUT to cease forwarding multicast
packets after a corresponding IGMP "Leave Group" message has been packets after a corresponding IGMP "Leave Group" message has been
successfully offered to the DUT/SUT. successfully offered to the DUT/SUT.
Discussion: Discussion:
While it is important to understand how quickly a system can While it is important to understand how quickly a system can
process multicast frames; it may be beneficial to understand process multicast frames; it may be beneficial to understand
how quickly that same system can stop the process as well. how quickly that same system can stop the process as well.
Measurement units: Measurement units:
Microseconds. Microseconds.
Issues: Methodology may need to consider protocol-specific timeout Issues: Methodology may need to consider protocol-specific timeout
values. values.
3.6 Capacity 3.5 Capacity
This section offers terms relating to the identification of multicast This section offers terms relating to the identification of multicast
group limits of a DUT/SUT. group limits of a DUT/SUT.
3.6.1 Multicast Group Capacity. 3.5.1 Multicast Group Capacity.
Definition: Definition:
The maximum number of multicast groups a SUT/DUT can support The maximum number of multicast groups a SUT/DUT can support
while maintaining the ability to forward multicast frames while maintaining the ability to forward multicast frames
to all multicast groups registered to that SUT/DUT. to all multicast groups registered to that SUT/DUT.
Discussion: Discussion:
Measurement units: Measurement units:
Multicast groups. Multicast groups.
Issues: Issues:
The related methodology may have to consider the impact of multicast The related methodology may have to consider the impact of multicast
sources per group on the ability of a SUT/DUT to "scale up" the sources per group on the ability of a SUT/DUT to "scale up" the
number of supportable multicast groups. number of supportable multicast groups.
3.6 Interaction
Network forwarding devices are generally required to provide more
functionality than than the forwarding of traffic. Moreover, network
forwarding devices may be asked to provide those functions in a variety of
environments. This section offers terms to assist in the charaterization
of DUT/SUT behavior in consideration of potentially interacting factors.
3.6.1 Burdened Response.
Definition:
A measured response collected from a DUT/SUT in light of
interacting, or potentially interacting, distinct stimulii.
Discussion:
Many metrics provide a one dimensional view into an operating
characteristic of a tested system. For example, the forwarding rate
metric may yield information about the packet processing ability
of a device. Collecting that same metric in view of another
control variable can oftentimes be very insightful. Taking that same
forwarding rate measurement, for instance, while the device's address
table is injected with an additional 50,000 entries may yield a
different perspective.
Measurement units:
While burdened response is not a specific metric, metrics of this
this type must follow guidelines when reporting results.
The metric's principal result MUST be reported in conjunction with the
contributing factors.
For example, in reporting a Forwarding Burdened Latency, the
latency measurement should be reported with respect to
corresponding Offered Load and Forwarding Rates.
Issues:
A Burdened response may be very illuminating when trying to
characterize a single device or system. Extreme care must
be exercised when attempting to use that characterization as
a basis of comparison with other devices or systems. Test agents
must ensure that the measured response is a function of the
controlled stimulii, and not secondary factors. An example of
of such an interfering factor would be configuration mismatch of
a timer impacting a response process.
3.6.2 Forwarding Burdened Multicast Latency.
Definition:
A multicast latency taken from a DUT/SUT in the presence of
a traffic forwarding requirement.
Discussion:
This burdened response metric builds on the Multicast Latency definition
offered in section 3.3.1. It mandates that the DUT be subjected to
an additional measure of traffic not required by the non-burdened
metric.
It attempts to provide a means to evaluate how traffic load may or
may not impact a device's or system's packet processing delay.
Measurement units:
Time units with enough precision to reflect the latencies measurements.
Latency measurements MUST be reported with the corresponding sustained
Forwarding Rate and associated Offered Load.
3.6.3 Forwarding Burdened Group Join Delay.
Definition:
A multicast Group Join Delay taken from a DUT/SUT in the presence of
a traffic forwarding requirement.
Discussion:
This burdened response metric builds on the Group Join Delay definition
offered in section 3.4.1. It mandates that the DUT be subjected to
an additional measure of traffic not required by the non-burdened
metric.
Many factors can contribute to different results, such as
the number or type of multicast-related protocols configured
on the system under test. Other factors are physical topology and
"tree" configuration.
Because of the number of variables that could impact this metric,
the metric may be a better characterization tool for a device or
system rather than a basis for comparisons with other devices.
Measurement units:
Time units with enough precision to reflect the delay measurements.
Delay measurements MUST be reported with the corresponding sustained
Forwarding Rate and associated Offered Load.
4. Security Considerations 4. Security Considerations
Security issues are not addressed in this memo. Security issues are not addressed in this memo.
5. References 5. References
[1] Bradner, S. Benchmarking Terminology for Network [1] Bradner, S. Benchmarking Terminology for Network
Interconnection Devices. RFC 1242. July, 1991. Interconnection Devices. RFC 1242. July, 1991.
[2] Bradner, S., McQuaid, J. Benchmarking Methodology for Network [2] Bradner, S., McQuaid, J. Benchmarking Methodology for Network
Interconnect Devices. RFC 1944. May, 1996. Interconnect Devices. RFC 1944. May, 1996.
[3] Craig, R. Terminology for Cell/Call Benchmarking. <draft-ietf- [3] Craig, R. Terminology for Cell/Call Benchmarking. <draft-ietf-
bmwg-call-01.txt> March, 1997. Work in progress. bmwg-call-01.txt> March, 1997.
[4] Mandeville, R. Benchmarking Terminology for LAN Switching [4] Mandeville, R. Benchmarking Terminology for LAN Switching Devices.
Devices. <draft-ietf-bmwg-lanswitch-06.txt> July, 1997. RFC 2285. February, 1998.
Work in progress.
5. Author's Address 5. Author's Address
Kevin Dubray Kevin Dubray
Bay Networks, Inc. Bay Networks, Inc.
2 Federal Street 600 Technology Park Drive
Billerica, MA 01984 M/S BL60-301
(508) 916-3862 Billerica, MA 01981
(978) 916-3862
kdubray@baynetworks.com kdubray@baynetworks.com
or direct discussion to the Benchmarking Methodology Working Group: or direct discussion to the Benchmarking Methodology Working Group:
bmwg@harvard.edu bmwg@harvard.edu
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/