draft-ietf-bmwg-mcast-01.txt   draft-ietf-bmwg-mcast-02.txt 
Network Working Group K. Dubray Network Working Group K. Dubray
INTERNET-DRAFT Bay Networks INTERNET-DRAFT Bay Networks
Expiration Date: September 1997 March 1997 Expiration Date: January 1998 July 1997
Terminology for IP Multicast Benchmarking Terminology for IP Multicast Benchmarking
<draft-ietf-bmwg-mcast-01.txt> <draft-ietf-bmwg-mcast-02.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts. working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference time. It is inappropriate to use Internet- Drafts as reference
material or to cite them other than as ``work in progress.'' material or to cite them other than as ``work in progress.''
To learn the current status of any Internet-Draft, please check the To view the entire list of current Internet-Drafts, please check
``1id-abstracts.txt'' listing contained in the Internet- Drafts the "1id-abstracts.txt" listing contained in the Internet-Drafts
Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East
ftp.isi.edu (US West Coast). Coast), or ftp.isi.edu (US West Coast).
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
Abstract Abstract
The purpose of this draft is to add terminology specific to the The purpose of this draft is to add terminology specific to the
benchmarking of multicast IP forwarding devices. It builds upon the benchmarking of multicast IP forwarding devices. It builds upon the
tenets set forth in RFC 1242, RFC 1944, and other IETF Benchmarking tenets set forth in RFC 1242, RFC 1944, and other IETF Benchmarking
Methodology Working Group (BMWG) effort and extends them to the Methodology Working Group (BMWG) effort and extends them to the
multicast paradigm. multicast paradigm.
1. Introduction 1. Introduction
skipping to change at page 2, line 40 skipping to change at page 2, line 40
2.1 Existing Terminology 2.1 Existing Terminology
This document draws on existing terminology defined in other This document draws on existing terminology defined in other
BMWG work. Examples include, but are not limited to: BMWG work. Examples include, but are not limited to:
Throughput (RFC 1242, section 3.17) Throughput (RFC 1242, section 3.17)
Latency (RFC 1242, section 3.8) Latency (RFC 1242, section 3.8)
Constant Load (RFC 1242, section 3.4) Constant Load (RFC 1242, section 3.4)
Frame Loss Rate (RFC 1242, section 3.6) Frame Loss Rate (RFC 1242, section 3.6)
Overhead behavior (RFC 1242, section 3.11) Overhead behavior (RFC 1242, section 3.11)
Forwarding Rates ([4], section 3.6)
Loads ([4], section 3.5)
Devices ([4], section 3.1)
3. Table of Defined Terms 3. Table of Defined Terms
3.1 General Nomenclature 3.1 General Nomenclature
3.1.1 Device Under Test (DUT). 3.1.1 Traffic Class.
3.1.2 System Under Test (SUT). 3.1.2 Group Class.
3.1.3 Target Rate. 3.1.3 Service Class.
3.1.4 Offered Rate.
3.1.5 Forwarding Rate. 3.2 Forwarding and Throughput
3.1.6 Maximum Forwarding Rate (MFR). 3.2.1 Mixed Class Throughput (MCT).
3.1.7 Flow. 3.2.2 Scaled Group Forwarding Matrix (SGFM).
3.1.8 Group Flow.
3.1.9 Service Flow.
3.2 Throughput
3.2.1 Mixed Flow Throughput (MFT).
3.2.2 Scaled Group Throughput (SGT).
3.2.3 Aggregated Multicast Throughput (AMT) 3.2.3 Aggregated Multicast Throughput (AMT)
3.2.4 Translational Throughput (TT) 3.2.4 Translational Throughput (TT)
3.3 Fairness 3.3 Fairness
3.4 Forwarding Latency 3.4 Forwarding Latency
3.4.1 Multicast Latency 3.4.1 Multicast Latency
3.4.2 Min/Max Multicast Latency 3.4.2 Min/Max Multicast Latency
3.5 Overhead 3.5 Overhead
3.5.1 Group Join Delay. 3.5.1 Group Join Delay.
3.5.2 Group Leave Delay. 3.5.2 Group Leave Delay.
3.6 Capacity 3.6 Capacity
3.6.1 Multicast Group Capacity. 3.6.1 Multicast Group Capacity.
3.1 General Nomenclature 3.1 General Nomenclature
This section will present general terminology to be used in This section will present general terminology to be used in
this and other documents. this and other documents.
3.1.1 Device Under Test (DUT). 3.1.1 Traffic Class.
Definition:
The network forwarding device being tested.
Discussion:
Measurement units:
Not applicable.
Issue:
This definition is being moved to the BMWG LAN switch
terminology draft. This item will be deleted from this
multicast terminology draft upon migration.
3.1.2 System Under Test (SUT).
Definition:
The collective set of network devices being tested as a singular
entity.
Discussion: A system under test may be comprised of a variety
of networking devices. Some devices may be active in the
forwarding decision making process, such as routers or switches;
other devices may be passive such as CSU/DSUs. Regardless
of constituent components, the system is treated as a "black box"
to which stimuli is offered and response measured.
Measurement units:
Not applicable.
Issue:
This definition is being moved to the BMWG LAN switch
terminology draft. This item will be deleted from this
multicast terminology draft upon migration.
3.1.3 Target Rate.
Definition:
The requested rate at which the test device attempts to offer the
DUT or SUT test traffic.
Discussion:
There are networks events (e.g., collisions) that may preclude the
test device from delivering the requested rate to the SUT. In this
case, differentiation is made between target rate and offered rate.
Measurement units:
Frames per second.
Issue:
This definition is being moved to the BMWG LAN switch
terminology draft as "intended rate". This item will be deleted
multicast terminology draft upon migration.
3.1.4 Offered Rate.
Definition:
The actual resultant rate at which the test device is successful
in offering test traffic to the SUT.
Discussion:
Contrast with Target Rate. Note relationship to Forwarding Rate.
Measurement units:
Frames per second.
Issue:
This definition is being moved to the BMWG LAN switch
terminology draft. This item will be deleted from this
multicast terminology draft upon migration.
3.1.5 Forwarding Rate.
Definition:
The rate at which the SUT has been observed to successfully
forward test traffic to the traffic's correct destination(s) in
response to a particular offered rate.
Discussion:
Note the specification of "correct destination(s)" in the
definition. The reporting of a forwarding rate MUST
correspond to an associated Offered Rate. Frame loss is not
a constraint when reporting Forwarding Rate.
Measurement units:
Frames per second.
Issue:
This definition is being moved to the BMWG LAN switch
terminology draft. This item will be deleted from this
multicast terminology draft upon migration.
3.1.6 Maximum Forwarding Rate (MFR).
Definition:
The rate at which the SUT has been observed to successfully
forward test traffic to the traffic's correct destination(s) in
response to the test device's maximum offered rate.
Discussion:
Because a DUT's maximum forwarding rate does not always equal
the largest forwarding rate of the DUT, this metric can sometimes
indicate oversubscription or congestion internal to the DUT/SUT.
For example, consider the following table:
Test Device DUT
Offered Rate Forwarding Rate
------------- ---------------
1. 14,880 fps 7,400 fps
2. 13,880 fps 8,472 fps
3. 12,880 fps 12,880 fps
The tester's maximum offered rate is 14,880 frames per second,
as indicated in line 1. Per the definition, the corresponding
MFR for the DUT is 7,440 fps - not the 12,880 fps indicated in
line 3.
When reporting the MFR, the corresponding test device's maximum
offered load MUST be cited. This is due to the fact that not
all test devices deliver the maximum usable bandwidth. In the
case when the test device is able to exceed the maximum, legal
bandwidth, the test results SHOULD reflect that the test was
conducted in a overload condition.
Measurement units:
Frames per second.
Issue:
This definition is being moved to the BMWG LAN switch
terminology draft as "Forwarding Rate at Maximum Offered Load."
This item will be deleted from this multicast terminology draft
upon migration.
3.1.7 Flow.
Definition: Definition:
An equivalence class of packets comprising one or more data An equivalence class of packets comprising one or more data
streams. streams.
Discussion: Discussion:
In the scope of this document, Flow will be considered a logical In the scope of this document, Traffic Class will be considered
identifier used to discriminate between a set or sets of packets a logical identifier used to discriminate between a set or sets
offered the DUT. of packets offered the DUT.
For example, one flow may identify a set of unicast packets For example, one Traffic Class may identify a set of unicast packets
offered to the DUT. Another flow may differentiate the offered to the DUT. Another Traffic Class may differentiate the
multicast packets destined to multicast group X. Yet another multicast packets destined to multicast group X. Yet another
flow may distinguish the set of multicast packets destined to Class may distinguish the set of multicast packets destined to
multicast group Y. multicast group Y.
Unless otherwise qualified, the usage of the word "Class" in this
document will refer simply to a Traffic Class.
Measurement units: Measurement units:
Not applicable. Not applicable.
Issue: This definition does not seem to be inconsistent with 3.1.2 Group Class.
section 1.2 of the RSVP draft, but may not map directly to
work being done in the RTFM space.
3.1.8 Group Flow.
Definition: Definition:
A specific type of flow where the packets comprising the flow A specific type of Traffic Class where the packets comprising the Class
are destined to a particular multicast group. are destined to a particular multicast group.
Discussion: Discussion:
Measurement units: Measurement units:
Not applicable. Not applicable.
3.1.9 Service Flow. 3.1.3 Service Class.
Definition: Definition:
A specific type of flow where the packets comprising the flow A specific type of Traffic Class where the packets comprising the Class
require particular treatment or treatments by the network require particular treatment or treatments by the network
forwarding devices along the path to the packets' destination(s). forwarding devices along the path to the packets' destination(s).
Discussion: Discussion:
Measurement units: Measurement units:
Not applicable. Not applicable.
3.2 Throughput. 3.2 Forwarding and Throughput.
This section presents terminology relating to the characterization of This section presents terminology relating to the characterization of
the packet forwarding ability of a DUT/SUT in a multicast environment. the packet forwarding ability of a DUT/SUT in a multicast environment.
It extends the concept of throughput presented in RFC 1242. Some metrics extend the concept of throughput presented in RFC 1242.
3.2.1 Mixed Flow Throughput (MFT). 3.2.1 Mixed Class Throughput (MCT).
Definition: Definition:
The maximum rate at which none of the offered frames, comprised The maximum rate at which none of the offered frames, comprised
from a unicast flow and a multicast flow, to be forwarded are from a unicast Class and a multicast Class, to be forwarded are
dropped by the device. dropped by the device.
Discussion: Discussion:
Often times, throughput is collected on a homogenous traffic Often times, throughput is collected on a homogenous traffic
type - though the packets' destinations may vary, the packets type - though the packets' destinations may vary, the packets
follow the same packet forwarding path through the DUT. follow the same packet forwarding path through the DUT.
Based on the RFC 1242 definition for throughput, the Mixed Based on the RFC 1242 definition for throughput, the Mixed
Flow Throughput benchmark attempts to characterize the DUT's Class Throughput benchmark attempts to characterize the DUT's
ability to process both unicast and multicast frames in the ability to process both unicast and multicast frames in the
same aggregated traffic stream. same aggregated traffic stream.
Measurement units: Measurement units:
Frames per second Frames per second
Issues: Issues:
Related methodology may have to address the ratio of unicast packets Related methodology may have to address the ratio of unicast packets
to multicast packets. to multicast packets.
3.2.2 Scaled Group Throughput (SGT). 3.2.2 Scaled Group Forwarding Matrix (SGFM).
Definition: Definition:
The maximum number of multicast groups that a DUT/SUT can A table that demonstrates Forwarding Rate as a function of
support and still yield the same throughput as supporting a tested multicast groups for a fixed number of tested
single multicast group. DUT/SUT ports.
Discussion: Discussion:
A desirable attribute of many Internet mechanisms is the ability A desirable attribute of many Internet mechanisms is the ability
to "scale." This benchmark seeks to demonstrate the ability to "scale." This benchmark seeks to demonstrate the ability
of a SUT to scale the number of multicast groups upwards while of a SUT to forward as the number of multicast groups is scaled
holding it to the RFC 1242 definition of throughput for a single upwards.
multicast group.
Measurement units: Measurement units:
Number of multicast groups. Packets per second, with corresponding tested multicast group
and port configurations.
Issues: Issues:
The corresponding methodology (or even the definition itself) may The corresponding methodology (or even the definition itself) may
have to reflect the impact that the pairing (source, group) has on have to reflect the impact that the pairing (source, group) has on
many multicast routing protocols. many multicast routing protocols.
Refers to the concept of Forwarding Rate originally defined in
this document. The definition of Forwarding Rate has been
moved to [4].
3.2.3 Aggregated Multicast Throughput (AMT) 3.2.3 Aggregated Multicast Throughput (AMT)
Definition: Definition:
The maximum rate at which none of the offered frames to be The maximum rate at which none of the offered frames to be
forwarded through N destination interfaces of the same multicast forwarded through N destination interfaces of the same multicast
group are dropped. group are dropped.
Discussion: Discussion:
Another "scaling" type of exercise, designed to identify the Another "scaling" type of exercise, designed to identify the
DUT/SUT's ability to handle traffic as a function of the DUT/SUT's ability to handle traffic as a function of the
skipping to change at page 9, line 30 skipping to change at page 6, line 30
Consideration may need to be given with respect to the impact Consideration may need to be given with respect to the impact
of different frame formats on usable bandwidth. of different frame formats on usable bandwidth.
Measurement units: Measurement units:
Frames per second. Frames per second.
3.3 Fairness. 3.3 Fairness.
Definition: Definition:
The ability of a SUT to fulfill the requirements of a flow The ability of a SUT to fulfill the requirements of a Traffic
without compromising the requirements, if any, of other flows. Class without compromising the requirements, if any, of other
Classes.
Discussion: Discussion:
Measurement units: Measurement units:
Not applicable. Not applicable.
3.4 Forwarding Latency. 3.4 Forwarding Latency.
This section presents terminology relating to the characterization of This section presents terminology relating to the characterization of
the forwarding latency of a DUT/SUT in a multicast environment. the forwarding latency of a DUT/SUT in a multicast environment.
skipping to change at page 10, line 17 skipping to change at page 7, line 10
Definition: Definition:
The set of individual latencies from a single input port on The set of individual latencies from a single input port on
the DUT or SUT to all tested ports belonging to the destination the DUT or SUT to all tested ports belonging to the destination
multicast group. multicast group.
Discussion: Discussion:
This benchmark is based on the RFC 1242 definition of latency. This benchmark is based on the RFC 1242 definition of latency.
While it is useful to collect latency between a pair of source While it is useful to collect latency between a pair of source
and destination multicast ports, it may be insightful to collect and destination multicast ports, it may be insightful to collect
the same type of measurements across a range of ports supporting the same type of measurements across a range of ports supporting
that group flow. that Group Class.
A variety of statistical exercises can be applied to the set of A variety of statistical exercises can be applied to the set of
latencies measurements. latencies measurements.
Measurement units: Measurement units:
Time units with enough precision to reflect measurement. Time units with enough precision to reflect measurement.
3.4.2 Min/Max Multicast Latency. 3.4.2 Min/Max Multicast Latency.
Definition: Definition:
skipping to change at page 13, line 35 skipping to change at page 9, line 9
5. References 5. References
[1] Bradner, S. Benchmarking Terminology for Network [1] Bradner, S. Benchmarking Terminology for Network
Interconnection Devices. RFC 1242. July, 1991. Interconnection Devices. RFC 1242. July, 1991.
[2] Bradner, S., McQuaid, J. Benchmarking Methodology for Network [2] Bradner, S., McQuaid, J. Benchmarking Methodology for Network
Interconnect Devices. RFC 1944. May, 1996. Interconnect Devices. RFC 1944. May, 1996.
[3] Craig, R. Terminology for Cell/Call Benchmarking. <draft-ietf- [3] Craig, R. Terminology for Cell/Call Benchmarking. <draft-ietf-
bmwg-call-00.txt> November, 1996. Work in progress. bmwg-call-01.txt> March, 1997. Work in progress.
[4] Mandeville, R. Benchmarking Terminology for LAN Switching [4] Mandeville, R. Benchmarking Terminology for LAN Switching
Devices. <draft-ietf-bmwg-lanswitch-03.txt> February, 1997. Devices. <draft-ietf-bmwg-lanswitch-06.txt> July, 1997.
Work in progress. Work in progress.
5. Author's Address 5. Author's Address
Kevin Dubray Kevin Dubray
Bay Networks, Inc. Bay Networks, Inc.
2 Federal Street 2 Federal Street
Billerica, MA 01984 Billerica, MA 01984
(508) 436-3862 (508) 916-3862
kdubray@baynetworks.com kdubray@baynetworks.com
or direct discussion to the Benchmarking Methodology Working Group: or direct discussion to the Benchmarking Methodology Working Group:
bmwg@harvard.edu bmwg@harvard.edu
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/