draft-ietf-bmwg-sip-bench-term-11.txt   draft-ietf-bmwg-sip-bench-term-12.txt 
Benchmarking Methodology Working Group C. Davids Benchmarking Methodology Working Group C. Davids
Internet-Draft Illinois Institute of Technology Internet-Draft Illinois Institute of Technology
Intended status: Informational V. Gurbani Intended status: Informational V. Gurbani
Expires: January 3, 2015 Bell Laboratories, Expires: May 16, 2015 Bell Laboratories,
Alcatel-Lucent Alcatel-Lucent
S. Poretsky S. Poretsky
Allot Communications Allot Communications
July 2, 2014 November 12, 2014
Terminology for Benchmarking Session Initiation Protocol (SIP) Devices: Terminology for Benchmarking Session Initiation Protocol (SIP) Devices:
Basic session setup and registration Basic session setup and registration
draft-ietf-bmwg-sip-bench-term-11 draft-ietf-bmwg-sip-bench-term-12
Abstract Abstract
This document provides a terminology for benchmarking the Session This document provides a terminology for benchmarking the Session
Initiation Protocol (SIP) performance of devices. Methodology Initiation Protocol (SIP) performance of devices. Methodology
related to benchmarking SIP devices is described in the companion related to benchmarking SIP devices is described in the companion
methodology document. Using these two documents, benchmarks can be methodology document. Using these two documents, benchmarks can be
obtained and compared for different types of devices such as SIP obtained and compared for different types of devices such as SIP
Proxy Servers, Registrars and Session Border Controllers. The term Proxy Servers, Registrars and Session Border Controllers. The term
"performance" in this context means the capacity of the device-under- "performance" in this context means the capacity of the device-under-
skipping to change at page 2, line 4 skipping to change at page 2, line 4
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 3, 2015. This Internet-Draft will expire on May 16, 2015.
Copyright Notice Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 17 skipping to change at page 3, line 17
1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 7 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Protocol Components . . . . . . . . . . . . . . . . . . . 7 3.1. Protocol Components . . . . . . . . . . . . . . . . . . . 7
3.1.1. Session . . . . . . . . . . . . . . . . . . . . . . . 7 3.1.1. Session . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.2. Signaling Plane . . . . . . . . . . . . . . . . . . . 8 3.1.2. Signaling Plane . . . . . . . . . . . . . . . . . . . 8
3.1.3. Media Plane . . . . . . . . . . . . . . . . . . . . . 8 3.1.3. Media Plane . . . . . . . . . . . . . . . . . . . . . 8
3.1.4. Associated Media . . . . . . . . . . . . . . . . . . . 9 3.1.4. Associated Media . . . . . . . . . . . . . . . . . . . 9
3.1.5. Overload . . . . . . . . . . . . . . . . . . . . . . . 9 3.1.5. Overload . . . . . . . . . . . . . . . . . . . . . . . 9
3.1.6. Session Attempt . . . . . . . . . . . . . . . . . . . 9 3.1.6. Session Attempt . . . . . . . . . . . . . . . . . . . 10
3.1.7. Established Session . . . . . . . . . . . . . . . . . 10 3.1.7. Established Session . . . . . . . . . . . . . . . . . 10
3.1.8. Session Attempt Failure . . . . . . . . . . . . . . . 11 3.1.8. Session Attempt Failure . . . . . . . . . . . . . . . 11
3.2. Test Components . . . . . . . . . . . . . . . . . . . . . 11 3.2. Test Components . . . . . . . . . . . . . . . . . . . . . 11
3.2.1. Emulated Agent . . . . . . . . . . . . . . . . . . . . 11 3.2.1. Emulated Agent . . . . . . . . . . . . . . . . . . . . 11
3.2.2. Signaling Server . . . . . . . . . . . . . . . . . . . 12 3.2.2. Signaling Server . . . . . . . . . . . . . . . . . . . 12
3.2.3. SIP Transport Protocol . . . . . . . . . . . . . . . . 12 3.2.3. SIP Transport Protocol . . . . . . . . . . . . . . . . 12
3.3. Test Setup Parameters . . . . . . . . . . . . . . . . . . 13 3.3. Test Setup Parameters . . . . . . . . . . . . . . . . . . 13
3.3.1. Session Attempt Rate . . . . . . . . . . . . . . . . . 13 3.3.1. Session Attempt Rate . . . . . . . . . . . . . . . . . 13
3.3.2. Establishment Threshold Time . . . . . . . . . . . . . 13 3.3.2. Establishment Threshold Time . . . . . . . . . . . . . 13
3.3.3. Session Duration . . . . . . . . . . . . . . . . . . . 14 3.3.3. Session Duration . . . . . . . . . . . . . . . . . . . 14
3.3.4. Media Packet Size . . . . . . . . . . . . . . . . . . 14 3.3.4. Media Packet Size . . . . . . . . . . . . . . . . . . 14
3.3.5. Codec Type . . . . . . . . . . . . . . . . . . . . . . 15 3.3.5. Codec Type . . . . . . . . . . . . . . . . . . . . . . 15
3.4. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15 3.4. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15
3.4.1. Session Establishment Rate . . . . . . . . . . . . . . 15 3.4.1. Session Establishment Rate . . . . . . . . . . . . . . 16
3.4.2. Registration Rate . . . . . . . . . . . . . . . . . . 16 3.4.2. Registration Rate . . . . . . . . . . . . . . . . . . 16
3.4.3. Registration Attempt Rate . . . . . . . . . . . . . . 17 3.4.3. Registration Attempt Rate . . . . . . . . . . . . . . 17
4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17
5. Security Considerations . . . . . . . . . . . . . . . . . . . 17 5. Security Considerations . . . . . . . . . . . . . . . . . . . 17
6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 18 6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 18
7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 18
7.1. Normative References . . . . . . . . . . . . . . . . . . . 18 7.1. Normative References . . . . . . . . . . . . . . . . . . . 18
7.2. Informational References . . . . . . . . . . . . . . . . . 19 7.2. Informational References . . . . . . . . . . . . . . . . . 19
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19
skipping to change at page 6, line 15 skipping to change at page 6, line 15
2.1. Scope 2.1. Scope
The scope of this document is summarized as follows: The scope of this document is summarized as follows:
o This terminology document describes SIP signaling performance o This terminology document describes SIP signaling performance
benchmarks for black-box measurements of SIP networking devices. benchmarks for black-box measurements of SIP networking devices.
Stress and debug scenarios are not addressed in this document. Stress and debug scenarios are not addressed in this document.
o The DUT must be RFC 3261 capable network equipment. This may be a o The DUT must be RFC 3261 capable network equipment. This may be a
Registrar, Redirect Server, or Stateful Proxy. This document does Registrar, Redirect Server, or Stateful Proxy. This document does
not require the intermediary to assume the role of a stateless not require the intermediary to assume the role of a stateless
proxy. A DUT MAY also include a B2BUA, SBC functionality. proxy. A DUT may also include a B2BUA, SBC functionality.
o The Tester acts as multiple "Emulated Agents" (EA) that initiate o The Tester acts as multiple "Emulated Agents" (EA) that initiate
(or respond to) SIP messages as session endpoints and source (or (or respond to) SIP messages as session endpoints and source (or
receive) associated media for established connections. receive) associated media for established connections.
o SIP Signaling in presence of media o SIP Signaling in presence of media
* The media performance is not benchmarked. * The media performance is not benchmarked.
* Some tests require media, but the use of media is limited to * Some tests require media, but the use of media is limited to
observing the performance of SIP signaling. Tests that require observing the performance of SIP signaling. Tests that require
media will annotate the media characteristics as a condition of media will annotate the media characteristics as a condition of
test. test.
* The type of DUT dictates whether the associated media streams * The type of DUT dictates whether the associated media streams
skipping to change at page 7, line 21 skipping to change at page 7, line 21
this document. This is because our goal is to determine the this document. This is because our goal is to determine the
maximum capacity of the device or system under test, that is the maximum capacity of the device or system under test, that is the
number of simultaneous SIP sessions that the device or system can number of simultaneous SIP sessions that the device or system can
support. It is true that there are BYE requests being created support. It is true that there are BYE requests being created
during the test process. These transactions do contribute to the during the test process. These transactions do contribute to the
load on the device or system under test and thus are accounted for load on the device or system under test and thus are accounted for
in the metric we derive. We do not seek a separate metric for the in the metric we derive. We do not seek a separate metric for the
number of BYE transactions a device or system can support. number of BYE transactions a device or system can support.
o IMS-specific scenarios are not considered, but test cases can be o IMS-specific scenarios are not considered, but test cases can be
applied with 3GPP-specific SIP signaling and the P-CSCF as a DUT. applied with 3GPP-specific SIP signaling and the P-CSCF as a DUT.
o The benchmarks described in this document are intended for a
laboratory environment and are not intended to be used on a
production network. Some of the benchmarks send enough traffic
that a denial of service attack is possible if used in production
networks.
3. Term Definitions 3. Term Definitions
3.1. Protocol Components 3.1. Protocol Components
3.1.1. Session 3.1.1. Session
Definition: Definition:
The combination of signaling and media messages and associated The combination of signaling and media messages and associated
processing that enable a single SIP-based audio or video call, or processing that enable a single SIP-based audio or video call, or
skipping to change at page 8, line 31 skipping to change at page 8, line 36
None. None.
See Also: See Also:
Media Plane Media Plane
EAs EAs
3.1.3. Media Plane 3.1.3. Media Plane
Definition: Definition:
The data plane in which one or more media streams and their The data plane in which one or more media streams and their
associated media control protocols are exchanged between User associated media control protocols (e.g., RTCP [RFC3550]) are
Agents after a media connection has been created by the exchange exchanged between User Agents after a media connection has been
of signaling messages in the Signaling Plane. created by the exchange of signaling messages in the Signaling
Plane.
Discussion: Discussion:
Media may also be known as the "bearer channel". The Media Plane Media may also be known as the "bearer channel". The Media Plane
MUST include the media control protocol, if one is used, and the MUST include the media control protocol, if one is used, and the
media stream(s). Examples of media are audio and video. The media stream(s). Examples of media are audio and video. The
media streams are described in the SDP of the Signaling Plane. media streams are described in the SDP of the Signaling Plane.
Measurement Units: Measurement Units:
N/A. N/A.
Issues: Issues:
None. None.
See Also: See Also:
Signaling Plane Signaling Plane
3.1.4. Associated Media 3.1.4. Associated Media
skipping to change at page 9, line 12 skipping to change at page 9, line 20
See Also: See Also:
Signaling Plane Signaling Plane
3.1.4. Associated Media 3.1.4. Associated Media
Definition: Definition:
Media that corresponds to an 'm' line in the SDP payload of the Media that corresponds to an 'm' line in the SDP payload of the
Signaling Plane. Signaling Plane.
Discussion: Discussion:
Any media protocol MAY be used. The format of the media is determined by the SDP attributes for
the corresponding 'm' line.
Measurement Units: Measurement Units:
N/A. N/A.
Issues: Issues:
None. None.
3.1.5. Overload 3.1.5. Overload
Definition: Definition:
skipping to change at page 13, line 47 skipping to change at page 14, line 10
Definition: Definition:
Configuration of the EA that represents the amount of time that an Configuration of the EA that represents the amount of time that an
EA client will wait for a response from an EA server before EA client will wait for a response from an EA server before
declaring a Session Attempt Failure. declaring a Session Attempt Failure.
Discussion: Discussion:
This time duration is test dependent. This time duration is test dependent.
It is RECOMMENDED that the Establishment Threshold Time value be It is RECOMMENDED that the Establishment Threshold Time value be
set to Timer B or Timer F as specified in RFC 3261, Table 4 set to Timer B or Timer F as specified in RFC 3261, Table 4
[RFC3261]. Following the default value of T1 (500ms) specified in [RFC3261].
the table and a constant multiplier of 64 gives a value of 32
seconds for this timer (i.e., 500ms * 64 = 32s).
Measurement Units: Measurement Units:
Seconds Seconds
Issues: Issues:
None. None.
See Also: See Also:
None. None.
skipping to change at page 18, line 13 skipping to change at page 18, line 19
of scope for this document. of scope for this document.
6. Acknowledgments 6. Acknowledgments
The authors would like to thank Keith Drage, Cullen Jennings, Daryl The authors would like to thank Keith Drage, Cullen Jennings, Daryl
Malas, Al Morton, and Henning Schulzrinne for invaluable Malas, Al Morton, and Henning Schulzrinne for invaluable
contributions to this document. Dale Worley provided an extensive contributions to this document. Dale Worley provided an extensive
review that lead to improvements in the documents. We are grateful review that lead to improvements in the documents. We are grateful
to Barry Constantine, William Cerveny and Robert Sparks for providing to Barry Constantine, William Cerveny and Robert Sparks for providing
valuable comments during the document's last calls and expert valuable comments during the document's last calls and expert
reviews. reviews. Al Morton and Sarah Banks have been exemplary working group
chairs, we thank them for tracking this work to completion.
7. References 7. References
7.1. Normative References 7.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
 End of changes. 13 change blocks. 
15 lines changed or deleted 22 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/