draft-ietf-bmwg-acc-bench-term-04.txt   draft-ietf-bmwg-acc-bench-term-05.txt 
Network Working Group Network Working Group
INTERNET-DRAFT INTERNET-DRAFT
Expires in: April 2005 Expires in: October 2005
Scott Poretsky Scott Poretsky
Quarry Technologies Quarry Technologies
Shankar Rao Shankar Rao
Qwest Communications Qwest Communications
October 2004 February 2005
Terminology for Accelerated Stress Benchmarking Terminology for Accelerated Stress Benchmarking
<draft-ietf-bmwg-acc-bench-term-04.txt> <draft-ietf-bmwg-acc-bench-term-05.txt>
Intellectual Property Rights (IPR) statement: Intellectual Property Rights (IPR) statement:
By submitting this Internet-Draft, I certify that any applicable By submitting this Internet-Draft, I certify that any applicable
patent or other IPR claims of which I am aware have been disclosed, or patent or other IPR claims of which I am aware have been disclosed, or
will be disclosed, and any of which I become aware will be disclosed, will be disclosed, and any of which I become aware will be disclosed,
in accordance with RFC 3668. in accordance with RFC 3668.
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
skipping to change at page 1, line 55 skipping to change at page 1, line 55
Benchmarking of networking devices. The three phases of the Stress Benchmarking of networking devices. The three phases of the Stress
Test: Startup, Instability and Recovery are defined along with the Test: Startup, Instability and Recovery are defined along with the
benchmarks and configuration terms associated with the each phase. benchmarks and configuration terms associated with the each phase.
Also defined are the Benchmark Planes fundamental to stress testing Also defined are the Benchmark Planes fundamental to stress testing
configuration, setup and measurement. The terminology is to be configuration, setup and measurement. The terminology is to be
used with the companion framework and methodology documents. used with the companion framework and methodology documents.
Table of Contents Table of Contents
1. Introduction ............................................... 3 1. Introduction ............................................... 3
2. Existing definitions ....................................... 3 2. Existing definitions ....................................... 3
3. Term definitions............................................ 3 3. Term definitions............................................ 4
3.1 General Terms............................................. 3 3.1 General Terms............................................. 4
3.1.1 Benchmark Planes...................................... 3 3.1.1 Benchmark Planes...................................... 4
3.1.2 Configuration Sets.................................... 4 3.1.2 Configuration Sets.................................... 5
3.1.3 Startup Conditions.................................... 4 3.1.3 Startup Conditions.................................... 5
3.1.4 Instability Conditions................................ 5 3.1.4 Instability Conditions................................ 6
3.1.5 Aggregate Forwarding Rate............................. 6 3.1.5 Aggregate Forwarding Rate............................. 6
3.1.6 Controlled Session Loss............................... 6 3.1.6 Controlled Session Loss............................... 7
3.1.7 Uncontrolled Session Loss............................. 6 3.1.7 Uncontrolled Session Loss............................. 7
3.2 Benchmark Planes.......................................... 7 3.2 Benchmark Planes.......................................... 8
3.2.1 Control Plane......................................... 7 3.2.1 Control Plane......................................... 8
3.2.2 Data Plane............................................ 7 3.2.2 Data Plane............................................ 8
3.2.3 Management Plane...................................... 8 3.2.3 Management Plane...................................... 8
3.2.4 Security Plane........................................ 8 3.2.4 Security Plane........................................ 9
3.3 Startup................................................... 9 3.3 Startup...................................................10
3.3.1 Startup Phase......................................... 9 3.3.1 Startup Phase.........................................10
3.3.2 Benchmarks............................................10 3.3.2 Benchmarks............................................10
3.3.2.1 Stable Aggregate Forwarding Rate..................10 3.3.2.1 Stable Aggregate Forwarding Rate..................10
3.3.2.2 Stable Latency....................................10 3.3.2.2 Stable Latency....................................11
3.3.2.3 Stable Session Count..............................11 3.3.2.3 Stable Session Count..............................11
3.3.3 Control Plane.........................................11 3.3.3 Control Plane.........................................12
3.3.3.1 Control Plane Configuration Set...................11 3.3.3.1 Control Plane Configuration Set...................12
3.3.3.2 Control Plane Startup Conditions..................12 3.3.3.2 Control Plane Startup Conditions..................13
3.3.4 Data Plane............................................12 3.3.4 Data Plane............................................13
3.3.4.1 Data Plane Configuration Set......................12 3.3.4.1 Data Plane Configuration Set......................13
3.3.4.2 Traffic Profile...................................13 3.3.4.2 Traffic Profile...................................13
3.3.5 Management Plane......................................13 3.3.5 Management Plane......................................14
3.3.5.1 Management Plane Configuration Set................13 3.3.5.1 Management Plane Configuration Set................14
3.3.6 Security Plane........................................14 3.3.6 Security Plane........................................15
3.3.6.1 Security Plane Configuration Set..................14 3.3.6.1 Security Plane Configuration Set..................15
3.3.6.2 Security Plane Startup Conditions.................15 3.3.6.2 Security Plane Startup Conditions.................16
3.4 Instability...............................................15 3.4 Instability...............................................16
3.4.1 Instability Phase.....................................15 3.4.1 Instability Phase.....................................16
3.4.2 Benchmarks............................................16 3.4.2 Benchmarks............................................17
3.4.2.1 Unstable Aggregate Forwarding Rate................16 3.4.2.1 Unstable Aggregate Forwarding Rate................17
3.4.2.2 Degraded Aggregate Forwarding Rate................17 3.4.2.2 Degraded Aggregate Forwarding Rate................17
3.4.2.3 Average Degraded Aggregate Forwarding Rate........17 3.4.2.3 Average Degraded Aggregate Forwarding Rate........18
3.4.2.4 Unstable Latency..................................17 3.4.2.4 Unstable Latency..................................18
3.4.2.5 Unstable Uncontrolled Sessions Lost...............18 3.4.2.5 Unstable Uncontrolled Sessions Lost...............19
3.4.3 Instability Conditions................................18 3.4.3 Instability Conditions................................19
3.4.3.1 Control Plane Instability Conditions..............18 3.4.3.1 Control Plane Instability Conditions..............19
3.4.3.2 Data Plane Instability Conditions.................19 3.4.3.2 Data Plane Instability Conditions.................20
3.4.3.3 Management Plane Instability Conditions...........19 3.4.3.3 Management Plane Instability Conditions...........20
3.4.3.4 Security Plane Instability Conditions.............20 3.4.3.4 Security Plane Instability Conditions.............20
3.5 Recovery..................................................20 3.5 Recovery..................................................21
3.5.1 Recovery Phase........................................20 3.5.1 Recovery Phase........................................21
3.5.2 Benchmarks............................................21 3.5.2 Benchmarks............................................21
3.5.2.1 Recovered Aggregate Forwarding Rate...............21 3.5.2.1 Recovered Aggregate Forwarding Rate...............21
3.5.2.2 Recovered Latency.................................21 3.5.2.2 Recovered Latency.................................22
3.5.2.3 Recovery Time.....................................22 3.5.2.3 Recovery Time.....................................22
3.5.2.4 Recovered Uncontrolled Sessions Lost..............22 3.5.2.4 Recovered Uncontrolled Sessions Lost..............23
3.5.2.5 Variability Benchmarks............................23 3.5.2.5 Variability Benchmarks............................23
4. Security Considerations.....................................23 4. Security Considerations.....................................24
5. References..................................................23 5. Normative References........................................24
6. Author's Address............................................24 6. Informative References......................................24
Appendix 1 - White Box Benchmarks..............................24 7. Author's Address............................................25
Appendix 1 - White Box Benchmarks..............................25
1. Introduction 1. Introduction
Routers in an operational network are simultaneously configured with Routers in an operational network are simultaneously configured with
multiple protocols and security policies while forwarding traffic and multiple protocols and security policies while forwarding traffic and
being managed. To accurately benchmark a router for deployment it is being managed. To accurately benchmark a router for deployment it is
necessary to test that router in operational conditions by necessary to test that router in operational conditions by
simultaneously configuring and scaling network protocols and security simultaneously configuring and scaling network protocols and security
policies, forwarding traffic, and managing the device. It is helpful policies, forwarding traffic, and managing the device. It is helpful
to accelerate these network operational conditions so that the to accelerate these network operational conditions so that the
router under test can be benchmarked with faster test duration. router under test can be benchmarked with faster test duration.
skipping to change at page 3, line 30 skipping to change at page 3, line 36
benchmark and configuration terms associated with the each phase. benchmark and configuration terms associated with the each phase.
Benchmarks for stress testing are defined using the Aggregate Benchmarks for stress testing are defined using the Aggregate
Forwarding Rate and control plane Session Count during each phase Forwarding Rate and control plane Session Count during each phase
of the test. Also defined are the Benchmark Planes fundamental to of the test. Also defined are the Benchmark Planes fundamental to
stress testing configuration, setup and measurement. These are stress testing configuration, setup and measurement. These are
the Control Plane, Data Plane, Management Plane and Security Plane the Control Plane, Data Plane, Management Plane and Security Plane
For each plane, the Configuration Set, Startup Conditions, and For each plane, the Configuration Set, Startup Conditions, and
Instability Conditions are defined. White Box benchmarks are Instability Conditions are defined. White Box benchmarks are
provided in Appendix 1 for additional DUT behavior measurements. provided in Appendix 1 for additional DUT behavior measurements.
The terminology is to be used with the companion methodology The terminology is to be used with the companion methodology
document [6]. document [4]. The sequence of phases, actions, and benchmarks
are shown in Table 1.
2. Existing definitions 2. Existing definitions
RFC 1242 "Benchmarking Terminology for Network Interconnect RFC 1242 "Benchmarking Terminology for Network Interconnect
Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching
Devices" should be consulted before attempting to make use of this Devices" should be consulted before attempting to make use of this
document. For the sake of clarity and continuity this RFC adopts
the template for definitions set out in Section 2 of RFC 1242.
Definitions are indexed and grouped together in sections for ease
of reference.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track
document. document.
For the sake of clarity and continuity this RFC adopts the template Table 1. Phase Sequence and Benchmarks
for definitions set out in Section 2 of RFC 1242. Definitions are III. Recovery Phase II. Instability Phase I. Startup Phase
indexed and grouped together in sections for ease of reference. <-----------------<---<-------------------<----<--------------<
Remove Instability Achieve Configuration Apply Startup
Conditions Set Conditions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", Benchmark: Benchmark: Benchmark:
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in Recovered Aggregate Unstable Aggregate Stable Aggregate
this document are to be interpreted as described in RFC 2119. Forwarding Rate Forwarding Rate Forwarding Rate
Degraded Aggregate
Forwarding Rate
Average Degraded
Forwarding Rate
Recovered Latency Unstable Latency Startup Latency
Recovered Uncontrolled Recovered Uncontrolled Stable Session Count
Sessions Lost Sessions Lost
Recovery Time
3. Term definitions 3. Term definitions
3.1 General Terms 3.1 General Terms
3.1.1 Benchmark Planes 3.1.1 Benchmark Planes
Definition: Definition:
The features, conditions, and behavior for the Accelerated Stress The features, conditions, and behavior for the Accelerated Stress
Benchmarking. Benchmarking.
Discussion: Discussion:
There are four Benchmark Planes: Control Plane, Data Plane, There are four Benchmark Planes: Control Plane, Data Plane,
Management Plane, and Security Plane as shown in Figure 1. The Management Plane, and Security Plane as shown in Figure 1. The
Benchmark Planes define the Configuration, Startup Conditions, Benchmark Planes define the Configuration, Startup Conditions,
Instability Conditions, and Failure Conditions used for the test. Instability Conditions, and Failure Conditions used for the test.
Measurement units:
N/A
Issues:
None
See Also:
Control Plane
Data Plane
Management Plane
Security Plane
___________ ___________ ___________ ___________
| Control | | Management| | Control | | Management|
| Plane |___ ___| Plane | | Plane |___ ___| Plane |
| | | | | | | | | | | |
----------- | | ----------- ----------- | | -----------
\/ \/ ___________ \/ \/ ___________
___________ | Security | ___________ | Security |
| |<-----------| Plane | | |<-----------| Plane |
| DUT | | | | DUT | | |
|--->| |<---| ----------- |--->| |<---| -----------
| ----------- | | ----------- |
| | | |
| ___________ | | ___________ |
| | Data | | | | Data | |
|--->| Plane |<---| |--->| Plane |<---|
| | | |
----------- -----------
Figure 1. Router Accelerated Stress Benchmarking Planes Figure 1. Router Accelerated Stress Benchmarking Planes
Measurement units:
N/A
Issues:
None
See Also:
Control Plane
Data Plane
Management Plane
Security Plane
3.1.2 Configuration Sets 3.1.2 Configuration Sets
Definition: Definition:
The features and scaling limits used during the Accelerated Stress The features and scaling limits used during the Accelerated Stress
Benchmarking. Benchmarking.
Discussion: Discussion:
There are four Configuration Sets: Control Plane Configuration Set, There are four Configuration Sets: Control Plane Configuration Set,
Data Plane Configuration Set, Management Plane Configuration Set, Data Plane Configuration Set, Management Plane Configuration Set,
and Security Plane Configuration Set. and Security Plane Configuration Set.
skipping to change at page 5, line 52 skipping to change at page 6, line 37
Discussion: Discussion:
Instability Conditions are applied to the DUT after the Instability Conditions are applied to the DUT after the
Startup Conditions have completed. Instability Conditions Startup Conditions have completed. Instability Conditions
occur for the Control Plane, Data Plane, Management Plane, occur for the Control Plane, Data Plane, Management Plane,
and Security Plane. and Security Plane.
Measurement units: Measurement units:
N/A N/A
Issues: Issues: None
None
See Also: See Also:
Control Plane Instability Conditions Control Plane Instability Conditions
Data Plane Instability Conditions Data Plane Instability Conditions
Management Plane Instability Conditions Management Plane Instability Conditions
Security Plane Instability Conditions Security Plane Instability Conditions
3.1.5 Aggregate Forwarding Rate 3.1.5 Aggregate Forwarding Rate
Definition: Definition:
Sum of forwarding rates for all interfaces on the Sum of forwarding rates for all interfaces on the
skipping to change at page 9, line 13 skipping to change at page 9, line 50
features and protocols. The Security Plane includes the features and protocols. The Security Plane includes the
ACLs, Firewall, Secure Protocols, and User Login. Tunnels ACLs, Firewall, Secure Protocols, and User Login. Tunnels
for those such as IPsec should be established and flapped. for those such as IPsec should be established and flapped.
Policies for Firewalls and ACLs should be repeatedly added Policies for Firewalls and ACLs should be repeatedly added
and removed from the configuration via telnet, SSH, or and removed from the configuration via telnet, SSH, or
serial management sessions. serial management sessions.
Measurement units: Measurement units:
N/A N/A
Issues: Issues: None
None
See Also: See Also:
Benchmark Planes Benchmark Planes
Security Plane Configuration Set Security Plane Configuration Set
Security Plane Startup Conditions Security Plane Startup Conditions
Security Plane Instability Conditions Security Plane Instability Conditions
3.3 Startup 3.3 Startup
3.3.1 Startup Phase 3.3.1 Startup Phase
Definition Definition
The portion of the benchmarking test in which the The portion of the benchmarking test in which the
Startup Conditions are generated with the DUT. This Startup Conditions are generated with the DUT. This
begins with the attempt to establish the first session begins with the attempt to establish the first session
and ends when the last Control Plane session is and ends when the last Control Plane session is
established. established.
skipping to change at page 13, line 8 skipping to change at page 13, line 42
The data traffic profile enabled for the Accelerated Stress The data traffic profile enabled for the Accelerated Stress
Benchmarking. Benchmarking.
Discussion: Discussion:
Data Plane Configuration Set includes the Traffic Profile and Data Plane Configuration Set includes the Traffic Profile and
interfaces used for the Accelerated Stress Benchmarking. interfaces used for the Accelerated Stress Benchmarking.
Measurement Units: Measurement Units:
N/A N/A
Issues: Issues: None
None
See Also: See Also:
Traffic Profile Traffic Profile
3.3.4.2 Traffic Profile 3.3.4.2 Traffic Profile
Definition Definition
The characteristics of the Offered Load to the DUT used for The characteristics of the Offered Load to the DUT used for
the Accelerated Stress Benchmarking. the Accelerated Stress Benchmarking.
Discussion Discussion
skipping to change at page 22, line 11 skipping to change at page 22, line 44
Recovery Phase. Unstable Latency is the difference Recovery Phase. Unstable Latency is the difference
between Stable Latency and the average Latency measured between Stable Latency and the average Latency measured
during the Recovery Phase. It is expected that there during the Recovery Phase. It is expected that there
be no increase in average latency from the Startup Phase be no increase in average latency from the Startup Phase
to the Recovery Phase. The Recovered Latency cannot be a to the Recovery Phase. The Recovered Latency cannot be a
negative number. negative number.
Measurement units: Measurement units:
seconds seconds
Issues: Issues: None
None
See Also: See Also:
Recovery Phase Recovery Phase
Stable Latency Stable Latency
3.5.2.3 Recovery Time 3.5.2.3 Recovery Time
Definition Definition
The amount of time for the Recovered Aggregate Forwarding The amount of time for the Recovered Aggregate Forwarding
Rate to become equal to the Stable Aggregate Forwarding Rate. Rate to become equal to the Stable Aggregate Forwarding Rate.
skipping to change at page 23, line 43 skipping to change at page 24, line 24
Startup Period Startup Period
Instability Period Instability Period
Recovery Period Recovery Period
4. Security Considerations 4. Security Considerations
Documents of this type do not directly effect the security of Documents of this type do not directly effect the security of
the Internet or of corporate networks as long as benchmarking the Internet or of corporate networks as long as benchmarking
is not performed on devices or systems connected to operating is not performed on devices or systems connected to operating
networks. networks.
5. References 5. Normative References
[1] Bradner, S., Editor, "Benchmarking Terminology for Network [1] Bradner, S., Editor, "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, October 1991. Interconnection Devices", RFC 1242, October 1991.
[2] Mandeville, R., "Benchmarking Terminology for LAN Switching [2] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, June 1998. Devices", RFC 2285, June 1998.
[3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[4] "Core Router Evaluation for Higher Availability", Scott [4] Poretsky, S. and Rao, S., "Methodology for Accelerated
Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-01,
work in progress, February 2005.
6. Informative References
[RFC3871] RFC 3871 "Operational Security Requirements for Large
Internet Service Provider (ISP) IP Network Infrastructure.
G. Jones, Ed.. IETF, September 2004.
[NANOG25] "Core Router Evaluation for Higher Availability", Scott
Poretsky, NANOG 25, June 8, 2002, Toronto, CA. Poretsky, NANOG 25, June 8, 2002, Toronto, CA.
[5] "Router Stress Testing to Validate Readiness for Network [IEEECQR] "Router Stress Testing to Validate Readiness for Network
Deployment", Scott Poretsky, IEEE CQR 2003. Deployment", Scott Poretsky, IEEE CQR 2003.
[6] Poretsky, S. and Rao, S., "Methodology for Accelerated [CONVMETH] Poretsky, S., "Benchmarking Methodology for IGP Data Plane
Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-01, Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-05,
work in progress, October 2004. work in progress, February 2005.
6. Author's Address [CONVTERM] Poretsky, S., "Benchmarking Terminology for IGP Data Plane
Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-05,
work in progress, February 2005.
7. Author's Address
Scott Poretsky Scott Poretsky
Quarry Technologies Quarry Technologies
8 New England Executive Park 8 New England Executive Park
Burlington, MA 01803 Burlington, MA 01803
USA USA
Phone: + 1 781 395 5090 Phone: + 1 781 395 5090
EMail: sporetsky@quarrytech.com EMail: sporetsky@quarrytech.com
Shankar Rao Shankar Rao
skipping to change at page 24, line 44 skipping to change at page 25, line 35
Minimum DUT Available Memory during the duration of the Minimum DUT Available Memory during the duration of the
Accelerated Stress Test. Accelerated Stress Test.
Discussion: Discussion:
It is necessary to monitor DUT memory to measure this It is necessary to monitor DUT memory to measure this
benchmark. benchmark.
Measurement units: Measurement units:
bytes bytes
Issues: Issues: None
None
See Also: See Also:
Maximum CPU Utilization Maximum CPU Utilization
Maximum CPU Utilization Maximum CPU Utilization
Definition: Definition:
Maximum DUT CPU utilization during the duration of the Maximum DUT CPU utilization during the duration of the
Accelerated Stress Test. Accelerated Stress Test.
Discussion: Discussion:
It is necessary to monitor DUT CPU Utilization to measure It is necessary to monitor DUT CPU Utilization to measure
this benchmark. this benchmark.
Measurement units: % Measurement units: %
skipping to change at page 25, line 11 skipping to change at page 25, line 51
Definition: Definition:
Maximum DUT CPU utilization during the duration of the Maximum DUT CPU utilization during the duration of the
Accelerated Stress Test. Accelerated Stress Test.
Discussion: Discussion:
It is necessary to monitor DUT CPU Utilization to measure It is necessary to monitor DUT CPU Utilization to measure
this benchmark. this benchmark.
Measurement units: % Measurement units: %
Issues: Issues: None
None
See Also: See Also:
Minimum Available Memory Minimum Available Memory
Intellectual Property Statement Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any Intel- The IETF takes no position regarding the validity or scope of any Intel-
lectual Property Rights or other rights that might be claimed to pertain lectual Property Rights or other rights that might be claimed to pertain
to the implementation or use of the technology described in this docu- to the implementation or use of the technology described in this docu-
ment or the extent to which any license under such rights might or might ment or the extent to which any license under such rights might or might
skipping to change at page 25, line 51 skipping to change at page 26, line 39
This document and the information contained herein are provided on an This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR
IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMA- INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMA-
TION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF TION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement Copyright Statement
Copyright (C) The Internet Society (2004). This document is subject to Copyright (C) The Internet Society (2005). This document is subject to
the rights, licenses and restrictions contained in BCP 78, and except as the rights, licenses and restrictions contained in BCP 78, and except as
set forth therein, the authors retain all their rights. set forth therein, the authors retain all their rights.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/