Routing Working Group                                              N. So
Internet Draft                                                  A. Malis
RTGWG                                                 C. Villamizar, Ed.
Internet-Draft                                      Infinera Corporation
Intended Status: Standard status: Informational                           D. McDysan McDysan, Ed.
Expires: January 9, 2011                                         S. Ning
                                                                A. Malis
                                                                 Verizon
                                                                 L. Yong
                                                              Huawei
                                                               F. Jounay
                                                          France Telecom
                                                               Y. Kamite
                                                                     NTT
                                                       February 17, USA
                                                            July 8, 2010

              Requirements for MPLS Over a Composite Link

                    draft-ietf-rtgwg-cl-requirement-00
                   draft-ietf-rtgwg-cl-requirement-01

Abstract

   There is often a need to provide large aggregates of bandwidth that
   is best provided using parallel links between routers or MPLS LSR.
   In core networks there is often no alternative since the aggregate
   capacities of core networks today far exceed the capacity of a single
   physical link or single packet processing element.  Furthermore,
   links may be composed of network elements operating across multiple
   layers.

   The presence of parallel links, potentially comprised of multiple
   layers has resulted in a additional requirements.  Certain services
   may benefit from being restricted to a subset of the set of composite
   link component links or a specific component link, where component
   link characteristics, such as latency, differ.  Certain services
   require that LSP be treated as atomic and avoid reordering.  Other
   services will continue to require only that reordering not occur with
   a microflow as is current practice.

   Current practice related to multipath is described briefly in an
   appendix.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts.
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on August 18, 2010. January 9, 2011.

Copyright Notice

   Copyright (c) 2010 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Abstract
   This document presents motivation, a problem statement and
   operational requirements for a traffic distribution problem in
   today's IP/MPLS network when multiple links are configured between
   two routers. It defines a composite link as a group of parallel links
   that can be considered as a single traffic engineering link or as an
   IP link, and used for MPLS.  The framework described in a companion
   document is used to organize the requirements.

Table of Contents

   1. Introduction...................................................3
   2. Conventions used in this document..............................4
      2.1. Acronyms..................................................4
      2.2. Terminology...............................................4
   3. Motivation and Summary Problem Statement.......................5
      3.1. Motivation................................................5
      3.2. Summary of Problems Requiring Solution....................7
   4. Requirements...................................................7
      4.1. Interior Functions........................................8
         4.1.1. Functions common to all LSP flows....................8
            4.1.1.1. Traffic Flow and Connection Mapping.............9
            4.1.1.2. Management of Other Operational Aspects.........9
               4.1.1.2.1. Resilience.................................9
               4.1.1.2.2. Fault management requirement...............9
               4.1.1.2.3. Flow/Connection Mapping Change Frequency..10
         4.1.2. Functions specific to LSP flows with TE information.11
         4.1.3. Functions specific to LSP flows without TE information12
         4.1.4. Sets of LSP flows with  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  Requirements Language  . . . . . . . . . . . . . . . . . .  4
   2.  Assumptions  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3.  Definitions  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   4.  Network Operator Functional Requirements . . . . . . . . . . .  5
     4.1.  Availability, Stability and without TE information...12
            4.1.4.1. Handling Bandwidth Shortage Events.............13 Transient Response . . . . . .  5
     4.2. Exterior Functions.......................................13
         4.2.1. Functions common to all LSP flows...................13
            4.2.1.1. Signaling Protocol Extensions..................13
            4.2.1.2. Routing Advertisement Extensions...............13
         4.2.2. Functions specific to LSP flows with TE information.14
            4.2.2.1. Signaling Protocol Extensions Requirements.....14
            4.2.2.2. Routing Advertisement Extensions...............15
         4.2.3. Functions specific to LSP flows without TE information15
            4.2.3.1. Signaling Protocol Extensions..................15
            4.2.3.2. Routing Advertisement Extensions...............16
         4.2.4. Functions specific to LSP flows  Component Links Provided by Lower Layer Networks . . . . .  6
     4.3.  Parallel Component Links with and without TE
         information................................................16
            4.2.4.1. Signaling Protocol Extensions..................16
            4.2.4.2. Routing Advertisement Extensions...............16 Different Characteristics  .  7
   5. Security Considerations.......................................16  Derived Requirements . . . . . . . . . . . . . . . . . . . . .  8
   6. IANA Considerations...........................................16  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 10
   7. References....................................................17
      7.1. Normative References.....................................17
      7.2. Informative References...................................17  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 10
   8. Acknowledgments...............................................18

1. Introduction

   IP/MPLS network traffic growth forces carriers to deploy multiple
   parallel physical/logical links between adjacent routers as the total
   capacity of all aggregated traffic flows exceed the capacity of a
   single link.  The network is expected to carry aggregated traffic
   flows some of which approach the capacity of any single link, and
   also some flows that may be very small compared to the capacity of a
   single link.

   Operating an MPLS network with multiple parallel links between all
   adjacent routers causes scaling problems in the routing protocols.
   This issue is addressed in [RFC4201] which defines the notion of a
   Link Bundle -- a set of identical parallel traffic engineered (TE)
   links (called component links) that are grouped together and
   advertised as a single TE link within the routing protocol.

   The Link Bundle concept is somewhat limited because of the
   requirement that all component links must have identical
   capabilities, and because it applies only to TE links.  This document
   sets out a more generic set of requirements for grouping together a
   set of parallel data links that may have different characteristics,
   and for advertising and operating them as a single TE or non-TE link
   called a Composite Link.

   First, there is a set of requirements related to the interior
   functioning of a router, of which the operational requirement is to
   have consistent configuration, reporting and alarm notification. The
   functions that impact the routers other than those connected by the
   composite link are control plane routing and signaling protocols. A
   further subdivision of the requirements is based upon characteristics
   of the combination of MPLS signaling protocols in use, namely RSVP-TE
   only, LDP only, or a combination of RSVP_TE and LDP. Extension
   requirements to IGP-related protocols are also described in this
   document. Furthermore, there are requirements that relate to use of
   signaling (and possibly routing) that can be used within the same
   layer or between layers.

   Applicability of the work within this document is focused  Security Considerations  . . . . . . . . . . . . . . . . . . . 10
   9.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 10
     9.1.  Normative References . . . . . . . . . . . . . . . . . . . 10
     9.2.  Informative References . . . . . . . . . . . . . . . . . . 11
     9.3.  Appendix References  . . . . . . . . . . . . . . . . . . . 11
   Appendix A.  More Details on MPLS
   traffic as controlled through control plane protocols.  Thus, this
   document describes requirements related to the routing protocols that
   advertise link parameters and the Resource Reservation Protocol
   (RSVP-TE) Existing Network Operator
                Practices and the Label Distribution Protocol (LDP) signaling
   protocols that distribute MPLS labels Usage  . . . . . . . . . . . . 12
   Appendix B.  Existing Multipath Standards and establish Label Switched Techniques . . . . . 14
     B.1.  Common Multpath Load Spliting Techniques . . . . . . . . . 15
     B.2.  Simple and Adaptive Load Balancing Multipath . . . . . . . 16
     B.3.  Traffic Split over Parallel Links  . . . . . . . . . . . . 16
     B.4.  Traffic Split over Multiple Paths (LSPs). Interactions between the control plane and the data  . . . . . . . . . . . . 17
   Appendix C.  ITU-T G.800 Composite Link Definitions and
   management planes are also addressed.
                Terminology . . . . . . . . . . . . . . . . . . . . . 17
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18

1.  Introduction

   The focus purpose of this document is
   on MPLS traffic either signaled by RSVP-TE or LDP. IP traffic over
   multiple parallel links is handled relatively well by ECMP or
   LAG/hashing methods. to describe why network operators
   require certain functions in order to solve certain business problems
   (Section 2).  The handling of IP control plane traffic intent is
   within the scope to first describe why things need to be
   done in terms of the functional requirements that are as independent as
   possible of protocol specifications (Section 4).  For certain
   functional requirements this document.

   A companion framework document [CLFRAMEWORK] further describes the
   overall context, a structure, set of derived
   protocol requirements (Section 5).  Three appendices provide
   supporting details as a summary of existing/prior operator
   approaches, a summary of implementation techniques and some standardization considerations.
   Specific relevant
   protocol solutions are outside the scope standards, and a summary of this document.

2. Conventions G.800 terminology used in this document to define
   the concept of a composite link.  (Appendix B).

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119.

2.1.  Acronyms

      BW: Bandwidth

      ECMP: Equal Cost Multi-Path

      FRR: Fast Re-Route

      LAG: Link Aggregation Group

      LDP: Label Distribution Protocol

      LSP: Label Switched Path

      MPLS: Multi-Protocol Label Switching

      OAM: Operation, Administration, RFC 2119 [RFC2119].

2.  Assumptions

   The services supported include L3VPN, L2VPN (VPWS and Management

      PDU: Protocol Data Unit

      PE: Provider Edge device

      RSVP: ReSource reserVation Protocol

      RTD: Real Time Delay

      TE: Traffic Engineering

      VRF: Virtual Routing VPLS), Internet
   traffic encapsulated by at least one MPLS label, and Forwarding

2.2. Terminology

   Composite Link: a group of component links, which can dynamically
   signaled MPLS-TP LSPs and pseudowires.  The MPLS LSPs supporting
   these services may be considered
   as pt-pt, pt-mpt, or mpt-mpt.

   The location in a single MPLS TE link network where these requirements apply are a Label
   Edge Router (LER) or as a single Label Switch Router (LSR) as defined in RFC
   3031 [RFC3031].

   The IP link DSCP cannot be used for MPLS. is The
   The ITU-T [ITU-T G.800] defines Composite Link Characteristics as
   those which makes multiple parallel component links between two
   transport nodes appear as a single logical link from the network
   perspective.  Each component link flow identification since L3VPN
   requires Diffserv transparency (see RFC 4031 5.5.2 [RFC4031]), and in a
   general network operators do not rely on the DSCP of Internet
   packets.

3.  Definitions

   Composite Link:
       Section 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite link can be
   supported by a separate server layer trail, i.e.,
       as summarized in Appendix Appendix C.  The following definitions
       map the ITU-T G.800 terminology into IETF terminology which is
       used in this document.

       Multiple parallel links:  When multiple parallel component links
   in a composite link can have
           between the same or different properties such as
   latency an LER/LSR and capacity. another LER/LSR.

       Multi-layer Component Link: a  A component link that is formed by
           other network elements at other layers.

   Component Link:  A physical link (e.g., Lambda, Ethernet PHY, SONET/
       SDH, OTN, etc.) with packet transport capability, or a logical
       link (e.g., MPLS LSP, Ethernet VLAN, MPLS-TP LSP, etc.)

   Connection: An aggregation of traffic flows which are treated
   together as a single unit by the composite link Interior Function for
   the purpose

   Flow:  A sequence of routing onto a specific packets that must be transferred on one
       component link link.

   Flow identification:  The label stack and measuring
   traffic volume.

   Exterior Functions: These are performed by an MPLS router other information that makes
       uniquely identifies a composite link useable by the network via flow.  Other information in flow
       identification may include an IP header, PW control protocols, word,
       Ethernet MAC address, etc.  Note that an LSP may contain one or
       more Flows or by an MPLS router that interacts with other routers LSP may be equivalent to dynamically
   control a component link as part a composite link. These functions
   are those that interact via routing and/or signaling protocols with
   other routers in the same layer network or other layer networks.
   Interior Functions: Actions performed by the MPLS routers directly
   connected by Flow.  Flow
       identification is used to locally select a composite link. This includes the determination of the
   connection and component link on which a traffic flow is placed.
   Although link, or a local implementation matter,
       path through the configuration control of
   certain aspects of these interior functions is an important
   operational requirement.

   Traffic Flow: A set of packets that with common identifier
   characteristics that network toward the composite link is able to use to aggregate
   traffic into Connections.  Identifiers can be an MPLS label stack or
   any combination of IP addresses and protocol types for routing,
   signaling and management packets.

   Virtual Interface: Composite link characteristics advertised destination.

4.  Network Operator Functional Requirements

   The Functional Requirements in IGP

3. Motivation and Summary Problem Statement

3.1. Motivation

   There this section are several established approaches to using multiple parallel
   links between a pair grouped in
   subsections starting with the highest priority.

4.1.  Availability, Stability and Transient Response

   Limiting the period of routers.  These have limitations unavailability in response to failures or
   transient events is extremely important as
   summarized below.

   o  ECMP/Hashing/LAG: IP traffic composed well as maintaining
   stability.  The transient period between some service disrupting
   event and the convergence of the routing and/or signaling protocols
   MUST occur within a large number time frame specified by SLA objectives.  The
   timeframes range from rapid restoration, on the order of flows
      with bandwidth that is small with respect 100 ms or
   less (e.g., for VPWS), to several minutes (e.g., for L3VPN) and may
   differ among the individual link
      capacity can be handled relatively well using ECMP/LAG approaches.
      However, these approaches do not make use set of MPLS control plane
      information nor traffic volume information. Distribution
      techniques applied only customers within a single service.

   FR#1  The solution SHALL provide a means to summarize routing
         advertisements regarding the data plane can result in less
      than ideal load balancing across component links characteristics of a composite
      link.

   o  Advertisement of each component
         link into such that the IGP. Although this
      would address routing protocol convergence within the problem, it has
         timeframe needed to meet the SLA objective..

   FR#2  The solution SHALL provide a scaling impact on IGP routing,
      and was an important motivation means for the specification of link
      bundling [RFC4201]. However, there are two gaps aggregating signaling
         such that in link bundling:

   o     1.  It only supports RSVP-TE, not LDP.

   o     2.  It does not support response to a set of component links with different
      characteristics (e.g., different bandwidth and/or latency).

      For example, failure in practice carriers commonly use link bandwidth and
      link latency to set link TE metrics for RSVP-TE.  For RSVP-TE,
      limiting the component links worst case cross
         section of the network that MPLS LSPs are restored within the
         timeframe needed to same TE metric has meet the practical
      effect SLA objective.

   FR#3  The solution SHALL provide to select a path for a flow across a
         network that contains a number of dis-allowing component links with different link
      bandwidth and latencies.

   o  Inverse Multiplexing: Making multiple parallel paths comprised of pairs of
         nodes connected by composite links to appear as
      a single link using inverse multiplexing techniques, in such a way as
      proposals under discussion in to
         automatically distribute the [PWBONDING]. However, load over the
      inverse multiplexed link will have a latency network nodes
         connected by composite links while meeting all of the link with the
      largest latency. When there is a mix of latency sensitive traffic
      with other traffic that is less sensitive
         mandatory requirements stated above.  The solution SHOULD work
         in a manner similar to latency, having all
      traffic experience that when the latency characteristics of the worst link is not acceptable
         individual component links are advertised.

   FR#4  If extensions to operators.

   o  Planning Tool LSP Assignment: Although theoretically optimal, an
      external system that participates in the IGP, measures traffic and
      assigns TE LSPs existing protocols are specified and/or adjusts IGP metrics has new
         protocols are defined, then the solution SHOULD provide a potentially large
      response time means
         for a network operator to certain failure scenarios. Furthermore, such migrate an existing deployment in a
      system could make use of more information than provided
         minimally disruptive manner.

   FR#5  Any automatic LSP routing and/or load balancing solutions MUST
         not oscillate such that performance observed by link
      bundling IGP advertisements and could make use of mechanisms users changes
         such that
      would allow pinning MPLS traffic an SLA is violated.  Since oscillation may cause
         reordering, there MUST be means to a particular component link in
      a composite link.

   o  In a multi-layer network, control the characteristics frequency of a
         changing the component link
      can over which a flow is placed.

   FR#6  Management and diagnostic protocols MUST be altered able to operate
         over composite links.

4.2.  Component Links Provided by Lower Layer Networks

   Case 3 as defined in [ITU-T.G.800] involves a component link
   supporting an MPLS layer network over another lower layer network and this can create
      significant operational impact in some cases. For example, if a
   (e.g., circuit switched or another MPLS network (e.g., MPLS-TP)).
   The lower layer network may change the latency (and/or other
   performance parameters) seen by the MPLS layer network.  Network
   Operators have SLAs of which some components are based on performance
   parameters.  Currently, there is no protocol for the lower layer
   network performs restoration and markedly increases to inform the latency higher layer network of a link change in a link bundle, the traffic placed on this
      longer latency link may generate user complaints and/or exceed the
      parameters
   performance parameter.  Communication of a Service Level Agreement (SLA).

   o  In the case where multiple routing instances could share a
      composite link, inefficiency can result if either 1) specific
      component links are assigned to an individual routing instance, or
      2) if statically assigned capacity latency performance
   parameter is made to a logical/sub
      interface in each component link very important requirement.  Communication of a composite link for each
      routing instance. In other words, the issue
   performance parameters (e.g., delay variation) is that unused
      capacity in one routing instance cannot be used by another in
      either of these cases.

   o  Unicast LDP signaled LSP flows follow desirable.

   FR#7   In order to support network SLAs and provide acceptable user
          experience, the IGP determined topology
      in solution SHALL specify a multipoint-to-point manner. The principal means of control of
      LDP flows is through adjustment of the IGP metric. The simplicity
      of this method is attractive. However, this protocol means that the LDP
      flow traffic level can change significantly in response to some
      link and/or node failures, thus significantly change the traffic
      level at points within
          allow a network. A means for operators lower layer server network to better
      manage LDP signaled flows is therefore highly desirable.

3.2. Summary of Problems Requiring Solution

   The following bullets highlight aspects of solutions for which
   detailed requirements are stated in Section 5.

   o  Ensure the ability communicate latency to transport both RSVP-TE and LDP signaled non-
      TE LSPs on
          the same composite link (i.e., a single set higher layer client network.

   FR#8   The precision of
      component links) while maintaining acceptable service quality latency reporting SHOULD be at least 10% of
          the one way latency for
      both RSVP-TE and LDP signaled LSPs.

   o  Extend latency of 1 ms or more.

   FR#9   The solution SHALL provide a link bundling type function means to scenarios with groups of
      links having different characteristics (e.g., bandwidth, latency).

   o  When an end-to-end limit the latency on a
          per LSP signaled by RSVP-TE uses basis between nodes within a composite link, network to meet an SLA
          target when the composite link must select path between these nodes contains one or more
          pairs of nodes connected via a component link that meets composite link.

          The SLAs differ across the
      end-to-end requirements services, and some services have
          different SLAs for the LSP. To perform different QoS classes, for example, one QoS
          class may have a much larger latency bound than another.
          Overload can occur which would violate an SLA parameter (e.g.,
          loss) and some remedy to handle this function, case for a composite
          link.

   FR#10  If the
      network elements at total demand offered by traffic flows exceeds the end
          capacity of a the composite link must be made aware
      of link, the required, desired, and acceptable link characteristics
      (e.g., latency, optimization frequency) solution SHOULD define a
          means to cause the LSPs for each hop some traffic flows to move to some
          other point in the path.
      The solution should network that is not congested.  These
          "preempted LSPs" may not be able restored if there is no
          uncongested path in the network.

4.3.  Parallel Component Links with Different Characteristics

   Corresponding to operate Case 1 of [ITU-T.G.800], as one means to provide
   high availability, network operators deploy a topology in the MPLS
   network using lower layer networks that have a statically configured
      or dynamically signaled manner.

   o  Support sets certain degree of component links between routers across
      intermediate nodes
   diversity at the same and/or lower layers where layer(s).  Many techniques have been developed
   to balance the
      characteristics (e.g., latency) distribution of said flows across component links may change
      dynamically. The solution should support the case where that
   connect the
      changes in characteristics same pair of these links are not communicated by nodes (See Appendix B.3).  When the IGP (e.g., path for
   a link in flow can be chosen from a lower layer network has set of candidate nodes connected via
   composite links, other techniques have been developed (See
   Appendix B.4).

   FR#11  The solution SHALL measure traffic on a change labeled traffic flow
          and dynamically select the component link on which to place
          this flow in
      latency or QoS due order to a restoration action). The solution could
      measure balance the performance (e.g., latency), and/or receive load so that no component
          link in the
      results composite link between a pair of nodes is
          overloaded.

   FR#12  When a measurement or computation traffic flow is moved from lower layer
      configuration data via signaling.

4. Requirements

   As defined one component link to
          another in the terminology section same composite link between a (traffic) flow is the
   smallest unit set of traffic assignable to nodes (or
          sites), it MUST be done so in a connection, and connections
   are assigned to minimally disruptive manner.

          When a component link that flow is part of moved from a composite link.
   The composite current link to a target link has routing information, normal IGP that has no TE
   information and IGP with TE extensions (IGP-TE) and signaling
   information with TE information (RSVP-TE) and without TE information.
   The following table summarizes the three cases covered in this
   requirements section.

   Flows                   IGP  IGP-TE  RSVP-TE  LDP

   ----------------------- ---  ------  -------  ---

   With TE Info             Y     Y       Y       N

   Wihtout TE Info          Y     N       N       Y

   With & Without TE Info   Y     Y       Y       Y

   Furthermore,
          different latency, reordering can occur if a requirement would be repeated for each of the above
   three cases (e.g., IGP related routing information) it target link
          latency is described
   in a section common to all flows.

   Therefore, the outline less than that of this section the current or clumping can occur
          if target link latency is structured as follows:

   o  Management/Measurement of Interior Functions

      - Functions common to all LSP flows

      - Functions specific to LSP flows with TE information

      - Functions specific to LSP flows without TE information

      - Sets greater than that of LSP flows with and without TE information

   o  Exterior Functions

      - Functions common to all LSP the current.
          Therefore, some flows

      - Functions specific (e.g., timing distribution, PW circuit
          emulation) are quite sensitive to LSP flows with TE information

      - Functions specific these effects, which may be
          specified in an SLA or are needed to LSP flows without TE information

      - Sets of LSP flows with and without TE information

4.1. Interior Functions

4.1.1. Functions common meet a user experience
          objective (e.g. jitter buffer under/overrun).

   FR#13  The solution SHALL provide a means to all LSP identify flows whose
          rearrangement frequency needs to be bounded by a configured
          value.

   FR#14  The operator solution SHALL provide a means that communicates whether
          the flows within an LSP can be able split across multiple component
          links.  The solution SHOULD provide a means to indicate the
          flow identification field(s) which can be used along the flow
          path which can be used to configure perform this function.

   FR#15  The solution SHALL provide a "virtual interface"
   corresponding means to indicate that a composite traffic
          flow shall select a component link that has at least all of with the normal
   IGP routing parameters . minimum latency
          value.

   FR#16  The solution SHALL allow configuration of virtual interface IGP
   parameters for provide a non-TE (i.e., normal IP routing) means to indicate that a traffic
          flow shall select a component link used for MPLS
   (e.g., administrative cost or weight).

   o with a maximum acceptable
          latency value as specified by protocol.

   FR#17  The solution SHALL support configuration of provide a composite means to indicate that a traffic
          flow shall select a component link
      composed of set of with a maximum acceptable
          delay variation value as specified by protocol.

   FR#18  The solution SHALL provide a local means to a node which
          automatically distribute flows across the component links in
          the composite link that may be logical or
      physical. connects to the other node such that
          SLA objectives are met.

   FR#19  The "virtual interface" solution SHALL appear as provide a fully-featured routing
   adjacency not just means to distribute flows from a
          single LSP across multiple component links to handle at least
          the case where the traffic carried in an LSP exceeds that of
          any component link in the composite link.

5.  Derived Requirements

   This section takes the next step and derives high-level requirements
   on protocol specification from the functional requirements.

   DR#1  The solution SHOULD attempt to extend existing protocols
         wherever possible, developing a new protocol only if this adds
         a significant set of capabilities.

         The vast majority of network operators have provisioned L3VPN
         services over LDP.  Many have deployed L2VPN services over LDP
         as an FA [RFC4206]. In particular, it needs well.  TE extensions to
   work with at least the following IP/MPLS control protocols: OSPF/IS-
   IS, LDP, IGPOSPF-TE/ISIS-TE, IGP and RSVP-TE.

   The solution SHALL accept a new component link or remove an existing
   component link RSVP-TE are viewed as being
         overly complex by operator provisioning or in response some operators.

   DR#2  A solution SHOULD extend LDP capabilities to signaling
   at a lower layer (e.g., meet functional
         requirements (without using GMPLS).

   The solution SHALL support derivation TE methods as decided in
         [RFC3468]).

   DR#3  Coexistence of the advertised interface
   parameters from configured component link parameters based LDP and RSVP-TE signaled LSPs MUST be supported
         on
   operator policy.

   A a composite link SHALL link.  Other functional requirements should be configurable
         supported as a numbered or unnumbered
   link (virtual interface in IP/MPLS).

   A component link SHALL be configurable independently of signaling protocol as possible.

   DR#4  When the nodes connected via a numbered link or
   unnumbered link. A component composite link should be not advertised are in IGP.

4.1.1.1. Traffic Flow and Connection Mapping

   The solution SHALL support operator assignment of traffic flows to
   specific connections.

   The the same
         MPLS network topology, the solution SHALL support operator assignment of connections MAY define extensions to
   specific component links.

   The solution SHALL support IP packet transport across
         the IGP.

   DR#5  When the nodes are connected via a composite link for control plane (signaling, routing) and management plane
   functions.

   In order to prevent packet loss, are in
         different MPLS network topologies, the solution must employ make-
   before-break when SHALL NOT rely
         on extensions to the IGP.

   DR#6  When a change worst case failure scenario occurs,the resulting number
         of links advertised in the mapping of a connection IGP causes IGP convergence to occur,
         causing a
   component link mapping change has to occur.

4.1.1.2. Management period of Other Operational Aspects

4.1.1.2.1. Resilience unavailability as perceived by users.  The component link recovery scheme SHALL perform equal to or better
   than existing local recovery methods.  A short service disruption may
   occur during
         convergence time of the recovery period. Fast ReRoute (FRR) SHALL be
   configurable solution MUST meet the SLA objective
         for a composite link.

   As part the duration of FRR, a method to recover from composite link node failure
   is desirable. OAM Messaging Support

4.1.1.2.2. Fault management requirement

   There are two aspects unavailability.

   DR#7  The Solution SHALL summarize the characteristics of fault management in the solution.  One is
   about
         component links as a composite link between two local adjacent routers.  The other
   is about IGP advertisement that
         results in convergence time better than that of advertising the
         individual component link.

   OAM protocols for fault management from the outside routers (e.g.,
   LSP-Ping/Trace, IP-ping/Trace) links.  This summary SHALL be transparently treated.

   For example, it is expected designed so
         that LSP-ping/trace message is able to
   diagnose composite link status and its associated virtual interface
   information; however, it is not required to directly treat represents the range of capabilities of the individual
         component link and CL-connection because they links such that functional requirements are local matter of two
   routers.

   The solution SHALL support fault notification mechanism (e.g.,
   syslog, SNMP trap to the management system/operators) with met, and
         also minimizes the
   granularity level frequency of affected part as detailed below:

   o  Data-plane advertisement updates which may
         cause IGP convergence to occur.  Examples of advertisement
         update tiggering events to be considered include: LSP
         establishment/release, changes in component link level

   o  Data-plane of composite link level (as
         characteristics (e.g., latency, up/down state), and/or
         bandwidth utilization.

   DR#8  When a whole, and as worst case failure scenario occurs,the resulting number
         of links advertised in the IGP causes IGP convergence to occur,
         causing a sum period of
      associated component links)

   o  Control-plane unavailability as perceived by users.  The
         convergence time of the virtual interface level (i.e.,
      routing/signaling on it)

   o  A composite link that believes that solution MUST meet the underlying component link
      server layers might not efficiently report failures, can run
      Bidirectional Forwarding Detection (BFD) at SLA objective
         for the component link
      layer.

   Race conditions: The solution shall support configuration duration of timers
   so that lower layer methods have time to detect/restore faults before unavailability.

   DR#9  When a composite link function would be invoked.

   The solution SHALL allow operator or control plane to query worst case failure scenario occurs, the
   component link number of
         RSVP-TE LSPs to which an LSP is assigned.

4.1.1.2.3. Flow/Connection Mapping Change Frequency be resignaled will cause a period of
         unavailability as perceived by users.  The resignaling time of
         the solution requires methods to dampen MUST meet the frequency of flow to
   connection mapping change, connection bandwidth change, and/or
   connection to component link mapping changes (e.g., SLA objective for re-
   optimization).  Operator imposed control policy regarding the
   frequency duration of
         unavailability.  The resignaling time of the solution MUST not
         increase significantly as compared with current methods.

6.  Acknowledgements

   Frederic Jounay of France Telecom and Yuji Kamite of NTT
   Communications Corporation co-authored a version of change for sets this document.

   A rewrite of flows SHALL be supported.

   The solution SHALL support latency this document occurred after the IETF77 meeting.
   Dimitri Papadimitriou, Lou Berger, Tony Li, the WG chairs John Scuder
   and delay variation sensitive
   traffic Alex Zinin, and limit others provided valuable guidance prior to and at
   the mapping change for these flows, IETF77 RTGWG meeting.

   Tony Li and place them
   on component links that John Drake have lower latency.

   The determination of latency sensitive traffic SHALL be determined by
   any of made numerous valuable comments on the
   RTGWG mailing list that are reflected in versions following methods:

   o  Use the
   IETF77 meeting.

7.  IANA Considerations

   This memo includes no request to IANA.

8.  Security Considerations

   This document specifies a set of requirements.  The requirements
   themselves do not pose a pre-defined local policy setting at composite link
      ingress security threat.  If these requirements are
   met using MPLS signaling as commonly practiced today with
   authenticated but unencrypted OSPF-TE, ISIS-TE, and RSVP-TE or LDP,
   then the requirement to provide additional information in this
   communication presents additional information that can could conceivably
   be mapped gathered in a man-in-the-middle confidentiality breach.  Such an
   attack would require a capability to monitor this signaling either
   through a flow

   o  A manually configured setting at composite link ingress that can
      be mapped provider breach or access to provider physical transmission
   infrastructure.  A provider breach already poses a flow

   The determination threat of latency sensitive traffic SHOULD be determined
   (if possible) by numerous
   tpes of attacks which are of far more serious consequence.  Encrption
   of the signaling can prevent or render more difficult any
   confidentiality breach that otherwise might occur by means of access
   to provider physical transmission infrastructure.

9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

9.2.  Informative References

   [ITU-T.G.800]
              ITU-T, "Unified functional architecture of transport
              networks", 2007, <http://www.itu.int/rec/T-REC-G/
              recommendation.asp?parent=T-REC-G.800>.

   [RFC2702]  Awduche, D., Malcolm, J., Agogbua, J., O'Dell, M., and J.
              McManus, "Requirements for Traffic Engineering Over MPLS",
              RFC 2702, September 1999.

   [RFC3031]  Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
              Label Switching Architecture", RFC 3031, January 2001.

   [RFC3468]  Andersson, L. and G. Swallow, "The Multiprotocol Label
              Switching (MPLS) Working Group decision on MPLS signaling
              protocols", RFC 3468, February 2003.

   [RFC3809]  Nagarajan, A., "Generic Requirements for Provider
              Provisioned Virtual Private Networks (PPVPN)", RFC 3809,
              June 2004.

   [RFC4031]  Carugi, M. and D. McDysan, "Service Requirements for Layer
              3 Provider Provisioned Virtual Private Networks (PPVPNs)",
              RFC 4031, April 2005.

   [RFC4665]  Augustyn, W. and Y. Serbest, "Service Requirements for
              Layer 2 Provider-Provisioned Virtual Private Networks",
              RFC 4665, September 2006.

   [RFC5254]  Bitar, N., Bocci, M., and L. Martini, "Requirements for
              Multi-Segment Pseudowire Emulation Edge-to-Edge (PWE3)",
              RFC 5254, October 2008.

9.3.  Appendix References

   [I-D.ietf-pwe3-fat-pw]
              Bryant, S., Filsfils, C., Drafz, U., Kompella, V., Regan,
              J., and S. Amante, "Flow Aware Transport of the following methods:

   o  Pre-set bits Pseudowires
              over an MPLS PSN", draft-ietf-pwe3-fat-pw-03 (work in the Payload (e.g., DSCP bits
              progress), January 2010.

   [IEEE-802.1AX]
              IEEE Standards Association, "IEEE Std 802.1AX-2008 IEEE
              Standard for IP or Ethernet
      user priority Local and Metropolitan Area Networks - Link
              Aggregation", 2006, <http://standards.ieee.org/getieee802/
              download/802.1AX-2008.pdf>.

   [ITU-T.Y.1541]
              ITU-T, "Network performance objectives for Ethernet payload) which are typically assigned
      by end-user

   o  MPLS Traffic-Class Field (aka EXP) which is typically mapped by
      the LER/LSR on the basis that its value is given IP-based
              services", 2006, <http://www.itu.int/rec/T-REC-Y.1541/en>.

   [RFC1717]  Sklower, K., Lloyd, B., McGregor, G., and D. Carr, "The
              PPP Multilink Protocol (MP)", RFC 1717, November 1994.

   [RFC2475]  Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
              and W. Weiss, "An Architecture for
      differentiating latency-sensitive traffic of end-users

4.1.2. Functions specific to LSP flows with TE information

   The following requirements apply to the case Differentiated
              Services", RFC 2475, December 1998.

   [RFC2615]  Malis, A. and W. Simpson, "PPP over SONET/SDH", RFC 2615,
              June 1999.

   [RFC2991]  Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
              Multicast Next-Hop Selection", RFC 2991, November 2000.

   [RFC2992]  Hopps, C., "Analysis of RSVP-TE signaled LSPs
   where an Equal-Cost Multi-Path
              Algorithm", RFC 2992, November 2000.

   [RFC3260]  Grossman, D., "New Terminology and Clarifications for
              Diffserv", RFC 3260, April 2002.

   [RFC4201]  Kompella, K., Rekhter, Y., and L. Berger, "Link Bundling
              in MPLS Traffic Engineering (TE)", RFC 4201, October 2005.

   [RFC4301]  Kent, S. and K. Seo, "Security Architecture for the virtual interface is configured with IGP TE extensions.

   The solution SHALL allow configuration of virtual interface
   parameters
              Internet Protocol", RFC 4301, December 2005.

   [RFC4385]  Bryant, S., Swallow, G., Martini, L., and D. McPherson,
              "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for TE extensions link (e.g., available bandwidth, maximum
   bandwidth, maximum allowable LSP bandwidth, TE metric,
              Use over an MPLS PSN", RFC 4385, February 2006.

   [RFC4928]  Swallow, G., Bryant, S., and resource
   classes (i.e., administrative groups) or link colors).

   The solution SHALL support configuration of a composite link composed L. Andersson, "Avoiding Equal
              Cost Multipath Treatment in MPLS Networks", BCP 128,
              RFC 4928, June 2007.

Appendix A.  More Details on Existing Network Operator Practices and
             Protocol Usage

   Network operators have SLAs for services that are comprised of set
   numerical values for performance measures, principally availability,
   latency, delay variation.  See [ITU-T.Y.1541], RFC 3809, Section 4.9
   [RFC3809] for examples of component links , with each component link potentially
   having at least the following characteristics which may differ:

   o  Bandwidth

   o  Latency

   o  QoS characteristics (e.g., jitter, error rate)

   The solution SHALL support the admission control by RSVP-TE that is
   signaled from the routers outside the composite link. Note that RSVP-
   TE signaling need not specify the actual component link to be used?
   because the selection form of component link is such SLAs.  Note that the local matter
   numerical values of two
   adjacent routers based upon signaled Y.1541 span multiple networks and locally configured
   information.

   The solution shall may be able to receive, interpret looser
   than network operator SLAs.  Applications and act upon at
   least acceptable user
   experience have a relationship to these performance parameters.

   Consider latency as an example.  In some cases, minimizing latency
   relates directly to the following RSVP-TE signaled parameters: bandwidth setup
   priority, and holding priority [RFC 3209, RFC 2215] preemption
   priority best customer experience (e.g., in TCP closer
   is faster).  I other cases, user experience is relatively insensitive
   to latency, up to a specific limit at which point user perception of
   quality degrades significantly (e.g., interactive human voice and traffic class [RFC 4124],
   multimedia conferencing).  A number of SLAs have. a bound on point-
   point latency, and apply them to as long as this bound is met, the CL
   connections where SLA is met --
   decreasing the LSP latency is mapped.

   The solution shall support configuration of at least not necessary.  In some SLAs, if the following
   parameters
   specified latency is not met, the user considers the service as
   unavailable.  An unprotected LSP can be manually provisioned on a per composite link basis:

   o  Local Bandwidth Oversubscription factor
   The determination set
   of to meet this type of SLA, but this lowers availability since an
   alternate route that meets the latency sensitive traffic SHALL SLA cannot be determined determined.

   Historically, when an IP/MPLS network was operated over a lower layer
   circuit switched network (e.g., SONET rings), a change in latency
   caused by
   any of the following methods:

   o lower layer network (e.g., due to a maintenance action
   or failure) this was not known to the MPLS traffic class network.  This resulted in
   latency affecting end user experience, sometimes violating SLAs or
   resulting in user complaints.

   A response to this problem was to provision IP/MPLS networks over
   unprotected circuits and set the metric and/or TE-metric proportional
   to latency.  This resulted in a RSVP-TE signaling message (i.e., Diffserv-
      TE traffic class [RFC 4124])

4.1.3. Functions specific being directed over the least
   latency path, even if this was not needed to LSP flows without TE information

   The following requirements apply meet an SLA or meet user
   experience objectives.  This results in reduced flexibility and
   increased cost for network operators.  Using lower layer networks to
   provide restoration and grooming is expected to be more efficient,
   but the inability to communicate performance parameters, in
   particular latency, from the lower layer network to the case of LDP signaled LSPs
   when no signaled TE information higher layer
   network is available.

   The solution shall map LDP-assigned labeled packets an important problem to be solved before this can be done.

   Latency SLAs for pt-pt services are often tied closely to geographic
   locations, while latency for multipoint services may be based upon local
   configuration (e.g., label stack depth) to define a connection that
   worst case within a region.

   The presence of only three Traffic Class (TC) bits (previously known
   as EXP bits) in the MPLS shim header is mapped limiting when a network
   operator needs to one support QoS classes for multiple services (e.g.,
   L2VPN VPWS, VPLS, L3VPN and Internet), each of the component link.

   The solution SHALL map LDP-assigned labeled packets which has a set of QoS
   classes that identify the
   outer label's FEC.

   The solution SHALL support entropy labels [Entropy Label] to map more
   granular flows need to connections.

   The solution SHALL be able to measure the bandwidth actually supported.  In some cases one bit is used by
   a particular connection and derive proper local to
   indicate conformance to some ingress traffic TE
   information classification, leaving
   only two bits for indicating the connection.

   When the connection bandwidth exceeds the component link capacity,
   the solution SHALL be able to reassign the traffic flows to several
   connections. service QoS classes.  The solution SHALL support management plane controlled parameters approach
   that define at least a minimum bandwidth, maximum bandwidth,
   preemption priority, has been taken is to aggregate these QoS classes into similar
   sets on LER-LSR and holding priority for each connection without
   TE information (i.e., LDP signaled LSP that does not contain the same
   information as an RSVP-TE signaled LSP).

4.1.4. Sets of LSP flows with LSR-LSR links.

   Labeled LSPs have been and without TE information

   The solution shall support separation use of resources for traffic flows
   mapped to connections that link layer encapsulation have access been
   standardized in order to TE information (e.g., RSVP-
   TE signaled flows) from those that provide a means to meet these needs.

   The IP DSCP cannot be used for flow identification since RFC 4301
   Section 5.5 [RFC4301] requires Diffserv transparency, and in general
   network operators do not have access to TE
   information (e.g., LDP-signaled flows or RSVP-TE LSP rely on the DSCP of Internet packets.

   A label is pushed onto Internet packets when they are carried along
   with Zero
   bandwidth).

   Component links in a composite L2/L3VPN packets on the same link may fail independently.  The
   failure of or lower layer network
   provides a component link may impact some connections.  The
   impacted connections shall be transferred mean to other active component
   links using distinguish between the same rules as QoS class for these
   packets.

   Operating an MPLS-TE network involves a different paradigm from
   operating an IGP metric-based LDP signaled MPLS network.  The mpt-pt
   LDP signaled MPLS LSPs occur automatically, and balancing across
   parallel links occurs if the original assignment IGP metrics are set "equally" (with
   equality a locally definable relation).

   Traffic is typically comprised of
   connections to component links. a few large (some very large) flows
   and many small flows.  In some cases, separate LSPs are established
   for very large flows.  This can occur even if the event that all connections
   cannot be recovered, configured priority and preemption parameters
   SHOULD be employed.

4.1.4.1. Handling Bandwidth Shortage Events

   The following requirements apply to IP header
   information is inspected by a virtual interface router, for example an IPsec tunnel
   that supports
   traffic carries a large amount of traffic.  An important example of
   large flows both with and without TE information, in response to is that of a
   bandwidth shortage event. A "bandwidth shortage" can arise in L2/L3 VPN customer who has an access line
   bandwdith comparable to a client-client composite link if the total bandwidth --
   there could be flows that are on the order of the connections with
   provisioned/signaled TE information access line
   bandwdith.

Appendix B.  Existing Multipath Standards and those signaled without TE
   information (but with measured bandwidth) exceeds Techniques

   Today the bandwidth requirement to handle large aggregations of
   the composite link that carries connections.

   The solution shall support traffic, much
   larger than a policy-based preemption capability such
   that, in the event of such single component link, can be handled by a "bandwidth shortage", number of
   techniques which we will collectively call multipath.  Multipath
   applied to parallel links between the signaled or
   configured preemption and holding parameters can same set of nodes includes
   Ethernet Link Aggregation [IEEE-802.1AX], link bundling [RFC4201], or
   other aggregation techniques some of which may be vendor specific.
   Multipath applied to the
   following treatments diverse paths rather than parallel links
   includes Equal Cost MultiPath (ECMP) as applied to the connections:

   o  For a connection that has RSVP-TE LSPs, signal the router that the
      LSP has been preempted. OSPF, ISIS, or
   even BGP, and equal cost LSP, as described in Appendix B.4.  Various
   mutilpath techniques have strengths and weaknesses.

   The solution shall support soft
      preemption (i.e., notify the preempted LSP source prior to
      preemption). [Soft Preemption]

   o  For a connection that has LDP(s), where the routers connected via
      the term composite link is aware of more general than terms such as link
   aggregate which is generally considered to be specific to Ethernet
   and its use here is consistent with the LDP signaling involved broad definition in
   [ITU-T.G.800].  The term multipath excludes inverse multiplexing and
   refers to techniques which only solve the
      preempted label stack depth, signal release problem of large
   aggregations of traffic, without addressing the label other requirements
   outlined in this document.

B.1.  Common Multpath Load Spliting Techniques

   Identical load balancing techniqes are used for multipath both over
   parallel links and over diverse paths.

   Large aggregates of IP traffic do not provide explicit signaling to
   indicate the
      router

   o  For a connection that has non-re-routable expected traffic loads.  Large aggregates of MPLS
   traffic are carried in MPLS tunnels supported by MPLS LSP.  LSP which
   are signaled using RSVP-TE LSPs or non-
      releasable LDP labels, signal extensions do provide explicit signaling
   which includes the router or operator that expected traffic load for the aggregate.  LSP
      or
   which are signaled using LDP label has been lost.

4.2. Exterior Functions

4.2.1. Functions common to all LSP flows

   The solution MUST be able to interoperate with exiting IETF do not provide an expected traffic load.

   MPLS
   [RFC3031] control and data planes where appropriate.

4.2.1.1. Signaling Protocol Extensions

   The solution SHALL support signaling a composite link between two
   routers (e.g., P, P/PE, or PE).

   The solution SHALL support signaling a component link as part of a
   composite link.

   The solution SHALL support signaling a composite link and
   automatically injecting it into the IGP [LSP Hieararchy bis] or
   connected two routers.

4.2.1.2. Routing Advertisement Extensions

   The solution SHALL support IGP extensions to identify that the
   advertised routing adjacency LSP may contain other MPLS LSP arranged hierarchically.  When an
   MPLS LSR serves as a composite link.

4.2.1.3. Multi-Layer Networking Aspects

   The solution MUST be able to interoperate midpoint LSR in an LSP carrying other LSP as
   payload, there is no signaling associated with existing IETF
   GMPLS/MPLS-TP control and data planes where appropriate, for example, these inner LSP.
   Therefore even when using RSVP-TE signaling there may be insufficient
   information provided by signaling to adequately distribute load
   across a component composite link.

   The solution SHALL support derivation

   Generally a set of label stack entries that is unique across the advertised interface
   parameters from signaled component link parameters from
   ordered set of label numbers can safely be assumed to contain a lower layer
   (e.g., latency, capacity) based on operator policy. group
   of flows.  The solution SHALL reordering of traffic can therefore be able considered to accept the GMPLS/MPLS-TP control plane
   signaled component link.  It SHALL
   be able to support the following
   component link parameters

   o  Maximum (estimated or measured) acceptable latency

   o  Actual(estimated or measured) assigned latency

   o  Bandwidth

   o  Delay Variation

   It SHOULD support the following component link parameter

   o  Loss rate

4.2.2. Functions specific unless reordering occurs within traffic containing a
   common unique set of label stack entries.  Existing load splitting
   techniques take advantage of this property in addition to LSP flows with TE information

4.2.2.1. Signaling Protocol Extensions Requirements

   The solution SHALL support per LSP signaling looking
   beyond the bottom of at least the
   following additional parameters for an LSP

   o  Maximum (estimated or measured) acceptable latency

   o  Actual (estimated label stack and determining if the payload
   is IPv4 or measured) accumulated latency based upon IPv6 to load balance traffic accordingly.

   MPLS-TP OAM violates the
      actual component link assigned by assumption that it is safe to reorder
   traffic within an LSP.  If MPLS-TP OAM is to be accommodated, then
   existing multipth techniques must be modified.  Such modifications
   are outside the composite link

   o  Bandwidth scope of the highest and lowest speed

   The solution SHOULD support per LSP signaling this document.

   For example a large aggregate of at least the
   following additional parameters:

   o  Delay Variation

   o  Loss rate

   As part IP traffic may be subdivided into a
   large number of determining the path groups of an LSP, flows using a hash on the client may query IP source and
   destination addresses.  This is as described in [RFC2475] and
   clarified in [RFC3260].  For MPLS traffic carrying IP, a
   Patch Computation Element (PCE) and use similar hash
   can be performed on the response set of labels in determining the (complete or partial) source route sent in label stack.  These
   techniques are both examples of means to subdivide traffic into
   groups of flows for the TE signaling
   message.

   Note that an LSP can be a component purpose of load balancing traffic across
   aggregated link or capacity.  The means of identifying a client LSP. Since
   component links may involve GMPLS, separate signaling protocol
   extensions may be needed for particular switching capabilities.

4.2.2.2. Routing Advertisement Extensions

   It SHALL flow should not
   be possible for confused with the solution to represent multiple values, definition of a flow.

   Discussion of whether a hash based approach provides a sufficiently
   even load balance using any particular hashing algorithm or method of
   distributing traffic across a range set of values, for the composite link interface parameters in
   order to communicate information about differences in the constituent component links is outside of
   the scope of this document.

   The current load balancing techniques are referenced in an exterior function route advertisement

   Some [RFC4385] and
   [RFC4928].  The use of these capabilities three hash based approaches are specified for link bundling [RFC
   4201], but some extensions may be needed. Techniques such as those described in [RFC 2676] for encoding latency
   [RFC2991] and using it in routing
   should be considered. LSA encoding techniques such as those [RFC2992].  A mechanism to identify flows within PW is
   described in [RFC 3630] should also be considered.

   For example, a range [I-D.ietf-pwe3-fat-pw].  The use of hash based
   approaches is mentioned as an example of an existing set of latencies,
   techniques to distribute traffic over a range set of capacities and/or other
   characteristics for the component links that comprise links.
   Other techniques are not precluded.

B.2.  Simple and Adaptive Load Balancing Multipath

   Simple multipath generally relies on the composite
   links could be advertised. When mathematical probability
   that given a range very large number of characteristics is
   advertised, small microflows, these can microflows
   will tend to be used in distributed evenly across a constrained path computation, hash space.  A common
   simple multipath implementation assumes that
   is, certain composite links can be excluded since no component link
   meets all members (component
   links) are of equal capacity and perform a modulo operation across
   the characteristic as part hashed value.  An alternate simple multipath technique uses a
   table generally with a power of two size, and distributes the overall path.

4.2.3. Functions specific table
   entries proportionally among members according to LSP flows without TE information

   A straightforward set the capacity of requirement would result
   each member.

   Simple load balancing works well if the same
   functions specified for RSVP-TE were also specified for LDP. there are a very large number of
   small microflows (i.e., microflow rate is much less than component
   link capacity).  However, the IETF MPLS Working Group's decision on MPLS signaling protocols
   [RFC 3468] to case where there are even a few large
   microflows is not pursue such an approach handled well by not adding functions simple load balancing.

   An adaptive multipath technique is one where the traffic bound to
   CR-LDP means that a different approach must be taken.
   each member (component link) is measured and the load split is
   adjusted accordingly.  As described in long as the Interior Function section, 4.1.3, adjustment is done within a basic
   composite link capability when
   single network element, then no protocol extensions are required and
   there is are no TE information interoperability issues.

   Note that if the load balancing algorithm and/or its parameters is to
   adjusted, then packets in some flows may be
   able to measure the amount delivered out of LDP signaled
   sequence.

B.3.  Traffic Split over Parallel Links

   The load spliting techniques defined in Appendix B.1 and Appendix B.2
   are both used in splitting traffic that is sent on an
   LSP. It over parallel links between the
   same pair of nodes.  The best known technique, though far from being
   the first, is highly desirable to be able to signal and advertise
   certain aspects Ethernet Link Aggregation [IEEE-802.1AX].  This same
   technique had been applied much earlier using OSPF or ISIS Equal Cost
   MultiPath (ECMP) over parallel links between the same nodes.
   Multilink PPP [RFC1717] uses a technique that provides inverse
   multiplexing, however a number of these measurements, if they cannot be explicitly
   signaled via vendors had provided proprietary
   extensions to LDP.

4.2.3.1. Signaling Protocol Extensions

   The solution SHOULD PPP over SONET/SDH [RFC2615] that predated Ethernet
   Link Aggregation but are no longer used.

   Link bundling [RFC4201] provides yet another means of handling
   parallel LSP.  RFC4201 explicitly allow addition a special value of an object to an LDP label
   mapping message that indicates how much measured capacity can be sent all ones
   to indicate a split across all members of the bundle.

B.4.  Traffic Split over Multiple Paths

   OSPF or ISIS Equal Cost MultiPath (ECMP) is a pair well known form of nodes connected via a composite link. This signaling
   should be conveyed at least
   traffic split over multiple paths that may traverse intermediate
   nodes.  ECMP is often incorrectly equated to nodes which only this case, and
   multipath over multiple diverse paths is often incorrectly equated to
   ECMP.

   Many implementations are adjacent able to the create more than one LSP between a
   pair of nodes connected via a composite link.

   Nodes SHOULD be able to use this information in conjunction with the
   IGP link information nodes, where these LSP are routed diversely to decide which label mappings it will better make
   use for
   forwarding LDP signaled flows toward a pair of nodes connected via a
   composite link.

4.2.3.2. Routing Advertisement Extensions

   Signaling via LDP the available capacity capacity.  The load on a composite link for
   flows without TE information is one method. A preferable method would these LSP can be distributed
   proportionally to extend the IGP to indicate the amount reserved bandwidth of capacity allocated by
   an Interior function to flows without TE information so that nodes in the network using LDP can make better informed decisions about which
   label mapping messages are used to form an LDP LSP.

4.2.4. Functions specific to  These multiple
   LSP flows with and without TE information

   As described in the Interior Function section, 4.1.4, the principal
   driver in this case is how to handle bandwidth shortage events when
   both RSVP-TE may be advertised as a single PSC FA and LDP signaled LSPs are present in the composite link.
   The requirements relevant to the nodes connected by any LSP making use of
   the composite
   link are described in section 4.1.4, and this section describes
   additional desirable functions beyond FA may be split over these nodes. Since RSVP-TE
   signaling defines parameters and procedures for preemption, the focus
   is on extensions to better support LDP signaled LSPs.

4.2.4.1. Signaling Protocol Extensions

   The solution SHOULD allow addition of an object to an LDP label
   mapping message that indicates that a node controlling multiple LSP.

   Link bundling [RFC4201] component links may themselves be LSP.  When
   this technique is used, any LSP which specifies the composite link wants to preempt the traffic offered by an LSP. This should have bundle may
   be split across the effect multiple paths of pruning label distribution for the LSP at that comprise the node pair
   connected via a
   bundle.

Appendix C.  ITU-T G.800 Composite Link Definitions and Terminology

   Composite Link:
       Section 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite link.

4.2.4.2. Routing Advertisement Extensions

   The solution SHOULD allow addition link
       in terms of an object to three cases, of which the IGP that
   indicates following two are relevant
       (the one describing inverse (TDM) multiplexing does not apply).
       Note that all LDP signaled traffic should avoid these case definitions are taken verbatim from section
       6.9, "Layer Relationships".

       Case 1:  "Multiple parallel links between the advertised same subnetworks
           can be bundled together into a single composite link.

5. Security Considerations

   The solution  Each
           component of the composite link is a local function on independent in the router to support traffic
   engineering management over multiple parallel links.  It does not
   introduce sense
           that each component link is supported by a security risk for control plane and data plane.

   The solution could change the frequency of routing update messages
   and therefore could change routing convergence time. separate server
           layer trail.  The solution
   MUST provide controls to dampen the frequency of such changes so as
   to not destabilize routing protocols.

6. IANA Considerations

   IANA actions to provide solutions are for further study.

7. References

7.1. Normative References

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2215] S. Shenker, J. Wroclawski, "General Characterization
             Parameters for Integrated Service Network Elements."
             September 1997

   [RFC3209]  D. Awduche, L. Berger, D. Gan, T. Li, V. Srinivasan, G.
   Swallow, "RSVP-TE: Extensions to RSVP for LSP Tunnels,"   December
   2001

   [RFC4206] Label Switched Paths (LSP) Hierarchy with Generalized
   Multi-Protocol Label Switching (GMPLS) Traffic Engineering (TE)  K.
   Kompella, Y. Rekhter October 2005

   [RFC4090]  Pan, P., "Fast Reroute Extensions to RSVP-TE for LSP
   Tunnels", RFC 4090, May 2005.

   [RFC4124] Protocol Extensions for Support of Diffserv-aware MPLS
   Traffic Engineering  F. Le Faucheur, Ed. June 2005

   [RFC4201]  Kompella, K., "Link Bundle in MPLS Traffic Engineering",
   RFC 4201, March 2005.

7.2. Informative References

   [Entropy Label] Kompella, K. and S. Amante, "The Use composite link conveys communication
           information using different server layer trails thus the
           sequence of Entropy
   Labels in MPLS Forwarding", November 2008, Work in Progress

   [LSP Hierarchy] Shiomoto, K. and A. Farrel, "Procedures for
   Dynamically               Signaled Hierarchical Label Switched
   Paths", November 2008, Work symbols crossing this link may not be preserved.
           This is illustrated in Progress

   [Soft Preemption] Meyer, M. Figure 14."

       Case 3:  "A link can also be constructed by a concatenation of
           component links and J. Vasseur, "MPLS Traffic Engineering
   Soft Preemption", February 2009, Work in Progress

   [CLFRAMEWORK] Yong, L. Ed, "Framework for MPLS Over Composite Link,"
   work in progress.

   [RFC3468] L. Andersson, G. Swallow, "The Multiprotocol Label
   Switching (MPLS) Working Group decision on MPLS signaling
   protocols."

   [PWBONDING] Stein, Y, "PW Bonding", Dec. 2008

   [RFC3630]  Katz, D., "Traffic Engineering (TE) Extension configured channel forwarding
           relationships.  The forwarding relationships must have a 1:1
           correspondence to OSPF
   Version 2", RFC 3630, September 2003.

   [RFC 2676]  G. Apostolopoulos, S. Kama, D. Williams, R. Guerin, A.
   Orda, T. Przygienda, "QoS Routing Mechanisms and OSPF Extensions,"
   August, 1999

   [ITU-T G.800]  ITU-T, "Unified functional architecture of transport
   networks," September, 2007.

8. Acknowledgments

   Authors would like the link connections that will be provided
           by the client link.  In this case, it is not possible to thank Adrian Farrel from Olddog for his
   extensive comments and suggestions, Ron Bonica from Juniper, Nabil
   Bitar from Verizon, Eric Gray from Ericsson, Lou Berger from LabN,
   and Kireeti Kompella from Juniper, for their reviews and great
   suggestions.

   This document was prepared using 2-Word-v2.0.template.dot.

   Copyright (c) 2010 IETF Trust and
           fully infer the persons identified as authors status of the code. All rights reserved.

   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. link by observing the server
           layer trails visible at the ends of the link.  This code was derived from IETF RFC [insert RFC number]. Please
   reproduce is
           illustrated in Figure 16."

   Subnetwork:  A set of one or more nodes (i.e., LER or LSR) and links.
       As a special case it can represent a site comprised of multiple
       nodes.

   Forwarding Relationship:  Configured forwarding between ports on a
       subnetwork.  It may be connectionless (e.g., IP, not considered
       in this note if possible. draft), or connection oriented (e.g., MPLS signaled or
       configured).

   Component Link:  A topolological relationship between subnetworks
       (i.e., a connection between nodes), which may be a wavelength,
       circuit, virtual circuit or an MPLS LSP.

Authors' Addresses

   Curtis Villamizar (editor)
   Infinera Corporation
   169 W. Java Drive
   Sunnyvale, CA  94089

   Email: cvillamizar@infinera.com

   Dave McDysan (editor)
   Verizon
   22001 Loudoun County PKWY
   Ashburn, VA  20147

   Email: dave.mcdysan@verizon.com

   So Ning
   Verizon
   2400 N. Glem Ave.,
      Richerdson, Glenville Ave.
   Richardson, TX  75082

   Phone: +1 972-729-7905
   Email: ning.so@verizonbusiness.com
   Andrew Malis
   Verizon
   117 West St.
   Waltham, MA  02451

   Phone: +1 781-466-2362
   Email: andrew.g.malis@verizon.com

      Dave McDysan
      Verizon
      22001 Loudoun County PKWY
      Ashburn, VA  20147
      Email: dave.mcdysan@verizon.com

   Lucy Yong
   Huawei USA
   1700 Alma Dr. Suite 500
   Plano, TX  75075

   Phone: +1 469-229-5387
   Email: lucyyong@huawei.com

      Frederic Jounay
      France Telecom
      2, avenue Pierre-Marzin
      22307 Lannion Cedex,
      FRANCE
      Email: frederic.jounay@orange-ftgroup.com

      Yuji Kamite
      NTT Communications Corporation
      Granpark Tower
      3-4-1 Shibaura, Minato-ku
      Tokyo  108-8118
      Japan
      Email: y.kamite@ntt.com