draft-ietf-avtcore-rtp-topologies-update-00.txt   draft-ietf-avtcore-rtp-topologies-update-01.txt 
Network Working Group M. Westerlund Network Working Group M. Westerlund
Internet-Draft Ericsson Internet-Draft Ericsson
Obsoletes: 5117 (if approved) S. Wenger Obsoletes: 5117 (if approved) S. Wenger
Intended status: Informational Vidyo Intended status: Informational Vidyo
Expires: October 24, 2013 April 22, 2013 Expires: April 25, 2014 October 22, 2013
RTP Topologies RTP Topologies
draft-ietf-avtcore-rtp-topologies-update-00 draft-ietf-avtcore-rtp-topologies-update-01
Abstract Abstract
This document discusses point to point and multi-endpoint topologies This document discusses point to point and multi-endpoint topologies
used in Real-time Transport Protocol (RTP)-based environments. In used in Real-time Transport Protocol (RTP)-based environments. In
particular, centralized topologies commonly employed in the video particular, centralized topologies commonly employed in the video
conferencing industry are mapped to the RTP terminology. conferencing industry are mapped to the RTP terminology.
This document is updated with additional topologies and are intended This document is updated with additional topologies and is intended
to replace RFC 5117. to replace RFC 5117.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on October 24, 2013. This Internet-Draft will expire on April 25, 2014.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 17 skipping to change at page 2, line 17
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Topologies . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Topologies . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.1. Point to Point . . . . . . . . . . . . . . . . . . . . . 4 3.1. Point to Point . . . . . . . . . . . . . . . . . . . . . 4
3.2. Point to Point via Middlebox . . . . . . . . . . . . . . 5 3.2. Point to Point via Middlebox . . . . . . . . . . . . . . 5
3.2.1. Translators . . . . . . . . . . . . . . . . . . . . . 5 3.2.1. Translators . . . . . . . . . . . . . . . . . . . . . 5
3.2.2. Back to Back RTP sessions . . . . . . . . . . . . . . 8 3.2.2. Back to Back RTP sessions . . . . . . . . . . . . . . 9
3.3. Point to Multipoint Using Multicast . . . . . . . . . . . 9 3.3. Point to Multipoint Using Multicast . . . . . . . . . . . 9
3.3.1. Any Source Multicast (ASM) . . . . . . . . . . . . . 9 3.3.1. Any Source Multicast (ASM) . . . . . . . . . . . . . 10
3.3.2. Source Specific Multicast (SSM) . . . . . . . . . . . 11 3.3.2. Source Specific Multicast (SSM) . . . . . . . . . . . 11
3.3.3. SSM with Local Unicast Resources . . . . . . . . . . 12 3.3.3. SSM with Local Unicast Resources . . . . . . . . . . 13
3.4. Point to Multipoint Using Mesh . . . . . . . . . . . . . 14 3.4. Point to Multipoint Using Mesh . . . . . . . . . . . . . 14
3.5. Point to Multipoint Using the RFC 3550 Translator . . . . 15 3.5. Point to Multipoint Using the RFC 3550 Translator . . . . 17
3.5.1. Relay - Transport Translator . . . . . . . . . . . . 15 3.5.1. Relay - Transport Translator . . . . . . . . . . . . 17
3.5.2. Media Translator . . . . . . . . . . . . . . . . . . 16 3.5.2. Media Translator . . . . . . . . . . . . . . . . . . 19
3.6. Point to Multipoint Using the RFC 3550 Mixer Model . . . 16 3.6. Point to Multipoint Using the RFC 3550 Mixer Model . . . 19
3.6.1. Media Mixing . . . . . . . . . . . . . . . . . . . . 18 3.6.1. Media Mixing . . . . . . . . . . . . . . . . . . . . 21
3.6.2. Media Switching . . . . . . . . . . . . . . . . . . . 21 3.6.2. Media Switching . . . . . . . . . . . . . . . . . . . 24
3.7. Source Projecting Middlebox . . . . . . . . . . . . . . . 23 3.7. Selective Forwarding Middlebox . . . . . . . . . . . . . 26
3.8. Point to Multipoint Using Video Switching MCUs . . . . . 25 3.8. Point to Multipoint Using Video Switching MCUs . . . . . 29
3.9. Point to Multipoint Using RTCP-Terminating MCU . . . . . 27 3.9. Point to Multipoint Using RTCP-Terminating MCU . . . . . 30
3.10. De-composite Endpoint . . . . . . . . . . . . . . . . . . 28 3.10. Split Component Endpoint . . . . . . . . . . . . . . . . 32
3.11. Non-Symmetric Mixer/Translators . . . . . . . . . . . . . 29 3.11. Non-Symmetric Mixer/Translators . . . . . . . . . . . . . 33
3.12. Combining Topologies . . . . . . . . . . . . . . . . . . 30 3.12. Combining Topologies . . . . . . . . . . . . . . . . . . 33
4. Comparing Topologies . . . . . . . . . . . . . . . . . . . . 30 4. Comparing Topologies . . . . . . . . . . . . . . . . . . . . 34
4.1. Topology Properties . . . . . . . . . . . . . . . . . . . 31 4.1. Topology Properties . . . . . . . . . . . . . . . . . . . 34
4.1.1. All to All Media Transmission . . . . . . . . . . . . 31 4.1.1. All to All Media Transmission . . . . . . . . . . . . 34
4.1.2. Transport or Media Interoperability . . . . . . . . . 31 4.1.2. Transport or Media Interoperability . . . . . . . . . 35
4.1.3. Per Domain Bit-Rate Adaptation . . . . . . . . . . . 31 4.1.3. Per Domain Bit-Rate Adaptation . . . . . . . . . . . 35
4.1.4. Aggregation of Media . . . . . . . . . . . . . . . . 32 4.1.4. Aggregation of Media . . . . . . . . . . . . . . . . 36
4.1.5. View of All Session Participants . . . . . . . . . . 32 4.1.5. View of All Session Participants . . . . . . . . . . 36
4.1.6. Loop Detection . . . . . . . . . . . . . . . . . . . 32 4.1.6. Loop Detection . . . . . . . . . . . . . . . . . . . 36
4.2. Comparison of Topologies . . . . . . . . . . . . . . . . 33 4.2. Comparison of Topologies . . . . . . . . . . . . . . . . 36
5. Security Considerations . . . . . . . . . . . . . . . . . . . 33 5. Security Considerations . . . . . . . . . . . . . . . . . . . 37
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 39
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 39
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 35 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 39
8.1. Normative References . . . . . . . . . . . . . . . . . . 35 8.1. Normative References . . . . . . . . . . . . . . . . . . 39
8.2. Informative References . . . . . . . . . . . . . . . . . 36 8.2. Informative References . . . . . . . . . . . . . . . . . 39
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 41
1. Introduction 1. Introduction
Real-time Transport Protocol (RTP) [RFC3550] topologies describe Real-time Transport Protocol (RTP) [RFC3550] topologies describe
methods for interconnecting RTP entities and their processing methods for interconnecting RTP entities and their processing
behavior of RTP and RTCP. This document tries to address past and behavior of RTP and RTCP. This document tries to address past and
existing confusion, especially with respect to terms not defined in existing confusion, especially with respect to terms not defined in
RTP but in common use in the conversational communication industry, RTP but in common use in the conversational communication industry,
such as MCU. In doing so, this memo provides a common information such as the Multipoint Control Unit or MCU.
basis for future discussion and specification work. It attempts to
clarify and explain sections of the Real-time Transport Protocol
(RTP) spec [RFC3550] in an informal way. It is not intended to
update or change what is normatively specified within RFC 3550.
When the Audio-Visual Profile with Feedback (AVPF) [RFC4585] was When the Audio-Visual Profile with Feedback (AVPF) [RFC4585] was
developed the main emphasis lay in the efficient support of point to developed the main emphasis lay in the efficient support of point to
point and small multipoint scenarios without centralized multipoint point and small multipoint scenarios without centralized multipoint
control. However, in practice, many small multipoint conferences control. In practice, however, most multipoint conferences operate
operate utilizing devices known as Multipoint Control Units (MCUs). utilizing centralized units referred to as MCUs. MCUs may implement
MCUs may implement Mixer or Translator (in RTP [RFC3550] terminology) Mixer or Translator functionality (in RTP [RFC3550] terminology), and
functionality and signalling support. They may also contain signalling support. They may also contain additional application
additional application functionality. This document focuses on the layer functionality. This document focuses on the media transport
media transport aspects of the MCU that can be realized using RTP, as aspects of the MCU that can be realized using RTP, as discussed
discussed below. Further considered are the properties of Mixers and below. Further considered are the properties of Mixers and
Translators, and how some types of deployed MCUs deviate from these Translators, and how some types of deployed MCUs deviate from these
properties. properties.
This document also codifies new multipoint architectures that have
recently been introduced and which were not anticipated in RFC 5117.
These architectures use scalable video coding and simulcasting, and
their associated centralized units are referred to as Selective
Forwarding Units (SFU). This codification provides a common
information basis for future discussion and specification work.
The document's attempt to clarify and explain sections of the Real-
time Transport Protocol (RTP) spec [RFC3550] is informal. It is not
intended to update or change what is normatively specified within RFC
3550.
2. Definitions 2. Definitions
2.1. Glossary 2.1. Glossary
ASM: Any Source Multicast ASM: Any Source Multicast
AVPF: The Extended RTP Profile for RTCP-based Feedback AVPF: The Extended RTP Profile for RTCP-based Feedback
CSRC: Contributing Source CSRC: Contributing Source
Link: The data transport to the next IP hop Link: The data transport to the next IP hop
Middlebox: A device that is on the Path that media travel between
two Endpoints
MCU: Multipoint Control Unit MCU: Multipoint Control Unit
Path: The concatenation of multiple links, resulting in an end-to- Path: The concatenation of multiple links, resulting in an end-to-
end data transfer. end data transfer.
PtM: Point to Multipoint PtM: Point to Multipoint
PtP: Point to Point PtP: Point to Point
SFU: Selective Forwarding Unit
SSM: Source-Specific Multicast SSM: Source-Specific Multicast
SSRC: Synchronization Source SSRC: Synchronization Source
3. Topologies 3. Topologies
This subsection defines several topologies that are relevant for This subsection defines several topologies that are relevant for
codec control but also RTP usage in other contexts. The section codec control but also RTP usage in other contexts. The section
starts with point to point cases, without and with middleboxes. Then starts with point to point cases, with or without middleboxes. Then
follows a number of different methods for establishing point to follows a number of different methods for establishing point to
multipoint communication. These are structure around the most multipoint communication. These are structured around the most
fundamental enabler, i.e. multicast, a mesh of connections, fundamental enabler, i.e., multicast, a mesh of connections,
translators, mixers and source projection middlebox, to finally translators, mixers and finally MCUs and SFUs. The section ends by
discuss MCUs. The section ends by discussing de-composed endpoints, discussing de-composited endpoints, asymmetric middlebox behaviors
asymmetric middlebox behaviors and combining topologies. and combining topologies.
The topologies may be referenced in other documents by a shortcut The topologies may be referenced in other documents by a shortcut
name, indicated by the prefix "Topo-". name, indicated by the prefix "Topo-".
For each of the RTP-defined topologies, we discuss how RTP, RTCP, and For each of the RTP-defined topologies, we discuss how RTP, RTCP, and
the carried media are handled. With respect to RTCP, we also discuss the carried media are handled. With respect to RTCP, we also discuss
the handling of RTCP feedback messages as defined in [RFC4585] and the handling of RTCP feedback messages as defined in [RFC4585] and
[RFC5104]. Any important differences between the two will be [RFC5104].
illuminated in the discussion. At this stage we don't intended to
discuss in detail how each and every feedback messages should be
treated in the various topologies.
3.1. Point to Point 3.1. Point to Point
Shortcut name: Topo-Point-to-Point Shortcut name: Topo-Point-to-Point
The Point to Point (PtP) topology (Figure 1) consists of two The Point to Point (PtP) topology (Figure 1) consists of two
endpoints, communicating using unicast. Both RTP and RTCP traffic endpoints, communicating using unicast. Both RTP and RTCP traffic
are conveyed endpoint-to-endpoint, using unicast traffic only (even are conveyed endpoint-to-endpoint, using unicast traffic only (even
if, in exotic cases, this unicast traffic happens to be conveyed over if, in exotic cases, this unicast traffic happens to be conveyed over
an IP-multicast address). an IP-multicast address).
+---+ +---+ +---+ +---+
| A |<------->| B | | A |<------->| B |
+---+ +---+ +---+ +---+
Figure 1: Point to Point Figure 1: Point to Point
The main property of this topology is that A sends to B, and only B, The main property of this topology is that A sends to B, and only B,
while B sends to A, and only A. This avoids all complexities of while B sends to A, and only A. This avoids all complexities of
handling multiple endpoints and combining the requirements from them. handling multiple endpoints and combining the requirements stemming
Note that an endpoint can still use multiple RTP Synchronization from them. Note that an endpoint can still use multiple RTP
Sources (SSRCs) in an RTP session. The number of RTP sessions in use Synchronization Sources (SSRCs) in an RTP session. The number of RTP
between A and B can also be of any number. sessions in use between A and B can also be of any number, subject
only to system level limitations like the number range of ports.
RTCP feedback messages for the indicated SSRCs are communicated RTCP feedback messages for the indicated SSRCs are communicated
directly between the endpoints. Therefore, this topology poses directly between the endpoints. Therefore, this topology poses
minimal (if any) issues for any feedback messages. For RTP sessions minimal (if any) issues for any feedback messages. For RTP sessions
which use multiple SSRC per endpoint it can be relevant to implement which use multiple SSRC per endpoint it can be relevant to implement
support for cross reporting suppression as defined in "Real-Time support for cross-reporting suppression as defined in "Sending
Transport Protocol (RTP) Considerations for Endpoints Sending Multiple Media Streams in a Single RTP Session"
Multiple Media Streams" [I-D.lennox-avtcore-rtp-multi-stream]. [I-D.ietf-avtcore-rtp-multi-stream].
3.2. Point to Point via Middlebox 3.2. Point to Point via Middlebox
This section discusses cases where two endpoints communicate but have This section discusses cases where two endpoints communicate but have
one or more middlebox involved in the RTP session. one or more middleboxes involved in the RTP session.
3.2.1. Translators 3.2.1. Translators
Shortcut name: Topo-PtP-Translator Shortcut name: Topo-PtP-Translator
Two main categories of Translators can be distinguished; Transport Two main categories of Translators can be distinguished; Transport
Translators and Media translators. Both Translator types share Translators and Media translators. Both Translator types share
common attributes that separate them from Mixers. For each media common attributes that separate them from Mixers. For each media
stream that the Translator receives, it generates an individual stream that the Translator receives, it generates an individual
stream in the other domain. A translator keeps the SSRC for a stream stream in the other domain. A translator keeps the SSRC for a stream
skipping to change at page 5, line 52 skipping to change at page 6, line 14
A Translator commonly does not use an SSRC of its own, and is not A Translator commonly does not use an SSRC of its own, and is not
visible as an active participant in the session. One reason to have visible as an active participant in the session. One reason to have
its own SSRC is when a Translator acts as a quality monitor that its own SSRC is when a Translator acts as a quality monitor that
sends RTCP reports and therefore is required to have an SSRC. sends RTCP reports and therefore is required to have an SSRC.
Another example is the case when a Translator is prepared to use RTCP Another example is the case when a Translator is prepared to use RTCP
feedback messages. This may, for example, occur in a translator feedback messages. This may, for example, occur in a translator
configured to detect packet loss of important video packets and wants configured to detect packet loss of important video packets and wants
to trigger repair by the media sender, by sending feedback messages. to trigger repair by the media sender, by sending feedback messages.
While such feedback could use the SSRC of the target for the While such feedback could use the SSRC of the target for the
translator, but this in turn would require translation of the targets translator, this in turn would require translation of the targets
RTCP reports to make them consistent. It may be simpler to expose an RTCP reports to make them consistent. It may be simpler to expose an
additional SSRC in the session, the only concern are endpoints additional SSRC in the session. The only concern is endpoints
failing to support the full RTP specification, thus having issues failing to support the full RTP specification, thus having issues
with multiple SSRCs reporting on the RTP streams sent by that with multiple SSRCs reporting on the RTP streams sent by that
endpoint. endpoint.
In general, a Translator implementation should consider which RTCP In general, a Translator implementation should consider which RTCP
feedback messages or codec-control messages it needs to understand in feedback messages or codec-control messages it needs to understand in
relation to the functionality of the Translator itself. This is relation to the functionality of the Translator itself. This is
completely in line with the requirement to also translate RTCP completely in line with the requirement to also translate RTCP
messages between the domains. messages between the domains.
3.2.1.1. Transport Relay/Anchoring 3.2.1.1. Transport Relay/Anchoring
There exist a number of different types of middleboxes that might be There exist a number of different types of middleboxes that might be
inserted between two RTP endpoints on the transport level, e.g. inserted between two RTP endpoints on the transport level, e.g., to
perform changes on the IP/UDP headers, and are, therefore, basic perform changes on the IP/UDP headers, and are, therefore, basic
transport translators. These middleboxes come in many variations transport translators. These middleboxes come in many variations
including NAT [RFC3022] traversal by pinning the media path to a including NAT [RFC3022] traversal by pinning the media path to a
public address domain relay, network topologies where the media flow public address domain relay, network topologies where the media flow
is required to pass a particular point for audit by employing is required to pass a particular point for audit by employing
relaying, or preserving privacy by hiding each peers transport relaying, or preserving privacy by hiding each peer's transport
addresses to the other party. Other protocols or functionalities addresses to the other party. Other protocols or functionalities
that provide this behavior are TURN [RFC5766] servers, Session Border that provide this behavior are TURN [RFC5766] servers, Session Border
Gateways and Media Processing Nodes with media anchoring Gateways and Media Processing Nodes with media anchoring
functionalities. functionalities.
+---+ +---+ +---+ +---+ +---+ +---+
| A |<------>| T |<------->| B | | A |<------>| T |<------->| B |
+---+ +---+ +---+ +---+ +---+ +---+
Figure 2: Point to Point with Translator Figure 2: Point to Point with Translator
What is common for these functions is that they are normally A common element in these functions is that they are normally
transparent on RTP level, i.e. they perform no changes on any RTP or transparent at the RTP level, i.e., they perform no changes on any
RTCP packet fields, only on the lower layers. However, they may RTP or RTCP packet fields and only affect the lower layers. They may
effect the path the RTP and RTCP packets are routed between the affect, however, the path the RTP and RTCP packets are routed between
endpoints in the RTP session, and thereby only indirectly affect the the endpoints in the RTP session, and thereby only indirectly affect
RTP session. For this reason, one could believe that transport the RTP session. For this reason, one could believe that transport
translator type middleboxes do not need to included in this document. translator-type middleboxes do not need to be included in this
However, this topology can raise additional requirements the RTP document. This topology, however, can raise additional requirements
implementation and its interactions with the signalling solution. in the RTP implementation and its interactions with the signalling
Both in signalling and in certain RTCP field other network addresses solution. Both in signalling and in certain RTCP fields, network
than those of the relay can occur, due to that B has different addresses other than those of the relay can occur since B has a
network address than the relay (T). However, implementation not different network address than the relay (T). Implementations that
capable of this will neither not work when endpoints are subject to can not support this will also not work correctly when endpoints are
NAT. subject to NAT.
The transport relay implementation also have some considerations,
where security considerations are an important aspect. Source
address filtering of incoming packets are usually important in
relays, to prevent attackers to inject traffic into a session, which
one peer will think comes from the other peer.
3.2.1.2. Transport Translator 3.2.1.2. Transport Translator
Transport Translators (Topo-Trn-Translator) do not modify the media Transport Translators (Topo-Trn-Translator) do not modify the media
stream itself, but are concerned with transport parameters. stream itself, but are concerned with transport parameters.
Transport parameters, in the sense of this section, comprise the Transport parameters, in the sense of this section, comprise the
transport addresses (to bridge different domains such unicast to transport addresses (to bridge different domains such unicast to
multicast) and the media packetization to allow other transport multicast) and the media packetization to allow other transport
protocols to be interconnected to a session (in gateways). Of the protocols to be interconnected to a session (in gateways). Of the
transport Translators, this memo is primarily interested in those transport Translators, this memo is primarily interested in those
skipping to change at page 7, line 15 skipping to change at page 7, line 30
3.2.1.2. Transport Translator 3.2.1.2. Transport Translator
Transport Translators (Topo-Trn-Translator) do not modify the media Transport Translators (Topo-Trn-Translator) do not modify the media
stream itself, but are concerned with transport parameters. stream itself, but are concerned with transport parameters.
Transport parameters, in the sense of this section, comprise the Transport parameters, in the sense of this section, comprise the
transport addresses (to bridge different domains such unicast to transport addresses (to bridge different domains such unicast to
multicast) and the media packetization to allow other transport multicast) and the media packetization to allow other transport
protocols to be interconnected to a session (in gateways). Of the protocols to be interconnected to a session (in gateways). Of the
transport Translators, this memo is primarily interested in those transport Translators, this memo is primarily interested in those
that use RTP on both sides, and this is assumed henceforth. that use RTP on both sides, and this is assumed henceforth.
Translators that bridge between different protocol worlds need to be Translators that bridge between different protocol worlds need to be
concerned about the mapping of the SSRC/CSRC (Contributing Source) concerned about the mapping of the SSRC/CSRC (Contributing Source)
concept to the non-RTP protocol. When designing a Translator to a concept to the non-RTP protocol. When designing a Translator to a
non-RTP-based media transport, one crucial factor lies in how to non-RTP-based media transport, an important consideration is how to
handle different sources and their identities. This problem space is handle different sources and their identities. This problem space is
not discussed henceforth. not discussed henceforth.
The most basic transport translators that operate below RTP level was The most basic transport translators that operate below the RTP level
already discussed in Section 3.2.1.1. were already discussed in Section 3.2.1.1.
3.2.1.3. Media Translator 3.2.1.3. Media Translator
Media Translators (Topo-Media-Translator), in contrast, modify the Media Translators (Topo-Media-Translator) modify the media stream
media stream itself. This process is commonly known as transcoding. itself. This process is commonly known as transcoding. The
The modification of the media stream can be as small as removing modification of the media stream can be as small as removing parts of
parts of the stream, and it can go all the way to a full transcoding the stream, and it can go all the way to a full decoding and re-
(down to the sample level or equivalent) utilizing a different media encoding (down to the sample level or equivalent) utilizing a
codec. Media Translators are commonly used to connect entities different media codec. Media Translators are commonly used to
without a common interoperability point in the media encoding. connect entities without a common interoperability point in the media
encoding.
Stand-alone Media Translators are rare. Most commonly, a combination Stand-alone Media Translators are rare. Most commonly, a combination
of Transport and Media Translators are used to translate both the of Transport and Media Translator is used to translate both the media
media stream and the transport aspects of a stream between two stream and the transport aspects of a stream between two transport
transport domains (or clouds). domains (or clouds).
When media translation occurs, the Translator's task regarding When media translation occurs, the Translator's task regarding
handling of RTCP traffic becomes substantially more complex. In this handling of RTCP traffic becomes substantially more complex. In this
case, the Translator needs to rewrite B's RTCP Receiver Report before case, the Translator needs to rewrite B's RTCP Receiver Report before
forwarding them to A. The rewriting is needed as the stream received forwarding them to A. The rewriting is needed as the stream received
by B is not the same stream as the other participants receive. For by B is not the same stream as the other participants receive. For
example, the number of packets transmitted to B may be lower than example, the number of packets transmitted to B may be lower than
what A sends, due to the different media format and data rate. what A sends, due to the different media format and data rate.
Therefore, if the Receiver Reports were forwarded without changes, Therefore, if the Receiver Reports were forwarded without changes,
the extended highest sequence number would indicate that B were the extended highest sequence number would indicate that B were
substantially behind in reception, while it most likely it would not substantially behind in reception, while most likely it would not be.
be. Therefore, the Translator must translate that number to a Therefore, the Translator must translate that number to a
corresponding sequence number for the stream the Translator received. corresponding sequence number for the stream the Translator received.
Similar arguments can be made for most other fields in the RTCP Similar arguments can be made for most other fields in the RTCP
Receiver Reports. Receiver Reports.
A media Translator may in some cases act on behalf of the "real" A media Translator may in some cases act on behalf of the "real"
source and respond to RTCP feedback messages. This may occur, for source and respond to RTCP feedback messages. This may occur, for
example, when a receiver requests a bandwidth reduction, and the example, when a receiver requests a bandwidth reduction, and the
media Translator has not detected any congestion or other reasons for media Translator has not detected any congestion or other reasons for
bandwidth reduction between the media source and itself. In that bandwidth reduction between the media source and itself. In that
case, it is sensible that the media Translator reacts to the codec case, it is sensible that the media Translator reacts to the codec
control messages itself, for example, by transcoding to a lower media control messages itself, for example, by transcoding to a lower media
rate. rate.
A variant of translator behaviour worth pointing out is the one A variant of translator behaviour worth pointing out is the one
depicted in Figure 3 of an endpoint A sends a media flow to B. On depicted in Figure 3 of an endpoint A sends a media flow to B. On the
the path there is a device T that on A's behalf does something with path there is a device T that on A's behalf does something with the
the media streams, for example adds an RTP session with FEC media streams, for example adds an RTP session with FEC information
information for A's media streams. In this case, T needs to bind the for A's media streams. In this case, T needs to bind the new FEC
new FEC streams to A's media stream, for example by using the same streams to A's media stream, for example by using the same CNAME as
CNAME as A. A.
+------+ +------+ +------+ +------+ +------+ +------+
| | | | | | | | | | | |
| A |------->| T |-------->| B | | A |------->| T |-------->| B |
| | | |---FEC-->| | | | | |---FEC-->| |
+------+ +------+ +------+ +------+ +------+ +------+
Figure 3: When De-composition is a Translator Figure 3: When De-composition is a Translator
This type of functionality where T does something with the media This type of functionality where T does something with the media
stream on behalf of A is covered under the media translator stream on behalf of A is covered under the media translator
definition. definition.
3.2.2. Back to Back RTP sessions 3.2.2. Back to Back RTP sessions
There exist middleboxes that interconnect two endpoints through There exist middleboxes that interconnect two endpoints through
themselves not by being part of a common RTP session. Instead they themselves, but not by being part of a common RTP session. They
establish two different RTP sessions, one between A and the middlebox establish instead two different RTP sessions, one between A and the
(MB) and another between the MB and B. middlebox and another between the middlebox and B.
|<--Session A-->| |<--Session B-->| |<--Session A-->| |<--Session B-->|
+------+ +------+ +------+ +------+ +------+ +------+
| A |------->| MB |-------->| B | | A |------->| MB |-------->| B |
+------+ +------+ +------+ +------+ +------+ +------+
Figure 4: When De-composition is a Translator Figure 4: When De-composition is a Translator
The MB acts as a application level gateway and bridges the two RTP The middlebox acts as an application-level gateway and bridges the
session. This bridging can be as basic as forwarding the RTP two RTP sessions. This bridging can be as basic as forwarding the
payloads between the sessions, or more complex including media RTP payloads between the sessions, or more complex including media
transcoding. The difference with the single RTP session context is transcoding. The difference with the single RTP session context is
the handling of the SSRCs and the other session related identifiers, the handling of the SSRCs and the other session-related identifiers,
such as CNAMEs. With two different RTP sessions these can be freely such as CNAMEs. With two different RTP sessions these can be freely
changed and it becomes the MB task to maintain the right relations. changed and it becomes the middlebox's task to maintain the correct
relations.
The signalling or other above-RTP level functionalities referencing The signalling or other above-RTP level functionalities referencing
RTP media streams may be what is most impacted by using two RTP RTP media streams may be what is most impacted by using two RTP
sessions and changing identifiers. The structure with two RTP sessions and changing identifiers. The structure with two RTP
sessions also puts a congestion control requirement on the middlebox, sessions also puts a congestion control requirement on the middlebox,
because it becomes fully responsible for the media stream it sources because it becomes fully responsible for the media stream it sources
into each of the sessions. into each of the sessions.
This can be solved locally or by bridging also statistics from the Adherence to congestion control can be solved locally or by bridging
receiving endpoint. However, from an implementation point this also statistics from the receiving endpoint. From an implementation
requires the implementation to support dealing with a number of point, however, this requires dealing with a number of
inconsistencies. First, packet loss must be detected for an RTP flow inconsistencies. First, packet loss must be detected for an RTP flow
sent from A to the MB, and that loss must be reported through a sent from A to the middlebox, and that loss must be reported through
skipped sequence number in the flow from the MB to B. This coupling a skipped sequence number in the flow from the middlebox to B. This
and the resulting inconsistencies is conceptually easier to handle coupling and the resulting inconsistencies is conceptually easier to
when considering the two flows as belonging to a single RTP session. handle when considering the two flows as belonging to a single RTP
session.
3.3. Point to Multipoint Using Multicast 3.3. Point to Multipoint Using Multicast
Multicast is an IP layer functionality that is available in some
Multicast is a IP layer functionality that is available in some
networks. Two main flavors can be distinguished: Any Source networks. Two main flavors can be distinguished: Any Source
Multicast (ASM) where any multicast group participant can send to the Multicast (ASM) [RFC1112] where any multicast group participant can
group address and expect the packet to reach all group participants; send to the group address and expect the packet to reach all group
and Source Specific Multicast (SSM), where only a particular IP host participants; and Source Specific Multicast (SSM) [RFC3569], where
sends to the multicast group. Both these models are discussed below only a particular IP host sends to the multicast group. Both these
in their respective section. models are discussed below in their respective sections.
3.3.1. Any Source Multicast (ASM) 3.3.1. Any Source Multicast (ASM)
Shortcut name: Topo-ASM (was Topo-Multicast) Shortcut name: Topo-ASM (was Topo-Multicast)
+-----+ +-----+
+---+ / \ +---+ +---+ / \ +---+
| A |----/ \---| B | | A |----/ \---| B |
+---+ / Multi- \ +---+ +---+ / Multi- \ +---+
+ Cast + + Cast +
+---+ \ Network / +---+ +---+ \ Network / +---+
| C |----\ /---| D | | C |----\ /---| D |
+---+ \ / +---+ +---+ \ / +---+
+-----+ +-----+
skipping to change at page 10, line 25 skipping to change at page 10, line 37
Point to Multipoint (PtM) is defined here as using a multicast Point to Multipoint (PtM) is defined here as using a multicast
topology as a transmission model, in which traffic from any topology as a transmission model, in which traffic from any
participant reaches all the other participants, except for cases such participant reaches all the other participants, except for cases such
as: as:
o packet loss, or o packet loss, or
o when a participant does not wish to receive the traffic for a o when a participant does not wish to receive the traffic for a
specific multicast group and, therefore, has not subscribed to the specific multicast group and, therefore, has not subscribed to the
IP-multicast group in question. This scenario can occur, for IP multicast group in question. This scenario can occur, for
example, where a multi-media session is distributed using two or example, where a multimedia session is distributed using two or
more multicast groups and a participant is subscribed only to a more multicast groups and a participant is subscribed only to a
subset of these sessions. subset of these sessions.
In the above context, "traffic" encompasses both RTP and RTCP In the above context, "traffic" encompasses both RTP and RTCP
traffic. The number of participants can vary between one and many, traffic. The number of participants can vary between one and many,
as RTP and RTCP scale to very large multicast groups (the theoretical as RTP and RTCP scale to very large multicast groups (the theoretical
limit of the number of participants in a single RTP session is in the limit of the number of participants in a single RTP session is in the
range of billions). The above can be realized using Any Source range of billions). The above can be realized using Any Source
Multicast (ASM). Multicast (ASM).
For feedback usage, it is useful to define a "small multicast group" For feedback usage, it is useful to define a "small multicast group"
as a group where the number of participants is so low (and other as a group where the number of participants is so low (and other
factors such as the connectivity is so good) that it allows the factors such as the connectivity is so good) that it allows the
participants to use early or immediate feedback, as defined in AVPF participants to use early or immediate feedback, as defined in AVPF
[RFC4585]. Even when the environment would allow for the use of a [RFC4585]. Even when the environment would allow for the use of a
small multicast group, some applications may still want to use the small multicast group, some applications may still want to use the
more limited options for RTCP feedback available to large multicast more limited options for RTCP feedback available to large multicast
groups, for example when there is a likelyhood that the threshold of groups, for example when there is a likelihood that the threshold of
the small multicast group (in terms of participants) may be exceeded the small multicast group (in terms of participants) may be exceeded
during the lifetime of a session. during the lifetime of a session.
RTCP feedback messages in multicast reach, like media, every RTCP feedback messages in multicast reach, like media data, every
subscriber (subject to packet losses and multicast group subscriber (subject to packet losses and multicast group
subscription). Therefore, the feedback suppression mechanism subscription). Therefore, the feedback suppression mechanism
discussed in [RFC4585] is typically required. Each individual node discussed in [RFC4585] is typically required. Each individual node
needs to process every feedback message it receives, not to determine needs to process every feedback message it receives, not to determine
if it is affected or if the feedback message applies only to some if it is affected or if the feedback message applies only to some
other participant, but also to derive timing restriction for the other participant, but also to derive timing restrictions for the
sending of its own feedback messages, if any. sending of its own feedback messages, if any.
3.3.2. Source Specific Multicast (SSM) 3.3.2. Source Specific Multicast (SSM)
In Any Source Multicast, any of the participants can send to all the In Any Source Multicast, any of the participants can send to all the
other participants, by sending a packet to the multicast group. In other participants, by sending a packet to the multicast group. In
contrast, Source Specific Multicast [RFC4607] refers to scenarios contrast, Source Specific Multicast [RFC3569][RFC4607] refers to
where only a single source (Distribution Source) can send to the scenarios where only a single source (Distribution Source) can send
multicast group, creating a topology that looks like the one below: to the multicast group, creating a topology that looks like the one
below:
+--------+ +-----+ +--------+ +-----+
|Media | | | Source-specific |Media | | | Source-specific
|Sender 1|<----->| D S | Multicast |Sender 1|<----->| D S | Multicast
+--------+ | I O | +--+----------------> R(1) +--------+ | I O | +--+----------------> R(1)
| S U | | | | | S U | | | |
+--------+ | T R | | +-----------> R(2) | +--------+ | T R | | +-----------> R(2) |
|Media |<----->| R C |->+ | : | | |Media |<----->| R C |->+ | : | |
|Sender 2| | I E | | +------> R(n-1) | | |Sender 2| | I E | | +------> R(n-1) | |
+--------+ | B | | | | | | +--------+ | B | | | | | |
skipping to change at page 11, line 42 skipping to change at page 12, line 30
+--------+ +-----+ RTCP Unicast +--------+ +-----+ RTCP Unicast
FT = Feedback Target FT = Feedback Target
Transport from the Feedback Target to the Distribution Transport from the Feedback Target to the Distribution
Source is via unicast or multicast RTCP if they are not Source is via unicast or multicast RTCP if they are not
co-located. co-located.
Figure 6: Point to Multipoint using Source Specific Multicast Figure 6: Point to Multipoint using Source Specific Multicast
In the SSM topology (Figure 6) a number of RTP sources (1 to M) are In the SSM topology (Figure 6) a number of RTP sources (1 to M) are
allowed to send media to the SSM group. These send media to a allowed to send media to the SSM group. These sources send media to
dedicated distribution source, which then forwards the media streams a dedicated distribution source, which forwards the media streams to
to the multicast group on behalf of the original senders. The media the multicast group on behalf of the original senders. The media
streams reach the Receivers (R(1) to R(n)). The Receivers' RTCP streams reach the Receivers (R(1) to R(n)). The Receivers' RTCP
cannot be sent to the multicast group, as the SSM multicast group by messages cannot be sent to the multicast group, as the SSM multicast
definition has only a single source. To support RTCP, an RTP group by definition has only a single source. To support RTCP, an
extension for SSM [RFC5760] was defined. It uses unicast RTP extension for SSM [RFC5760] was defined. It uses unicast
transmission to send RTCP from each of the receivers to one or more transmission to send RTCP from each of the receivers to one or more
Feedback Targets (FT). The feedback targets relay the RTCP Feedback Targets (FT). The feedback targets relay the RTCP
unmodified, or provide summary of the participants RTCP reports unmodified, or provide a summary of the participants RTCP reports
towards the whole group by forwarding the RTCP traffic to the towards the whole group by forwarding the RTCP traffic to the
distribution source. Figure 6 only shows a single feedback target distribution source. Figure 6 only shows a single feedback target
integrated in the distribution source, but for scalability the FT can integrated in the distribution source, but for scalability the FT can
be many and have responsibility for sub-groups of the receivers. For be many and have responsibility for sub-groups of the receivers. For
summary reports, however, there must be a single feedback aggregating summary reports, however, there must be a single feedback aggregating
all the summaries to a common message to the whole receiver group. all the summaries to a common message to the whole receiver group.
The RTP extension for SSM specifies how feedback (both reception The RTP extension for SSM specifies how feedback (both reception
information and specific feedback events) are handled. The more information and specific feedback events) are handled. The more
general problems associated with the use of multicast, where everyone general problems associated with the use of multicast, where everyone
receives what the distribution source sends needs to be accounted receives what the distribution source sends needs to be accounted
for. for.
The result of this is some common behaviours for RTP multicast: Aforementioned situation results in common behavior for RTP
multicast:
1. Multicast applications often use a group of RTP sessions, not 1. Multicast applications often use a group of RTP sessions, not
one. Each endpoint needs to be a member of most or all of these one. Each endpoint needs to be a member of most or all of these
RTP sessions in order to perform well. RTP sessions in order to perform well.
2. Within each RTP session, the number of media sinks is likely to 2. Within each RTP session, the number of media sinks is likely to
be much larger than the number of RTP sources. be much larger than the number of RTP sources.
3. Multicast applications need signalling functions to identify the 3. Multicast applications need signalling functions to identify the
relationships between RTP sessions. relationships between RTP sessions.
skipping to change at page 13, line 37 skipping to change at page 14, line 25
-------> Multicast RTP Flow -------> Multicast RTP Flow
.-.-.-.> Multicast RTCP Flow .-.-.-.> Multicast RTCP Flow
.=.=.=.> Unicast RTCP Reports .=.=.=.> Unicast RTCP Reports
~~~~~~~> Unicast RTCP Feedback Messages ~~~~~~~> Unicast RTCP Feedback Messages
.......> Unicast RTP Flow .......> Unicast RTP Flow
Figure 7 Figure 7
The Rapid acquisition extension allows an endpoint joining an SSM The Rapid acquisition extension allows an endpoint joining an SSM
multicast session to request media starting with the last sync-point multicast session to request media starting with the last sync-point
(from where media can be decoded without prior packets) to be sent at (from where media can be decoded without requiring context
high speed until such time where, after decoding of these bursted established by the decoding of prior packets) to be sent at high
media packets, the correct media timing is established, i.e. media speed until such time where, after decoding of these burst-delivered
media packets, the correct media timing is established, i.e. media
packets are received within adequate buffer intervals for this packets are received within adequate buffer intervals for this
application. This is accomplished by first establishing an unicast application. This is accomplished by first establishing a unicast
PtP RTP session between the BRS (Figure 7) and the RTP Receiver. PtP RTP session between the Burst/Retransmission Source (BRS, Figure
That session is used to transmit cached packets from the multicast 7) and the RTP Receiver. The unicast session is used to transmit
group at higher then nominal speed so to synchronize the receiver to cached packets from the multicast group at higher then normal speed
the ongoing multicast packet flow. Once the RTP receiver and its in order to synchronize the receiver to the ongoing multicast packet
decoder have caught up with the multicast session's current delivery, flow. Once the RTP receiver and its decoder have caught up with the
the receiver switches over to receiving from the multicast group multicast session's current delivery, the receiver switches over to
directly. The (still existing) PtP RTP session can be used as a receiving directly from the multicast group. The (still existing)
repair channel, i.e. for RTP Retransmission traffic of those packets PtP RTP session is, in many deployed applications, be used as a
repair channel, i.e., for RTP Retransmission traffic of those packets
that were not received from the multicast group. that were not received from the multicast group.
3.4. Point to Multipoint Using Mesh 3.4. Point to Multipoint Using Mesh
Shortcut name: Topo-Mesh Shortcut name: Topo-Mesh
+---+ +---+ +---+ +---+
| A |<---->| B | | A |<---->| B |
+---+ +---+ +---+ +---+
^ ^ ^ ^
skipping to change at page 14, line 24 skipping to change at page 15, line 13
\ / \ /
v v v v
+---+ +---+
| C | | C |
+---+ +---+
Figure 8: Point to Multi-Point using Mesh Figure 8: Point to Multi-Point using Mesh
Based on the RTP session definition, it is clearly possible to have a Based on the RTP session definition, it is clearly possible to have a
joint RTP session over multiple unicast transport flows like the joint RTP session over multiple unicast transport flows like the
above three endpoint joint session. In this case, A needs to send above joint three endpoint session. In this case, A needs to send
its' media streams and RTCP packets to both B and C over their its' media streams and RTCP packets to both B and C over their
respective transport flows. As long as all participants do the same, respective transport flows. As long as all participants do the same,
everyone will have a joint view of the RTP session. everyone will have a joint view of the RTP session.
This doesn't create any additional requirements beyond the need to This does not create any additional requirements beyond the need to
have multiple transport flows associated with a single RTP session. have multiple transport flows associated with a single RTP session.
Note that an endpoint may use a single local port to receive all Note that an endpoint may use a single local port to receive all
these transport flows, or it might have separate local reception these transport flows, or it might have separate local reception
ports for each of the endpoints. ports for each of the endpoints.
+-A--------------------+ +-B-----------+
|+---+ | | |
||CAM| | | |
|+---+ +-UDP1------| |-UDP1------+ |
| | | +-RTP1----| |-RTP1----+ | |
| V | | +-Video-| |-Video-+ | | |
|+----+ | | | |<----------------|BV1 | | | |
||ENC |----+-+-+--->AV1|---------------->| | | | |
|+----+ | | +-------| |-------+ | | |
| | | +---------| |---------+ | |
| | +-----------| |-----------+ |
| | ------------| |------------ |
| | | |-------------+
| | |
| | | +-C-----------+
| | | | |
| | +-UDP2------| |-UDP2------+ |
| | | +-RTP1----| |-RTP1----+ | |
| | | | +-Video-| |-Video-+ | | |
| +-------+-+-+--->AV1|---------------->| | | | |
| | | | |<----------------|CV1 | | | |
| | | +-------| |-------+ | | |
| | +---------| |---------+ | |
| +-----------| |-----------+ |
| ------------| |------------ |
+----------------------+ +-------------+
Figure 9: An Multi-unicast Mesh with a joint RTP session
A joint RTP session from A's perspective for the Mesh depicted in
Figure 8 with a joint RTP session have multiple transport flows, here
enumerated as UDP1 and UDP2. However, there is only one RTP session
(RTP1). The media source (CAM) is encoded and transmitted over the
SSRC (AV1) across both transport layers. However, as this is a joint
RTP session, the two streams must be the same. Thus, an congestion
control adaptation needed for the paths A to B and A to C needs to
use the most restricting path's properties.
An alternative structure for establishing the above topology is to An alternative structure for establishing the above topology is to
use independent RTP sessions between each pair of peers, i.e. three use independent RTP sessions between each pair of peers, i.e., three
different RTP sessions. In some scenarios, the same RTP media stream different RTP sessions. In some scenarios, the same RTP media stream
is being sent from each sending endpoint. In others, some form of may be sent from transmitting endpoint, however it also supports
local adaptation takes place in one or more of the RTP media streams, local adaptation taking place in one or more of the RTP media
rendering them non-identical. From a topologies viewpoint, a streams, rendering them non-identical.
difference exists in the behaviours around RTCP. For example, when a
single RTP session spans all three endpoints and their connecting +-A----------------------+ +-B-----------+
flows, a RTCP bandwidth is calculated and used for this single one |+---+ | | |
joint session. In contrast, when there are multiple independent RTP ||MIC| +-UDP1------| |-UDP1------+ |
sessions, each has its local RTCP bandwidth allocation. Also, when |+---+ | +-RTP1----| |-RTP1----+ | |
multiple sessions are used, endpoints not directly involved in these | | +----+ | | +-Audio-| |-Audio-+ | | |
sessions do not have any awareness of the conditions occurring in | +->|ENC1|--+-+-+--->AA1|------------->| | | | |
sessions not involving that endpoint. For example, in case of the | | +----+ | | | |<-------------|BA1 | | | |
three endpoint configuration above, endpoint A has no awareness of | | | | +-------| |-------+ | | |
the conditions occurring in the session between endpoints B and C | | | +---------| |---------+ | |
| | +-----------| |-----------+ |
| | ------------| |-------------|
| | | |-------------+
| | |
| | | +-C-----------+
| | | | |
| | +-UDP2------| |-UDP2------+ |
| | | +-RTP2----| |-RTP2----+ | |
| | +----+ | | +-Audio-| |-Audio-+ | | |
| +->|ENC2|--+-+-+--->AA2|------------->| | | | |
| +----+ | | | |<-------------|CA1 | | | |
| | | +-------| |-------+ | | |
| | +---------| |---------+ | |
| +-----------| |-----------+ |
+------------------------+ +-------------+
Figure 10: An Multi-unicast Mesh with independent RTP session
Lets review the topology when independent RTP sessions are used, from
A's perspective in Figure 8 by considering both how the media is a
handled and the RTP sessions that are set-up in Figure 10. A's
microphone is captured and the digital audio can then be feed into
two different encoder instances, as each beeing associated with two
independent RTP sessions (RTP1 and RTP2). The SSRCs (AA1 and AA2) in
each RTP session will be completely independent and the media bit-
rate produced by the encoders can also be tuned differently to
address any congestion control requirements differing for the paths A
to B compared to A to C.
From a topologies viewpoint, an important difference exists in the
behavior around RTCP. First, when a single RTP session spans all
three endpoints and their connecting flows, an common RTCP bandwidth
is calculated and used for this single joint session. In contrast,
when there are multiple independent RTP sessions, each RTP session
has its local RTCP bandwidth allocation.
Further, when multiple sessions are used, endpoints not directly
involved in a session, do not have any awareness of the conditions in
those sessions. For example, in the case of the three endpoint
configuration in Figure 8, endpoint A has no awareness of the
conditions occurring in the session between endpoints B and C
(whereas, if a single RTP session were used, it would have such (whereas, if a single RTP session were used, it would have such
awareness). Loop detection is also affected. With independent RTP awareness).
sessions, the SSRC/CSRC can't be used to determine when a endpoint
receives its own media stream or a mixed media stream including its Loop detection is also affected. With independent RTP sessions, the
own media stream a condition known as a loop. The identification of SSRC/CSRC cannot be used to determine when an endpoint receives its
loops and, in most cases, its avoidance, has to be achieved by other own media stream, or a mixed media stream including its own media
means, for example through signaling, or the use of an RTP external stream (a condition known as a loop). The identification of loops
and, in most cases, their avoidance, has to be achieved by other
means, for example through signaling or the use of an RTP external
name space binding SSRC/CSRC among any communicating RTP sessions in name space binding SSRC/CSRC among any communicating RTP sessions in
the mesh. the mesh.
3.5. Point to Multipoint Using the RFC 3550 Translator 3.5. Point to Multipoint Using the RFC 3550 Translator
This section discusses some additional usages related to point to This section discusses some additional usages related to point to
multipoint of Translators compared to the point to point only cases multipoint of Translators compared to the point to point only cases
in Section 3.2.1. in Section 3.2.1.
3.5.1. Relay - Transport Translator 3.5.1. Relay - Transport Translator
Shortcut name: Topo-PtM-Trn-Translator Shortcut name: Topo-PtM-Trn-Translator
This section discusses Transport Translator only usages to enable This section discusses Transport Translator only usages to enable
multipoint sessions. multipoint sessions.
+-----+ +-----+
+---+ / \ +------------+ +---+ +---+ / \ +------------+ +---+
| A |<---/ \ | |<---->| B | | A |<---/ \ | |<---->| B |
+---+ / Multi- \ | | +---+ +---+ / Multi- \ | | +---+
+ Cast +->| Translator | + cast +->| Translator |
+---+ \ Network / | | +---+ +---+ \ Network / | | +---+
| C |<---\ / | |<---->| D | | C |<---\ / | |<---->| D |
+---+ \ / +------------+ +---+ +---+ \ / +------------+ +---+
+-----+ +-----+
Figure 9: Point to Multipoint Using Multicast Figure 11: Point to Multipoint Using Multicast
Figure 9 depicts an example of a Transport Translator performing at Figure 11 depicts an example of a Transport Translator performing at
least IP address translation. It allows the (non-multicast-capable) least IP address translation. It allows the (non-multicast-capable)
participants B and D to take part in an any source multicast session participants B and D to take part in an any source multicast session
by having the Translator forward their unicast traffic to the by having the Translator forward their unicast traffic to the
multicast addresses in use, and vice versa. It must also forward B's multicast addresses in use, and vice versa. It must also forward B's
traffic to D, and vice versa, to provide each of B and D with a traffic to D, and vice versa, to provide each of B and D with a
complete view of the session. complete view of the session.
+---+ +------------+ +---+ +---+ +------------+ +---+
| A |<---->| |<---->| B | | A |<---->| |<---->| B |
+---+ | | +---+ +---+ | | +---+
| Translator | | Translator |
+---+ | | +---+ +---+ | | +---+
| C |<---->| |<---->| D | | C |<---->| |<---->| D |
+---+ +------------+ +---+ +---+ +------------+ +---+
Figure 10: RTP Translator (Relay) with Only Unicast Paths Figure 12: RTP Translator (Relay) with Only Unicast Paths
Another Translator scenario is depicted in Figure 10. Herein, the Another Translator scenario is depicted in Figure 12. The Translator
Translator connects multiple users of a conference through unicast. in this case connects multiple users of a conference through unicast.
This can be implemented using a very simple transport Translator, This can be implemented using a very simple transport Translator
which in this document is called a relay. The relay forwards all which, in this document, is called a relay. The relay forwards all
traffic it receives, both RTP and RTCP, to all other participants. traffic it receives, both RTP and RTCP, to all other participants.
In doing so, a multicast network is emulated without relying on a In doing so, a multicast network is emulated without relying on a
multicast-capable network infrastructure. multicast-capable network infrastructure.
For RTCP feedback this results in a similar set of considerations For RTCP feedback this results in a similar set of considerations to
those described in the ASM RTP topology. It also puts some those described in the ASM RTP topology. It also puts some
additional signalling requirements onto the session establishment; additional signalling requirements onto the session establishment;
for example, a common configuration of RTP payload types is required. for example, a common configuration of RTP payload types is required.
Transport translators and relays should always consider doing source
address filtering, to prevent attackers to inject traffic using the
listening ports on the translator. The translator can however go one
step further, and especially if explicit SSRC signalling is used,
prevent other session participants to send SSRCs that are used by
other participants in the session. This can improve the security
properties of the session, despite the use of group keys that on
cryptographic level allows anyone to impersonate another in the same
RTP session.
A Translator that doesn't change the RTP/RTCP packets content can be
operated without the requiring the translator to have access to the
security contexts used to protect the RTP/RTCP traffic between the
participants.
3.5.2. Media Translator 3.5.2. Media Translator
In the context of multipoint communications a Media Translator is not In the context of multipoint communications a Media Translator is not
providing new mechanisms to establish a multipoint session. It is providing new mechanisms to establish a multipoint session. It is
much more an enabler or facilitator that ensures one or some sub-set more of an enabler, or facilitator, that ensures one or some sub-set
of session participants can participate in the session. of session participants can participate in the session.
If B in Figure 9 were behind a limited network path, the Translator If B in Figure 11 were behind a limited network path, the Translator
may perform media transcoding to allow the traffic received from the may perform media transcoding to allow the traffic received from the
other participants to reach B without overloading the path. This other participants to reach B without overloading the path. This
transcoding can help the other participants in the Multicast part of transcoding can help the other participants in the Multicast part of
the session, by not requiring the quality transmitted by A to be the session, by not requiring the quality transmitted by A to be
lowered to the nitrates that B is actually capable of receiving. lowered to the bitrates that B is actually capable of receiving.
3.6. Point to Multipoint Using the RFC 3550 Mixer Model 3.6. Point to Multipoint Using the RFC 3550 Mixer Model
Shortcut name: Topo-Mixer Shortcut name: Topo-Mixer
A Mixer is a middlebox that aggregates multiple RTP streams, which A Mixer is a middlebox that aggregates multiple RTP streams that are
are part of a session, by generating a new RTP stream and, in most part of a session by generating a new RTP stream and, in most cases,
cases, by manipulation of the media data. One common application for by manipulating the media data. One common application for a Mixer
a Mixer is to allow a participant to receive a session with a reduced is to allow a participant to receive a session with a reduced amount
amount of resources. of resources.
+-----+ +-----+
+---+ / \ +-----------+ +---+ +---+ / \ +-----------+ +---+
| A |<---/ \ | |<---->| B | | A |<---/ \ | |<---->| B |
+---+ / Multi- \ | | +---+ +---+ / Multi- \ | | +---+
+ Cast +->| Mixer | + cast +->| Mixer |
+---+ \ Network / | | +---+ +---+ \ Network / | | +---+
| C |<---\ / | |<---->| D | | C |<---\ / | |<---->| D |
+---+ \ / +-----------+ +---+ +---+ \ / +-----------+ +---+
+-----+ +-----+
Figure 11: Point to Multipoint Using the RFC 3550 Mixer Model Figure 13: Point to Multipoint Using the RFC 3550 Mixer Model
A Mixer can be viewed as a device terminating the media streams A Mixer can be viewed as a device terminating the media streams
received from other session participants. Using the media data from received from other session participants. Using the media data from
the received media streams, a Mixer generates a media stream that is the received media streams, a Mixer generates a media stream that is
sent to the session participant. sent to the session participant.
The content that the Mixer provides is the mixed aggregate of what The content that the Mixer provides is the mixed aggregate of what
the Mixer receives over the PtP or PtM paths, which are part of the the Mixer receives over the PtP or PtM paths, which are part of the
same conference session. same conference session.
The Mixer is the content source, as it mixes the content (often in The Mixer is the content source, as it mixes the content (often in
the uncompressed domain) and then encodes it for transmission to a the uncompressed domain) and then encodes it for transmission to a
participant. The CSRC Count (CC) and CSRC fields in the RTP header participant. The CSRC Count (CC) and CSRC fields in the RTP header
can be used to indicate the contributors of to the newly generated can be used to indicate the contributors to the newly generated
stream. The SSRCs of the to-be-mixed streams on the Mixer input stream. The SSRCs of the to-be-mixed streams on the Mixer input
appear as the CSRCs at the Mixer output. That output stream uses a appear as the CSRCs at the Mixer output. That output stream uses a
unique SSRC that identifies the Mixer's stream. The CSRC should be unique SSRC that identifies the Mixer's stream. The CSRC should be
forwarded between the different conference participants to allow for forwarded between the different conference participants to allow for
loop detection and identification of sources that are part of the loop detection and identification of sources that are part of the
global session. Note that Section 7.1 of RFC 3550 requires the SSRC global session. Note that Section 7.1 of RFC 3550 requires the SSRC
space to be shared between domains for these reasons. This also space to be shared between domains for these reasons. This also
implies that any SDES information normally needs to be forwarded implies that any SDES information normally needs to be forwarded
across the mixer. across the mixer.
The Mixer is responsible for generating RTCP packets in accordance The Mixer is responsible for generating RTCP packets in accordance
with its role. It is a receiver and should therefore send receiver with its role. It is a receiver and should therefore send receiver
reports for the media streams it receives. In its role as a media reports for the media streams it receives. In its role as a media
sender, it should also generate sender reports for those media sender, it should also generate sender reports for those media
streams it sends. As specified in Section 7.3 of RFC 3550, a Mixer streams it sends. As specified in Section 7.3 of RFC 3550, a Mixer
must not forward RTCP unaltered between the two domains. must not forward RTCP unaltered between the two domains.
The Mixer depicted in Figure 11 is involved in three domains that The Mixer depicted in Figure 13 is involved in three domains that
need to be separated: the any source multicast network (including need to be separated: the any source multicast network (including
participants A and C), participant B, and participant D. Assuming participants A and C), participant B, and participant D. Assuming all
all four participants in the conference are interested in receiving four participants in the conference are interested in receiving
content from each other participant, the Mixer produces different content from each other participant, the Mixer produces different
mixed streams for B and D, as the one to B may contain content mixed streams for B and D, as the one to B may contain content
received from D, and vice versa. However, the Mixer may only need received from D, and vice versa. However, the Mixer may only need
one SSRC per media type in each domain that is the receiving entity one SSRC per media type in each domain where it is the receiving
and transmitter of mixed content. entity and transmitter of mixed content.
In the multicast domain, a Mixer still needs to provide a mixed view In the multicast domain, a Mixer still needs to provide a mixed view
of the other domains. This makes the Mixer simpler to implement and of the other domains. This makes the Mixer simpler to implement and
avoids any issues with advanced RTCP handling or loop detection, avoids any issues with advanced RTCP handling or loop detection,
which would be problematic if the Mixer were providing non-symmetric which would be problematic if the Mixer were providing non-symmetric
behavior. Please see Section 3.11 for more discussion on this topic. behavior. Please see Section 3.11 for more discussion on this topic.
However, the mixing operation in each domain could potentially be The mixing operation, however, in each domain could potentially be
different. different.
A Mixer is responsible for receiving RTCP feedback messages and A Mixer is responsible for receiving RTCP feedback messages and
handling them appropriately. The definition of "appropriate" depends handling them appropriately. The definition of "appropriate" depends
on the message itself and the context. In some cases, the reception on the message itself and the context. In some cases, the reception
of a codec-control message by the Mixer may result in the generation of a codec-control message by the Mixer may result in the generation
and transmission of RTCP feedback messages by the Mixer to the and transmission of RTCP feedback messages by the Mixer to the
participants in the other domain(s). In other cases, a message is participants in the other domain(s). In other cases, a message is
handled by the Mixer itself and therefore not forwarded to any other handled by the Mixer itself and therefore not forwarded to any other
domain. domain.
When replacing the multicast network in Figure 11 (to the left of the When replacing the multicast network in Figure 13 (to the left of the
Mixer) with individual unicast paths as depicted in Figure 12, the Mixer) with individual unicast paths as depicted in Figure 14, the
Mixer model is very similar to the one discussed in Section 3.9 Mixer model is very similar to the one discussed in Section 3.9
below. Please see the discussion in Section 3.9 about the below. Please see the discussion in Section 3.9 about the
differences between these two models. differences between these two models.
+---+ +------------+ +---+ +---+ +------------+ +---+
| A |<---->| |<---->| B | | A |<---->| |<---->| B |
+---+ | | +---+ +---+ | | +---+
| Mixer | | Mixer |
+---+ | | +---+ +---+ | | +---+
| C |<---->| |<---->| D | | C |<---->| |<---->| D |
+---+ +------------+ +---+ +---+ +------------+ +---+
Figure 12: RTP Mixer with Only Unicast Paths Figure 14: RTP Mixer with Only Unicast Paths
Lets now discuss in more detail different mixing operations that a We now discuss in more detail the different mixing operations that a
mixer can perform and how they can affect the RTP and RTCP. mixer can perform and how they can affect RTP and RTCP behavior.
3.6.1. Media Mixing 3.6.1. Media Mixing
The media mixing mixer is likely the one that most think of when they The media mixing mixer is likely the one that most think of when they
hear the term "mixer". Its basic pattern of operation is that it hear the term "mixer". Its basic mode of operation is that it
receives media streams from (typically several) participants. Of receives media streams from several participants and selects the
those, it selects (either through static configuration or by dynamic, stream(s) to be included in a media-domain mix. The selection can be
content dependent means such as voice activation) the stream(s) to be through static configuration or by dynamic, content dependent means
included in a media domain mix. Then it creates a single outgoing such as voice activation. The mixer then creates a single outgoing
stream from this mix. stream from this mix.
The most commonly deployed media mixer is probably the audio mixer, The most commonly deployed media mixer is probably the audio mixer,
used in voice conferencing, where the output consists of a mixture of used in voice conferencing, where the output consists of a mixture of
all the input streams; this needs minimal signalling to be all the input streams; this needs minimal signalling to be
successfully set up. Audio mixing is relatively straightforward and successfully set up. Audio mixing is relatively straightforward and
commonly possible for a reasonable number of participants. Lets commonly possible for a reasonable number of participants. Assume,
assume that you want to mix N streams from different participants. for example, that one wants to mix N streams from different
The mixer needs to decode those N streams, typically into the sample participants. The mixer needs to decode those N streams, typically
domain. Then it needs to produce N or N+1 mixes, the reasons that into the sample domain, and then produce N or N+1 mixes. Different
different mixes are needed being that each contributing source get a mixes are needed so that each contributing source gets a mix of all
mix of all other sources except its own, as this would result in an other sources except its own, as this would result in an echo. When
echo. When N is lower than the number of all participants one may N is lower than the number of all participants one may produce a Mix
produce a Mix of all N streams for the group that are currently not of all N streams for the group that are currently not included in the
included in the mix, thus N+1 mixes. These audio streams are then mix, thus N+1 mixes. These audio streams are then encoded again, RTP
encoded again, RTP packetized and sent out. In many cases, audio packetized and sent out. In many cases, audio level normalization is
level normalization is also required before the actual mixing also required before the actual mixing process.
process.
Video can't really be "mixed" and produce something particularly In video, the term "mixing" has a different interpretation than
useful for the users, however creating an composition out of the audio. It is commonly used to refer to the process of spatially
contributed video streams is possible and known as "tiling". For combining contributed video streams is known as "tiling". The
example the reconstructed, appropriately scaled down videos can be reconstructed, appropriately scaled down videos can be spatially
spatially arranged in a set of tiles, each tile containing the video arranged in a set of tiles, each tile containing the video from a
from a participant. Tiles can be of different sizes, so that, for participant. Tiles can be of different sizes, so that, for example,
example, a particularly important participant, or the loudest a particularly important participant, or the loudest speaker, is
speaker, is being shown on in larger tile than other participants. A being shown on in larger tile than other participants. A self-view
self-picture can be included in the tiling, which can either be picture can be included in the tiling, which can either be locally
locally produced or be a feedback from a received and reconstructed produced or be a feedback from a received and reconstructed video
video image (allowing for confidence monitoring, the participant sees image. Such remote loopback allows for confidence monitoring, i.e.,
himself/herself just as other participants see him/her). The tiling it enables the participant to see himself/herself just as other
normally operates on reconstructed video in the sample domain. The participants see him/her. The tiling normally operates on
tiled image is encoded, packetized, and sent by the mixer. It is reconstructed video in the sample domain. The tiled image is
possible that a middlebox with media mixing duties contains only a encoded, packetized, and sent by the mixer. It is possible that a
single mixer of the aforementioned type, in which case all middlebox with media mixing duties contains only a single mixer of
participants necessarily see the same tiled video, even if it is the aforementioned type, in which case all participants necessarily
being sent over different RTP streams. More common, however, are see the same tiled video, even if it is being sent over different RTP
mixing arrangement where an individual mixer is available for each streams. More common, however, are mixing arrangement where an
outgoing port of the middlebox, allowing individual compositions for individual mixer is available for each outgoing port of the
each participant. middlebox, allowing individual compositions for each participant (a
feature referred to as personalized layout).
One problem with media mixing is that it consumes both large amount One problem with media mixing is that it consumes both large amount
of media processing (for the actual mixing process in the of media processing (for the actual mixing process in the
uncompressed domain) and encoding resources (for the encoding of the uncompressed domain) and encoding resources (for the encoding of the
mixed signal). Another problem is the quality degradation created by mixed signal). Another problem is the quality degradation created by
decoding and re-encoding the media that is encapsulated in the RTP decoding and re-encoding the media that is encapsulated in the RTP
media stream, which is the result of the lossy nature of most, if not media stream, which is the result of the lossy nature of most
all, commonly used media codecs. A third problem is the latency commonly used media codecs. A third problem is the latency
introduced by the media mixing, which can be substantial and introduced by the media mixing, which can be substantial and
annoyingly noticeable in case of video. The advantage of media annoyingly noticeable in case of video, or in case of audio if that
mixing is that it is quite simplistic for the clients to handle the mixed audio is lip-sychronized with high latency video. The
single media stream (which includes the mixed aggregate of many advantage of media mixing is that it is straightforward for the
sources), as they don't need to handle multiple decodings, local clients to handle the single media stream (which includes the mixed
mixing and composition. In fact, mixers were introduced in pre-RTP aggregate of many sources), as they don't need to handle multiple
times so that legacy, single stream receiving endpoints can decodings, local mixing and composition. In fact, mixers were
successfully participate in what a user would recognize as a introduced in pre-RTP times so that legacy, single stream receiving
multiparty session. endpoints could successfully participate in what a user would
recognize as a multiparty video conference.
+-A---------+ +-MIXER----------------------+ +-A---------+ +-MIXER----------------------+
| +-RTP1----| |-RTP1------+ +-----+ | | +-RTP1----| |-RTP1------+ +-----+ |
| | +-Audio-| |-Audio---+ | +---+ | | | | | +-Audio-| |-Audio---+ | +---+ | | |
| | | AA1|--------->|---------+-+-|DEC|->| | | | | | AA1|--------->|---------+-+-|DEC|->| | |
| | | |<---------|MA1 <----+ | +---+ | | | | | | |<---------|MA1 <----+ | +---+ | | |
| | | | |(BA1+CA1)|\| +---+ | | | | | | | |(BA1+CA1)|\| +---+ | | |
| | +-------| |---------+ +-|ENC|<-| B+C | | | | +-------| |---------+ +-|ENC|<-| B+C | |
| +---------| |-----------+ +---+ | | | | +---------| |-----------+ +---+ | | |
+-----------+ | | | | +-----------+ | | | |
skipping to change at page 20, line 39 skipping to change at page 23, line 26
+-C---------+ | | I | | +-C---------+ | | I | |
| +-RTP3----| |-RTP3------+ | X | | | +-RTP3----| |-RTP3------+ | X | |
| | +-Audio-| |-Audio---+ | +---+ | E | | | | +-Audio-| |-Audio---+ | +---+ | E | |
| | | CA1|--------->|---------+-+-|DEC|->| R | | | | | CA1|--------->|---------+-+-|DEC|->| R | |
| | | |<---------|MA3 <----+ | +---+ | | | | | | |<---------|MA3 <----+ | +---+ | | |
| | +-------| |(BA1+CA1)|\| +---+ | | | | | +-------| |(BA1+CA1)|\| +---+ | | |
| +---------| |---------+ +-|ENC|<-| A+B | | | +---------| |---------+ +-|ENC|<-| A+B | |
+-----------+ |-----------+ +---+ +-----+ | +-----------+ |-----------+ +---+ +-----+ |
+----------------------------+ +----------------------------+
Figure 13: Session and SSRC details for Media Mixer Figure 15: Session and SSRC details for Media Mixer
From an RTP perspective media mixing can be very straightforward as From an RTP perspective media mixing can be a very simple process, as
can be seen in Figure 13. The mixer presents one SSRC towards the can be seen in Figure 15. The mixer presents one SSRC towards the
receiving client, e.g. MA1 to Peer A; the associated stream of which receiving client, e.g., MA1 to Peer A, where the associated stream is
is the media mix of the other participants. As, in this example, the media mix of the other participants. As each peer, in this
each peer receives a different version produced by the mixer, there example, receives a different version of a mix from the mixer, there
is no actual relation between the different RTP sessions in the is no actual relation between the different RTP sessions in terms of
actual media or the transport level information. There are, however, actual media or transport level information. There are, however,
common relationships between RTP1-RTP3 namely SSRC space and identity common relationships between RTP1-RTP3, namely SSRC space and
information. When A receives the MA1 stream which is a combination identity information. When A receives the MA1 stream which is a
of BA1 and CA1 streams, the mixer may include CSRC information in the combination of BA1 and CA1 streams, the mixer may include CSRC
MA1 stream to identify the contributing source BA1 and CA1, allowing information in the MA1 stream to identify the contributing source BA1
the receiver to identify the contributing sources even if this were and CA1, allowing the receiver to identify the contributing sources
not possible through the media itself or other signaling means. even if this were not possible through the media itself or through
other signaling means.
The CSRC has, in turn, utility in RTP extensions, like the Mixer to The CSRC has, in turn, utility in RTP extensions, like the Mixer to
Client audio levels RTP header extension [RFC6465]. If the SSRC from Client audio levels RTP header extension [RFC6465]. If the SSRCs
endpoint to mixer leg are used as CSRC in another RTP session, then from the endpoint to mixer paths are used as CSRCs in another RTP
RTP1, RTP2 and RTP3 become one joint session as they have a common session, then RTP1, RTP2 and RTP3 become one joint session as they
SSRC space. At this stage, the mixer also need to consider which have a common SSRC space. At this stage, the mixer also needs to
RTCP information it needs to expose in the different legs. In the consider which RTCP information it needs to expose in the different
above scenario, commonly, a mixer would expose nothing more than the paths. In the above scenario, a mixer would normally expose nothing
Source Description (SDES) information and RTCP BYE for CSRC leaving more than the Source Description (SDES) information and RTCP BYE for
the session. The main goal would be to enable the correct binding a CSRC leaving the session. The main goal would be to enable the
against the application logic and other information sources. This correct binding against the application logic and other information
also enables loop detection in the RTP session. sources. This also enables loop detection in the RTP session.
3.6.2. Media Switching 3.6.2. Media Switching
Media switching mixers are commonly used in such limited Media switching mixers are used from limited functionality scenarios
functionality scenarios where no, or only very limited, concurrent where no, or only very limited, concurrent presentation of multiple
presentation of multiple sources is required by the application. An sources is required by the application to more complex multi-stream
RTP Mixer based on media switching avoids the media decoding and usages with receiver mixing or tiling, including combined with
encoding cycle in the mixer, as it conceptually forwards the encoded simulcast and/or scalability between source and mixer. An RTP Mixer
media stream as it was being sent to the mixer, but not the based on media switching avoids the media decoding and encoding
decryption and re-encryption cycle as it rewrites RTP headers. operations in the mixer, as it conceptually forwards the encoded
Forwarding media (in contrast to reconstructing-mixing-encoding media stream as it was being sent to the mixer. It does not avoid,
media) reduces the amount of computational resources needed in the however, the decryption and re-encryption cycle as it rewrites RTP
mixer and increases the media quality (both in terms of fidelity and headers. Forwarding media (in contrast to reconstructing-mixing-
reduced latency) per transmitted bit. encoding media) reduces the amount of computational resources needed
in the mixer and increases the media quality (both in terms of
fidelity and reduced latency).
A media switching mixer maintains a pool of SSRCs representing A media switching mixer maintains a pool of SSRCs representing
conceptual or functional streams the mixer can produce. These conceptual or functional streams that the mixer can produce. These
streams are created by selecting media from one of RTP media streams streams are created by selecting media from one of the RTP media
received by the mixer and forwarded to the peer using the mixer's own streams received by the mixer and forwarded to the peer using the
SSRCs. The mixer can switch between available sources if that is mixer's own SSRCs. The mixer can switch between available sources if
required by the concept for the source, like currently active that is required by the concept for the source, like the currently
speaker. Note that the mixer, in most cases, still need to perform a active speaker. Note that the mixer, in most cases, still needs to
certain amount of media processing, as many media formats do not perform a certain amount of media processing, as many media formats
allow to "tune" into the stream at arbitrary points of their do not allow to "tune into" the stream at arbitrary points of their
bitstream. bitstream.
To achieve a coherent RTP media stream from the mixer's SSRC, the To achieve a coherent RTP media stream from the mixer's SSRC, the
mixer needs to rewrite the incoming RTP packet's header. First the mixer needs to rewrite the incoming RTP packet's header. First the
SSRC field must be set to the value of the Mixer's SSRC. Secondly, SSRC field must be set to the value of the Mixer's SSRC. Second, the
the sequence number must be the next in the sequence of outgoing sequence number must be the next in the sequence of outgoing packets
packets it sent. Thirdly the RTP timestamp value needs to be it sent. Third, the RTP timestamp value needs to be adjusted using
adjusted using an offset that changes each time one switch media an offset that changes each time one switches media source. Finally,
source. Finally depending on the negotiation the RTP payload type depending on the negotiation of the RTP payload type, the value
value representing this particular RTP payload configuration may have representing this particular RTP payload configuration may have to be
to be changed if the different endpoint mixer legs have not arrived changed if the different endpoint mixer paths have not arrived on the
on the same numbering for a given configuration. This also requires same numbering for a given configuration. This also requires that
that the different end-points do support a common set of codecs, the different endpoints support a common set of codecs, otherwise
otherwise media transcoding for codec compatibility is still media transcoding for codec compatibility would still be required.
required.
Lets consider the operation of media switching mixer that supports a We now consider the operation of a media switching mixer that
video conference with six participants (A-F) where the two latest supports a video conference with six participants (A-F) where the two
speakers in the conference are shown to each participants. Thus the most recent speakers in the conference are shown to each participant.
mixer has two SSRCs sending video to each peer, and each peer is The mixer has thus two SSRCs sending video to each peer, and each
capable of locally handling two video streams simultaneously. peer is capable of locally handling two video streams simultaneously.
+-A---------+ +-MIXER----------------------+ +-A---------+ +-MIXER----------------------+
| +-RTP1----| |-RTP1------+ +-----+ | | +-RTP1----| |-RTP1------+ +-----+ |
| | +-Video-| |-Video---+ | | | | | | +-Video-| |-Video---+ | | | |
| | | AV1|------------>|---------+-+------->| S | | | | | AV1|------------>|---------+-+------->| S | |
| | | |<------------|MV1 <----+-+-BV1----| W | | | | | |<------------|MV1 <----+-+-BV1----| W | |
| | | |<------------|MV2 <----+-+-EV1----| I | | | | | |<------------|MV2 <----+-+-EV1----| I | |
| | +-------| |---------+ | | T | | | | +-------| |---------+ | | T | |
| +---------| |-----------+ | C | | | +---------| |-----------+ | C | |
+-----------+ | | H | | +-----------+ | | H | |
skipping to change at page 22, line 47 skipping to change at page 25, line 36
+-F---------+ | | | | +-F---------+ | | | |
| +-RTP6----| |-RTP6------+ | | | | +-RTP6----| |-RTP6------+ | | |
| | +-Video-| |-Video---+ | | | | | | +-Video-| |-Video---+ | | | |
| | | CV1|------------>|---------+-+------->| | | | | | CV1|------------>|---------+-+------->| | |
| | | |<------------|MV11 <---+-+-AV1----| | | | | | |<------------|MV11 <---+-+-AV1----| | |
| | | |<------------|MV12 <---+-+-EV1----| | | | | | |<------------|MV12 <---+-+-EV1----| | |
| | +-------| |---------+ | | | | | | +-------| |---------+ | | | |
| +---------| |-----------+ +-----+ | | +---------| |-----------+ +-----+ |
+-----------+ +----------------------------+ +-----------+ +----------------------------+
Figure 14: Media Switching RTP Mixer Figure 16: Media Switching RTP Mixer
The Media Switching RTP mixer can, similarly to the Media Mixing The Media Switching RTP mixer can, similarly to the Media Mixing
Mixer, reduce the bit-rate required for media transmission towards Mixer, reduce the bit-rate required for media transmission towards
the different peers by selecting and forwarding only a sub-set of RTP the different peers by selecting and forwarding only a sub-set of RTP
media streams it receives from the conference participants. In many media streams it receives from the conference participants. In cases
practical cases, the link capacities of either direction between the mixer receives simulcast transmissions or a scalable encoding of
peers and mixer are the same, which effectively limits the subset to the media source, the mixer has more degrees of freedom to select
a single media stream. streams or sub-sets of stream to forward to a receiver, both based on
transport or client restrictions as well as application logic.
To ensure that a media receiver can correctly decode the RTP media To ensure that a media receiver can correctly decode the RTP media
stream after a switch, a state saving (frame-based) codec needs to stream after a switch, a codec that uses temporal prediction needs to
start its decoding from independent refresh points or similar points start its decoding from independent refresh points, or similar points
in the bitstream. For some codecs, for example frame based speech in the bitstream. For some codecs, for example frame based speech
and audio codecs, this is easily achieved by starting the decoding at and audio codecs, this is easily achieved by starting the decoding at
RTP packet boundaries (proper packetization on the encoder side RTP packet boundaries, as each packet boundary provides a refresh
assumed), as each packet boundary provides a refresh point. For point (assuming proper packetization on the encoder side). For other
other (mostly video-) codecs, refresh points are less common in the codecs, particularly in video, refresh points are less common in the
bitstream or may not be present at all without an explicit request to bitstream or may not be present at all without an explicit request to
the respective encoder. For this purpose there exists the Full Intra the respective encoder. The Full Intra Request [RFC5104] RTCP codec
Request [RFC5104] RTCP codec control message. control message has been defined for this purpose.
Also in this type of mixer one could consider to terminate the RTP In this type of mixer one could consider to fully terminate the RTP
sessions fully between the different end-point and mixer legs. The sessions between the different endpoint and mixer paths. The same
same arguments and considerations as discussed in Section 3.9 need to arguments and considerations as discussed in Section 3.9 need to be
be taken into consideration and apply here. taken into consideration and apply here.
3.7. Source Projecting Middlebox 3.7. Selective Forwarding Middlebox
Another method for handling media in the RTP mixer is to project all Another method for handling media in the RTP mixer is to "project",
potential RTP sources (SSRCs) into a per end-point independent RTP or make available, all potential RTP sources (SSRCs) into a per-
session. The middlebox can select which of the potential sources endpoint, independent RTP session. The middlebox can select which of
that are currently actively transmitting media, despite that the the potential sources that are currently actively transmitting media
middlebox, in another RTP session, may receive media from that end- will be sent to each of the endpoints. This is similar to the media
point. This is similar to the media switching Mixer but has some switching Mixer but has some important differences in RTP details.
important differences in RTP details.
+-A---------+ +-Middlebox-----------------+ +-A---------+ +-Middlebox-----------------+
| +-RTP1----| |-RTP1------+ +-----+ | | +-RTP1----| |-RTP1------+ +-----+ |
| | +-Video-| |-Video---+ | | | | | | +-Video-| |-Video---+ | | | |
| | | AV1|------------>|---------+-+------>| | | | | | AV1|------------>|---------+-+------>| | |
| | | |<------------|BV1 <----+-+-------| S | | | | | |<------------|BV1 <----+-+-------| S | |
| | | |<------------|CV1 <----+-+-------| W | | | | | |<------------|CV1 <----+-+-------| W | |
| | | |<------------|DV1 <----+-+-------| I | | | | | |<------------|DV1 <----+-+-------| I | |
| | | |<------------|EV1 <----+-+-------| T | | | | | |<------------|EV1 <----+-+-------| T | |
| | | |<------------|FV1 <----+-+-------| C | | | | | |<------------|FV1 <----+-+-------| C | |
skipping to change at page 24, line 26 skipping to change at page 27, line 14
| +-RTP6----| |-RTP6------+ | | | | +-RTP6----| |-RTP6------+ | | |
| | +-Video-| |-Video---+ | | | | | | +-Video-| |-Video---+ | | | |
| | | FV1|------------>|---------+-+------>| | | | | | FV1|------------>|---------+-+------>| | |
| | | |<------------|AV1 <----+-+-------| | | | | | |<------------|AV1 <----+-+-------| | |
| | | | : : : |: : : : : : : : :| | | | | | | : : : |: : : : : : : : :| | |
| | | |<------------|EV1 <----+-+-------| | | | | | |<------------|EV1 <----+-+-------| | |
| | +-------| |---------+ | | | | | | +-------| |---------+ | | | |
| +---------| |-----------+ +-----+ | | +---------| |-----------+ +-----+ |
+-----------+ +---------------------------+ +-----------+ +---------------------------+
Figure 15: Media Projecting Middlebox Figure 17: Selective Forwarding Middlebox
In the six participant conference depicted above in (Figure 15) one In the six participant conference depicted above in (Figure 17) one
can see that end-point A is aware of five incoming SSRCs, BV1-FV1. can see that end-point A is aware of five incoming SSRCs, BV1-FV1.
If this middlebox intends to have a similar behaviour as in If this middlebox intends to have a similar behavior as in
Section 3.6.2 where the mixer provides the end-points with the two Section 3.6.2 where the mixer provides the end-points with the two
latest speaking end-points, then only two out of these five SSRCs latest speaking end-points, then only two out of these five SSRCs
need concurrently transmit media to A. As the middlebox selects the need concurrently transmit media to A. As the middlebox selects the
source in the different RTP sessions that transmit media to the end- source in the different RTP sessions that transmit media to the end-
points, each RTP media stream requires some rewriting of RTP header points, each RTP media stream requires some rewriting of RTP header
fields when being projected from one session into another. In fields when being projected from one session into another. In
particular, the sequence number needs to be consecutively incremented particular, the sequence number needs to be consecutively incremented
based on the packet actually being transmitted in each RTP session. based on the packet actually being transmitted in each RTP session.
Therefore, the RTP sequence number offset will change each time a Therefore, the RTP sequence number offset will change each time a
source is turned on in a RTP session. The timestamp (possibly source is turned on in a RTP session. The timestamp (possibly
offset) stays the same. offset) stays the same.
As the RTP sessions are independent, the SSRC numbers used can also As the RTP sessions are independent, the SSRC numbers used can also
skipping to change at page 25, line 26 skipping to change at page 28, line 14
session that contains the media source. Both end-points and the session that contains the media source. Both end-points and the
middlebox need to implement conference related codec control middlebox need to implement conference related codec control
functionalities to provide a good experience. Commonly used are Full functionalities to provide a good experience. Commonly used are Full
Intra Request to request from the media source to provide switching Intra Request to request from the media source to provide switching
points between the sources, and Temporary Maximum Media Bit-rate points between the sources, and Temporary Maximum Media Bit-rate
Request (TMMBR) to enable the middlebox to aggregate congestion Request (TMMBR) to enable the middlebox to aggregate congestion
control responses towards the media source so to enable it to adjust control responses towards the media source so to enable it to adjust
its bit-rate (obviously only in case the limitation is not in the its bit-rate (obviously only in case the limitation is not in the
source to middlebox link). source to middlebox link).
The selective forwarding middlebox has been introduced in recently
developed videoconferencing systems in conjunction with, and to
capitalize on, scalable video coding as well as simulcasting. An
example of scalable video coding is Annex G of H.264, but other
codecs, including H.264 AVC and VP8 also exhibit scalability, albeit
only in the temporal dimension. In both scalable coding and
simulcast cases the video signal is represented by a set of two or
more bitstreams, providing a corresponding number of distinct
fidelity points. The middlebox selects which parts of a scalable
bitstream (or which bitstream, in the case of simulcasting) to
forward to each of the receiving endpoints. The decision may be
driven by a number of factors, such as available bit rate, desired
layout, etc. Contrary to transcoding MCUs, these "Selective
Forwarding Units" (SFUs) have extremely low delay, and provide
features that are typically associated with high-end systems
(personalized layout, error localization) without any signal
processing at the middlebox. They are also capable of scaling to a
large number of concurrent users, and--due to their very low delay--
can also be cascaded.
This version of the middlebox also puts different requirements on the This version of the middlebox also puts different requirements on the
end-point when it comes to decoder instances and handling of the RTP endpoint when it comes to decoder instances and handling of the RTP
media streams providing media. As each projected SSRC can, at any media streams providing media. As each projected SSRC can, at any
time, provide media, the end-point either needs to be able to handle time, provide media, the endpoint either needs to be able to handle
as many decoder instances as the middlebox received, or have as many decoder instances as the middlebox received, or have
efficient switching of decoder contexts in a more limited set of efficient switching of decoder contexts in a more limited set of
actual decoder instances to cope with the switches. The application actual decoder instances to cope with the switches. The application
also gets more responsibility to update how the media provided is to also gets more responsibility to update how the media provided is to
be presented to the user. be presented to the user.
Note, this could potentially be seen as a media translator which Note that this topology could potentially be seen as a media
include an on/off logic as part of its media translation. The main translator which include an on/off logic as part of its media
difference would be a common global SSRC space in the case of the translation. The main difference would be a common global SSRC space
Media Translator and the mapped one used in the above. It also has in the case of the Media Translator and the mapped one used in the
mixer aspects, as the streams it provides are not basically above. It also has mixer aspects, as the streams it provides are not
translated version, but instead they have conceptual property basically translated version, but instead they have conceptual
assigned to them. Thus this topology appears to be some hybrid property assigned to them. Thus this topology appears to be some
between the translator and mixer model. hybrid between the translator and mixer model.
The differences between selective forwarding middlebox and a
switching mixer (Section 3.6.2) are minor, and they share most
properties. The above requirement on having a large number of
decoding instances or requiring efficient switching of decoder
contexts, are one point of difference. The other is how the
identification is performed, where the Mixer uses CSRC to provide
info what is included in a particular RTP packet stream that
represent a particular concept. Selective forwarding gets the source
information through the SSRC, and instead have to use other mechanism
to make clear the streams current purpose.
3.8. Point to Multipoint Using Video Switching MCUs 3.8. Point to Multipoint Using Video Switching MCUs
Shortcut name: Topo-Video-switch-MCU Shortcut name: Topo-Video-switch-MCU
+---+ +------------+ +---+ +---+ +------------+ +---+
| A |------| Multipoint |------| B | | A |------| Multipoint |------| B |
+---+ | Control | +---+ +---+ | Control | +---+
| Unit | | Unit |
+---+ | (MCU) | +---+ +---+ | (MCU) | +---+
| C |------| |------| D | | C |------| |------| D |
+---+ +------------+ +---+ +---+ +------------+ +---+
Figure 16: Point to Multipoint Using a Video Switching MCU Figure 18: Point to Multipoint Using a Video Switching MCU
This PtM topology was common before, although the RTCP-terminating This PtM topology was popular in early implementations of multipoint
MCUs, as discussed in the next section, where perhaps even more videoconferencing systems due to its simplicity, and the
common. This topology, as well as the following one, was a result of corresponding middlebox design has been known as a "video switching
lack of wide availability of IP multicast technologies, as well as MCU". The more complex RTCP-terminating MCUs, discussed in the next
the simplicity of content switching when compared to content mixing. section, became the norm, however, when technology allowed
The technology is commonly implemented in what is known as "Video implementations at acceptable costs.
Switching MCUs".
A video switching MCU forwards to a participant a single media A video switching MCU forwards to a participant a single media
stream, selected from the available streams. The criteria for stream, selected from the available streams. The criteria for
selection are often based on voice activity in the audio-visual selection are often based on voice activity in the audio-visual
conference, but other conference management mechanisms (like conference, but other conference management mechanisms (like
presentation mode or explicit floor control) are known to exist as presentation mode or explicit floor control) are known to exist as
well. well.
The video switching MCU may also perform media translation to modify The video switching MCU may also perform media translation to modify
the content in bit-rate, encoding, or resolution. However, it still the content in bit-rate, encoding, or resolution. However, it still
skipping to change at page 27, line 22 skipping to change at page 30, line 44
Shortcut name: Topo-RTCP-terminating-MCU Shortcut name: Topo-RTCP-terminating-MCU
+---+ +------------+ +---+ +---+ +------------+ +---+
| A |<---->| Multipoint |<---->| B | | A |<---->| Multipoint |<---->| B |
+---+ | Control | +---+ +---+ | Control | +---+
| Unit | | Unit |
+---+ | (MCU) | +---+ +---+ | (MCU) | +---+
| C |<---->| |<---->| D | | C |<---->| |<---->| D |
+---+ +------------+ +---+ +---+ +------------+ +---+
Figure 17: Point to Multipoint Using Content Modifying MCUs Figure 19: Point to Multipoint Using Content Modifying MCUs
In this PtM scenario, each participant runs an RTP point-to-point In this PtM scenario, each participant runs an RTP point-to-point
session between itself and the MCU. This is a very commonly deployed session between itself and the MCU. This is a very commonly deployed
topology in multipoint video conferencing. The content that the MCU topology in multipoint video conferencing. The content that the MCU
provides to each participant is either: provides to each participant is either:
a. a selection of the content received from the other participants, a. a selection of the content received from the other participants,
or or
b. the mixed aggregate of what the MCU receives from the other PtP b. the mixed aggregate of what the MCU receives from the other PtP
paths, which are part of the same conference session. paths, which are part of the same conference session.
In case a), the MCU may modify the content in bit-rate, encoding, or In case (a), the MCU may modify the content in terms of bit-rate,
resolution. No explicit RTP mechanism is used to establish the encoding format, or resolution. No explicit RTP mechanism is used to
relationship between the original media sender and the version the establish the relationship between the original media sender and the
MCU sends. In other words, the outgoing sessions typically use a version the MCU sends. In other words, the outgoing sessions
different SSRC, and may well use a different payload type (PT), even typically use a different SSRC, and may well use a different payload
if this different PT happens to be mapped to the same media type. type (PT), even if this different PT happens to be mapped to the same
This is a result of the individually negotiated session for each media type. This is a result of the individually negotiated session
participant. for each participant.
In case b), the MCU is the content source as it mixes the content and In case (b), the MCU is the content source as it mixes the content
then encodes it for transmission to a participant. According to RTP and then encodes it for transmission to a participant. According to
[RFC3550], the SSRC of the contributors are to be signalled using the RTP [RFC3550], the SSRC of the contributors are to be signalled using
CSRC/CC mechanism. In practice, today, most deployed MCUs do not the CSRC/CC mechanism. In practice, today, most deployed MCUs do not
implement this feature. Instead, the identification of the implement this feature. Instead, the identification of the
participants whose content is included in the Mixer's output is not participants whose content is included in the Mixer's output is not
indicated through any explicit RTP mechanism. That is, most deployed indicated through any explicit RTP mechanism. That is, most deployed
MCUs set the CSRC Count (CC) field in the RTP header to zero, thereby MCUs set the CSRC Count (CC) field in the RTP header to zero, thereby
indicating no available CSRC information, even if they could identify indicating no available CSRC information, even if they could identify
the content sources as suggested in RTP. the content sources as suggested in RTP.
The main feature that sets this topology apart from what RFC 3550 The main feature that sets this topology apart from what RFC 3550
describes is the breaking of the common RTP session across the describes is the breaking of the common RTP session across the
centralized device, such as the MCU. This results in the loss of centralized device, such as the MCU. This results in the loss of
skipping to change at page 28, line 32 skipping to change at page 32, line 5
currently active senders in a machine-usable way, thus preventing currently active senders in a machine-usable way, thus preventing
clients from indicating currently active speakers in user clients from indicating currently active speakers in user
interfaces, etc. interfaces, etc.
Note that deployed MCUs (and endpoints) rely on signalling layer Note that deployed MCUs (and endpoints) rely on signalling layer
mechanisms for the identification of the contributing sources, for mechanisms for the identification of the contributing sources, for
example, a SIP conferencing package [RFC4575]. This alleviates, to example, a SIP conferencing package [RFC4575]. This alleviates, to
some extent, the aforementioned issues resulting from ignoring RTP's some extent, the aforementioned issues resulting from ignoring RTP's
CSRC mechanism. CSRC mechanism.
As a result of the shortcomings of this topology, it is recommended 3.10. Split Component Endpoint
to instead implement the Mixer concept as specified by RFC 3550.
3.10. De-composite Endpoint Shortcut name: Topo-Split-Endpoint
The implementation of an application may desire to send a subset of The implementation of an application may desire to send a subset of
the application's data to each of multiple devices, each with its own the application's data to each of multiple devices, each with its own
network address. A very basic use case for this would be to separate network address. A very basic use case for this would be to separate
audio and video processing for a particular endpoint, like a audio and video processing for a particular endpoint into different
conference room, into one device handling the audio and another components. For example, in a video conference room system the
handling the video, being interconnected by some control functions endpoint could be considered as being composed of one device handling
allowing them to behave as a single endpoint in all aspects except the audio and another handling the video, interconnected by some
for transport Figure 18. control functions allowing them to behave as a single endpoint in all
aspects except for transport as depicted in Figure 20.
Which decomposition scheme is possible is highly dependent on the RTP Which decomposition scheme is possible is highly dependent on the RTP
session usage. It is not really feasible to decompose one logical session usage. It is not really feasible to decompose one logical
end-point into two different transport nodes in one RTP session. A end-point into two different transport nodes in one RTP session. A
third party monitor would report such an attempt as two entities third party monitor would report such an attempt as two entities
being two different end-points with a CNAME collision. As a result, being two different end-points with a CNAME collision. As a result,
a fully RTP conformant de-composited endpoint is one where the a fully RTP conformant de-composited endpoint is one where the
different decomposed parts use separate RTP sessions to send and/or different decomposed parts use separate RTP sessions to send and/or
receive media streams intended for them. receive media streams intended for them.
skipping to change at page 29, line 21 skipping to change at page 32, line 42
| +->| Audio |<+-RTP---\ | +->| Audio |<+-RTP---\
| | +------------+ | \ +------+ | | +------------+ | \ +------+
| | +------------+ | +-->| | | | +------------+ | +-->| |
| +->| Video |<+-RTP-------->| B | | +->| Video |<+-RTP-------->| B |
| | +------------+ | +-->| | | | +------------+ | +-->| |
| | +------------+ | / +------+ | | +------------+ | / +------+
| +->| Control |<+-SIP---/ | +->| Control |<+-SIP---/
| +------------+ | | +------------+ |
+---------------------+ +---------------------+
Figure 18: De-composite End-Point Figure 20: Split Component Endpoint
In the above usage, let us assume that the different RTP sessions are In the above usage, let us assume that the different RTP sessions are
used for audio and video. The audio and video parts, however, use a used for audio and video. The audio and video parts, however, use a
common CNAME and also have a common clock to ensure that common CNAME and also have a common clock to ensure that
synchronization and clock drift handling works, despite the synchronization and clock drift handling works, despite the fact that
decomposition. Also, the RTCP handling works correctly as long as the components are separated. Also, RTCP handling works correctly as
only one part of the de-composite is part of each RTP session. That long as only one part of the split endpoint is part of each RTP
way any differences in the path between A's audio entity and B and session. That way any differences in the path between A's audio
A's video and B are related to different SSRCs in different RTP entity and B and A's video and B are related to different SSRCs in
sessions. different RTP sessions.
The requirement that can be derived from the above usage is that the The requirement that can be derived from the above usage is that the
transport flows for each RTP session might be under common control, transport flows for each RTP session might be under common control,
but still are addressed to what looks like different endpoints (based but still are addressed to what looks like different endpoints (based
on addresses and ports). This geometry cannot be accomplished using on addresses and ports). This connection diagram cannot be
one RTP session, so in this case, multiple RTP sessions are needed. accomplished using one RTP session and thus multiple RTP sessions are
needed.
3.11. Non-Symmetric Mixer/Translators 3.11. Non-Symmetric Mixer/Translators
Shortcut name: Topo-Asymmetric Shortcut name: Topo-Asymmetric
It is theoretically possible to construct an MCU that is a Mixer in It is theoretically possible to construct an MCU that is a Mixer in
one direction and a Translator in another. The main reason to one direction and a Translator in another. The main reason to
consider this would be to allow topologies similar to Figure 11, consider this would be to allow topologies similar to Figure 13,
where the Mixer does not need to mix in the direction from B or D where the Mixer does not need to mix in the direction from B or D
towards the multicast domains with A and C. Instead, the media towards the multicast domains with A and C. Instead, the media
streams from B and D are forwarded without changes. Avoiding this streams from B and D are forwarded without changes. Avoiding this
mixing would save media processing resources that perform the mixing mixing would save media processing resources that perform the mixing
in cases where it isn't needed. However, there would still be a need in cases where it isn't needed. However, there would still be a need
to mix B's stream towards D. Only in the direction B -> multicast to mix B's stream towards D. Only in the direction B -> multicast
domain or D -> multicast domain would it be possible to work as a domain or D -> multicast domain would it be possible to work as a
Translator. In all other directions, it would function as a Mixer. Translator. In all other directions, it would function as a Mixer.
The Mixer/Translator would still need to process and change the RTCP The Mixer/Translator would still need to process and change the RTCP
before forwarding it in the directions of B or D to the multicast before forwarding it in the directions of B or D to the multicast
domain. One issue is that A and C do not know about the mixed-media domain. One issue is that A and C do not know about the mixed-media
stream the Mixer sends to either B or D. Therefore, any reports stream the Mixer sends to either B or D. Therefore, any reports
related to these streams must be removed. Also, receiver reports related to these streams must be removed. Also, receiver reports
related to A and C's media stream would be missing. To avoid A and C related to A and C's media stream would be missing. To avoid A and C
thinking that B and D aren't receiving A and C at all, the Mixer thinking that B and D aren't receiving A and C at all, the Mixer
needs to insert locally generated reports reflecting the situation needs to insert locally generated reports reflecting the situation
for the streams from A and C into B and D's Sender Reports. In the for the streams from A and C into B and D's Sender Reports. In the
opposite direction, the Receiver Reports from A and C about B's and opposite direction, the Receiver Reports from A and C about B's and
D's stream also need to be aggregated into the Mixer's Receiver D's stream also need to be aggregated into the Mixer's Receiver
Reports sent to B and D. Since B and D only have the Mixer as source Reports sent to B and D. Since B and D only have the Mixer as source
for the stream, all RTCP from A and C must be suppressed by the for the stream, all RTCP from A and C must be suppressed by the
Mixer. Mixer.
This topology is so problematic and it is so easy to get the RTCP This topology is so problematic and it is so easy to get the RTCP
processing wrong, that it is not recommended to implement this processing wrong, that it is not recommended for implementation.
topology.
3.12. Combining Topologies 3.12. Combining Topologies
Topologies can be combined and linked to each other using Mixers or Topologies can be combined and linked to each other using Mixers or
Translators. However, care must be taken in handling the SSRC/CSRC Translators. However, care must be taken in handling the SSRC/CSRC
space. A Mixer does not forward RTCP from sources in other domains, space. A Mixer does not forward RTCP from sources in other domains,
but instead generates its own RTCP packets for each domain it mixes but instead generates its own RTCP packets for each domain it mixes
into, including the necessary Source Description (SDES) information into, including the necessary Source Description (SDES) information
for both the CSRCs and the SSRCs. Thus, in a mixed domain, the only for both the CSRCs and the SSRCs. Thus, in a mixed domain, the only
SSRCs seen will be the ones present in the domain, while there can be SSRCs seen will be the ones present in the domain, while there can be
CSRCs from all the domains connected together with a combination of CSRCs from all the domains connected together with a combination of
Mixers and Translators. The combined SSRC and CSRC space is common Mixers and Translators. The combined SSRC and CSRC space is common
over any Translator or Mixer. This is important to facilitate loop over any Translator or Mixer. It is important to facilitate loop
detection, something that is likely to be even more important in detection, something that is likely to be even more important in
combined topologies due to the mixed behavior between the domains. combined topologies due to the mixed behavior between the domains.
Any hybrid, like the Topo-Video-switch-MCU or Topo-Asymmetric, Any hybrid, like the Topo-Video-switch-MCU or Topo-Asymmetric,
requires considerable thought on how RTCP is dealt with. requires considerable thought on how RTCP is dealt with.
4. Comparing Topologies 4. Comparing Topologies
The topologies discussed in Section 3 have different properties. The topologies discussed in Section 3 have different properties.
This section first lists these properties and maps the different This section first describes these properties and then analyzes how
topologies to them. Please note that even if a certain property is these properties are supported by the different topologies. Note
supported within a particular topology concept, the necessary that, even if a certain property is supported within a particular
functionality may, in many cases, be optional to implement. topology concept, the necessary functionality may be optional to
implement.
Note: This section has not yet been updated with the new additions of Note: This section has not yet been updated with the new additions of
topologies. topologies.
4.1. Topology Properties 4.1. Topology Properties
4.1.1. All to All Media Transmission 4.1.1. All to All Media Transmission
Multicast, at least Any Source Multicast (ASM), provides the Multicast, at least Any Source Multicast (ASM), provides the
functionality that everyone may send to, or receive from, everyone functionality that everyone may send to, or receive from, everyone
else within the session. MCUs, Mixers, and Translators may all else within the session. Mesh, MCUs, Mixers, and Translators may all
provide that functionality at least on some basic level. However, provide that functionality at least on some basic level. However,
there are some differences in which type of reachability they there are some differences in which type of reachability they
provide. provide.
The transport Translator function called "relay", in Section 3.5, is The transport Translator function called "relay", in Section 3.5, as
the one that provides the emulation of ASM that is closest to true well as the Mesh is the ones that provides the emulation of ASM that
IP-multicast-based, all to all transmission. Media Translators, is closest to true IP-multicast-based, all to all transmission.
Mixers, and the MCU variants do not provide a fully meshed forwarding Media Translators, Mixers, and the MCU variants do not provide a
on the transport level; instead, they only allow limited forwarding fully meshed forwarding on the transport level; instead, they only
of content from the other session participants. allow limited forwarding of content from the other session
participants.
The "all to all media transmission" requires that any media The "all to all media transmission" requires that any media
transmitting entity considers the path to the least capable receiver. transmitting entity considers the path to the least capable receiver.
Otherwise, the media transmissions may overload that path. Otherwise, the media transmissions may overload that path.
Therefore, a media sender needs to monitor the path from itself to Therefore, a media sender needs to monitor the path from itself to
any of the participants, to detect the currently least capable any of the participants, to detect the currently least capable
receiver, and adapt its sending rate accordingly. As multiple receiver, and adapt its sending rate accordingly. As multiple
participants may send simultaneously, the available resources may participants may send simultaneously, the available resources may
vary. RTCP's Receiver Reports help performing this monitoring, at vary. RTCP's Receiver Reports help performing this monitoring, at
least on a medium time scale. least on a medium time scale.
The resource consumption for performing all to all transmission
varies, where the benefit of ASM is that only one copy of each packet
traverse a particular link. Using a relay, causes one copy per
client to relay path and packet transmitted, however, in most cases
the links with the multiple copies will be the ones close to the
relay, rather than the clients unless they share LAN segment. The
Mesh causes N-1 copies of of each transmitted packet to traverse the
first hop link from the client, in a N client mesh. How long the
different paths are common, is highly situation dependent.
The transmission of RTCP automatically adapts to any changes in the The transmission of RTCP automatically adapts to any changes in the
number of participants due to the transmission algorithm, defined in number of participants due to the transmission algorithm, defined in
the RTP specification [RFC3550], and the extensions in AVPF [RFC4585] the RTP specification [RFC3550], and the extensions in AVPF [RFC4585]
(when applicable). That way, the resources utilized for RTCP stay (when applicable). That way, the resources utilized for RTCP stay
within the bounds configured for the session. within the bounds configured for the session.
4.1.2. Transport or Media Interoperability 4.1.2. Transport or Media Interoperability
Translators, Mixers, and RTCP-terminating MCU all allow changing the Translators, Mixers, and RTCP-terminating MCU, and Mesh with
media encoding or the transport to other properties of the other individual RTP sessions, all allow changing the media encoding or the
domain, thereby providing extended interoperability in cases where transport to other properties of the other domain, thereby providing
the participants lack a common set of media codecs and/or transport extended interoperability in cases where the participants lack a
protocols. common set of media codecs and/or transport protocols.
4.1.3. Per Domain Bit-Rate Adaptation 4.1.3. Per Domain Bit-Rate Adaptation
Participants are most likely to be connected to each other with a Participants are most likely to be connected to each other with a
heterogeneous set of paths. This makes congestion control in a Point heterogeneous set of paths. This makes congestion control in a Point
to Multipoint set problematic. For the ASM and "relay" scenario, to Multipoint set problematic. For the ASM, Mesh with common RTP
each individual sender has to adapt to the receiver with the least session, and Relay scenario, each individual sender has to adapt to
capable path. This is no longer necessary when Media Translators, the receiver with the least capable path. This is no longer
Mixers, or MCUs are involved, as each participant only needs to adapt necessary when Media Translators, Mixers, or MCUs are involved, as
to the slowest path within its own domain. The Translator, Mixer, or each participant only needs to adapt to the slowest path within its
MCU topologies all require their respective outgoing streams to own domain. The Translator, Mixer, or MCU topologies all require
adjust the bit-rate, packet-rate, etc., to adapt to the least capable their respective outgoing streams to adjust the bit-rate, packet-
path in each of the other domains. That way one can avoid lowering rate, etc., to adapt to the least capable path in each of the other
the quality to the least-capable participant in all the domains at domains. That way one can avoid lowering the quality to the least-
the cost (complexity, delay, equipment) of the Mixer or Translator. capable participant in all the domains at the cost (complexity,
delay, equipment) of the Mixer or Translator.
4.1.4. Aggregation of Media 4.1.4. Aggregation of Media
In the all to all media property mentioned above and provided by ASM, In the all to all media property mentioned above and provided by ASM,
all simultaneous media transmissions share the available bit-rate. all simultaneous media transmissions share the available bit-rate.
For participants with limited reception capabilities, this may result For participants with limited reception capabilities, this may result
in a situation where even a minimal acceptable media quality cannot in a situation where even a minimal acceptable media quality cannot
be accomplished. This is the result of multiple media streams be accomplished. This is the result of multiple media streams
needing to share the available resources. The solution to this needing to share the available resources. The solution to this
problem is to provide for a Mixer or MCU to aggregate the multiple problem is to provide for a Mixer or MCU to aggregate the multiple
skipping to change at page 35, line 8 skipping to change at page 38, line 38
allow the negotiation of security parameters resulting in a different allow the negotiation of security parameters resulting in a different
strength of the security, then this system should notify the strength of the security, then this system should notify the
participants in the other domains about this. participants in the other domains about this.
Second, the number of crypto contexts (keys and security related Second, the number of crypto contexts (keys and security related
state) needed (for example, in SRTP [RFC3711]) may vary between state) needed (for example, in SRTP [RFC3711]) may vary between
Mixers and Translators. A Mixer normally needs to represent only a Mixers and Translators. A Mixer normally needs to represent only a
single SSRC per domain and therefore needs to create only one single SSRC per domain and therefore needs to create only one
security association (SRTP crypto context) per domain. In contrast, security association (SRTP crypto context) per domain. In contrast,
a Translator needs one security association per participant it a Translator needs one security association per participant it
translates towards, in the opposite domain. Considering Figure 9, translates towards, in the opposite domain. Considering Figure 11,
the Translator needs two security associations towards the multicast the Translator needs two security associations towards the multicast
domain, one for B and one for D. It may be forced to maintain a set domain, one for B and one for D. It may be forced to maintain a set
of totally independent security associations between itself and B and of totally independent security associations between itself and B and
D respectively, so as to avoid two-time pad occurrences. These D respectively, so as to avoid two-time pad occurrences. These
contexts must also be capable of handling all the sources present in contexts must also be capable of handling all the sources present in
the other domains. Hence, using completely independent security the other domains. Hence, using completely independent security
associations (for certain keying mechanisms) may force a Translator associations (for certain keying mechanisms) may force a Translator
to handle N*DM keys and related state; where N is the total number of to handle N*DM keys and related state; where N is the total number of
SSRCs used over all domains and DM is the total number of domains. SSRCs used over all domains and DM is the total number of domains.
There exist a number of different mechanisms to provide keys to the There exist a number of different mechanisms to provide keys to the
different participants. One example is the choice between group keys different participants. One example is the choice between group keys
skipping to change at page 35, line 40 skipping to change at page 39, line 22
6. IANA Considerations 6. IANA Considerations
This document makes no request of IANA. This document makes no request of IANA.
Note to RFC Editor: this section may be removed on publication as an Note to RFC Editor: this section may be removed on publication as an
RFC. RFC.
7. Acknowledgements 7. Acknowledgements
The authors would like to thank Bo Burman, Umesh Chandra, Roni Even, The authors would like to thank Bo Burman, Umesh Chandra, Roni Even,
Keith Lantz, Ladan Gharai, Geoff Hunt, and Mark Baugher for their Keith Lantz, Ladan Gharai, Geoff Hunt, Mark Baugher, and Alex
help in reviewing this document. Eleftheriadis for their help in reviewing this document.
8. References 8. References
8.1. Normative References 8.1. Normative References
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003. Applications", STD 64, RFC 3550, July 2003.
[RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
skipping to change at page 36, line 20 skipping to change at page 39, line 48
Initiation Protocol (SIP) Event Package for Conference Initiation Protocol (SIP) Event Package for Conference
State", RFC 4575, August 2006. State", RFC 4575, August 2006.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control "Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July
2006. 2006.
8.2. Informative References 8.2. Informative References
[I-D.ietf-avtcore-rtp-multi-stream]
Lennox, J., Westerlund, M., Wu, W., and C. Perkins,
"Sending Multiple Media Streams in a Single RTP Session",
draft-ietf-avtcore-rtp-multi-stream-01 (work in progress),
July 2013.
[I-D.ietf-avtcore-rtp-security-options] [I-D.ietf-avtcore-rtp-security-options]
Westerlund, M. and C. Perkins, "Options for Securing RTP Westerlund, M. and C. Perkins, "Options for Securing RTP
Sessions", draft-ietf-avtcore-rtp-security-options-02 Sessions", draft-ietf-avtcore-rtp-security-options-08
(work in progress), February 2013. (work in progress), October 2013.
[I-D.lennox-avtcore-rtp-multi-stream] [RFC1112] Deering, S., "Host extensions for IP multicasting", STD 5,
Lennox, J., Westerlund, M., Wu, W., and C. Perkins, "RTP RFC 1112, August 1989.
Considerations for Endpoints Sending Multiple Media
Streams", draft-lennox-avtcore-rtp-multi-stream-02 (work
in progress), February 2013.
[RFC3022] Srisuresh, P. and K. Egevang, "Traditional IP Network [RFC3022] Srisuresh, P. and K. Egevang, "Traditional IP Network
Address Translator (Traditional NAT)", RFC 3022, January Address Translator (Traditional NAT)", RFC 3022, January
2001. 2001.
[RFC3569] Bhattacharyya, S., "An Overview of Source-Specific
Multicast (SSM)", RFC 3569, July 2003.
[RFC4607] Holbrook, H. and B. Cain, "Source-Specific Multicast for [RFC4607] Holbrook, H. and B. Cain, "Source-Specific Multicast for
IP", RFC 4607, August 2006. IP", RFC 4607, August 2006.
[RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
"Codec Control Messages in the RTP Audio-Visual Profile "Codec Control Messages in the RTP Audio-Visual Profile
with Feedback (AVPF)", RFC 5104, February 2008. with Feedback (AVPF)", RFC 5104, February 2008.
[RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control
Protocol (RTCP) Extensions for Single-Source Multicast Protocol (RTCP) Extensions for Single-Source Multicast
Sessions with Unicast Feedback", RFC 5760, February 2010. Sessions with Unicast Feedback", RFC 5760, February 2010.
 End of changes. 141 change blocks. 
445 lines changed or deleted 621 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/