draft-ietf-cdni-logging-02.txt   draft-ietf-cdni-logging-03.txt 
Internet Engineering Task Force G. Bertrand, Ed. Internet Engineering Task Force G. Bertrand, Ed.
Internet-Draft I. Oprescu, Ed. Internet-Draft I. Oprescu, Ed.
Intended status: Informational France Telecom - Orange Intended status: Informational France Telecom - Orange
Expires: November 28, 2013 F. Le Faucheur, Ed. Expires: December 02, 2013 F. Le Faucheur, Ed.
Cisco Systems Cisco Systems
R. Peterkofsky R. Peterkofsky
Skytide, Inc. Skytide, Inc.
May 27, 2013 May 31, 2013
CDNI Logging Interface CDNI Logging Interface
draft-ietf-cdni-logging-02 draft-ietf-cdni-logging-03
Abstract Abstract
This memo specifies the Logging interface between a downstream CDN This memo specifies the Logging interface between a downstream CDN
(dCDN) and an upstream CDN (uCDN) that are interconnected as per the (dCDN) and an upstream CDN (uCDN) that are interconnected as per the
CDN Interconnection (CDNI) framework. First, it describes a CDN Interconnection (CDNI) framework. First, it describes a
reference model for CDNI logging. Then, it specifies the CDNI reference model for CDNI logging. Then, it specifies the CDNI
Logging File format and the actual protocol for exchange of CDNI Logging File format and the actual protocol for exchange of CDNI
Logging Files. Logging Files.
skipping to change at page 1, line 39 skipping to change at page 1, line 39
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 28, 2013. This Internet-Draft will expire on December 02, 2013.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 43 skipping to change at page 2, line 43
2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12
2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12
2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12
2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13
2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13
2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13
2.2.5.6. Notions common to multiple Log Consuming 2.2.5.6. Notions common to multiple Log Consuming
Applications . . . . . . . . . . . . . . . . . . 13 Applications . . . . . . . . . . . . . . . . . . 13
3. CDNI Logging File Format . . . . . . . . . . . . . . . . . . 15 3. CDNI Logging File Format . . . . . . . . . . . . . . . . . . 15
3.1. CDNI Logging File Directives . . . . . . . . . . . . . . 16 3.1. CDNI Logging File Directives . . . . . . . . . . . . . . 16
3.2. Logging Records . . . . . . . . . . . . . . . . . . . . . 19 3.2. Logging Records . . . . . . . . . . . . . . . . . . . . . 20
3.2.1. HTTP Request Logging Record . . . . . . . . . . . . . 20 3.2.1. HTTP Request Logging Record . . . . . . . . . . . . . 21
3.2.2. CDNI Logging File Example . . . . . . . . . . . . . . 26 3.2.2. CDNI Logging File Example . . . . . . . . . . . . . . 27
3.3. Fields and Directives Formats . . . . . . . . . . . . . . 27 3.3. Fields and Directives Formats . . . . . . . . . . . . . . 28
4. CDNI Logging File Exchange Protocol . . . . . . . . . . . . . 27 4. CDNI Logging File Exchange Protocol . . . . . . . . . . . . . 28
4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 28 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 29
4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 28 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 29
5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 29 5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 30
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31
7. Security Considerations . . . . . . . . . . . . . . . . . . . 31 7. Security Considerations . . . . . . . . . . . . . . . . . . . 31
7.1. Authentication, Confidentiality, Integrity Protection . . 31 7.1. Authentication, Confidentiality, Integrity Protection . . 31
7.2. Non Repudiation . . . . . . . . . . . . . . . . . . . . . 32 7.2. Non Repudiation . . . . . . . . . . . . . . . . . . . . . 32
7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 32 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 32
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 32 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 33
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 33 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.1. Normative References . . . . . . . . . . . . . . . . . . 33 9.1. Normative References . . . . . . . . . . . . . . . . . . 33
9.2. Informative References . . . . . . . . . . . . . . . . . 33 9.2. Informative References . . . . . . . . . . . . . . . . . 34
Appendix A. Requirements . . . . . . . . . . . . . . . . . . . . 34 Appendix A. Requirements . . . . . . . . . . . . . . . . . . . . 34
A.1. Compliance with cdni-requirements . . . . . . . . . . . . 34 A.1. Compliance with cdni-requirements . . . . . . . . . . . . 34
A.2. Additional Requirements . . . . . . . . . . . . . . . . . 34 A.2. Additional Requirements . . . . . . . . . . . . . . . . . 35
A.2.1. Timeliness . . . . . . . . . . . . . . . . . . . . . 34 A.2.1. Timeliness . . . . . . . . . . . . . . . . . . . . . 35
A.2.2. Reliability . . . . . . . . . . . . . . . . . . . . . 35 A.2.2. Reliability . . . . . . . . . . . . . . . . . . . . . 35
A.2.3. Security . . . . . . . . . . . . . . . . . . . . . . 35 A.2.3. Security . . . . . . . . . . . . . . . . . . . . . . 35
A.2.4. Scalability . . . . . . . . . . . . . . . . . . . . . 35 A.2.4. Scalability . . . . . . . . . . . . . . . . . . . . . 35
A.2.5. Consistency between CDNI Logging and CDN Logging . . 35 A.2.5. Consistency between CDNI Logging and CDN Logging . . 36
A.2.6. Dispatching/Filtering . . . . . . . . . . . . . . . . 35 A.2.6. Dispatching/Filtering . . . . . . . . . . . . . . . . 36
Appendix B. Analysis of candidate protocols for Logging Appendix B. Analysis of candidate protocols for Logging
Transport . . . . . . . . . . . . . . . . . . . . . 36 Transport . . . . . . . . . . . . . . . . . . . . . 36
B.1. Syslog . . . . . . . . . . . . . . . . . . . . . . . . . 36 B.1. Syslog . . . . . . . . . . . . . . . . . . . . . . . . . 36
B.2. XMPP . . . . . . . . . . . . . . . . . . . . . . . . . . 36 B.2. XMPP . . . . . . . . . . . . . . . . . . . . . . . . . . 36
B.3. SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . 36 B.3. SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37
1. Introduction 1. Introduction
This memo specifies the Logging interface between a downstream CDN This memo specifies the Logging interface between a downstream CDN
(dCDN) and an upstream CDN (uCDN). First, it describes a reference (dCDN) and an upstream CDN (uCDN). First, it describes a reference
model for CDNI logging. Then, it specifies the CDNI Logging File model for CDNI logging. Then, it specifies the CDNI Logging File
format and the actual protocol for exchange of CDNI Logging Files. format and the actual protocol for exchange of CDNI Logging Files.
The reader should be familiar with the following documents: The reader should be familiar with the following documents:
skipping to change at page 17, line 29 skipping to change at page 17, line 29
o UUID: o UUID:
* format: <string> * format: <string>
* semantic: this is Universally Unique IDentifier for the CDNI * semantic: this is Universally Unique IDentifier for the CDNI
Logging File as specified in [RFC4122]. Logging File as specified in [RFC4122].
* occurrence: there MUST be one and only one instance of this * occurrence: there MUST be one and only one instance of this
directive. directive.
o Origin: o Claimed-Origin:
* format: <host> * format: <host>
* semantic: this identifies the entity transmitting the CDNI * semantic: this contains the claimed identification of the
Logging File (e.g. the host in a dCDN supporting the CDNI entity transmitting the CDNI Logging File (e.g. the host in a
Logging interface) or the entity responsible for transmitting dCDN supporting the CDNI Logging interface) or the entity
the CDNI Logging File (e.g. the dCDN). responsible for transmitting the CDNI Logging File (e.g. the
dCDN).
* occurrence: there MUST be zero or one instance of this * occurrence: there MUST be zero or one instance of this
directive. This directive MAY be included by the directive. This directive MAY be included by the dCDN. It
implementation transmitting the CDNI Logging file. When MUST NOT be included or modified by the uCDN.
included by the transmitting side, it MUST be validated or
over-written by the receiving side. When, it is not included o Verified-Origin:
by the transmitting side, it MAY be added locally by the
receiving side. [Editor's Note if we include a non-repudiation * format: <host>
mechanism: discuss the fact that this will provide incentive to * semantic: this contains the identification, as established by
dCDN to not cheat , as it can be detected] the entity receiving the CDNI Logging file, of the entity
transmitting the CDNI Logging File (e.g. the host in a dCDN
supporting the CDNI Logging interface) or the entity
responsible for transmitting the CDNI Logging File (e.g. the
dCDN).
* occurrence: there MUST be zero or one instance of this
directive. This directive MAY be added by the uCDN. It MUST
NOT be included by the dCDN. The mechanisms used by the uCDN
to establih and validate the entity respondible for the CDNI
Logging File is outside the scope of the present document. We
observe that, in particular, this may be achieved through
authentication mechanisms that are part of the CDNI Logging
File pull mechanism (Section 4.2).
o Record-Type: o Record-Type:
* format: <string> * format: <string>
* semantic: indicates the type of the CDNI Logging Records that * semantic: indicates the type of the CDNI Logging Records that
follow this directive, until another Record-Type directive (or follow this directive, until another Record-Type directive (or
the end of the CDNI Logging File). "cdni_http_request_v1" MUST the end of the CDNI Logging File). "cdni_http_request_v1" MUST
be indicated in the Record-Type directive for CDNI Logging be indicated in the Record-Type directive for CDNI Logging
records corresponding to HTTP request (e.g. a HTTP delivery records corresponding to HTTP request (e.g. a HTTP delivery
request) as specified in Section 3.2.1. request) as specified in Section 3.2.1.
* occurrence: there MUST be at least one instance of this * occurrence: there MUST be at least one instance of this
directive. The first instance of this directive MUST precede a directive. The first instance of this directive MUST precede a
Fields directive and precede any CDNI Logging Record. Fields directive and precede any CDNI Logging Record.
skipping to change at page 18, line 25 skipping to change at page 18, line 45
o Fields: o Fields:
* format: <field-name>[ <field-name>], where the allowed list of * format: <field-name>[ <field-name>], where the allowed list of
<field-name> are specified for each Record-Type in Section 3.2. <field-name> are specified for each Record-Type in Section 3.2.
* semantic: this lists the names of all the fields for which a * semantic: this lists the names of all the fields for which a
value is to appear in the CDNI Logging Records that are after value is to appear in the CDNI Logging Records that are after
this directive. The names of the fields, as well as their this directive. The names of the fields, as well as their
possible occurrences, are specified for each type of CDNI possible occurrences, are specified for each type of CDNI
Logging Records in Section 3.2. The field names listed in this Logging Records in Section 3.2. The field names listed in this
directive MUST be separated by a whitespace (" "). directive MUST be separated by the "horizontal tabulation
(TAB)" character.
* occurrence: there MUST be at least one instance of this * occurrence: there MUST be at least one instance of this
directive per Record-Type directive. The first instance of directive per Record-Type directive. The first instance of
this directive for a given Record-Type MUST precede any CDNI this directive for a given Record-Type MUST precede any CDNI
Logging Record for this Record-Type. Logging Record for this Record-Type.
o Integrity-Hash: o Integrity-Hash:
* format: <string> * format: <string>
* semantic: This directive permits the detection of a corrupted * semantic: This directive permits the detection of a corrupted
CDNI Logging File. This can be useful, for instance, if a CDNI Logging File. This can be useful, for instance, if a
problem occurs on the filesystem of the dCDN Logging system and problem occurs on the filesystem of the dCDN Logging system and
leads to a truncation of a logging file. The Integrity-Hash leads to a truncation of a logging file. The Integrity-Hash
value is computed, and included in this directive by the entity value is computed, and included in this directive by the entity
that transmits the CDNI Logging File, by applying the MD5 that transmits the CDNI Logging File. It is computed by
([RFC1321]) cryptographic hash function on the CDNI Logging applying the MD5 ([RFC1321]) cryptographic hash function on the
File, including all the directives and logging records, up to CDNI Logging File, including all the directives and logging
the Intergrity-Hash directive itself, excluding the Integrity- records, up to the Intergrity-Hash directive itself, excluding
Hash directive itself and, when present, also excluding the the Integrity-Hash directive itself and, when present, also
Non-Repudiation-Hash directive. The Integrity-Hash value is excluding the Non-Repudiation-Hash directive. The Integrity-
represented as a US-ASCII encoded hexadecimal number, 32 digits Hash value is represented as a US-ASCII encoded hexadecimal
long (representing a 128 bit hash value). The entity receiving number, 32 digits long (representing a 128 bit hash value).
the CDNI Logging File also computes in a similar way the MD5 The entity receiving the CDNI Logging File also computes in a
hash on the received CDNI Logging File and compares this hash similar way the MD5 hash on the received CDNI Logging File and
to the value of the Integrity-Hash directive. If the two compares this hash to the value of the Integrity-Hash
values are equal, then the received CDNI Logging File MUST be directive. If the two values are equal, then the received CDNI
considered non-corrupted. If the two values are different, the Logging File MUST be considered non-corrupted. If the two
received CDNI Logging File MUST be considered corrupted. The values are different, the received CDNI Logging File MUST be
behavior of the entity that received a corrupted CDNI Logging considered corrupted. The behavior of the entity that received
File is outside the scope of this specification; we note that a corrupted CDNI Logging File is outside the scope of this
the entity MAY attempt to pull again the same CDNI Logging file specification; we note that the entity MAY attempt to pull
from the transmitting entity. again the same CDNI Logging file from the transmitting entity.
If the entity receiving the CDNI Logging File adds a Verified-
Origin directive, it MUST recompute and update the Integrity-
Hash directive so it also protects the added Verified-Origin
directive.
* occurrence: there MUST be one and only one instance of this * occurrence: there MUST be zero or one instance of this
directive. This field MUST be the last line of the CDNI directive. There SHOULD be one instance of this directive.
Logging File when the Non-Repudiation-Hash is absent, and MUST One situation where that directive could be omitted is where
be the one before last line of the CDNI Logging File when the integrity protection is already provided via another mechanism
Non-Repudiation-Hash is present. (for example if an integrity hash is associated to the CDNI
Logging file out of band through the CDNI Logging Logging Feed
Section 4.1 leveraging ATOM extensions such as those proposed
in [I-D.snell-atompub-link-extensions]. When present, this
field MUST be the last line of the CDNI Logging File when the
Non-Repudiation-Hash is absent, and MUST be the one before last
line of the CDNI Logging File when the Non-Repudiation-Hash is
present.
o Non-Repudiation-Hash: o Non-Repudiation-Signature:
* format: <string> * format: <string>
* semantic: This field contains a signature that supports the
* semantic: This hash field permits the non-repudiation of the non-repudiation of the CDNI Logging File by the entity that
CDNI Logging File by the entity that transmitted the CDNI transmitted the CDNI Logging File. [Editor's Note: Detailed
Logging File. [Editor's Note: I need help for specifying the description To Be Added. David Mandelberg has the lead for
appropriate hash - ie hash must be signed with private-key of drafting text.The text needs to indicate that the Claimed-
entity transmitting the CDNI Logging File] Origin directive MUST be covered and the Verified-Origin
directive MUST NOT be covered by the signature. We may want to
refer to RFC4949 for terminology around Non-Redudiation]
* occurrence: there MAY be one and only one instance of this * occurrence: there MAY be one and only one instance of this
directive. When present, this directive MUST be the last line directive. When present, this directive MUST be the last line
of the CDNI Logging File. of the CDNI Logging File.
3.2. Logging Records 3.2. Logging Records
A CDNI Logging Record consists of a sequence of CDNI Logging Fields A CDNI Logging Record consists of a sequence of CDNI Logging Fields
relating to that single CDNI Logging Record. relating to that single CDNI Logging Record.
CDNI Logging Fields MUST be separated by the "horizontal tabulation CDNI Logging Fields MUST be separated by the "horizontal tabulation
(TAB)" character. (TAB)" character.
Some CDNI Logging field names use a prefix scheme similar to the one Some CDNI Logging field names use a prefix scheme similar to the one
used in W3C Extended Log File Format [ELF] to facilitate readability. used in W3C Extended Log File Format [ELF] to facilitate readability.
The semantics of the prefix in the present document is: The semantics of the prefix in the present document is:
o c: refers to the User Agent that issues the request (corresponds o c: refers to the User Agent that issues the request (corresponds
to the "client" of W3C Extended Log Format) to the "client" of W3C Extended Log Format)
o d: refers to the dCDN (relative to a given CDN acting as a uCDN)
o s: refers to the dCDN Surrogate that serves the request o s: refers to the dCDN Surrogate that serves the request
(corresponds to the "server" of W3C Extended Log Format) (corresponds to the "server" of W3C Extended Log Format)
o u: refers to the uCDN (relative to a given CDN acting as a dCDN)
o cs: refers to communication from the dCDN Surrogate towards the o cs: refers to communication from the dCDN Surrogate towards the
User-Agent User-Agent
o sc: refers to communication from the User-Agent towards the dCDN o sc: refers to communication from the User-Agent towards the dCDN
Surrogate Surrogate
[Editor's Note: see discussion with Rob about adding definition for
"r"]
An implementation of the CDNI Logging interface as per the present An implementation of the CDNI Logging interface as per the present
specification MUST support the CDNI HTTP Delivery Records as specification MUST support the CDNI HTTP Delivery Records as
specified in Section 3.2.1. [Editor's Note": other types of delivery specified in Section 3.2.1. [Editor's Note": other types of delivery
records will be listed here if we specify other types for this records will be listed here if we specify other types for this
version eg Request Routing]. version eg Request Routing].
The formats listed in this section in the form <...> are specified in The formats listed in this section in the form <...> are specified in
Section 3.3). Section 3.3).
3.2.1. HTTP Request Logging Record 3.2.1. HTTP Request Logging Record
The HTTP Request Logging Record contains the following CDNI Logging The HTTP Request Logging Record contains the following CDNI Logging
Fields, listed by their field name: Fields, listed by their field name:
o date: o date:
* format: <date> * format: <date>
* semantic: the date at which the processing of request started * semantic: the date at which the processing of request completed
on the Surrogate. on the Surrogate.
* occurrence: there MUST be one and only one instance of this * occurrence: there MUST be one and only one instance of this
field. field.
o time: o time:
* format: <time> * format: <time>
* semantic: the time at which the processing of request started * semantic: the time at which the processing of request completed
on the Surrogate. on the Surrogate.
* occurrence: there MUST be one and only one instance of this * occurrence: there MUST be one and only one instance of this
field. field.
o time-taken: o time-taken:
* format: <fixed> * format: <fixed>
* semantic: duration, in seconds, between the start of the * semantic: duration, in seconds, between the start of the
skipping to change at page 21, line 18 skipping to change at page 22, line 5
o c-ip: o c-ip:
* format: <address> * format: <address>
* semantic: the source IPv4 or IPv6 address (i.e. the "client" * semantic: the source IPv4 or IPv6 address (i.e. the "client"
address) in the request received by the Surrogate. address) in the request received by the Surrogate.
* occurrence: there MUST be one and only one instance of this * occurrence: there MUST be one and only one instance of this
field. field.
o c-ip-anonimizing:
* format: <integer>
* semantic: the number of bits of the address in the c-ip field
that are zeroed-out in order to anonymize the logging record.
The mechanism by which the two ends of teh CDNI Logging
nterafce agree on whether anonimization is to be supported and
the number of bits that need to be zeroed-out for this purpose
are outside the scope of the present document.
* occurrence: there MUST be zero or one instance of this field.
o c-port: o c-port:
* format: <integer> * format: <integer>
* semantic: the source TCP port (i.e. the "client" port) in the * semantic: the source TCP port (i.e. the "client" port) in the
request received by the Surrogate. request received by the Surrogate.
* occurrence: there MUST be zero or exactly one instance of this * occurrence: there MUST be zero or exactly one instance of this
field. field.
skipping to change at page 22, line 20 skipping to change at page 23, line 20
o cs-method: o cs-method:
* format: <string> * format: <string>
* semantic: this is the HTTP method of the HTTP request received * semantic: this is the HTTP method of the HTTP request received
by the Surrogate. by the Surrogate.
* occurrence: There MUST be one and only one instance of this * occurrence: There MUST be one and only one instance of this
field. field.
o cs-uri: [Editor's note: rename "sr-uri" ?] o cs-uri:
* format: <uri> * format: <uri>
* semantic: this is the absolute-URI of the request received by * semantic: this is the complete http_URL (as specified in
the Surrogate. [Editor's Note: do we agree this should be an [RFC2616]) of the request received by the Surrogate.
absolute-URI even if teh request uses a relative-URI?]
* occurrence: there MUST be zero or exactly one instance of this * occurrence: there MUST be zero or exactly one instance of this
field. field.
o ucdn-centric-uri: o u-uri:
* format: <uri> * format: <uri>
* semantic: this is an absolute URI derived from the absolute-URI * semantic: this is a complete http_URL (as specified in
of the request received by the Surrogate but modified by the [RFC2616]) derived from the complete URI of the request
entity generating or transmitting the CDNI Logging Record, in a received by the Surrogate but transformed by the entity
way that is agreed upon between the two ends of the CDNI generating or transmitting the CDNI Logging Record, in a way
Logging interface. For example, the two ends of the CDNI that is agreed upon between the two ends of the CDNI Logging
Logging interface could agree that the ucdn-centric-uri strips interface, so the transformed URI is meaningful to the uCDN.
the part of the delivery-uri that exposes which individual For example, the two ends of the CDNI Logging interface could
agree that the u-uri is constructed from the cs-uri by removing
the part of the hostname that exposes which individual
Surrogate actually performed the delivery. The details of Surrogate actually performed the delivery. The details of
modification performed to generate the ucdn-centric-uri, as modification performed to generate the u-uri, as well as the
well as the mechanism to agree on these modifications between mechanism to agree on these modifications between the two sides
the two sides of the CDNI Logging interface are outside the of the CDNI Logging interface are outside the scope of the
scope of the present document. [Editor's Note: do we agree present document.
this should be an absolute-URI even if the request uses a
relative-URI?]
* occurrence: there MUST be one and only one instance of this * occurrence: there MUST be one and only one instance of this
field. field.
o protocol: o protocol:
* format: <string> * format: <string>
* semantic: this is value of the HTTP-Version field as specified * semantic: this is value of the HTTP-Version field as specified
in [RFC2616] of the Request-Line of the request received by the in [RFC2616] of the Request-Line of the request received by the
skipping to change at page 26, line 18 skipping to change at page 27, line 18
available, this MUST be conveyed via a dash character ("-"). available, this MUST be conveyed via a dash character ("-").
The fields listed in the "Fields" directive can be listed in the The fields listed in the "Fields" directive can be listed in the
order in which they are listed in Section 3.2.1 or in any other order in which they are listed in Section 3.2.1 or in any other
order. order.
[Editor's Note: discuss private fields ] [Editor's Note: discuss private fields ]
3.2.2. CDNI Logging File Example 3.2.2. CDNI Logging File Example
#Version: 1.0 #Version:1.0<CRLF>
#UUID: urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6??? #UUID: urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6???<CRLF>
#Origin: cdni-logging-entity.dcdn.example.com #Claimed-Origin: cdni-logging-entity.dcdn.example.com<CRLF>
#Record-Type: cdni_http_request_v1 #Record-Type: cdni_http_request_v1<CRLF>
#Fields: date time time-taken c-ip cs-method ucdn-centric-uri #Fields:<TAB>date<TAB>time<TAB>time-taken<TAB>c-ip<TAB>cs-
protocol sc-status sc-total-bytes cs(User-Agent) cs(Referer) s-cached method<TAB>u-uri<TAB>protocol<TAB>sc-status<TAB>sc-total-bytes<TAB>cs
(User-Agent)<TAB>cs(Referer)<TAB>s-cached<CRLF>
2013-05-17 00:38:06.825 88.958 10.5.7.1 GET http://cdni- 2013-05-17<TAB>00:38:06.825<TAB>88.958<TAB>10.5.7.1<TAB>GET<TAB>http
ucdn.dcdn.example.com/video/movie100.mp4 HTTP/1.1 200 672989 Mozilla/ ://cdni-ucdn.dcdn.example.com/video/movie100.mp4<TAB>HTTP/
5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, 1.1<TAB>200<TAB>672989<TAB>Mozilla/5.0 (Windows; U; Windows NT 6.0;
like Gecko) Chrome/5.0.375.127 Safari /533.4 host1.example.com 1 en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127
Safari /533.4<TAB>host1.example.com <TAB>1<CRLF>
2013-05-17 00:39:09.145 169.790 10.5.10.5 GET http://cdni- 2013-05-17<TAB>00:39:09.145<TAB>169.790<TAB>10.5.10.5<TAB>GET<TAB>htt
ucdn.dcdn.example.com/video/movie118.mp4 HTTP/1.1200 1579920 Mozilla/ p://cdni-ucdn.dcdn.example.com/video/movie118.mp4<TAB>HTTP/
5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, 1.1<TAB>200<TAB>1579920<TAB>Mozilla/5.0 (Windows; U; Windows NT 6.0;
like Gecko) Chrome/5.0.375.127 Safari /533.4 host1.example.com 1 en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127
Safari /533.4<TAB>host1.example.com<TAB>1<CRLF>
2013-05-17 00:42:53.437 2.879 10.5.10.5 GET http://cdni- 2013-05-17<TAB>00:42:53.437<TAB>2.879<TAB>10.5.10.5<TAB>GET<TAB>http
ucdn.dcdn.example.com/video/picture11.mp4 HTTP/1.0 200 17724 Mozilla/ ://cdni-ucdn.dcdn.example.com/video/picture11.mp4<TAB>HTTP/
5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, 1.0<TAB>200<TAB>17724<TAB>Mozilla/5.0 (Windows; U; Windows NT 6.0;
like Gecko) Chrome/5.0.375.127 Safari /533.4 host5.example.com 0 en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127
Safari /533.4<TAB>host5.example.com<TAB>0<CRLF>
#Integrity-Hash: 9e107d9d372bb6826bd81d3542a419d6 [Editor's Note: #Integrity-Hash: 9e107d9d372bb6826bd81d3542a419d6 [Editor's Note:
include the correct MD5-hash value for the actual example] include the correct MD5-hash value for the actual example]<CRLF>
3.3. Fields and Directives Formats 3.3. Fields and Directives Formats
[Editor's Note: still needs work to minimise the number of types [Editor's Note: still needs work to minimise the number of types
defined across this section and specific types defined inside the defined across this section and specific types defined inside the
field definitions themselves] field definitions themselves]
o <digit> = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | o <digit> = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" |
"9" "9"
skipping to change at page 27, line 32 skipping to change at page 28, line 32
* Dates are recorded in the format YYYY-MM-DD where YYYY, MM and * Dates are recorded in the format YYYY-MM-DD where YYYY, MM and
DD stand for the numeric year, month and day respectively. All DD stand for the numeric year, month and day respectively. All
dates are specified in Universal Time Coordinated (UTC). dates are specified in Universal Time Coordinated (UTC).
o <time> = 2<digit> ":" 2<digit> ":" 2<digit> ["." *<digit>] o <time> = 2<digit> ":" 2<digit> ":" 2<digit> ["." *<digit>]
* Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH * Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH
is the hour in 24 hour format, MM is minutes and SS is seconds. is the hour in 24 hour format, MM is minutes and SS is seconds.
All times are specified in Universal Time Coordinated (UTC). All times are specified in Universal Time Coordinated (UTC).
o <uri> = <string> containing a URI as specified in [RFC3986]. o <uri> = <string> containing a http_URL as specified in [RFC2616].
o <fixed> = Fixed Format Float = 1*<digit> [. *<digit>] o <fixed> = Fixed Format Float = 1*<digit> [. *<digit>]
o <HTTP-header> = <string> containing a HTTP header field name (e.g. o <HTTP-header> = <string> containing a HTTP header field name (e.g.
"User-Agent", "Referer") as specified in [RFC2616]. "User-Agent", "Referer") as specified in [RFC2616].
4. CDNI Logging File Exchange Protocol 4. CDNI Logging File Exchange Protocol
This document specifies a protocol for the exchange of CDNI Logging This document specifies a protocol for the exchange of CDNI Logging
Files as specified in Section 3. Files as specified in Section 3.
skipping to change at page 29, line 36 skipping to change at page 30, line 36
o when the client-side request indicates client-supported o when the client-side request indicates client-supported
compression schemes, SHOULD use a compression scheme that it compression schemes, SHOULD use a compression scheme that it
supports and is supported by the client-side supports and is supported by the client-side
[Editor's Note: discuss Non-Repudiation : it is a nice to have and [Editor's Note: discuss Non-Repudiation : it is a nice to have and
how it could be supported, via a different digest than the one for how it could be supported, via a different digest than the one for
integrity] integrity]
5. Open Issues 5. Open Issues
o The proposed format for Date and Time is based on W3C and is only o What separator should be used between a directive name and a
in UTC. Is this all OK? RFC 5322 (Section 3.3) format could be directive value e.g. "#Version:1.0" or "#Version: 1.0" or
used or ISO 8601 formatted date and time in UTC (same format as "#Version:<TAB>1.0"?
proposed in [draft-caulfield-cdni-metadata-core-00]). Also see
RFC5424 Section 6.2.3. We currently use same field names as W3C
since we have same definition.
o (comment from Kevin) how are errors handled ? If the client gets
handed a bunch of 403s and 404s, but still gets the content
eventually, without triggering an event, are those still logged?
For Bytes-Sent, if there were aborted requests, do those get
counted as well? Not all client behavior can be correlated with
the simplified log
o Do we need to specify Logs for Request Routing performed by dCDN?
Observation: Probably can be generalized to the requirement for
"event" logging (e.g. dCDN request Router not able to redirect,
dCDN cannot acquire metadata, dCDN cannot aquire content, "dCDN
Busy Tone" ) Recommendation: Try first specify what events and
what information needs to be exchanged. Depending on progress
include in initial logging spec or not i.e. handle as a [MED]
requirement.
o Privacy: do we need some explicit support of IP address masking by
dCDN to uCDN, or is it OK to assume that uCDN is to keep this info
confidential (like dCDN is assumed to do already)?
o definition of field prefixes: add "r" is uCDN. This one is less
clear to me. I need to see how you propose to use "r" below,
before I can agree. (Just for my own notes, I thought "r" could
be used if the dCDN Surrogate was going to Log something related
to acquisition of content by the dCDN Surrogate from some content
source. Also, in a delivery log generated by a dCDN Surrogate ,
how can it know about acquisition from uCDN that can be done by
other devices than the dCDN Surrogate). "ucdn-centric-uri": ROB>
going back to the definitions of s/c/r suggested above, for a CDNI
logfile field would then just be "sr-uri". So we don't need to
invent a new prefix for CDNI, we can use the basic w3c naming?
FRANCOIS: I am OK to use "sr-uri" as long as we feel confident
that we will never need Surrogate to log information about how it
acquires from within the dCDN (ie regular use of "r" prefix). Are
we confident?
o Do we need Record-Type as File Directive?: ROB> Is this needed -
would a record type per file do the job? ... if we don't allow
mixed record types, we can include the record type in the ATOM
feed (to allow the reader to decide whether there might be records
it's interested in without getting the logfile). I can't think of
a reason to mix, (for example) http/rtmp records, or delivery/req-
routing. Different things are likely to be generating those
records anyway. A version change can always be done by starting a
new file. <Francois> Here are a couple potential use cases for
mixing record types in a single file: * we later define
"cdni_has_delivery_v1" record types for HTTP Adaptive BitRate
sessions. Then a dCDN Surrogate will be generating a continuous
mixture of "cdni_http_request_v1" records for PDL requests and
"cdni_has_request_v1" records for HAS sessions. Why should we be
forced to break those? * we later define some record types for
events taking place on Surrogates , which can happen any time in
the middle of sessions. Why shoudl we be forced to break those
into separate files. It seems wise to keep the flexibility in the
File structure to allow the mix in the future. And the overhead
is very small since it is encoded in a Directive.
o Integrity-Hash:ROB> draft-snell-atompub-link-extensions adds a
hash of the resource to the ATOM feed (not sure about the status
of that doc, looks like it's stalled a bit). But if we include
that in the ATOM feed, the value in the feed would need to include
this Integrity-Hash in the log file itself, which might mean re-
calculating the hash (especially if the feed is not generated in
the same place as the logfile). So we probably only want one of
the two? I think my preference would be to keep it in the feed,
saves any complications about what to hash (just running "md5sum"
on a downloaded logfile would work, rather than needing to remove
the last line). The draft-snell also allows other hashes, "sha1"
and so on - for cdni interoperability, we could limit it to md5 or
stick with draft-snell's base set. <Francois> Very good point. I
agree we should probably want one of the two in a typical
deployment. Leveraging draft-snell-atompub-link-extensions is
attractive because it leverages generic ATOM features and
expertise. It has the potential drawback of introducing a
dependency on a document that may be published later (or
potentially never since it is not even a WG doc). Defining our
own hash in the file is attractive because we can be done right
away, and there could be simple short term implementation that
start using the CDNI Logging File without relying on the ATOM
Feed. At the same time we don't want to end up with two redundant
hashes eventually. How about an approach where : * we define a
simple MD5 has only, and make it optional * when there is no other
mechanism to get the hash, it can be included in the file * when
there are other mechanism (e.g. draft-snell-atompub-link-
extensions), it is not included in the file.
o Compression: <Ben>When we say the server MUST support gzip & o Compression: <Ben>When we say the server MUST support gzip &
deflate we probably need to think through whether we mean content- deflate we probably need to think through whether we mean content-
encoding, transfer-encoding or both. The semantics get a little encoding, transfer-encoding or both. The semantics get a little
confusing so we probably just need to think them through to ensure confusing so we probably just need to think them through to ensure
we allow a server to store compressed logs as transmit them we allow a server to store compressed logs as transmit them
compressed. compressed.
o Handling of Event logs and notifications: There are two aspects of
that question:
* non-real-time exchange of event logs from dCDN to uCDN for
audit purposes. This could be added to current spec presumably
in the form of additional Record-Types and without requiring a
significant change to the current CDNI LOgging file exchange
approach. It is proposed that this be handled as a [MED]
requirement. e.g. try first specify what events and what
information needs to be exchanged; and depending on progress,
decide to include in initial logging spec or not
* real-time exchange of event notification from dCDN to uCDN for
immediate operational action (eg on notification by dCDN that
dCDN request routing is down, uCDN stops redirecting to this
dCDN). This would presumably require definition/extension of
another CDNI interface or significant change/extension to the
current CDNI logging spec. It is proposed that thisbe kept out
of the scope of the current cdni-logging spec .
Another question is what set of events should be logged/notified.
The first type of events realtes to "service-level" events i.e.
high level events that affect the service that the dCDN is
providing to the uCDN (e.g.dCDN request routing is down, dCDN is
overloaded). There is general agreements that it is desirable to
be able to log/notify such service-level events. The second type
of events is "atomic-level" events i.e. low level events that may
be useful to identify or track a component issue or a delivery
issue. logging/notifying about such events may be useful in some
situations (eg uCDN and dCDN have a particular relationship
allowing them to share detailed operational information) and may
not be useful in some situations (because the dCDN does not want
to expose details of its CDN operation). Ideal approach is to
define both types of events and have the first type as MUST and
the second type as MAY. Fall back approach woudl be to only
define the first type initially.
o Add precise definition of what must be supported by transmitting
implementation and what must be implemented by receiving
application (regardless of what may actually be used in a given
deployment). For example, it may be reasonable to mandate that a
receiving implementaton but be able to receive all the directives
specified in the doc and all fields.
6. IANA Considerations 6. IANA Considerations
TBD TBD
7. Security Considerations 7. Security Considerations
7.1. Authentication, Confidentiality, Integrity Protection 7.1. Authentication, Confidentiality, Integrity Protection
The use of TLS for transport of the CDNI Logging feed mechanism The use of TLS for transport of the CDNI Logging feed mechanism
(Section 4.1) and CDNI Logging File pull mechanism (Section 4.2) (Section 4.1) and CDNI Logging File pull mechanism (Section 4.2)
allows: allows:
o the dCDN and uCDN to authenticate each other (to ensure they are o the dCDN and uCDN to authenticate each other (to ensure they are
transmitting/receiving CDNI Logging File from an authenticated transmitting/receiving CDNI Logging File from an authenticated
CDN) CDN)
o the CDNI Logging information to be transmitted with o the CDNI Logging information to be transmitted with
confidentiality confidentiality
skipping to change at page 32, line 26 skipping to change at page 32, line 29
additional integrity protection, this time targeting potential additional integrity protection, this time targeting potential
corruption of the CDNI logging information during the CDNI Logging corruption of the CDNI logging information during the CDNI Logging
File generation. This mechanism does not allow restoration of the File generation. This mechanism does not allow restoration of the
corrupted CDNI Logging information, but it allows detection of such corrupted CDNI Logging information, but it allows detection of such
corruption and therefore triggering of appropraite correcting actions corruption and therefore triggering of appropraite correcting actions
(e.g. discard of corrupted information, attempt to re-obtain the (e.g. discard of corrupted information, attempt to re-obtain the
CDNI Logging information). CDNI Logging information).
7.2. Non Repudiation 7.2. Non Repudiation
The Non-Repudiation-Hash directive in the CDNI Logging File allows The Non-Repudiation-Signature directive in the CDNI Logging File
support of non-repudiation of the CDNI Logging File by the dCDN. The allows support of non-repudiation of the CDNI Logging File by the
optional Non-Repudiation-Hash can be used on the CDNI Logging dCDN. The optional Non-Repudiation-Hash can be used on the CDNI
interface where needed. Logging interface where needed.
7.3. Privacy 7.3. Privacy
CDNs have the opportunity to collect detailed information about the CDNs have the opportunity to collect detailed information about the
downloads performed by End-Users. The provision of this information downloads performed by End-Users. The provision of this information
to another CDN introduces End-Users privacy protection concerns. to another CDN introduces potential End-Users privacy protection
[Editor's Note: see list of open questions] concerns. We observe that when CDNI interconnection is realised as
per [I-D.ietf-cdni-framework], the uCDN handles the initial End-User
requests (before it is redirected to the dCDN) so, regardless of
which information is, or is not, communicated to the uCDN through the
CDNI Logging interface, the uCDN has visibility on significant
information such as the IP address of the End-User request and the
URL of the request. Nonetheless, if the dCDN and uCDN agree that
anonymization is required to avoid making some detailed information
available to the uCDN (such as how much bytes of the content has been
watched by an enduser and/or at what time) or is required to meet
some legal obligations, then the uCDN and dCDN can agree to exchange
anonymized End-User IP addresses in CDNI Logging files and the c-ip-
anonymization field can be used to convey the number of bits that
have been anonymized so that the meaningful information can still be
easily extracted from the anonymized addressses (e.g. for
geolocation aware analytics).
8. Acknowledgments 8. Acknowledgments
This document borrows from the W3C Extended Log Format [ELF]. This document borrows from the W3C Extended Log Format [ELF].
The authors would like to thank Sebastien Cubaud, Pawel Grochocki, The authors would like to thank Sebastien Cubaud, Pawel Grochocki,
Christian Jacquenet, Yannick Le Louedec, Anne Marrec and Emile Christian Jacquenet, Yannick Le Louedec, Anne Marrec and Emile
Stephan for their contributions on early versions of this document. Stephan for their contributions on early versions of this document.
The authors would like also to thank Rob Murray, Fabio Costa, Sara The authors would like also to thank Rob Murray, Fabio Costa, Sara
skipping to change at page 33, line 14 skipping to change at page 33, line 32
9. References 9. References
9.1. Normative References 9.1. Normative References
[I-D.ietf-cdni-metadata] [I-D.ietf-cdni-metadata]
Niven-Jenkins, B., Murray, R., Watson, G., Caulfield, M., Niven-Jenkins, B., Murray, R., Watson, G., Caulfield, M.,
Leung, K., and K. Ma, "CDN Interconnect Metadata", draft- Leung, K., and K. Ma, "CDN Interconnect Metadata", draft-
ietf-cdni-metadata-01 (work in progress), February 2013. ietf-cdni-metadata-01 (work in progress), February 2013.
[I-D.snell-atompub-link-extensions]
Snell, J., "Atom Link Extensions", draft-snell-atompub-
link-extensions-09 (work in progress), June 2012.
[RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321,
April 1992. April 1992.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
 End of changes. 46 change blocks. 
206 lines changed or deleted 225 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/