draft-ietf-trans-gossip-04.txt   draft-ietf-trans-gossip-05.txt 
TRANS L. Nordberg TRANS L. Nordberg
Internet-Draft NORDUnet Internet-Draft NORDUnet
Intended status: Experimental D. Gillmor Intended status: Experimental D. Gillmor
Expires: July 14, 2017 ACLU Expires: July 18, 2018 ACLU
T. Ritter T. Ritter
January 14, 2018
January 10, 2017
Gossiping in CT Gossiping in CT
draft-ietf-trans-gossip-04 draft-ietf-trans-gossip-05
Abstract Abstract
The logs in Certificate Transparency are untrusted in the sense that The logs in Certificate Transparency are untrusted in the sense that
the users of the system don't have to trust that they behave the users of the system don't have to trust that they behave
correctly since the behavior of a log can be verified to be correct. correctly since the behavior of a log can be verified to be correct.
This document tries to solve the problem with logs presenting a This document tries to solve the problem with logs presenting a
"split view" of their operations. It describes three gossiping "split view" of their operations or failing to incorporate a
mechanisms for Certificate Transparency: SCT Feedback, STH submission within MMD. It describes three gossiping mechanisms for
Pollination and Trusted Auditor Relationship. Certificate Transparency: SCT Feedback, STH Pollination and Trusted
Auditor Relationship.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on July 14, 2017. This Internet-Draft will expire on July 18, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 2, line 24 skipping to change at page 2, line 24
2. Defining the problem . . . . . . . . . . . . . . . . . . . . 4 2. Defining the problem . . . . . . . . . . . . . . . . . . . . 4
3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.1. Pre-Loaded vs Locally Added Anchors . . . . . . . . . . . 5 4.1. Pre-Loaded vs Locally Added Anchors . . . . . . . . . . . 5
5. Who gossips with whom . . . . . . . . . . . . . . . . . . . . 5 5. Who gossips with whom . . . . . . . . . . . . . . . . . . . . 5
6. What to gossip about and how . . . . . . . . . . . . . . . . 6 6. What to gossip about and how . . . . . . . . . . . . . . . . 6
7. Data flow . . . . . . . . . . . . . . . . . . . . . . . . . . 6 7. Data flow . . . . . . . . . . . . . . . . . . . . . . . . . . 6
8. Gossip Mechanisms . . . . . . . . . . . . . . . . . . . . . . 7 8. Gossip Mechanisms . . . . . . . . . . . . . . . . . . . . . . 7
8.1. SCT Feedback . . . . . . . . . . . . . . . . . . . . . . 7 8.1. SCT Feedback . . . . . . . . . . . . . . . . . . . . . . 7
8.1.1. SCT Feedback data format . . . . . . . . . . . . . . 8 8.1.1. SCT Feedback data format . . . . . . . . . . . . . . 8
8.1.2. HTTPS client to server . . . . . . . . . . . . . . . 8 8.1.2. HTTPS client to server . . . . . . . . . . . . . . . 9
8.1.3. HTTPS server operation . . . . . . . . . . . . . . . 11 8.1.3. HTTPS server operation . . . . . . . . . . . . . . . 11
8.1.4. HTTPS server to auditors . . . . . . . . . . . . . . 13 8.1.4. HTTPS server to auditors . . . . . . . . . . . . . . 13
8.2. STH pollination . . . . . . . . . . . . . . . . . . . . . 14 8.2. STH pollination . . . . . . . . . . . . . . . . . . . . . 14
8.2.1. HTTPS Clients and Proof Fetching . . . . . . . . . . 15 8.2.1. HTTPS Clients and Proof Fetching . . . . . . . . . . 16
8.2.2. STH Pollination without Proof Fetching . . . . . . . 17 8.2.2. STH Pollination without Proof Fetching . . . . . . . 17
8.2.3. Auditor Action . . . . . . . . . . . . . . . . . . . 17 8.2.3. Auditor Action . . . . . . . . . . . . . . . . . . . 17
8.2.4. STH Pollination data format . . . . . . . . . . . . . 17 8.2.4. STH Pollination data format . . . . . . . . . . . . . 18
8.3. Trusted Auditor Stream . . . . . . . . . . . . . . . . . 17 8.3. Trusted Auditor Stream . . . . . . . . . . . . . . . . . 18
8.3.1. Trusted Auditor data format . . . . . . . . . . . . . 18 8.3.1. Trusted Auditor data format . . . . . . . . . . . . . 19
9. 3-Method Ecosystem . . . . . . . . . . . . . . . . . . . . . 19 9. 3-Method Ecosystem . . . . . . . . . . . . . . . . . . . . . 20
9.1. SCT Feedback . . . . . . . . . . . . . . . . . . . . . . 19 9.1. SCT Feedback . . . . . . . . . . . . . . . . . . . . . . 20
9.2. STH Pollination . . . . . . . . . . . . . . . . . . . . . 20 9.2. STH Pollination . . . . . . . . . . . . . . . . . . . . . 20
9.3. Trusted Auditor Relationship . . . . . . . . . . . . . . 21 9.3. Trusted Auditor Relationship . . . . . . . . . . . . . . 21
9.4. Interaction . . . . . . . . . . . . . . . . . . . . . . . 22 9.4. Interaction . . . . . . . . . . . . . . . . . . . . . . . 22
10. Security considerations . . . . . . . . . . . . . . . . . . . 22 10. Security considerations . . . . . . . . . . . . . . . . . . . 23
10.1. Attacks by actively malicious logs . . . . . . . . . . . 22 10.1. Attacks by actively malicious logs . . . . . . . . . . . 23
10.2. Dual-CA Compromise . . . . . . . . . . . . . . . . . . . 23 10.2. Dual-CA Compromise . . . . . . . . . . . . . . . . . . . 23
10.3. Censorship/Blocking considerations . . . . . . . . . . . 24 10.3. Censorship/Blocking considerations . . . . . . . . . . . 24
10.4. Flushing Attacks . . . . . . . . . . . . . . . . . . . . 25 10.4. Flushing Attacks . . . . . . . . . . . . . . . . . . . . 25
10.4.1. STHs . . . . . . . . . . . . . . . . . . . . . . . . 25 10.4.1. STHs . . . . . . . . . . . . . . . . . . . . . . . . 25
10.4.2. SCTs & Certificate Chains on HTTPS Servers . . . . . 26 10.4.2. SCTs & Certificate Chains on HTTPS Servers . . . . . 26
10.4.3. SCTs & Certificate Chains on HTTPS Clients . . . . . 26 10.4.3. SCTs & Certificate Chains on HTTPS Clients . . . . . 27
10.5. Privacy considerations . . . . . . . . . . . . . . . . . 27 10.5. Privacy considerations . . . . . . . . . . . . . . . . . 27
10.5.1. Privacy and SCTs . . . . . . . . . . . . . . . . . . 27 10.5.1. Privacy and SCTs . . . . . . . . . . . . . . . . . . 27
10.5.2. Privacy in SCT Feedback . . . . . . . . . . . . . . 27 10.5.2. Privacy in SCT Feedback . . . . . . . . . . . . . . 27
10.5.3. Privacy for HTTPS clients performing STH Proof 10.5.3. Privacy for HTTPS clients performing STH Proof
Fetching . . . . . . . . . . . . . . . . . . . . . . 28 Fetching . . . . . . . . . . . . . . . . . . . . . . 28
10.5.4. Privacy in STH Pollination . . . . . . . . . . . . . 28 10.5.4. Privacy in STH Pollination . . . . . . . . . . . . . 29
10.5.5. Privacy in STH Interaction . . . . . . . . . . . . . 29 10.5.5. Privacy in STH Interaction . . . . . . . . . . . . . 29
10.5.6. Trusted Auditors for HTTPS Clients . . . . . . . . . 29 10.5.6. Trusted Auditors for HTTPS Clients . . . . . . . . . 30
10.5.7. HTTPS Clients as Auditors . . . . . . . . . . . . . 30 10.5.7. HTTPS Clients as Auditors . . . . . . . . . . . . . 30
11. Policy Recommendations . . . . . . . . . . . . . . . . . . . 30 11. Policy Recommendations . . . . . . . . . . . . . . . . . . . 31
11.1. Blocking Recommendations . . . . . . . . . . . . . . . . 31 11.1. Blocking Recommendations . . . . . . . . . . . . . . . . 31
11.1.1. Frustrating blocking . . . . . . . . . . . . . . . . 31 11.1.1. Frustrating blocking . . . . . . . . . . . . . . . . 31
11.1.2. Responding to possible blocking . . . . . . . . . . 31 11.1.2. Responding to possible blocking . . . . . . . . . . 31
11.2. Proof Fetching Recommendations . . . . . . . . . . . . . 32 11.2. Proof Fetching Recommendations . . . . . . . . . . . . . 33
11.3. Record Distribution Recommendations . . . . . . . . . . 33 11.3. Record Distribution Recommendations . . . . . . . . . . 33
11.3.1. Mixing Algorithm . . . . . . . . . . . . . . . . . . 34 11.3.1. Mixing Algorithm . . . . . . . . . . . . . . . . . . 34
11.3.2. The Deletion Algorithm . . . . . . . . . . . . . . . 35 11.3.2. The Deletion Algorithm . . . . . . . . . . . . . . . 35
11.4. Concrete Recommendations . . . . . . . . . . . . . . . . 36 11.4. Concrete Recommendations . . . . . . . . . . . . . . . . 36
11.4.1. STH Pollination . . . . . . . . . . . . . . . . . . 36 11.4.1. STH Pollination . . . . . . . . . . . . . . . . . . 36
11.4.2. SCT Feedback . . . . . . . . . . . . . . . . . . . . 39 11.4.2. SCT Feedback . . . . . . . . . . . . . . . . . . . . 40
12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 53 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 53
13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 53 13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 53
14. ChangeLog . . . . . . . . . . . . . . . . . . . . . . . . . . 53 14. ChangeLog . . . . . . . . . . . . . . . . . . . . . . . . . . 53
14.1. Changes between ietf-03 and ietf-04 . . . . . . . . . . 53 14.1. Changes between ietf-04 and ietf-05 . . . . . . . . . . 54
14.2. Changes between ietf-02 and ietf-03 . . . . . . . . . . 54 14.2. Changes between ietf-03 and ietf-04 . . . . . . . . . . 54
14.3. Changes between ietf-01 and ietf-02 . . . . . . . . . . 54 14.3. Changes between ietf-02 and ietf-03 . . . . . . . . . . 54
14.4. Changes between ietf-00 and ietf-01 . . . . . . . . . . 54 14.4. Changes between ietf-01 and ietf-02 . . . . . . . . . . 54
14.5. Changes between -01 and -02 . . . . . . . . . . . . . . 55 14.5. Changes between ietf-00 and ietf-01 . . . . . . . . . . 54
14.6. Changes between -00 and -01 . . . . . . . . . . . . . . 55 14.6. Changes between -01 and -02 . . . . . . . . . . . . . . 55
14.7. Changes between -00 and -01 . . . . . . . . . . . . . . 55
15. References . . . . . . . . . . . . . . . . . . . . . . . . . 55 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 55
15.1. Normative References . . . . . . . . . . . . . . . . . . 55 15.1. Normative References . . . . . . . . . . . . . . . . . . 55
15.2. Informative References . . . . . . . . . . . . . . . . . 56 15.2. Informative References . . . . . . . . . . . . . . . . . 56
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 56 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 57
1. Introduction 1. Introduction
The purpose of the protocols in this document, collectively referred The purpose of the protocols in this document, collectively referred
to as CT Gossip, is to detect certain misbehavior by CT logs. In to as CT Gossip, is to detect certain misbehavior by CT logs. In
particular, CT Gossip aims to detect logs that are providing particular, CT Gossip aims to detect logs that are providing
inconsistent views to different log clients, and logs failing to inconsistent views to different log clients, and logs failing to
include submitted certificates within the time period stipulated by include submitted certificates within the time period stipulated by
MMD. MMD.
skipping to change at page 4, line 23 skipping to change at page 4, line 26
case, each client might see a unique view of the log. case, each client might see a unique view of the log.
The CT logs are public, append-only and untrusted and thus have to be The CT logs are public, append-only and untrusted and thus have to be
audited for consistency, i.e., they should never rewrite history. audited for consistency, i.e., they should never rewrite history.
Additionally, auditors and other log clients need to exchange Additionally, auditors and other log clients need to exchange
information about logs in order to be able to detect a partitioning information about logs in order to be able to detect a partitioning
attack (as described above). attack (as described above).
Gossiping about log behavior helps address the problem of detecting Gossiping about log behavior helps address the problem of detecting
malicious or compromised logs with respect to a partitioning attack. malicious or compromised logs with respect to a partitioning attack.
We want some side of the partitioned tree, and ideally both sides, to We want some side of the partitioned tree, and ideally all sides, to
see the other side. see at least one other side.
Disseminating information about a log poses a potential threat to the Disseminating information about a log poses a potential threat to the
privacy of end users. Some data of interest (e.g. SCTs) is linkable privacy of end users. Some data of interest (e.g., SCTs) is linkable
to specific log entries and thereby to specific websites, which makes to specific log entries and thereby to specific websites, which makes
sharing them with others a privacy concern. Gossiping about this sharing them with others a privacy concern. Gossiping about this
data has to take privacy considerations into account in order not to data has to take privacy considerations into account in order not to
expose associations between users of the log (e.g., web browsers) and expose associations between users of the log (e.g., web browsers) and
certificate holders (e.g., web sites). Even sharing STHs (which do certificate holders (e.g., web sites). Even sharing STHs (which do
not link to specific log entries) can be problematic - user tracking not link to specific log entries) can be problematic - user tracking
by fingerprinting through rare STHs is one potential attack (see by fingerprinting through rare STHs is one potential attack (see
Section 8.2). Section 8.2).
3. Overview 3. Overview
This document presents three gossiping mechanisms: SCT Feedback, STH This document presents three gossiping mechanisms: SCT Feedback, STH
Pollination, and a Trusted Auditor Relationship. Pollination, and a Trusted Auditor Relationship.
SCT Feedback enables HTTPS clients to share Signed Certificate SCT Feedback enables HTTPS clients to share Signed Certificate
Timestamps (SCTs) (Section 3.3 of [RFC-6962-BIS-09]) with CT auditors Timestamps (SCTs) (Section 4.8 of [RFC-6962-BIS-27]) with CT auditors
in a privacy-preserving manner by sending SCTs to originating HTTPS in a privacy-preserving manner by sending SCTs to originating HTTPS
servers, who in turn share them with CT auditors. servers, which in turn share them with CT auditors.
In STH Pollination, HTTPS clients use HTTPS servers as pools to share In STH Pollination, HTTPS clients use HTTPS servers as pools to share
Signed Tree Heads (STHs) (Section 3.6 of [RFC-6962-BIS-09]) with Signed Tree Heads (STHs) (Section 4.10 of [RFC-6962-BIS-27]) with
other connecting clients in the hope that STHs will find their way to other connecting clients in the hope that STHs will find their way to
CT auditors. CT auditors.
HTTPS clients in a Trusted Auditor Relationship share SCTs and STHs HTTPS clients in a Trusted Auditor Relationship share SCTs and STHs
with trusted CT auditors directly, with expectations of privacy with trusted CT auditors directly, with expectations of privacy
sensitive data being handled according to whatever privacy policy is sensitive data being handled according to whatever privacy policy is
agreed on between client and trusted party. agreed on between client and trusted party.
Despite the privacy risks with sharing SCTs there is no loss in Despite the privacy risks with sharing SCTs there is no loss in
privacy if a client sends SCTs for a given site to the site privacy if a client sends SCTs for a given site to the site
corresponding to the SCT. This is because the site's logs would corresponding to the SCT. This is because the site's cookies could
already indicate that the client is accessing that site. In this way already indicate that the client had accessed that site. In this way
a site can accumulate records of SCTs that have been issued by a site can accumulate records of SCTs that have been issued by
various logs for that site, providing a consolidated repository of various logs for that site, providing a consolidated repository of
SCTs that could be shared with auditors. Auditors can use this SCTs that could be shared with auditors. Auditors can use this
information to detect a misbehaving log that fails to include a information to detect a misbehaving log that fails to include a
certificate within the time period stipulated by its MMD metadata. certificate within the time period stipulated by its MMD log
parameter.
Sharing an STH is considered reasonably safe from a privacy Sharing an STH is considered reasonably safe from a privacy
perspective as long as the same STH is shared by a large number of perspective as long as the same STH is shared by a large number of
other log clients. This safety in numbers can be achieved by only other log clients. This safety in numbers can be achieved by only
allowing gossiping of STHs issued in a certain window of time, while allowing gossiping of STHs issued in a certain window of time, while
also refusing to gossip about STHs from logs with too high an STH also refusing to gossip about STHs from logs with too high an STH
issuance frequency (see Section 8.2). issuance frequency (see Section 8.2).
4. Terminology 4. Terminology
This document relies on terminology and data structures defined in This document relies on terminology and data structures defined in
[RFC-6962-BIS-09], including MMD, STH, SCT, Version, LogID, SCT [RFC-6962-BIS-27], including MMD, STH, SCT, Version, LogID, SCT
timestamp, CtExtensions, SCT signature, Merkle Tree Hash. timestamp, CtExtensions, SCT signature, Merkle Tree Hash.
This document relies on terminology defined in This document relies on terminology defined in
[draft-ietf-trans-threat-analysis-03], including Auditing. [draft-ietf-trans-threat-analysis-12], including Auditing.
4.1. Pre-Loaded vs Locally Added Anchors 4.1. Pre-Loaded vs Locally Added Anchors
Through the document, we refer to both Trust Anchors (Certificate Through the document, we refer to both Trust Anchors (Certificate
Authorities) and Logs. Both Logs and Trust Anchors may be locally Authorities) and Logs. Both Logs and Trust Anchors may be locally
added by an administrator. Unless otherwise clarified, in both cases added by an administrator. Unless otherwise clarified, in both cases
we refer to the set of Trust Anchors and Logs that come pre-loaded we refer to the set of Trust Anchors and Logs that come pre-loaded
and pre-trusted in a piece of client software. and pre-trusted in a piece of client software.
5. Who gossips with whom 5. Who gossips with whom
skipping to change at page 5, line 49 skipping to change at page 6, line 4
Authorities) and Logs. Both Logs and Trust Anchors may be locally Authorities) and Logs. Both Logs and Trust Anchors may be locally
added by an administrator. Unless otherwise clarified, in both cases added by an administrator. Unless otherwise clarified, in both cases
we refer to the set of Trust Anchors and Logs that come pre-loaded we refer to the set of Trust Anchors and Logs that come pre-loaded
and pre-trusted in a piece of client software. and pre-trusted in a piece of client software.
5. Who gossips with whom 5. Who gossips with whom
o HTTPS clients and servers (SCT Feedback and STH Pollination) o HTTPS clients and servers (SCT Feedback and STH Pollination)
o HTTPS servers and CT auditors (SCT Feedback and STH Pollination) o HTTPS servers and CT auditors (SCT Feedback and STH Pollination)
o CT auditors (Trusted Auditor Relationship) o CT auditors (Trusted Auditor Relationship)
Additionally, some HTTPS clients may engage with an auditor who they
trust with their privacy: Additionally, some HTTPS clients may engage with an auditor which
they trust with their privacy:
o HTTPS clients and CT auditors (Trusted Auditor Relationship) o HTTPS clients and CT auditors (Trusted Auditor Relationship)
6. What to gossip about and how 6. What to gossip about and how
There are three separate gossip streams: There are three separate gossip streams:
o SCT Feedback - transporting SCTs and certificate chains from HTTPS o SCT Feedback - transporting SCTs and certificate chains from HTTPS
clients to CT auditors via HTTPS servers. clients to CT auditors via HTTPS servers.
skipping to change at page 7, line 16 skipping to change at page 7, line 16
| | CA | ----------+ | | CA | ----------+
| + SCT -> +----------+ | | + SCT -> +----------+ |
v | Cert [& SCT] v | Cert [& SCT]
+----------+ | +----------+ |
| Log | ---------- SCT -----------+ | Log | ---------- SCT -----------+
+----------+ v +----------+ v
| ^ +----------+ | ^ +----------+
| | SCTs & Certs --- | Website | | | SCTs & Certs --- | Website |
| |[1] | +----------+ | |[1] | +----------+
| |[2] STHs ^ | | |[2] STHs ^ |
| |[3] v | | | |[3] v | HTTPS traffic
| | +----------+ | | | | +----------+ | |
| +--------> | Auditor | | HTTPS traffic | +--------> | Auditor | | SCT & Cert
| +----------+ | | | +----------+ | |
STH | SCT & Cert STH | STH & Inclusion proof
| SCTs & Certs | | | |
Log entries | | Log entries SCTs & Certs |
| STHs STHs | | |
v | | v STHs |
+----------+ | v +----------+ | v
| Monitor | +----------+ | Monitor | +----------+
+----------+ | Browser | +----------+ | Browser |
+----------+ +----------+
# Auditor Log # Auditor Log
[1] |--- get-sth ------------------->| [1] |--- get-sth ------------------->|
|<-- STH ------------------------| |<-- STH ------------------------|
[2] |--- leaf hash + tree size ----->| [2] |--- leaf hash + tree size ----->|
|<-- index + inclusion proof --->| |<-- index + inclusion proof --->|
skipping to change at page 8, line 26 skipping to change at page 8, line 26
HTTPS servers store SCTs and certificate chains received from HTTPS servers store SCTs and certificate chains received from
clients, as described in Section 8.1.3. They later share them with clients, as described in Section 8.1.3. They later share them with
CT auditors by either posting them to auditors or making them CT auditors by either posting them to auditors or making them
available via a well-known URL. This is described in Section 8.1.4. available via a well-known URL. This is described in Section 8.1.4.
8.1.1. SCT Feedback data format 8.1.1. SCT Feedback data format
The data shared between HTTPS clients and servers, as well as between The data shared between HTTPS clients and servers, as well as between
HTTPS servers and CT auditors, is a JSON array [RFC7159]. Each item HTTPS servers and CT auditors, is a JSON array [RFC7159]. Each item
in the array is a JSON object with the following content: in the array is a JSON object containing at least the first of the
following members:
o x509_chain: An array of PEM-encoded X.509 certificates. The first o "x509_chain" : An array of PEM-encoded X.509 certificates. The
element is the end-entity certificate, the second certifies the first element is the end-entity certificate, the second certifies
first and so on. the first and so on. The "x509_chain" member is mandatory to
include.
o sct_data: An array of objects consisting of the base64 o "sct_data_v1" : An array of base64 encoded
representation of the binary SCT data as defined in "SignedCertificateTimestampList"s as defined in [RFC6962] section
[RFC-6962-BIS-09] Section 3.3. 3.3. The "sct_data_v1" member is optional.
o "sct_data_v2" : An array of base64 encoded "TransItem" structures
of type "x509_sct_v2" or "precert_sct_v2" as defined in
[RFC-6962-BIS-27] section 4.8. The "sct_data_v2" member is
optional.
We will refer to this object as 'sct_feedback'. We will refer to this object as 'sct_feedback'.
The x509_chain element always contains a full chain from a leaf The x509_chain element always contains a full chain from a leaf
certificate to a self-signed trust anchor. certificate to a self-signed trust anchor.
See Section 8.1.2 for details on what the sct_data element contains See Section 8.1.2 for details on what the sct_data element contains
as well as more details about the x509_chain element. as well as more details about the x509_chain element.
8.1.2. HTTPS client to server 8.1.2. HTTPS client to server
When an HTTPS client connects to an HTTPS server, the client receives When an HTTPS client connects to an HTTPS server, the client receives
a set of SCTs as part of the TLS handshake. SCTs are included in the a set of SCTs as part of the TLS handshake. SCTs are included in the
TLS handshake using one or more of the three mechanisms described in TLS handshake using one or more of the three mechanisms described in
[RFC-6962-BIS-09] section 3.4 - in the server certificate, in a TLS [RFC-6962-BIS-27] section 6 - in the server certificate, in a TLS
extension, or in an OCSP extension. The client MUST discard SCTs extension, or in an OCSP extension. The client MUST discard SCTs
that are not signed by a log known to the client and SHOULD store the that are not signed by a log known to the client and SHOULD store the
remaining SCTs together with a locally constructed certificate chain remaining SCTs together with a locally constructed certificate chain
which is trusted (i.e. terminated in a pre-loaded or locally which is trusted (i.e., terminated in a pre-loaded or locally
installed Trust Anchor) in an sct_feedback object or equivalent data installed Trust Anchor) in an sct_feedback object or equivalent data
structure for later use in SCT Feedback. structure for later use in SCT Feedback.
The SCTs stored on the client MUST be keyed by the exact domain name The SCTs stored on the client MUST be keyed by the exact domain name
the client contacted. They MUST NOT be sent to any domain not the client contacted. They MUST NOT be sent to the well-known URI of
matching the original domain (e.g. if the original domain is any domain not matching the original domain (e.g., if the original
sub.example.com they must not be sent to sub.sub.example.com or to domain is sub.example.com they must not be sent to
example.com.) They MUST NOT be sent to any Subject Alternate Names sub.sub.example.com or to example.com.) In particular, they MUST NOT
be sent to the well-known URI of any Subject Alternate Names
specified in the certificate. In the case of certificates that specified in the certificate. In the case of certificates that
validate multiple domain names, the same SCT is expected to be stored validate multiple domain names, after visiting a second domain name
multiple times. specified in the certificate, the same SCT is expected to be stored
once under each domain name's key. If Connection Reuse as defined in
[RFC7540] is available, reusing an existing connection to
sub.example.com to send data to sub.sub.example.com is permitted.
Not following these constraints would increase the risk for two types Not following these constraints would increase the risk for two types
of privacy breaches. First, the HTTPS server receiving the SCT would of privacy breaches. First, the HTTPS server receiving the SCT would
learn about other sites visited by the HTTPS client. Second, learn about other sites visited by the HTTPS client. Second,
auditors receiving SCTs from the HTTPS server would learn information auditors receiving SCTs from the HTTPS server would learn information
about other HTTPS servers visited by its clients. about other HTTPS servers visited by its clients.
If the client later again connects to the same HTTPS server, it again If the client later again connects to the same HTTPS server, it again
receives a set of SCTs and calculates a certificate chain, and again receives a set of SCTs and calculates a certificate chain, and again
creates an sct_feedback or similar object. If this object does not creates an sct_feedback or similar object. If this object does not
skipping to change at page 10, line 4 skipping to change at page 10, line 17
Refer to Section 11.3 for recommendations for implementation. Refer to Section 11.3 for recommendations for implementation.
Because SCTs can be used as a tracking mechanism (see Because SCTs can be used as a tracking mechanism (see
Section 10.5.2), they deserve special treatment when they are Section 10.5.2), they deserve special treatment when they are
received from (and provided to) domains that are loaded as received from (and provided to) domains that are loaded as
subresources from an origin domain. Such domains are commonly called subresources from an origin domain. Such domains are commonly called
'third party domains'. An HTTPS client SHOULD store SCT Feedback 'third party domains'. An HTTPS client SHOULD store SCT Feedback
using a 'double-keying' approach, which isolates third party domains using a 'double-keying' approach, which isolates third party domains
by the first party domain. This is described in [double-keying]. by the first party domain. This is described in [double-keying].
Gossip would be performed normally for third party domains only when Gossip would be performed normally for third party domains only when
the user revisits the first party domain. In lieu of 'double- the user revisits the first party domain. In lieu of 'double-
keying', an HTTPS client MAY treat SCT Feedback in the same manner it keying', an HTTPS client MAY treat SCT Feedback in the same manner it
treats other security mechanisms that can enable tracking (such as treats other security mechanisms that can enable tracking (such as
HSTS and HPKP.) HSTS and HPKP.)
If the HTTPS client has configuration options for not sending cookies SCT Feedback is only performed when a user connects to a site via
to third parties, SCTs of third parties MUST be treated as cookies intentional web browsing or normal third party resource inclusion.
with respect to this setting. This prevents third party tracking It MUST NOT be performed automatically as part of some sort of
through the use of SCTs/certificates, which would bypass the cookie background process.
policy. For domains that are only loaded as third party domains, the
client may never perform SCT Feedback; however the client may perform Finally, if the HTTPS client has configuration options for not
STH Pollination after fetching an inclusion proof, as specified in sending cookies to third parties, SCTs of third parties MUST be
Section 8.2. treated as cookies with respect to this setting. This prevents third
party tracking through the use of SCTs/certificates, which would
bypass the cookie policy. For domains that are only loaded as third
party domains, the client may never perform SCT Feedback; however the
client may perform STH Pollination after fetching an inclusion proof,
as specified in Section 8.2.
SCTs and corresponding certificates are POSTed to the originating SCTs and corresponding certificates are POSTed to the originating
HTTPS server at the well-known URL: HTTPS server at the well-known URL:
https://<domain>/.well-known/ct-gossip/v1/sct-feedback https://<domain>/.well-known/ct-gossip/v1/sct-feedback
The data sent in the POST is defined in Section 8.1.1. This data The data sent in the POST is defined in Section 8.1.1. This data
SHOULD be sent in an already-established TLS session. This makes it SHOULD be sent in an already-established TLS session. This makes it
hard for an attacker to disrupt SCT Feedback without also disturbing hard for an attacker to disrupt SCT Feedback without also disturbing
ordinary secure browsing (https://). This is discussed more in ordinary secure browsing (https://). This is discussed more in
Section 11.1.1. Section 11.1.1.
The HTTPS server SHOULD respond with an HTTP 200 response code and an The HTTPS server SHOULD respond with an HTTP 200 response code and an
empty body if it was able to process the request. An HTTPS client empty body if it was able to process the request. An HTTPS client
who receives any other response SHOULD consider it an error. which receives any other response SHOULD consider it an error.
Some clients have trust anchors or logs that are locally added (e.g. Some clients have trust anchors or logs that are locally added (e.g.,
by an administrator or by the user themselves). These additions are by an administrator or by the user themselves). These additions are
potentially privacy-sensitive because they can carry information potentially privacy-sensitive because they can carry information
about the specific configuration, computer, or user. about the specific configuration, computer, or user.
Certificates validated by locally added trust anchors will commonly Certificates validated by locally added trust anchors will commonly
have no SCTs associated with them, so in this case no action is have no SCTs associated with them, so in this case no action is
needed with respect to CT Gossip. SCTs issued by locally added logs needed with respect to CT Gossip. SCTs issued by locally added logs
MUST NOT be reported via SCT Feedback. MUST NOT be reported via SCT Feedback.
If a certificate is validated by SCTs that are issued by publicly If a certificate is validated by SCTs that are issued by publicly
skipping to change at page 11, line 31 skipping to change at page 11, line 50
will only track leaf certificates and SCTs applicable to those leaf will only track leaf certificates and SCTs applicable to those leaf
certificates. In the more complex mode, the server will confirm the certificates. In the more complex mode, the server will confirm the
client's chain validation and store the certificate chain. The client's chain validation and store the certificate chain. The
latter mode requires more configuration, but is necessary to prevent latter mode requires more configuration, but is necessary to prevent
denial of service (DoS) attacks on the server's storage space. denial of service (DoS) attacks on the server's storage space.
In the simple mode of operation, upon receiving a submission at the In the simple mode of operation, upon receiving a submission at the
sct-feedback well-known URL, an HTTPS server will perform a set of sct-feedback well-known URL, an HTTPS server will perform a set of
operations, checking on each sct_feedback object before storing it: operations, checking on each sct_feedback object before storing it:
1. the HTTPS server MAY modify the sct_feedback object, and discard o (1) the HTTPS server MAY modify the sct_feedback object, and
all items in the x509_chain array except the first item (which is discard all items in the x509_chain array except the first item
the end-entity certificate) (which is the end-entity certificate)
2. if a bit-wise compare of the sct_feedback object matches one o (2) if a bit-wise compare of the sct_feedback object matches one
already in the store, this sct_feedback object SHOULD be already in the store, this sct_feedback object SHOULD be discarded
discarded
3. if the leaf cert is not for a domain for which the server is o (3) if the leaf cert is not for a domain for which the server is
authoritative, the SCT MUST be discarded authoritative, the SCT MUST be discarded
4. if an SCT in the sct_data array can't be verified to be a valid o (4) if an SCT in the sct_data array can't be verified to be a
SCT for the accompanying leaf cert, and issued by a known log, valid SCT for the accompanying leaf cert, and issued by a known
the individual SCT SHOULD be discarded log, the individual SCT SHOULD be discarded
The modification in step number 1 is necessary to prevent a malicious The modification in step number 1 is necessary to prevent a malicious
client from exhausting the server's storage space. A client can client from exhausting the server's storage space. A client can
generate their own issuing certificate authorities, and create an generate their own issuing certificate authorities, and create an
arbitrary number of chains that terminate in an end-entity arbitrary number of chains that terminate in an end-entity
certificate with an existing SCT. By discarding all but the end- certificate with an existing SCT. By discarding all but the end-
entity certificate, we prevent a simple HTTPS server from storing entity certificate, we prevent a simple HTTPS server from storing
this data. Note that operation in this mode will not prevent the this data. Note that operation in this mode will not prevent the
attack described in [dual-ca-compromise-attack]. Skipping this step attack described in [dual-ca-compromise-attack]. Skipping this step
requires additional configuration as described below. requires additional configuration as described below.
skipping to change at page 12, line 22 skipping to change at page 12, line 39
MAY be performed. If the certificate chain exactly matches an MAY be performed. If the certificate chain exactly matches an
existing certificate chain, the server MAY store the union of the existing certificate chain, the server MAY store the union of the
SCTs from the two objects in the first (existing) object. If the SCTs from the two objects in the first (existing) object. If the
validity check on any of the SCTs fails, the server SHOULD NOT store validity check on any of the SCTs fails, the server SHOULD NOT store
the union of the SCTs. the union of the SCTs.
The check in step 3 is to help malfunctioning clients from exposing The check in step 3 is to help malfunctioning clients from exposing
which sites they visit. It additionally helps prevent DoS attacks on which sites they visit. It additionally helps prevent DoS attacks on
the server. the server.
[ Note: Thinking about building this, how does the SCT Feedback app
know which sites it's authoritative for? It will need that amount of
configuration at least. ]
The check in step 4 is to prevent DoS attacks where an adversary The check in step 4 is to prevent DoS attacks where an adversary
fills up the store prior to attacking a client (thus preventing the fills up the store prior to attacking a client (thus preventing the
client's feedback from being recorded), or an attack where an client's feedback from being recorded), or an attack where an
adversary simply attempts to fill up server's storage space. adversary simply attempts to fill up server's storage space.
The above describes the simpler mode of operation. In the more The above describes the simpler mode of operation. In the more
advanced server mode, the server will detect the attack described in advanced server mode, the server will detect the attack described in
[dual-ca-compromise-attack]. In this configuration the server will [dual-ca-compromise-attack]. In this configuration the server will
not modify the sct_feedback object prior to performing checks 2, 3, not modify the sct_feedback object prior to performing checks 2, 3,
and 4. and 4. Instead, to prevent a malicious client from filling the
server's data store, the HTTPS server SHOULD perform an additional
To prevent a malicious client from filling the server's data store, check in the more advanced mode:
the HTTPS server SHOULD perform an additional check in the more
advanced mode:
o if the x509_chain consists of an invalid certificate chain, or the o (5) if the x509_chain consists of an invalid certificate chain, or
culminating trust anchor is not recognized by the server, the the culminating trust anchor is not recognized by the server, the
server SHOULD modify the sct_feedback object, discarding all items server SHOULD modify the sct_feedback object, discarding all items
in the x509_chain array except the first item in the x509_chain array except the first item
The HTTPS server MAY choose to omit checks 4 or 5. This will place The HTTPS server MAY choose to omit checks 4 or 5. This will place
the server at risk of having its data store filled up by invalid the server at risk of having its data store filled up by invalid
data, but can also allow a server to identify interesting certificate data, but can also allow a server to identify interesting certificate
or certificate chains that omit valid SCTs, or do not chain to a or certificate chains that omit valid SCTs, or do not chain to a
trusted root. This information may enable an HTTPS server operator trusted root. This information may enable an HTTPS server operator
to detect attacks or unusual behavior of Certificate Authorities even to detect attacks or unusual behavior of Certificate Authorities even
outside the Certificate Transparency ecosystem. outside the Certificate Transparency ecosystem.
skipping to change at page 15, line 16 skipping to change at page 15, line 28
happen is regarded as undefined policy with the exception of privacy happen is regarded as undefined policy with the exception of privacy
concerns explained below. Suggestions for the policy can be found in concerns explained below. Suggestions for the policy can be found in
Section 11.3. Section 11.3.
An HTTPS client could be tracked by giving it a unique or rare STH. An HTTPS client could be tracked by giving it a unique or rare STH.
To address this concern, we place restrictions on different To address this concern, we place restrictions on different
components of the system to ensure an STH will not be rare. components of the system to ensure an STH will not be rare.
o HTTPS clients silently ignore STHs from logs with an STH issuance o HTTPS clients silently ignore STHs from logs with an STH issuance
frequency of more than one STH per hour. Logs use the STH frequency of more than one STH per hour. Logs use the STH
Frequency Count metadata to express this ([RFC-6962-BIS-09] Frequency Count log parameter to express this ([RFC-6962-BIS-27]
sections 3.6 and 5.1). section 4.1).
o HTTPS clients silently ignore STHs which are not fresh. o HTTPS clients silently ignore STHs which are not fresh.
An STH is considered fresh iff its timestamp is less than 14 days in An STH is considered fresh iff its timestamp is less than 14 days in
the past. Given a maximum STH issuance rate of one per hour, an the past. Given a maximum STH issuance rate of one per hour, an
attacker has 336 unique STHs per log for tracking. Clients MUST attacker has 336 unique STHs per log for tracking. Clients MUST
ignore STHs older than 14 days. We consider STHs within this ignore STHs older than 14 days. We consider STHs within this
validity window not to be personally identifiable data, and STHs validity window not to be personally identifiable data, and STHs
outside this window to be personally identifiable. outside this window to be personally identifiable.
skipping to change at page 15, line 39 skipping to change at page 15, line 51
STHs, this number of unique STHs grow and the negative privacy STHs, this number of unique STHs grow and the negative privacy
implications grow with it. It's important that this is taken into implications grow with it. It's important that this is taken into
account when logs are chosen for default settings in HTTPS clients. account when logs are chosen for default settings in HTTPS clients.
This concern is discussed upon in Section 10.5.5. This concern is discussed upon in Section 10.5.5.
A log may cease operation, in which case there will soon be no STH A log may cease operation, in which case there will soon be no STH
within the validity window. Clients SHOULD perform all three methods within the validity window. Clients SHOULD perform all three methods
of gossip about a log that has ceased operation since it is possible of gossip about a log that has ceased operation since it is possible
the log was still compromised and gossip can detect that. STH the log was still compromised and gossip can detect that. STH
Pollination is the one mechanism where a client must know about a log Pollination is the one mechanism where a client must know about a log
shutdown. A client who does not know about a log shutdown MUST NOT shutdown. A client which does not know about a log shutdown MUST NOT
attempt any heuristic to detect a shutdown. Instead the client MUST attempt any heuristic to detect a shutdown. Instead the client MUST
be informed about the shutdown from a verifiable source (e.g. a be informed about the shutdown from a verifiable source (e.g., a
software update). The client SHOULD be provided the final STH issued software update), and be provided the final STH issued by the log.
by the log and SHOULD resolve SCTs and STHs to this final STH. If an The client SHOULD resolve SCTs and STHs to this final STH. If an SCT
SCT or STH cannot be resolved to the final STH, clients SHOULD follow or STH cannot be resolved to the final STH, clients SHOULD follow the
the requirements and recommendations set forth in Section 11.1.2. requirements and recommendations set forth in Section 11.1.2.
8.2.1. HTTPS Clients and Proof Fetching 8.2.1. HTTPS Clients and Proof Fetching
There are two types of proofs a client may retrieve; inclusion proofs There are two types of proofs a client may retrieve; inclusion proofs
and consistency proofs. and consistency proofs.
An HTTPS client will retrieve SCTs together with certificate chains An HTTPS client will retrieve SCTs together with certificate chains
from an HTTPS server. Using the timestamp in the SCT together with from an HTTPS server. Using the timestamp in the SCT together with
the end-entity certificate and the issuer key hash, it can obtain an the end-entity certificate and the issuer key hash, it can obtain an
inclusion proof to an STH in order to verify the promise made by the inclusion proof to an STH in order to verify the promise made by the
skipping to change at page 16, line 20 skipping to change at page 16, line 30
An HTTPS client will have STHs from performing STH Pollination, and An HTTPS client will have STHs from performing STH Pollination, and
may obtain a consistency proof to a more recent STH. may obtain a consistency proof to a more recent STH.
An HTTPS client may also receive an SCT bundled with an inclusion An HTTPS client may also receive an SCT bundled with an inclusion
proof to a historical STH via an unspecified future mechanism. proof to a historical STH via an unspecified future mechanism.
Because this historical STH is considered personally identifiable Because this historical STH is considered personally identifiable
information per above, the client needs to obtain a consistency proof information per above, the client needs to obtain a consistency proof
to a more recent STH. to a more recent STH.
A client SHOULD perform proof fetching. A client MUST NOT perform A client SHOULD attempt proof fetching. A client MAY do network
proof fetching for any SCTs or STHs issued by a locally added log. A probing to determine if proof fetching may succeed, and if it learns
client MAY fetch an inclusion proof for an SCT (issued by a pre- that it does not, SHOULD periodically re-probe (especially after
loaded log) that validates a certificate chaining to a locally added network change, if it is aware of these events.) If it does succeed,
trust anchor. queued events can be processed.
A client MUST NOT perform proof fetching for any SCTs or STHs issued
by a locally added log. A client MAY fetch an inclusion proof for an
SCT (issued by a pre-loaded log) that validates a certificate
chaining to a locally added trust anchor.
If a client requested either proof directly from a log or auditor, it If a client requested either proof directly from a log or auditor, it
would reveal the client's browsing habits to a third party. To would reveal the client's browsing habits to a third party. To
mitigate this risk, an HTTPS client MUST retrieve the proof in a mitigate this risk, an HTTPS client MUST retrieve the proof in a
manner that disguises the client. manner that disguises the client.
Depending on the client's DNS provider, DNS may provide an Depending on the client's DNS provider, DNS may provide an
appropriate intermediate layer that obfuscates the linkability appropriate intermediate layer that obfuscates the linkability
between the user of the client and the request for inclusion (while between the user of the client and the request for inclusion (while
at the same time providing a caching layer for oft-requested at the same time providing a caching layer for oft-requested
skipping to change at page 17, line 22 skipping to change at page 17, line 37
8.2.2. STH Pollination without Proof Fetching 8.2.2. STH Pollination without Proof Fetching
An HTTPS client MAY participate in STH Pollination without fetching An HTTPS client MAY participate in STH Pollination without fetching
proofs. In this situation, the client receives STHs from a server, proofs. In this situation, the client receives STHs from a server,
applies the same validation logic to them (signed by a known log, applies the same validation logic to them (signed by a known log,
within the validity window) and will later pass them to another HTTPS within the validity window) and will later pass them to another HTTPS
server. server.
When operating in this fashion, the HTTPS client is promoting gossip When operating in this fashion, the HTTPS client is promoting gossip
for Certificate Transparency, but derives no direct benefit itself. for Certificate Transparency, but derives no direct benefit itself.
In comparison, a client who resolves SCTs or historical STHs to In comparison, a client which resolves SCTs or historical STHs to
recent STHs and pollinates them is assured that if it was attacked, recent STHs and pollinates them is assured that if it was attacked,
there is a probability that the ecosystem will detect and respond to there is a probability that the ecosystem will detect and respond to
the attack (by distrusting the log). the attack (by distrusting the log).
8.2.3. Auditor Action 8.2.3. Auditor Action
CT auditors participate in STH pollination by retrieving STHs from CT auditors participate in STH pollination by retrieving STHs from
HTTPS servers. They verify that the STH is valid by checking the HTTPS servers. They verify that the STH is valid by checking the
signature, and requesting a consistency proof from the STH to the signature, and requesting a consistency proof from the STH to the
most recent STH. most recent STH.
After retrieving the consistency proof to the most recent STH, they After retrieving the consistency proof to the most recent STH, they
SHOULD pollinate this new STH among participating HTTPS servers. In SHOULD pollinate this new STH among participating HTTPS servers. In
this way, as STHs "age out" and are no longer fresh, their "lineage" this way, as STHs "age out" and are no longer fresh, their "lineage"
continues to be tracked in the system. continues to be tracked in the system.
8.2.4. STH Pollination data format 8.2.4. STH Pollination data format
The data sent from HTTPS clients and CT auditors to HTTPS servers is The data sent from HTTPS clients and CT auditors to HTTPS servers is
a JSON object [RFC7159] with the following content: a JSON object [RFC7159] with one or both of the following two
members:
o sths - an array of 0 or more fresh SignedTreeHeads as defined in o "v1" : array of 0 or more objects each containing an STH as
[RFC-6962-BIS-09] Section 3.6.1. returned from ct/v1/get-sth, see [RFC6962] section 4.3
o "v2" : array of 0 or more objects each containing an STH as
returned from ct/v2/get-sth, see [RFC-6962-BIS-27] section 5.2
Note that all STHs MUST be fresh as defined in Section 8.2.
8.3. Trusted Auditor Stream 8.3. Trusted Auditor Stream
HTTPS clients MAY send SCTs and cert chains, as well as STHs, HTTPS clients MAY send SCTs and cert chains, as well as STHs,
directly to auditors. If sent, this data MAY include data that directly to auditors. If sent, this data MAY include data that
reflects locally added logs or trust anchors. Note that there are reflects locally added logs or trust anchors. Note that there are
privacy implications in doing so, these are outlined in privacy implications in doing so, these are outlined in
Section 10.5.1 and Section 10.5.6. Section 10.5.1 and Section 10.5.6.
The most natural trusted auditor arrangement arguably is a web The most natural trusted auditor arrangement arguably is a web
browser that is "logged in to" a provider of various internet browser that is "logged in to" a provider of various internet
services. Another equivalent arrangement is a trusted party like a services. Another equivalent arrangement is a trusted party like a
corporation to which an employee is connected through a VPN or by corporation to which an employee is connected through a VPN or by
other similar means. A third might be individuals or smaller groups other similar means. A third might be individuals or smaller groups
of people running their own services. In such a setting, retrieving of people running their own services. In such a setting, retrieving
proofs from that third party could be considered reasonable from a proofs from that third party could be considered reasonable from a
privacy perspective. The HTTPS client may also do its own auditing privacy perspective. The HTTPS client may also do its own auditing
and might additionally share SCTs and STHs with the trusted party to and might additionally share SCTs and STHs with the trusted party to
contribute to herd immunity. Here, the ordinary [RFC-6962-BIS-09] contribute to herd immunity. Here, the ordinary [RFC-6962-BIS-27]
protocol is sufficient for the client to do the auditing while SCT protocol is sufficient for the client to do the auditing while SCT
Feedback and STH Pollination can be used in whole or in parts for the Feedback and STH Pollination can be used in whole or in parts for the
gossip part. gossip part.
Another well established trusted party arrangement on the internet Another well established trusted party arrangement on the internet
today is the relation between internet users and their providers of today is the relation between internet users and their providers of
DNS resolver services. DNS resolvers are typically provided by the DNS resolver services. DNS resolvers are typically provided by the
internet service provider (ISP) used, which by the nature of name internet service provider (ISP) used, which by the nature of name
resolving already know a great deal about which sites their users resolving already know a great deal about which sites their users
visit. As mentioned in Section 8.2.1, in order for HTTPS clients to visit. As mentioned in Section 8.2.1, in order for HTTPS clients to
skipping to change at page 20, line 14 skipping to change at page 20, line 35
o Maximize its chance of detecting a misissued certificate by o Maximize its chance of detecting a misissued certificate by
configuring a trust store of CAs configuring a trust store of CAs
o Establish a "push" mechanism for POSTing SCTs to CT auditors o Establish a "push" mechanism for POSTing SCTs to CT auditors
These configuration needs, and the simple fact that it would require These configuration needs, and the simple fact that it would require
some deployment of software, means that some percentage of HTTPS some deployment of software, means that some percentage of HTTPS
servers will not deploy SCT Feedback. servers will not deploy SCT Feedback.
It is worthwhile to note that an attacker may be able to prevent
detection of an attack on a webserver (in all cases) if SCT Feedback
is not implemented. This attack is detailed in Section 10.1).
If SCT Feedback was the only mechanism in the ecosystem, any server If SCT Feedback was the only mechanism in the ecosystem, any server
that did not implement the feature would open itself and its users to that did not implement the feature would open itself and its users to
attack without any possibility of detection. attack without any possibility of detection.
If SCT Feedback is not deployed by a webserver, malicious logs will A webserver not deploying SCT Feedback (or an alternative method
be able to attack all users of the webserver (who do not have a providing equivalent functionality) may never learn that it was a
Trusted Auditor relationship) with impunity. Additionally, users who target of an attack by a malicious log, as described in Section 10.1,
wish to have the strongest measure of privacy protection (by although the presence of an attack by the log could be learned
disabling STH Pollination Proof Fetching and forgoing a Trusted through STH Pollination. Additionally, users who wish to have the
Auditor) could be attacked without risk of detection. strongest measure of privacy protection (by disabling STH Pollination
Proof Fetching and forgoing a Trusted Auditor) could be attacked
without risk of detection.
9.2. STH Pollination 9.2. STH Pollination
STH Pollination requires the cooperation of HTTPS clients, HTTPS STH Pollination requires the cooperation of HTTPS clients, HTTPS
servers, and logs. servers, and logs.
For a client to fully participate in STH Pollination, and have this For a client to fully participate in STH Pollination, and have this
mechanism detect attacks against it, the client must have a way to mechanism detect attacks against it, the client must have a way to
safely perform Proof Fetching in a privacy preserving manner. (The safely perform Proof Fetching in a privacy preserving manner. (The
client may pollinate STHs it receives without performing Proof client may pollinate STHs it receives without performing Proof
skipping to change at page 21, line 4 skipping to change at page 21, line 23
amount of disk space to the endeavor. amount of disk space to the endeavor.
Logs (or a third party mirroring the logs) must provide access to Logs (or a third party mirroring the logs) must provide access to
clients to query proofs in a privacy preserving manner, most likely clients to query proofs in a privacy preserving manner, most likely
through DNS. through DNS.
Unlike SCT Feedback, the STH Pollination mechanism is not hampered if Unlike SCT Feedback, the STH Pollination mechanism is not hampered if
only a minority of HTTPS servers deploy it. However, it makes an only a minority of HTTPS servers deploy it. However, it makes an
assumption that an HTTPS client performs Proof Fetching (such as the assumption that an HTTPS client performs Proof Fetching (such as the
DNS mechanism discussed). Unfortunately, any manner that is DNS mechanism discussed). Unfortunately, any manner that is
anonymous for some (such as clients who use shared DNS services such anonymous for some (such as clients which use shared DNS services
as a large ISP), may not be anonymous for others. such as a large ISP), may not be anonymous for others.
For instance, DNS requests expose a considerable amount of sensitive For instance, DNS requests expose a considerable amount of sensitive
information (including what data is already present in the cache) in information (including what data is already present in the cache) in
plaintext over the network. For this reason, some percentage of plaintext over the network. For this reason, some percentage of
HTTPS clients may choose to not enable the Proof Fetching component HTTPS clients may choose to not enable the Proof Fetching component
of STH Pollination. (Although they can still request and send STHs of STH Pollination. (Although they can still request and send STHs
among participating HTTPS servers, even when this affords them no among participating HTTPS servers, even when this affords them no
direct benefit.) direct benefit.)
If STH Pollination was the only mechanism deployed, users that If STH Pollination was the only mechanism deployed, users that
disable it would be able to be attacked without risk of detection. disable it would be able to be attacked without risk of detection.
If STH Pollination was not deployed, HTTPS clients visiting HTTPS If STH Pollination (or an alternative method providing equivalent
Servers who did not deploy SCT Feedback could be attacked without functionality) was not deployed, HTTPS clients visiting HTTPS Servers
risk of detection. which did not deploy SCT Feedback could be attacked without risk of
detection.
9.3. Trusted Auditor Relationship 9.3. Trusted Auditor Relationship
The Trusted Auditor Relationship is expected to be the rarest gossip The Trusted Auditor Relationship is expected to be the rarest gossip
mechanism, as an HTTPS client is providing an unadulterated report of mechanism, as an HTTPS client is providing an unadulterated report of
its browsing history to a third party. While there are valid and its browsing history to a third party. While there are valid and
common reasons for doing so, there is no appropriate way to enter common reasons for doing so, there is no appropriate way to enter
into this relationship without retrieving informed consent from the into this relationship without retrieving informed consent from the
user. user.
skipping to change at page 22, line 14 skipping to change at page 22, line 31
If the Trusted Auditor relationship was not deployed, crawlers and If the Trusted Auditor relationship was not deployed, crawlers and
organizations would build it themselves for their own needs. By organizations would build it themselves for their own needs. By
standardizing it, users who wish to opt-in (for instance those standardizing it, users who wish to opt-in (for instance those
unwilling to participate fully in STH Pollination) can have an unwilling to participate fully in STH Pollination) can have an
interoperable standard they can use to choose and change their interoperable standard they can use to choose and change their
trusted auditor. trusted auditor.
9.4. Interaction 9.4. Interaction
The interactions of the mechanisms is thus outlined: Assuming no other log consistency measures exist, clients who perform
only a subset of the mechanisms described in this document are
exposed to the following vulnerabilities:
HTTPS clients can be attacked without risk of detection if they do HTTPS clients can be attacked without risk of detection if they do
not participate in any of the three mechanisms. not participate in any of the three mechanisms.
HTTPS clients are afforded the greatest chance of detecting an attack HTTPS clients are afforded the greatest chance of detecting an attack
when they either participate in both SCT Feedback and STH Pollination when they either participate in both SCT Feedback and STH Pollination
with Proof Fetching or if they have a Trusted Auditor relationship. with Proof Fetching or if they have a Trusted Auditor relationship.
(Participating in SCT Feedback is required to prevent a malicious log (Participating in SCT Feedback is the only way specified in this
from refusing to ever resolve an SCT to an STH, as put forward in document to prevent a malicious log from refusing to ever resolve an
Section 10.1). Additionally, participating in SCT Feedback enables SCT to an STH, as put forward in Section 10.1). Additionally,
an HTTPS client to assist in detecting the exact target of an attack. participating in SCT Feedback enables an HTTPS client to assist in
detecting the exact target of an attack.
HTTPS servers that omit SCT Feedback enable malicious logs to carry HTTPS servers that omit SCT Feedback enable malicious logs to carry
out attacks without risk of detection. If these servers are targeted out attacks without risk of detection. If these servers are targeted
specifically, even if the attack is detected, without SCT Feedback specifically, even if the attack is detected, without SCT Feedback
they may never learn that they were specifically targeted. HTTPS they may never learn that they were specifically targeted. HTTPS
servers without SCT Feedback do gain some measure of herd immunity, servers without SCT Feedback do gain some measure of herd immunity,
but only because their clients participate in STH Pollination (with but only because their clients participate in STH Pollination (with
Proof Fetching) or have a Trusted Auditor Relationship. Proof Fetching) or have a Trusted Auditor Relationship.
When HTTPS servers omit SCT feedback, it allows their users to be When HTTPS servers omit SCT feedback, it allows their users to be
attacked without detection by a malicious log; the vulnerable users attacked without detection by a malicious log; the vulnerable users
are those who do not have a Trusted Auditor relationship. are those who do not have a Trusted Auditor relationship.
10. Security considerations 10. Security considerations
10.1. Attacks by actively malicious logs 10.1. Attacks by actively malicious logs
One of the most powerful attacks possible in the CT ecosystem is a One of the most powerful attacks possible in the CT ecosystem is a
trusted log that has actively decided to be malicious. It can carry trusted log that has actively decided to be malicious. It can carry
out an attack in two ways: out an attack in at least two ways:
In the first attack, the log can present a split view of the log for In the first attack, the log can present a split view of the log for
all time. The only way to detect this attack is to resolve each view all time. This attack can be detected by CT auditors, but a naive
of the log to the two most recent STHs and then force the log to auditor implementation may fail to do so. The simplest, least
present a consistency proof. (Which it cannot.) This attack can be efficient way to detect the attack is to mirror the entire log and
detected by CT auditors participating in STH Pollination, as long as assert inclusion of every peice of data. If an auditor does not
they are explicitly built to handle the situation of a log mirror the log, one way to detect this attack is to resolve each view
continuously presenting a split view. of the log to the most recent STHs available and then force the log
to present a consistency proof. (Which it cannot.) We highly
recommend auditors plan for this attack scenario and ensure it will
be detected.
In the second attack, the log can sign an SCT, and refuse to ever In the second attack, the log can sign an SCT, and refuse to ever
include the certificate that the SCT refers to in the tree. include the certificate that the SCT refers to in the tree.
(Alternately, it can include it in a branch of the tree and issue an (Alternately, it can include it in a branch of the tree and issue an
STH, but then abandon that branch.) Whenever someone requests an STH, but then abandon that branch.) Whenever someone requests an
inclusion proof for that SCT (or a consistency proof from that STH), inclusion proof for that SCT (or a consistency proof from that STH),
the log would respond with an error, and a client may simply regard the log would respond with an error, and a client may simply regard
the response as a transient error. This attack can be detected using the response as a transient error. This attack can be detected using
SCT Feedback, or an Auditor of Last Resort, as presented in SCT Feedback, or an Auditor of Last Resort, as presented in
Section 11.1.2. Section 11.1.2.
Both of these attack variants can be detected by CT auditors who have
obtained an STH of an 'abnormal' view of the log. However, they may
not be able to link the STH to any particular SCT or Certificate.
This means that while the log misbehavior was successfully detected,
the target of the attack was not identified. To assertively identify
the target(s) of the attack, SCT Feedback is necessary.
10.2. Dual-CA Compromise 10.2. Dual-CA Compromise
[dual-ca-compromise-attack] describes an attack possible by an [dual-ca-compromise-attack] describes an attack possible by an
adversary who compromises two Certificate Authorities and a Log. This adversary who compromises two Certificate Authorities and a Log. This
attack is difficult to defend against in the CT ecosystem, and attack is difficult to defend against in the CT ecosystem, and
[dual-ca-compromise-attack] describes a few approaches to doing so. [dual-ca-compromise-attack] describes a few approaches to doing so.
We note that Gossip is not intended to defend against this attack, We note that Gossip is not intended to defend against this attack,
but can in certain modes. but can in certain modes.
Defending against the Dual-CA Compromise attack requires SCT Defending against the Dual-CA Compromise attack requires SCT
Feedback, and explicitly requires the server to save full certificate Feedback, and explicitly requires the server to save full certificate
chains (described in Section 8.1.3 as the 'complex' configuration.) chains (described in Section 8.1.3 as the 'complex' configuration.)
After CT auditors receive the full certificate chains from servers, After CT auditors receive the full certificate chains from servers,
they MAY compare the chain built by clients to the chain supplied by they MAY compare the chain built by clients to the chain supplied by
the log. If the chains differ significantly, the auditor SHOULD the log. If the chains differ significantly, the auditor SHOULD
skipping to change at page 23, line 36 skipping to change at page 24, line 19
Defending against the Dual-CA Compromise attack requires SCT Defending against the Dual-CA Compromise attack requires SCT
Feedback, and explicitly requires the server to save full certificate Feedback, and explicitly requires the server to save full certificate
chains (described in Section 8.1.3 as the 'complex' configuration.) chains (described in Section 8.1.3 as the 'complex' configuration.)
After CT auditors receive the full certificate chains from servers, After CT auditors receive the full certificate chains from servers,
they MAY compare the chain built by clients to the chain supplied by they MAY compare the chain built by clients to the chain supplied by
the log. If the chains differ significantly, the auditor SHOULD the log. If the chains differ significantly, the auditor SHOULD
raise a concern. A method of determining if chains differ raise a concern. A method of determining if chains differ
significantly is by asserting that one chain is not a subset of the significantly is by asserting that one chain is not a subset of the
other and that the roots of the chains are different. other and that the roots of the chains are different.
[Note: Justification for this algorithm:
Cross-Signatures could result in a different org being treated as the
'root', but in this case, one chain would be a subset of the other.
Intermediate swapping (e.g. different signature algorithms) could
result in different chains, but the root would be the same.
(Hitting both those cases at once would cause a false positive
though, but this would likely be rare.)
Are there other cases that could occur? (Left for the purposes of
reading during pre-Last Call, to be removed by Editor)]
10.3. Censorship/Blocking considerations 10.3. Censorship/Blocking considerations
We assume a network attacker who is able to fully control the We assume a network attacker who is able to fully control the
client's internet connection for some period of time, including client's internet connection for some period of time, including
selectively blocking requests to certain hosts and truncating TLS selectively blocking requests to certain hosts and truncating TLS
connections based on information observed or guessed about client connections based on information observed or guessed about client
behavior. In order to successfully detect log misbehavior, the behavior. In order to successfully detect log misbehavior, the
gossip mechanisms must still work even in these conditions. gossip mechanisms must still work even in these conditions.
There are several gossip connections that can be blocked: There are several gossip connections that can be blocked:
skipping to change at page 25, line 39 skipping to change at page 26, line 4
window SHOULD NOT be deleted. An attacker cannot flush an item from window SHOULD NOT be deleted. An attacker cannot flush an item from
the cache if it is never removed so flushing attacks are completely the cache if it is never removed so flushing attacks are completely
mitigated. mitigated.
The required disk space for all STHs within the validity window is The required disk space for all STHs within the validity window is
336 STHs per log that is trusted. If 20 logs are trusted, and each 336 STHs per log that is trusted. If 20 logs are trusted, and each
STH takes 1 Kilobytes, this is 6.56 Megabytes. STH takes 1 Kilobytes, this is 6.56 Megabytes.
Note that it is important that implementors do not calculate the Note that it is important that implementors do not calculate the
exact size of cache expected - if an attack does occur, a small exact size of cache expected - if an attack does occur, a small
number of additional STHs will enter into the cache. These STHs will number of additional, fraudulent STHs will enter into the cache.
be in addition to the expected set, and will be evidence of the These STHs will be in addition to the expected set, and will be
attack. evidence of the attack. Flooding the cache will not work, as an
attacker would have to include fraudulent STHs in the flood.
If an HTTPS client or HTTPS server is operating in a constrained If an HTTPS client or HTTPS server is operating in a constrained
environment and cannot devote enough storage space to hold all STHs environment and cannot devote enough storage space to hold all STHs
within the validity window it is recommended to use the below within the validity window it is recommended to use the below
Deletion Algorithm Section 11.3.2 to make it more difficult for the Deletion Algorithm in section Section 11.3.2 to make it more
attacker to perform a flushing attack. difficult for the attacker to perform a flushing attack.
10.4.2. SCTs & Certificate Chains on HTTPS Servers 10.4.2. SCTs & Certificate Chains on HTTPS Servers
An HTTPS server will only accept SCTs and Certificate Chains for An HTTPS server will only accept SCTs and Certificate Chains for
domains it is authoritative for. Therefore the storage space needed domains it is authoritative for. Therefore the storage space needed
is bound by the number of logs it accepts, multiplied by the number is bound by the number of logs it accepts, multiplied by the number
of domains it is authoritative for, multiplied by the number of of domains it is authoritative for, multiplied by the number of
certificates issued for those domains. certificates issued for those domains.
Imagine a server authoritative for 10,000 domains, and each domain Imagine a server authoritative for 10,000 domains, and each domain
skipping to change at page 26, line 35 skipping to change at page 26, line 45
SCT(s) and Certificate Chain(s) will enter into the cache. This data SCT(s) and Certificate Chain(s) will enter into the cache. This data
will be in addition to the expected set, and will be evidence of the will be in addition to the expected set, and will be evidence of the
attack. attack.
If an HTTPS server is operating in a constrained environment and If an HTTPS server is operating in a constrained environment and
cannot devote enough storage space to hold all SCTs and Certificate cannot devote enough storage space to hold all SCTs and Certificate
Chains it is authoritative for it is recommended to configure the SCT Chains it is authoritative for it is recommended to configure the SCT
Feedback mechanism to allow only certain certificates that are known Feedback mechanism to allow only certain certificates that are known
to be valid. These chains and SCTs can then be discarded without to be valid. These chains and SCTs can then be discarded without
being stored or subsequently provided to any clients or auditors. If being stored or subsequently provided to any clients or auditors. If
the allowlist is not sufficient, the below Deletion Algorithm the allowlist is not sufficient, the below Deletion Algorithm in
Section 11.3.2 is recommended to make it more difficult for the Section 11.3.2 is recommended to make it more difficult for the
attacker to perform a flushing attack. attacker to perform a flushing attack.
10.4.3. SCTs & Certificate Chains on HTTPS Clients 10.4.3. SCTs & Certificate Chains on HTTPS Clients
HTTPS clients will accumulate SCTs and Certificate Chains without HTTPS clients will accumulate SCTs and Certificate Chains without
bound. It is expected they will choose a particular cache size and bound. It is expected they will choose a particular cache size and
delete entries when the cache size meets its limit. This does not delete entries when the cache size meets its limit. This does not
mitigate flushing attacks, and such an attack is documented in mitigate flushing attacks, and such an attack is documented in
[gossip-mixing]. [gossip-mixing].
skipping to change at page 27, line 29 skipping to change at page 27, line 40
is better to err on the side of safety, and not assume a client is is better to err on the side of safety, and not assume a client is
okay with giving up its privacy. okay with giving up its privacy.
10.5.1. Privacy and SCTs 10.5.1. Privacy and SCTs
An SCT contains information that links it to a particular web site. An SCT contains information that links it to a particular web site.
Because the client-server relationship is sensitive, gossip between Because the client-server relationship is sensitive, gossip between
clients and servers about unrelated SCTs is risky. Therefore, a clients and servers about unrelated SCTs is risky. Therefore, a
client with an SCT for a given server SHOULD NOT transmit that client with an SCT for a given server SHOULD NOT transmit that
information in any other than the following two channels: to the information in any other than the following two channels: to the
server associated with the SCT itself; or to a Trusted Auditor, if server associated with the SCT itself (via a TLS connection with a
one exists. certificate identifying the Domain Name of the web site with a Host
header specifying the domain name); or to a Trusted Auditor, if one
exists.
10.5.2. Privacy in SCT Feedback 10.5.2. Privacy in SCT Feedback
SCTs introduce yet another mechanism for HTTPS servers to store state SCTs introduce yet another mechanism for HTTPS servers to store state
on an HTTPS client, and potentially track users. HTTPS clients which on an HTTPS client, and potentially track users. HTTPS clients which
allow users to clear history or cookies associated with an origin allow users to clear history or cookies associated with an origin
MUST clear stored SCTs and certificate chains associated with the MUST clear stored SCTs and certificate chains associated with the
origin as well. origin as well.
Auditors should treat all SCTs as sensitive data. SCTs received Auditors should treat all SCTs as sensitive data. SCTs received
skipping to change at page 28, line 9 skipping to change at page 28, line 23
There is a possible fingerprinting attack where a log issues a unique There is a possible fingerprinting attack where a log issues a unique
SCT for targeted log client(s). A colluding log and HTTPS server SCT for targeted log client(s). A colluding log and HTTPS server
operator could therefore be a threat to the privacy of an HTTPS operator could therefore be a threat to the privacy of an HTTPS
client. Given all the other opportunities for HTTPS servers to client. Given all the other opportunities for HTTPS servers to
fingerprint clients - TLS session tickets, HPKP and HSTS headers, fingerprint clients - TLS session tickets, HPKP and HSTS headers,
HTTP Cookies, etc. - this is considered acceptable. HTTP Cookies, etc. - this is considered acceptable.
The fingerprinting attack described above would be mitigated by a The fingerprinting attack described above would be mitigated by a
requirement that logs must use a deterministic signature scheme when requirement that logs must use a deterministic signature scheme when
signing SCTs ([RFC-6962-BIS-09] Section 2.1.4). A log signing using signing SCTs ([RFC-6962-BIS-27] section 2.2). A log signing using
RSA is not required to use a deterministic signature scheme. RSA is not required to use a deterministic signature scheme.
Since logs are allowed to issue a new SCT for a certificate already Since logs are allowed to issue a new SCT for a certificate already
present in the log, mandating deterministic signatures does not stop present in the log, mandating deterministic signatures does not stop
this fingerprinting attack altogether. It does make the attack this fingerprinting attack altogether. It does make the attack
harder to pull off without being detected though. harder to pull off without being detected though.
There is another similar fingerprinting attack where an HTTPS server There is another similar fingerprinting attack where an HTTPS server
tracks a client by using a unique certificate or a variation of cert tracks a client by using a unique certificate or a variation of cert
chains. The risk for this attack is accepted on the same grounds as chains. The risk for this attack is accepted on the same grounds as
skipping to change at page 28, line 31 skipping to change at page 28, line 45
10.5.3. Privacy for HTTPS clients performing STH Proof Fetching 10.5.3. Privacy for HTTPS clients performing STH Proof Fetching
An HTTPS client performing Proof Fetching SHOULD NOT request proofs An HTTPS client performing Proof Fetching SHOULD NOT request proofs
from a CT log that it doesn't accept SCTs from. An HTTPS client from a CT log that it doesn't accept SCTs from. An HTTPS client
SHOULD regularly request an STH from all logs it is willing to SHOULD regularly request an STH from all logs it is willing to
accept, even if it has seen no SCTs from that log. accept, even if it has seen no SCTs from that log.
The time between two polls for new STH's SHOULD NOT be significantly The time between two polls for new STH's SHOULD NOT be significantly
shorter than the MMD of the polled log divided by its STH Frequency shorter than the MMD of the polled log divided by its STH Frequency
Count ([RFC-6962-BIS-09] section 5.1). Count ([RFC-6962-BIS-27] section 4.1).
The actual mechanism by which Proof Fetching is done carries The actual mechanism by which Proof Fetching is done carries
considerable privacy concerns. Although out of scope for the considerable privacy concerns. Although out of scope for the
document, DNS is a mechanism currently discussed. DNS exposes data document, DNS is a mechanism currently discussed. DNS exposes data
in plaintext over the network (including what sites the user is in plaintext over the network (including what sites the user is
visiting and what sites they have previously visited) and may not be visiting and what sites they have previously visited) and may not be
suitable for some. suitable for some.
10.5.4. Privacy in STH Pollination 10.5.4. Privacy in STH Pollination
skipping to change at page 29, line 16 skipping to change at page 29, line 29
There is a possible fingerprinting attack where a log issues a unique There is a possible fingerprinting attack where a log issues a unique
STH for a targeted HTTPS client. This is similar to the STH for a targeted HTTPS client. This is similar to the
fingerprinting attack described in Section 10.5.2, but can operate fingerprinting attack described in Section 10.5.2, but can operate
cross-origin. If a log (or HTTPS server cooperating with a log) cross-origin. If a log (or HTTPS server cooperating with a log)
provides a unique STH to a client, the targeted client will be the provides a unique STH to a client, the targeted client will be the
only client pollinating that STH cross-origin. only client pollinating that STH cross-origin.
It is mitigated partially because the log is limited in the number of It is mitigated partially because the log is limited in the number of
STHs it can issue. It must 'save' one of its STHs each MMD to STHs it can issue. It must 'save' one of its STHs each MMD to
perform the attack. perform the attack. A log violating its STH Frequency Count
([RFC-6962-BIS-27] section 4.1) can be identified as non-compliant by
CT auditors following the procedure described in [RFC-6962-BIS-27]
section 8.3.
10.5.5. Privacy in STH Interaction 10.5.5. Privacy in STH Interaction
An HTTPS client may pollinate any STH within the last 14 days. An An HTTPS client may pollinate any STH within the last 14 days. An
HTTPS client may also pollinate an STH for any log that it knows HTTPS client may also pollinate an STH for any log that it knows
about. When a client pollinates STHs to a server, it will release about. When a client pollinates STHs to a server, it will release
more than one STH at a time. It is unclear if a server may 'prime' a more than one STH at a time. It is unclear if a server may 'prime' a
client and be able to reliably detect the client at a later time. client and be able to reliably detect the client at a later time.
It's clear that a single site can track a user any way they wish, but It's clear that a single site can track a user any way they wish, but
skipping to change at page 30, line 11 skipping to change at page 30, line 27
to the auditor. In particular, it will identify the web sites that to the auditor. In particular, it will identify the web sites that
the client has visited to the auditor. Some clients may already the client has visited to the auditor. Some clients may already
share this information to a third party, for example, when using a share this information to a third party, for example, when using a
server to synchronize browser history across devices in a server- server to synchronize browser history across devices in a server-
visible way, or when doing DNS lookups through a trusted DNS visible way, or when doing DNS lookups through a trusted DNS
resolver. For clients with such a relationship already established, resolver. For clients with such a relationship already established,
sending SCTs to a trusted auditor run by the same organization does sending SCTs to a trusted auditor run by the same organization does
not appear to expose any additional information to the trusted third not appear to expose any additional information to the trusted third
party. party.
Clients who wish to contact a CT auditor without associating their Clients which wish to contact a CT auditor without associating their
identities with their SCTs may wish to use an anonymizing network identities with their SCTs may wish to use an anonymizing network
like Tor to submit SCT Feedback to the auditor. Auditors SHOULD like Tor to submit SCT Feedback to the auditor. Auditors SHOULD
accept SCT Feedback that arrives over such anonymizing networks. accept SCT Feedback that arrives over such anonymizing networks.
Clients sending feedback to an auditor may prefer to reduce the Clients sending feedback to an auditor may prefer to reduce the
temporal granularity of the history exposure to the auditor by temporal granularity of the history exposure to the auditor by
caching and delaying their SCT Feedback reports. This is elaborated caching and delaying their SCT Feedback reports. This is elaborated
upon in Section 11.3. This strategy is only as effective as the upon in Section 11.3. This strategy is only as effective as the
granularity of the timestamps embedded in the SCTs and STHs. granularity of the timestamps embedded in the SCTs and STHs.
skipping to change at page 30, line 46 skipping to change at page 31, line 14
consistency proof between two valid STHs or (b) an inclusion proof consistency proof between two valid STHs or (b) an inclusion proof
for a certificate to an STH any time after the log's MMD has for a certificate to an STH any time after the log's MMD has
elapsed from the issuance of the SCT. The log's inability to elapsed from the issuance of the SCT. The log's inability to
provide either proof will not be externally cryptographically- provide either proof will not be externally cryptographically-
verifiable, as it may be indistinguishable from a network error. verifiable, as it may be indistinguishable from a network error.
11. Policy Recommendations 11. Policy Recommendations
This section is intended as suggestions to implementors of HTTPS This section is intended as suggestions to implementors of HTTPS
Clients, HTTPS servers, and CT auditors. It is not a requirement for Clients, HTTPS servers, and CT auditors. It is not a requirement for
technique of implementation, so long as privacy considerations technique of implementation, so long as the privacy considerations
established above are obeyed. established above are obeyed.
11.1. Blocking Recommendations 11.1. Blocking Recommendations
11.1.1. Frustrating blocking 11.1.1. Frustrating blocking
When making gossip connections to HTTPS servers or Trusted Auditors, When making gossip connections to HTTPS servers or Trusted Auditors,
it is desirable to minimize the plaintext metadata in the connection it is desirable to minimize the plaintext metadata in the connection
that can be used to identify the connection as a gossip connection that can be used to identify the connection as a gossip connection
and therefore be of interest to block. Additionally, introducing and therefore be of interest to block. Additionally, introducing
skipping to change at page 36, line 25 skipping to change at page 36, line 33
11.4. Concrete Recommendations 11.4. Concrete Recommendations
We present the following pseudocode as a concrete outline of our We present the following pseudocode as a concrete outline of our
policy recommendations. policy recommendations.
Both suggestions presented are applicable to both clients and Both suggestions presented are applicable to both clients and
servers. Servers may not perform proof fetching, in which case large servers. Servers may not perform proof fetching, in which case large
portions of the pseudocode are not applicable. But it should work in portions of the pseudocode are not applicable. But it should work in
either case. either case.
Note that we use a function 'rand()' in the pseudocode, this function
is assumed to be a cryptographically secure pseudorandom number
generator. Additionally, when N unique items are needed, they are
chosen at random by drawing a random index repeatedly until the N
unique items from an array have been chosen. Although simple, when
the array is N or near-N items in length this is inefficient. A
secure shuffle algorithm followed by selecting the first N items may
be more efficient, especially when N is large.
11.4.1. STH Pollination 11.4.1. STH Pollination
The STH class contains data pertaining specifically to the STH The STH class contains data pertaining specifically to the STH
itself. itself.
class STH class STH
{ {
uint16 proof_attempts uint16 proof_attempts
uint16 proof_failure_count uint16 proof_failure_count
uint32 num_reports_to_thirdparty uint32 num_reports_to_thirdparty
skipping to change at page 37, line 9 skipping to change at page 37, line 27
complicated locking that would likely be required for a data complicated locking that would likely be required for a data
structure being accessed by multiple threads. Something to note structure being accessed by multiple threads. Something to note
about this pseudocode is that it does not remove STHs once they have about this pseudocode is that it does not remove STHs once they have
been resolved to a newer STH. Doing so might make older STHs within been resolved to a newer STH. Doing so might make older STHs within
the validity window rarer and thus enable tracking. the validity window rarer and thus enable tracking.
class STHStore class STHStore
{ {
STH[] sth_list STH[] sth_list
// This function is run after receiving a set of STHs from // This function is run after receiving a set of STHs from
// a third party in response to a pollination submission // a third party in response to a pollination submission
def insert(STH[] new_sths) { def insert(STH[] new_sths) {
foreach(new in new_sths) { foreach(new in new_sths) {
if(this.sth_list.contains(new)) if(this.sth_list.contains(new))
continue continue
this.sth_list.insert(new) this.sth_list.insert(new)
} }
} }
// This function is called to delete the given STH // This function is called to delete the given STH
// from the data store // from the data store
def delete_now(STH s) { def delete_now(STH s) {
this.sth_list.remove(s) this.sth_list.remove(s)
} }
// When it is time to perform STH Pollination, the HTTPS client // When it is time to perform STH Pollination, the HTTPS client
// calls this function to get a selection of STHs to send as // calls this function to get a selection of STHs to send as
// feedback // feedback
def get_pollination_selection() { def get_pollination_selection() {
if(len(this.sth_list) < MAX_STH_TO_GOSSIP) if(len(this.sth_list) < MAX_STH_TO_GOSSIP)
return this.sth_list return this.sth_list
else { else {
indexes = set() indexes = set()
modulus = len(this.sth_list) modulus = len(this.sth_list)
while(len(indexes) < MAX_STH_TO_GOSSIP) { outdated_sths = 0
while(len(indexes) + outdated_sths < MAX_STH_TO_GOSSIP) {
r = randomInt() % modulus r = randomInt() % modulus
// Ignore STHs that are past the validity window but not if(r not in indexes)
// yet removed. // Ignore STHs that are past the validity window but not
if(r not in indexes // yet removed.
&& now() - this.sth_list[i].timestamp < TWO_WEEKS) if(now() - this.sth_list[i].timestamp < TWO_WEEKS)
indexes.insert(r) outdated_sths++;
else
indexes.insert(r)
} }
return_selection = [] return_selection = []
foreach(i in indexes) { foreach(i in indexes) {
return_selection.insert(this.sth_list[i]) return_selection.insert(this.sth_list[i])
} }
return return_selection return return_selection
} }
} }
} }
skipping to change at page 39, line 5 skipping to change at page 40, line 5
// already in the queue.) // already in the queue.)
&& sth.proof_attempts == sth.proof_failure_count) { && sth.proof_attempts == sth.proof_failure_count) {
sth.proof_attempts++ sth.proof_attempts++
queue_consistency_proof(sth, consistency_proof_callback) queue_consistency_proof(sth, consistency_proof_callback)
} }
} }
} }
These functions also exist in the STHStore class. These functions also exist in the STHStore class.
// This function is called after successfully pollinating STHs // This function is called after successfully pollinating STHs
// to a third party. It is passed the STHs sent to the third // to a third party. It is passed the STHs sent to the third
// party, which is the output of get_gossip_selection(), as well // party, which is the output of get_gossip_selection(), as well
// as the STHs received in the response. // as the STHs received in the response.
def successful_thirdparty_submission_callback(STH[] submitted_sth_list, def successful_thirdparty_submission_callback(STH[] submitted_sth_list,
STH[] new_sths) STH[] new_sths)
{ {
foreach(sth in submitted_sth_list) { foreach(sth in submitted_sth_list) {
sth.num_reports_to_thirdparty++ sth.num_reports_to_thirdparty++
} }
this.insert(new_sths); this.insert(new_sths);
} }
// Attempt auditor of last resort submissions until it succeeds // Attempt auditor of last resort submissions until it succeeds
def auditor_of_last_resort_callback(original_sth, error) { def auditor_of_last_resort_callback(original_sth, error) {
if(!error) { if(!error) {
delete_now(original_sth) delete_now(original_sth)
} }
} }
def consistency_proof_callback(consistency_proof, original_sth, error) { def consistency_proof_callback(consistency_proof, original_sth, error) {
if(!error) { if(!error) {
insert(consistency_proof.current_sth) insert(consistency_proof.current_sth)
} else { } else {
skipping to change at page 41, line 31 skipping to change at page 42, line 31
counters are reset counters are reset
4. If a domain succeeds, but then begins failing, it must fail more 4. If a domain succeeds, but then begins failing, it must fail more
than 90% of the time (configurable) and then the process begins than 90% of the time (configurable) and then the process begins
at (2). at (2).
If a domain is visited infrequently (say, once every 7 months) then If a domain is visited infrequently (say, once every 7 months) then
it will be evicted from the cache and start all over again (according it will be evicted from the cache and start all over again (according
to the suggestion values in the below pseudocode). to the suggestion values in the below pseudocode).
[ Note: To be certain the logic is correct I give the following test
cases which illustrate the intended behavior. Hopefully the code
matches!
Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0
Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0
... wait a month ...
Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1
... wait a month ...
Succeed 1 month later num_submissions_attempted=13 num_submissions_succeeded=2 num_feedback_loop_failures=0(r) indicates (Reset)
-> Feedback is attempted regularly.
Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0
Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0
... wait a month ...
Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1
... wait a month ...
Fail 1 month later num_submissions_attempted=13 num_submissions_succeeded=1 num_feedback_loop_failures=2
... wait a month ...
Succeed 1 month later num_submissions_attempted=14 num_submissions_succeeded=2 num_feedback_loop_failures=0(r)
-> Feedback is attempted regularly.
Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0
Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0
... wait a month ...
Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1
... wait a month ...
Fail 1 month later num_submissions_attempted=13 num_submissions_succeeded=1 num_feedback_loop_failures=2
... wait a month ...
Fail 1 month later num_submissions_attempted=14 num_submissions_succeeded=2 num_feedback_loop_failures=3
... clear_old_data() is run every hour ...
num_submissions_attempted=0 num_submissions_succeeded=0 num_feedback_loop_failures=3
sct_feedback_failing_longterm=True
Fail 1 month later num_submissions_attempted=1 num_submissions_succeeded=0 num_feedback_loop_failures=4
sct_feedback_failing_longterm=True
... clear_old_data() is run every hour ...
num_submissions_attempted=0(r) num_submissions_succeeded=0 num_feedback_loop_failures=3
sct_feedback_failing_longterm=True
Succeed 1 month later num_submissions_attempted=2 num_submissions_succeeded=1 num_feedback_loop_failures=0(r)
sct_feedback_failing_longterm=False
-> Feedback is attempted regularly.
Note above that the second run of clear_old_data() will reset num_submissions_attempted from 1 to 0. This is
CRITICAL. Otherwise, we would have the below bug (where after 10 months of failures, a success would not hit
the required ratio to keep going)
//The below represents a bug.
Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0
Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0
... wait a month ...
Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1
... wait a month ...
Fail 1 month later num_submissions_attempted=13 num_submissions_succeeded=1 num_feedback_loop_failures=2
... wait a month ...
Fail 1 month later num_submissions_attempted=14 num_submissions_succeeded=2 num_feedback_loop_failures=3
... clear_old_data() is run every hour ...
num_submissions_attempted=0 num_submissions_succeeded=0 num_feedback_loop_failures=3
sct_feedback_failing_longterm=True
Fail 1 month later num_submissions_attempted=1 num_submissions_succeeded=0 num_feedback_loop_failures=4
sct_feedback_failing_longterm=True
Fail 9 times for 9 months
num_submissions_attempted=10 num_submissions_succeeded=0 num_feedback_loop_failures=13
sct_feedback_failing_longterm=True
Succeed 1 month later num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0(r)
sct_feedback_failing_longterm=False
-> Feedback is NOT attempted regularly. \]
//Suggestions: //Suggestions:
// After concluding a domain doesn't support feedback, we try again // After concluding a domain doesn't support feedback, we try again
// after WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time to see if // after WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time to see if
// they added support // they added support
#define WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS 1 month #define WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS 1 month
// If we've waited MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE // If we've waited MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE
// multiplied by WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time, we // multiplied by WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time, we
// still attempt SCT Feedback, but no longer bother storing any data // still attempt SCT Feedback, but no longer bother storing any data
// until the domain supports SCT Feedback // until the domain supports SCT Feedback
#define MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE 3 #define MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE 3
// If this percentage of SCT Feedback attempts previously succeeded, // If this percentage of SCT Feedback attempts previously succeeded,
// we consider the domain as supporting feedback and is just having // we consider the domain as supporting feedback and is just having
// transient errors // transient errors
#define MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING .10 #define MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING .10
class SCTDomainEntry class SCTDomainEntry
{ {
// This is the primary key of the object, the exact domain name it // This is the primary key of the object, the exact domain name it
// is valid for // is valid for
string domain string domain
// This is the last time the domain was contacted. For client // This is the last time the domain was contacted. For client
// operations it is updated whenever the client makes any request // operations it is updated whenever the client makes any request
// (not just feedback) to the domain. For server operations, it is // (not just feedback) to the domain. For server operations, it is
// updated whenever any client contacts the domain. Responsibility // updated whenever any client contacts the domain. Responsibility
// for updating lies OUTSIDE of the class // for updating lies OUTSIDE of the class
public datetime last_contact_for_domain public datetime last_contact_for_domain
// This is the last time SCT Feedback was attempted for the domain. // This is the last time SCT Feedback was attempted for the domain.
// It is updated whenever feedback is attempted - responsibility for // It is updated whenever feedback is attempted - responsibility for
// updating lies OUTSIDE of the class // updating lies OUTSIDE of the class
// This is not used when this algorithm runs on servers // This is not used when this algorithm runs on servers
public datetime last_sct_feedback_attempt public datetime last_sct_feedback_attempt
// This is the number of times we have waited an // This is the number of times we have waited an
// WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time, and still failed // WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time, and still failed
// e.g. 10 months of failures // e.g., 10 months of failures
// This is not used when this algorithm runs on servers // This is not used when this algorithm runs on servers
private uint16 num_feedback_loop_failures private uint16 num_feedback_loop_failures
// This is whether or not SCT Feedback has failed enough times that we // This is whether or not SCT Feedback has failed enough times that we
// should not bother storing data for it anymore. It is a small function // should not bother storing data for it anymore. It is a small
// used for illustrative purposes // function used for illustrative purposes.
// This is not used when this algorithm runs on servers // This is not used when this algorithm runs on servers
private bool sct_feedback_failing_longterm() private bool sct_feedback_failing_longterm()
{ num_feedback_loop_failures >= MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE } { num_feedback_loop_failures >=
MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE }
// This is the number of SCT Feedback submissions attempted.
// Responsibility for incrementing lies OUTSIDE of the class // This is the number of SCT Feedback submissions attempted.
// (And watch for integer overflows) // Responsibility for incrementing lies OUTSIDE of the class
// This is not used when this algorithm runs on servers // (And watch for integer overflows)
// This is not used when this algorithm runs on servers
public uint16 num_submissions_attempted public uint16 num_submissions_attempted
// This is the number of successful SCT Feedback submissions. This // This is the number of successful SCT Feedback submissions. This
// variable is updated by the class. // variable is updated by the class.
// This is not used when this algorithm runs on servers // This is not used when this algorithm runs on servers
private uint16 num_submissions_succeeded private uint16 num_submissions_succeeded
// This contains all the bundles of SCT data we have observed for // This contains all the bundles of SCT data we have observed for
// this domain // this domain
SCTBundle[] observed_records SCTBundle[] observed_records
// This function can be called to determine if we should attempt // This function can be called to determine if we should attempt
// SCT Feedback for this domain. // SCT Feedback for this domain.
def should_attempt_feedback() { def should_attempt_feedback() {
// Servers always perform feedback! // Servers always perform feedback!
if(operator_is_server) if(operator_is_server)
return true return true
// If we have not tried in a month, try again // If we have not tried in a month, try again
if(now() - last_sct_feedback_attempt > WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS) if(now() - last_sct_feedback_attempt >
WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS)
return true return true
// If we have tried recently, and it seems to be working, go for it! // If we have tried recently, and it seems to be working, go for it!
if((num_submissions_succeeded / num_submissions_attempted) > if((num_submissions_succeeded / num_submissions_attempted) >
MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING) MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING)
return true return true
// Otherwise don't try // Otherwise don't try
return false return false
} }
// For Clients, this function is called after a successful // For Clients, this function is called after a successful
// connection to an HTTPS server, with a single SCTBundle // connection to an HTTPS server, with a single SCTBundle
// constructed from that connection's certificate chain and SCTs. // constructed from that connection's certificate chain and SCTs.
// For Servers, this is called after receiving SCT Feedback with // For Servers, this is called after receiving SCT Feedback with
// all the bundles sent in the feedback. // all the bundles sent in the feedback.
def insert(SCTBundle[] bundles) { def insert(SCTBundle[] bundles) {
// Do not store data for long-failing domains // Do not store data for long-failing domains
if(sct_feedback_failing_longterm()) { if(sct_feedback_failing_longterm()) {
return return
} }
foreach(b in bundles) { foreach(b in bundles) {
if(operator_is_server) { if(operator_is_server) {
if(!passes_validity_checks(b)) if(!passes_validity_checks(b))
return return
skipping to change at page 45, line 21 skipping to change at page 45, line 4
return return
else if(e.approx_equals(b)) { else if(e.approx_equals(b)) {
have_inserted = true have_inserted = true
e.insert_scts(b.sct_list) e.insert_scts(b.sct_list)
} }
} }
if(!have_inserted) if(!have_inserted)
this.observed_records.insert(b) this.observed_records.insert(b)
} }
SCTStoreManager.update_cache_percentage() SCTStoreManager.update_cache_percentage()
} }
// When it is time to perform SCT Feedback, the HTTPS client // When it is time to perform SCT Feedback, the HTTPS client
// calls this function to get a selection of SCTBundles to send // calls this function to get a selection of SCTBundles to send
// as feedback // as feedback
def get_gossip_selection() { def get_gossip_selection() {
if(len(observed_records) > MAX_SCT_RECORDS_TO_GOSSIP) { if(len(observed_records) > MAX_SCT_RECORDS_TO_GOSSIP) {
indexes = set() indexes = set()
modulus = len(observed_records) modulus = len(observed_records)
while(len(indexes) < MAX_SCT_RECORDS_TO_GOSSIP) { while(len(indexes) < MAX_SCT_RECORDS_TO_GOSSIP) {
r = randomInt() % modulus r = randomInt() % modulus
if(r not in indexes) if(r not in indexes)
indexes.insert(r) indexes.insert(r)
} }
skipping to change at page 45, line 48 skipping to change at page 45, line 32
return_selection.insert(this.observed_records[i]) return_selection.insert(this.observed_records[i])
} }
return return_selection return return_selection
} }
else else
return this.observed_records return this.observed_records
} }
def passes_validity_checks(SCTBundle b) { def passes_validity_checks(SCTBundle b) {
// This function performs the validity checks specified in // This function performs the validity checks specified in
// {{feedback-srvop}} // {{feedback-srvop}}
} }
} }
The SCTDomainEntry is responsible for handling the outcome of a The SCTDomainEntry is responsible for handling the outcome of a
submission report for that domain using its member function: submission report for that domain using its member function:
// This function is called after providing SCT Feedback // This function is called after providing SCT Feedback
// to a server. It is passed the feedback sent to the other party, which // to a server. It is passed the feedback sent to the other party, which
// is the output of get_gossip_selection(), and also the SCTBundle // is the output of get_gossip_selection(), and also the SCTBundle
// representing the connection the data was sent on. // representing the connection the data was sent on.
// (When this code runs on the server, connectionBundle is NULL) // (When this code runs on the server, connectionBundle is NULL)
// If the Feedback was not sent successfully, error is True // If the Feedback was not sent successfully, error is True
def after_submit_to_thirdparty(error, SCTBundle[] submittedBundles, def after_submit_to_thirdparty(error, SCTBundle[] submittedBundles,
SCTBundle connectionBundle) SCTBundle connectionBundle)
{ {
// Server operation in this instance is exceedingly simple // Server operation in this instance is exceedingly simple
if(operator_is_server) { if(operator_is_server) {
if(error) if(error)
return return
foreach(bundle in submittedBundles) foreach(bundle in submittedBundles)
bundle.num_reports_to_thirdparty++ bundle.num_reports_to_thirdparty++
return return
} }
// Client behavior is much more complicated // Client behavior is much more complicated
if(error) { if(error) {
if(sct_feedback_failing_longterm()) { if(sct_feedback_failing_longterm()) {
num_feedback_loop_failures++ num_feedback_loop_failures++
} }
skipping to change at page 50, line 36 skipping to change at page 50, line 36
original_sct.has_been_resolved_to_sth = True original_sct.has_been_resolved_to_sth = True
insert_to_sth_datastore(inclusion_proof.new_sth) insert_to_sth_datastore(inclusion_proof.new_sth)
} else { } else {
original_sct.proof_failure_count++ original_sct.proof_failure_count++
} }
} }
If the cache is getting full, these three member functions of the If the cache is getting full, these three member functions of the
SCTStoreManager class will be used. SCTStoreManager class will be used.
// ----------------------------------------------------------------- // -----------------------------------------------------------------
// This function is called when the cache is not yet full, but is // This function is called when the cache is not yet full, but is
// nearing it. It prioritizes deleting data that should be safe // nearing it. It prioritizes deleting data that should be safe
// to delete (because it has been shared with the site or resolved // to delete (because it has been shared with the site or resolved
// to a STH) // to an STH)
def cache_pressure_is_imminent() def cache_pressure_is_imminent()
{ {
bundlesToDelete = [] bundlesToDelete = []
foreach(domainEntry in all_sct_stores) { foreach(domainEntry in all_sct_stores) {
foreach(sctBundle in domainEntry.observed_records) { foreach(sctBundle in domainEntry.observed_records) {
if(proof_fetching_enabled) { if(proof_fetching_enabled) {
// First, queue proofs for anything not already queued. // First, queue proofs for anything not already queued.
if(!sctBundle.has_been_fully_resolved_to_sths()) { if(!sctBundle.has_been_fully_resolved_to_sths()) {
foreach(sct in bundle.sct_list) { foreach(sct in bundle.sct_list) {
if(!sct.has_been_resolved_to_sth if(!sct.has_been_resolved_to_sth
&& !sct.proof_outstanding) { && !sct.proof_outstanding) {
sct.proof_outstanding = True sct.proof_outstanding = True
queue_inclusion_proof(sct, inclusion_proof_callback) queue_inclusion_proof(sct, inclusion_proof_callback)
} }
} }
} }
// Second, consider deleting entries that have been fully // Second, consider deleting entries that have been fully
// resolved. // resolved.
else { else {
bundlesToDelete.append( Struct(domainEntry, sctBundle) ) bundlesToDelete.append( Struct(domainEntry, sctBundle) )
} }
} }
// Third, consider deleting entries that have been successfully // Third, consider deleting entries that have been successfully
// reported // reported
if(sctBundle.num_reports_to_thirdparty > 0) { if(sctBundle.num_reports_to_thirdparty > 0) {
bundlesToDelete.append( Struct(domainEntry, sctBundle) ) bundlesToDelete.append( Struct(domainEntry, sctBundle) )
} }
} }
} }
// Third, delete the eligible entries at random until the cache is // Third, delete the eligible entries at random until the cache is
// at a safe level // at a safe level
uint recalculateIndex = 0 uint recalculateIndex = 0
#define RECALCULATE_EVERY_N_OPERATIONS 50 #define RECALCULATE_EVERY_N_OPERATIONS 50
while(bundlesToDelete.length > 0 && while(bundlesToDelete.length > 0 &&
current_cache_percentage() > CACHE_PRESSURE_SAFE) { current_cache_percentage() > CACHE_PRESSURE_SAFE) {
uint rndIndex = rand() % bundlesToDelete.length uint rndIndex = rand() % bundlesToDelete.length
bundlesToDelete[rndIndex].domainEntry.observed_records.remove(bundlesToDelete[rndIndex].sctBundle) bundlesToDelete[rndIndex].domainEntry.observed_records.remove(
bundlesToDelete.removeAt(rndIndex) bundlesToDelete[rndIndex].sctBundle)
bundlesToDelete.removeAt(rndIndex)
recalculateIndex++ recalculateIndex++
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) { if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) {
update_cache_percentage() update_cache_percentage()
} }
} }
// Finally, tell the proof fetching engine to go faster // Finally, tell the proof fetching engine to go faster
if(proof_fetching_enabled) { if(proof_fetching_enabled) {
// This function would speed up proof fetching until an // This function would speed up proof fetching until an
// arbitrary time has passed. Perhaps until it has fetched // arbitrary time has passed. Perhaps until it has fetched
// proofs for the number of items currently in its queue? Or // proofs for the number of items currently in its queue? Or
// a percentage of them? // a percentage of them?
proof_fetch_faster_please() proof_fetch_faster_please()
} }
update_cache_percentage(); update_cache_percentage();
} }
// ----------------------------------------------------------------- // -----------------------------------------------------------------
// This function is called when the cache is almost full. It will // This function is called when the cache is almost full. It will
// evict entries at random, while attempting to save entries that // evict entries at random, while attempting to save entries that
// appear to have proof fetching failures // appear to have proof fetching failures
def cache_pressure_almost_full() def cache_pressure_almost_full()
{ {
uint recalculateIndex = 0 uint recalculateIndex = 0
uint savedRecords = 0 uint savedRecords = 0
#define RECALCULATE_EVERY_N_OPERATIONS 50 #define RECALCULATE_EVERY_N_OPERATIONS 50
while(all_sct_stores.length > savedRecords && while(all_sct_stores.length > savedRecords &&
current_cache_percentage() > CACHE_PRESSURE_SAFE) { current_cache_percentage() > CACHE_PRESSURE_SAFE) {
uint rndIndex1 = rand() % all_sct_stores.length uint rndIndex1 = rand() % all_sct_stores.length
uint rndIndex2 = rand() % all_sct_stores[rndIndex1].observed_records.length uint rndIndex2 = rand() %
all_sct_stores[rndIndex1].observed_records.length
if(proof_fetching_enabled) { if(proof_fetching_enabled) {
if(all_sct_stores[rndIndex1].observed_records[rndIndex2].max_proof_failures() > if(all_sct_stores[rndIndex1].observed_records[
MIN_PROOF_FAILURES_CONSIDERED_SUSPICIOUS) { rndIndex2].max_proof_failures() >
savedRecords++ MIN_PROOF_FAILURES_CONSIDERED_SUSPICIOUS) {
continue savedRecords++
} continue
} }
}
// If proof fetching is not enabled we need some other logic // If proof fetching is not enabled we need some other logic
else { else {
if(sctBundle.num_reports_to_thirdparty == 0) { if(sctBundle.num_reports_to_thirdparty == 0) {
savedRecords++ savedRecords++
continue continue
} }
} }
all_sct_stores[rndIndex1].observed_records.removeAt(rndIndex2) all_sct_stores[rndIndex1].observed_records.removeAt(rndIndex2)
if(all_sct_stores[rndIndex1].observed_records.length == 0) { if(all_sct_stores[rndIndex1].observed_records.length == 0) {
all_sct_stores.removeAt(rndIndex1) all_sct_stores.removeAt(rndIndex1)
} }
recalculateIndex++ recalculateIndex++
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) { if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) {
update_cache_percentage() update_cache_percentage()
} }
} }
update_cache_percentage(); update_cache_percentage();
}
// -----------------------------------------------------------------
// This function is called when the cache is full, and will evict
// cache entries at random
def cache_is_full()
{
uint recalculateIndex = 0
#define RECALCULATE_EVERY_N_OPERATIONS 50
while(all_sct_stores.length > 0 && }
current_cache_percentage() > CACHE_PRESSURE_SAFE) {
uint rndIndex1 = rand() % all_sct_stores.length
uint rndIndex2 = rand() % all_sct_stores[rndIndex1].observed_records.length
all_sct_stores[rndIndex1].observed_records.removeAt(rndIndex2) // -----------------------------------------------------------------
if(all_sct_stores[rndIndex1].observed_records.length == 0) { // This function is called when the cache is full, and will evict
all_sct_stores.removeAt(rndIndex1) // cache entries at random
} def cache_is_full()
{
uint recalculateIndex = 0
#define RECALCULATE_EVERY_N_OPERATIONS 50
recalculateIndex++ while(all_sct_stores.length > 0 &&
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) { current_cache_percentage() > CACHE_PRESSURE_SAFE) {
update_cache_percentage() uint rndIndex1 = rand() % all_sct_stores.length
} uint rndIndex2 = rand() %
} all_sct_stores[rndIndex1].observed_records.length
update_cache_percentage(); all_sct_stores[rndIndex1].observed_records.removeAt(rndIndex2)
} if(all_sct_stores[rndIndex1].observed_records.length == 0) {
all_sct_stores.removeAt(rndIndex1)
}
recalculateIndex++
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) {
update_cache_percentage()
}
}
update_cache_percentage();
}
12. IANA considerations 12. IANA considerations
[ TBD ] There are no IANA considerations.
13. Contributors 13. Contributors
The authors would like to thank the following contributors for The authors would like to thank the following contributors for
valuable suggestions: Al Cutter, Ben Laurie, Benjamin Kaduk, Josef valuable suggestions: Al Cutter, Andrew Ayer, Ben Laurie, Benjamin
Gustafsson, Karen Seo, Magnus Ahltorp, Steven Kent, Yan Zhu. Kaduk, Graham Edgecombe, Josef Gustafsson, Karen Seo, Magnus Ahltorp,
Steven Kent, Yan Zhu.
14. ChangeLog 14. ChangeLog
14.1. Changes between ietf-04 and ietf-05
14.1. Changes between ietf-03 and ietf-04 o STH and SCT data formats changed to support CT v1 and v2.
o Address ED review comments.
14.2. Changes between ietf-03 and ietf-04
o No changes. o No changes.
14.2. Changes between ietf-02 and ietf-03 14.3. Changes between ietf-02 and ietf-03
o TBD's resolved. o TBD's resolved.
o References added. o References added.
o Pseduocode changed to work for both clients and servers. o Pseduocode changed to work for both clients and servers.
14.3. Changes between ietf-01 and ietf-02 14.4. Changes between ietf-01 and ietf-02
o Requiring full certificate chain in SCT Feedback. o Requiring full certificate chain in SCT Feedback.
o Clarifications on what clients store for and send in SCT Feedback o Clarifications on what clients store for and send in SCT Feedback
added. added.
o SCT Feedback server operation updated to protect against DoS o SCT Feedback server operation updated to protect against DoS
attacks on servers. attacks on servers.
o Pre-Loaded vs Locally Added Anchors explained. o Pre-Loaded vs Locally Added Anchors explained.
o Base for well-known URL's changed. o Base for well-known URL's changed.
o Remove all mentions of monitors - gossip deals with auditors. o Remove all mentions of monitors - gossip deals with auditors.
o New sections added: Trusted Auditor protocol, attacks by actively o New sections added: Trusted Auditor protocol, attacks by actively
malicious log, the Dual-CA compromise attack, policy malicious log, the Dual-CA compromise attack, policy
recommendations, recommendations,
14.4. Changes between ietf-00 and ietf-01 14.5. Changes between ietf-00 and ietf-01
o Improve language and readability based on feedback from Stephen o Improve language and readability based on feedback from Stephen
Kent. Kent.
o STH Pollination Proof Fetching defined and indicated as optional. o STH Pollination Proof Fetching defined and indicated as optional.
o 3-Method Ecosystem section added. o 3-Method Ecosystem section added.
o Cases with Logs ceasing operation handled. o Cases with Logs ceasing operation handled.
o Text on tracking via STH Interaction added. o Text on tracking via STH Interaction added.
o Section with some early recommendations for mixing added. o Section with some early recommendations for mixing added.
o Section detailing blocking connections, frustrating it, and the o Section detailing blocking connections, frustrating it, and the
implications added. implications added.
14.5. Changes between -01 and -02 14.6. Changes between -01 and -02
o STH Pollination defined. o STH Pollination defined.
o Trusted Auditor Relationship defined. o Trusted Auditor Relationship defined.
o Overview section rewritten. o Overview section rewritten.
o Data flow picture added. o Data flow picture added.
o Section on privacy considerations expanded. o Section on privacy considerations expanded.
14.6. Changes between -00 and -01 14.7. Changes between -00 and -01
o Add the SCT feedback mechanism: Clients send SCTs to originating o Add the SCT feedback mechanism: Clients send SCTs to originating
web server which shares them with auditors. web server which shares them with auditors.
o Stop assuming that clients see STHs. o Stop assuming that clients see STHs.
o Don't use HTTP headers but instead .well-known URL's - avoid that o Don't use HTTP headers but instead .well-known URL's - avoid that
battle. battle.
o Stop referring to trans-gossip and trans-gossip-transport-https - o Stop referring to trans-gossip and trans-gossip-transport-https -
skipping to change at page 55, line 41 skipping to change at page 55, line 48
back and add more later. back and add more later.
o Add more reasoning about privacy. o Add more reasoning about privacy.
o Do specify data formats. o Do specify data formats.
15. References 15. References
15.1. Normative References 15.1. Normative References
[RFC-6962-BIS-09] [RFC-6962-BIS-27]
Laurie, B., Langley, A., Kasper, E., Messeri, E., and R. Laurie, B., Langley, A., Kasper, E., Messeri, E., and R.
Stradling, "Certificate Transparency", October 2015, Stradling, "Certificate Transparency", October 2017,
<https://datatracker.ietf.org/doc/draft-ietf-trans- <https://datatracker.ietf.org/doc/draft-ietf-trans-
rfc6962-bis/>. rfc6962-bis/>.
[RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data [RFC6962] Laurie, B., Langley, A., and E. Kasper, "Certificate
Interchange Format", RFC 7159, March 2014. Transparency", RFC 6962, DOI 10.17487/RFC6962, June 2013,
<https://www.rfc-editor.org/info/rfc6962>.
[RFC7159] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March
2014, <https://www.rfc-editor.org/info/rfc7159>.
[RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext
Transfer Protocol Version 2 (HTTP/2)", RFC 7540,
DOI 10.17487/RFC7540, May 2015, <https://www.rfc-
editor.org/info/rfc7540>.
15.2. Informative References 15.2. Informative References
[double-keying] [double-keying]
Perry, M., Clark, E., and S. Murdoch, "Cross-Origin Perry, M., Clark, E., and S. Murdoch, "Cross-Origin
Identifier Unlinkability", May 2015, Identifier Unlinkability", May 2015,
<https://www.torproject.org/projects/torbrowser/ <https://www.torproject.org/projects/torbrowser/
design/#identifier-linkability>. design/#identifier-linkability>.
[draft-ct-over-dns] [draft-ct-over-dns]
Laurie, B., Phaneuf, P., and A. Eijdenberg, "Certificate Laurie, B., Phaneuf, P., and A. Eijdenberg, "Certificate
Transparency over DNS", February 2016, Transparency over DNS", February 2016,
<https://github.com/google/certificate-transparency- <https://github.com/google/certificate-transparency-
rfcs/blob/master/dns/draft-ct-over-dns.md>. rfcs/blob/master/dns/draft-ct-over-dns.md>.
[draft-ietf-trans-threat-analysis-03] [draft-ietf-trans-threat-analysis-12]
Kent, S., "Attack Model and Threat for Certificate Kent, S., "Attack and Threat Model for Certificate
Transparency", October 2015, Transparency", October 2017,
<https://datatracker.ietf.org/doc/draft-ietf-trans-threat- <https://datatracker.ietf.org/doc/draft-ietf-trans-threat-
analysis/>. analysis/>.
[dual-ca-compromise-attack] [dual-ca-compromise-attack]
Gillmor, D., "can CT defend against dual CA compromise?", Gillmor, D., "can CT defend against dual CA compromise?",
n.d., <https://www.ietf.org/mail- n.d., <https://www.ietf.org/mail-
archive/web/trans/current/msg01984.html>. archive/web/trans/current/msg01984.html>.
[gossip-mixing] [gossip-mixing]
Ritter, T., "A Bit on Certificate Transparency Gossip", Ritter, T., "A Bit on Certificate Transparency Gossip",
 End of changes. 140 change blocks. 
450 lines changed or deleted 447 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/