draft-ietf-dnsop-dns-capture-format-06.txt   draft-ietf-dnsop-dns-capture-format-07.txt 
dnsop J. Dickinson dnsop J. Dickinson
Internet-Draft J. Hague Internet-Draft J. Hague
Intended status: Standards Track S. Dickinson Intended status: Standards Track S. Dickinson
Expires: September 6, 2018 Sinodun IT Expires: November 9, 2018 Sinodun IT
T. Manderson T. Manderson
J. Bond J. Bond
ICANN ICANN
March 5, 2018 May 8, 2018
C-DNS: A DNS Packet Capture Format C-DNS: A DNS Packet Capture Format
draft-ietf-dnsop-dns-capture-format-06 draft-ietf-dnsop-dns-capture-format-07
Abstract Abstract
This document describes a data representation for collections of DNS This document describes a data representation for collections of DNS
messages. The format is designed for efficient storage and messages. The format is designed for efficient storage and
transmission of large packet captures of DNS traffic; it attempts to transmission of large packet captures of DNS traffic; it attempts to
minimize the size of such packet capture files but retain the full minimize the size of such packet capture files but retain the full
DNS message contents along with the most useful transport metadata. DNS message contents along with the most useful transport metadata.
It is intended to assist with the development of DNS traffic It is intended to assist with the development of DNS traffic
monitoring applications. monitoring applications.
skipping to change at page 1, line 40 skipping to change at page 1, line 40
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 6, 2018. This Internet-Draft will expire on November 9, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 21 skipping to change at page 2, line 21
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Data collection use cases . . . . . . . . . . . . . . . . . . 5 3. Data collection use cases . . . . . . . . . . . . . . . . . . 5
4. Design considerations . . . . . . . . . . . . . . . . . . . . 7 4. Design considerations . . . . . . . . . . . . . . . . . . . . 7
5. Choice of CBOR . . . . . . . . . . . . . . . . . . . . . . . 8 5. Choice of CBOR . . . . . . . . . . . . . . . . . . . . . . . 8
6. C-DNS format conceptual overview . . . . . . . . . . . . . . 9 6. C-DNS format conceptual overview . . . . . . . . . . . . . . 9
6.1. Block Parameters . . . . . . . . . . . . . . . . . . . . 10 6.1. Block Parameters . . . . . . . . . . . . . . . . . . . . 10
6.2. Storage Parameters . . . . . . . . . . . . . . . . . . . 10 6.2. Storage Parameters . . . . . . . . . . . . . . . . . . . 10
6.2.1. Optional data items . . . . . . . . . . . . . . . . . 10 6.2.1. Optional data items . . . . . . . . . . . . . . . . . 11
6.2.2. Optional RRs and OPCODES . . . . . . . . . . . . . . 11 6.2.2. Optional RRs and OPCODEs . . . . . . . . . . . . . . 12
6.2.3. Storage flags . . . . . . . . . . . . . . . . . . . . 12 6.2.3. Storage flags . . . . . . . . . . . . . . . . . . . . 12
6.2.4. IP Address storage . . . . . . . . . . . . . . . . . 12 6.2.4. IP Address storage . . . . . . . . . . . . . . . . . 12
7. C-DNS format detailed description . . . . . . . . . . . . . . 12 7. C-DNS format detailed description . . . . . . . . . . . . . . 13
7.1. Map quantities and indexes . . . . . . . . . . . . . . . 12 7.1. Map quantities and indexes . . . . . . . . . . . . . . . 13
7.2. Tabular representation . . . . . . . . . . . . . . . . . 13 7.2. Tabular representation . . . . . . . . . . . . . . . . . 13
7.3. "File" . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.3. "File" . . . . . . . . . . . . . . . . . . . . . . . . . 14
7.4. "FilePreamble" . . . . . . . . . . . . . . . . . . . . . 14 7.4. "FilePreamble" . . . . . . . . . . . . . . . . . . . . . 14
7.4.1. "BlockParameters" . . . . . . . . . . . . . . . . . . 15 7.4.1. "BlockParameters" . . . . . . . . . . . . . . . . . . 15
7.4.2. "CollectionParameters" . . . . . . . . . . . . . . . 18 7.4.2. "CollectionParameters" . . . . . . . . . . . . . . . 18
7.5. "Block" . . . . . . . . . . . . . . . . . . . . . . . . . 19 7.5. "Block" . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.5.1. "BlockPreamble" . . . . . . . . . . . . . . . . . . . 20 7.5.1. "BlockPreamble" . . . . . . . . . . . . . . . . . . . 20
7.5.2. "BlockStatistics" . . . . . . . . . . . . . . . . . . 21 7.5.2. "BlockStatistics" . . . . . . . . . . . . . . . . . . 21
7.5.3. "BlockTables" . . . . . . . . . . . . . . . . . . . . 22 7.5.3. "BlockTables" . . . . . . . . . . . . . . . . . . . . 22
7.6. "QueryResponse" . . . . . . . . . . . . . . . . . . . . . 27 7.6. "QueryResponse" . . . . . . . . . . . . . . . . . . . . . 28
7.6.1. "ResponseProcessingData" . . . . . . . . . . . . . . 29 7.6.1. "ResponseProcessingData" . . . . . . . . . . . . . . 30
7.6.2. "QueryResponseExtended" . . . . . . . . . . . . . . . 29 7.6.2. "QueryResponseExtended" . . . . . . . . . . . . . . . 30
7.7. "AddressEventCount" . . . . . . . . . . . . . . . . . . . 30 7.7. "AddressEventCount" . . . . . . . . . . . . . . . . . . . 31
7.8. "MalformedMessage" . . . . . . . . . . . . . . . . . . . 31 7.8. "MalformedMessage" . . . . . . . . . . . . . . . . . . . 32
8. Malformed messages . . . . . . . . . . . . . . . . . . . . . 32 8. C-DNS to PCAP . . . . . . . . . . . . . . . . . . . . . . . . 33
9. C-DNS to PCAP . . . . . . . . . . . . . . . . . . . . . . . . 33 8.1. Name compression . . . . . . . . . . . . . . . . . . . . 34
9.1. Name compression . . . . . . . . . . . . . . . . . . . . 34 9. Data collection . . . . . . . . . . . . . . . . . . . . . . . 34
10. Data collection . . . . . . . . . . . . . . . . . . . . . . . 35 9.1. Matching algorithm . . . . . . . . . . . . . . . . . . . 35
10.1. Matching algorithm . . . . . . . . . . . . . . . . . . . 35 9.2. Message identifiers . . . . . . . . . . . . . . . . . . . 36
10.2. Message identifiers . . . . . . . . . . . . . . . . . . 36 9.2.1. Primary ID (required) . . . . . . . . . . . . . . . . 36
10.2.1. Primary ID (required) . . . . . . . . . . . . . . . 36 9.2.2. Secondary ID (optional) . . . . . . . . . . . . . . . 36
10.2.2. Secondary ID (optional) . . . . . . . . . . . . . . 36 9.3. Algorithm parameters . . . . . . . . . . . . . . . . . . 36
10.3. Algorithm parameters . . . . . . . . . . . . . . . . . . 36 9.4. Algorithm requirements . . . . . . . . . . . . . . . . . 36
10.4. Algorithm requirements . . . . . . . . . . . . . . . . . 36 9.5. Algorithm limitations . . . . . . . . . . . . . . . . . . 37
10.5. Algorithm limitations . . . . . . . . . . . . . . . . . 37 9.6. Workspace . . . . . . . . . . . . . . . . . . . . . . . . 37
10.6. Workspace . . . . . . . . . . . . . . . . . . . . . . . 37 9.7. Output . . . . . . . . . . . . . . . . . . . . . . . . . 37
10.7. Output . . . . . . . . . . . . . . . . . . . . . . . . . 37 9.8. Post processing . . . . . . . . . . . . . . . . . . . . . 37
10.8. Post processing . . . . . . . . . . . . . . . . . . . . 37 10. Implementation guidance . . . . . . . . . . . . . . . . . . . 38
11. Implementation guidance . . . . . . . . . . . . . . . . . . . 37 10.1. Optional data . . . . . . . . . . . . . . . . . . . . . 38
11.1. Optional data . . . . . . . . . . . . . . . . . . . . . 38 10.2. Trailing bytes . . . . . . . . . . . . . . . . . . . . . 38
11.2. Trailing data in TCP . . . . . . . . . . . . . . . . . . 38 10.3. Limiting collection of RDATA . . . . . . . . . . . . . . 39
11.3. Limiting collection of RDATA . . . . . . . . . . . . . . 38 11. Implementation status . . . . . . . . . . . . . . . . . . . . 39
12. Implementation status . . . . . . . . . . . . . . . . . . . . 38 11.1. DNS-STATS Compactor . . . . . . . . . . . . . . . . . . 39
12.1. DNS-STATS Compactor . . . . . . . . . . . . . . . . . . 39 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 40
13. IANA considerations . . . . . . . . . . . . . . . . . . . . . 39 13. Security considerations . . . . . . . . . . . . . . . . . . . 40
14. Security considerations . . . . . . . . . . . . . . . . . . . 39 14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 40
15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 39 15. Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . 40
16. Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . 40 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 43
17. References . . . . . . . . . . . . . . . . . . . . . . . . . 42 16.1. Normative References . . . . . . . . . . . . . . . . . . 43
17.1. Normative References . . . . . . . . . . . . . . . . . . 42 16.2. Informative References . . . . . . . . . . . . . . . . . 43
17.2. Informative References . . . . . . . . . . . . . . . . . 42 16.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 44
17.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Appendix A. CDDL . . . . . . . . . . . . . . . . . . . . . . . . 45 Appendix A. CDDL . . . . . . . . . . . . . . . . . . . . . . . . 45
Appendix B. DNS Name compression example . . . . . . . . . . . . 54 Appendix B. DNS Name compression example . . . . . . . . . . . . 55
B.1. NSD compression algorithm . . . . . . . . . . . . . . . . 55 B.1. NSD compression algorithm . . . . . . . . . . . . . . . . 56
B.2. Knot Authoritative compression algorithm . . . . . . . . 56 B.2. Knot Authoritative compression algorithm . . . . . . . . 56
B.3. Observed differences . . . . . . . . . . . . . . . . . . 56 B.3. Observed differences . . . . . . . . . . . . . . . . . . 57
Appendix C. Comparison of Binary Formats . . . . . . . . . . . . 56 Appendix C. Comparison of Binary Formats . . . . . . . . . . . . 57
C.1. Comparison with full PCAP files . . . . . . . . . . . . . 59 C.1. Comparison with full PCAP files . . . . . . . . . . . . . 60
C.2. Simple versus block coding . . . . . . . . . . . . . . . 60 C.2. Simple versus block coding . . . . . . . . . . . . . . . 60
C.3. Binary versus text formats . . . . . . . . . . . . . . . 60 C.3. Binary versus text formats . . . . . . . . . . . . . . . 61
C.4. Performance . . . . . . . . . . . . . . . . . . . . . . . 60 C.4. Performance . . . . . . . . . . . . . . . . . . . . . . . 61
C.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . 61 C.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . 61
C.6. Block size choice . . . . . . . . . . . . . . . . . . . . 61 C.6. Block size choice . . . . . . . . . . . . . . . . . . . . 62
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 62 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 63
1. Introduction 1. Introduction
There has long been a need to collect DNS queries and responses on There has long been a need to collect DNS queries and responses on
authoritative and recursive name servers for monitoring and analysis. authoritative and recursive name servers for monitoring and analysis.
This data is used in a number of ways including traffic monitoring, This data is used in a number of ways including traffic monitoring,
analyzing network attacks and "day in the life" (DITL) [ditl] analyzing network attacks and "day in the life" (DITL) [ditl]
analysis. analysis.
A wide variety of tools already exist that facilitate the collection A wide variety of tools already exist that facilitate the collection
skipping to change at page 4, line 17 skipping to change at page 4, line 17
these are largely aimed at producing convenient representations of these are largely aimed at producing convenient representations of
single messages. single messages.
Many DNS operators may receive hundreds of thousands of queries per Many DNS operators may receive hundreds of thousands of queries per
second on a single name server instance so a mechanism to minimize second on a single name server instance so a mechanism to minimize
the storage size (and therefore upload overhead) of the data the storage size (and therefore upload overhead) of the data
collected is highly desirable. collected is highly desirable.
The format described in this document, C-DNS (Compacted-DNS), The format described in this document, C-DNS (Compacted-DNS),
focusses on the problem of capturing and storing large packet capture focusses on the problem of capturing and storing large packet capture
files of DNS traffic. with the following goals in mind: files of DNS traffic with the following goals in mind:
o Minimize the file size for storage and transmission o Minimize the file size for storage and transmission.
o Minimizing the overhead of producing the packet capture file and o Minimize the overhead of producing the packet capture file and the
the cost of any further (general purpose) compression of the file cost of any further (general purpose) compression of the file.
This document contains: This document contains:
o A discussion of the some common use cases in which such DNS data o A discussion of some common use cases in which DNS data is
is collected Section 3 collected, see Section 3.
o A discussion of the major design considerations in developing an o A discussion of the major design considerations in developing an
efficient data representation for collections of DNS messages efficient data representation for collections of DNS messages, see
Section 4 Section 4.
o A description of why CBOR [RFC7049] was chosen for this format o A description of why CBOR [RFC7049] was chosen for this format,
Section 5 see Section 5.
o A conceptual overview of the C-DNS format Section 6 o A conceptual overview of the C-DNS format, see Section 6.
o The definition of the C-DNS format for the collection of DNS o The definition of the C-DNS format for the collection of DNS
messages Section 7. messages, see Section 7.
o Notes on converting C-DNS data to PCAP format Section 9 o Notes on converting C-DNS data to PCAP format, see Section 8.
o Some high level implementation considerations for applications o Some high level implementation considerations for applications
designed to produce C-DNS Section 10 designed to produce C-DNS, see Section 9.
2. Terminology 2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
"Packet" refers to individual IPv4 or IPv6 packets. Typically these "Packet" refers to an individual IPv4 or IPv6 packet. Typically
are UDP, but may be constructed from a TCP packet. "Message", unless packets are UDP datagrams, but may also be part of a TCP data stream.
otherwise qualified, refers to a DNS payload extracted from a UDP or "Message", unless otherwise qualified, refers to a DNS payload
TCP data stream. extracted from a UDP datagram or a TCP data stream.
The parts of DNS messages are named as they are in [RFC1035]. In The parts of DNS messages are named as they are in [RFC1035].
specific, the DNS message has five sections: Header, Question, Specifically, the DNS message has five sections: Header, Question,
Answer, Authority, and Additional. Answer, Authority, and Additional.
Pairs of DNS messages are called a Query and a Response. Pairs of DNS messages are called a Query and a Response.
3. Data collection use cases 3. Data collection use cases
In an ideal world, it would be optimal to collect full packet In an ideal world, it would be optimal to collect full packet
captures of all packets going in or out of a name server. However, captures of all packets going in or out of a name server. However,
there are several design choices or other limitations that are common there are several design choices or other limitations that are common
to many DNS installations and operators. to many DNS installations and operators.
o DNS servers are hosted in a variety of situations o DNS servers are hosted in a variety of situations:
* Self-hosted servers * Self-hosted servers
* Third party hosting (including multiple third parties) * Third party hosting (including multiple third parties)
* Third party hardware (including multiple third parties) * Third party hardware (including multiple third parties)
o Data is collected under different conditions o Data is collected under different conditions:
* On well-provisioned servers running in a steady state * On well-provisioned servers running in a steady state
* On heavily loaded servers * On heavily loaded servers
* On virtualized servers * On virtualized servers
* On servers that are under DoS attack * On servers that are under DoS attack
* On servers that are unwitting intermediaries in DoS attacks * On servers that are unwitting intermediaries in DoS attacks
o Traffic can be collected via a variety of mechanisms o Traffic can be collected via a variety of mechanisms:
* Within the name server implementation itself
* On the same hardware as the name server itself * On the same hardware as the name server itself
* Using a network tap on an adjacent host to listen to DNS * Using a network tap on an adjacent host to listen to DNS
traffic traffic
* Using port mirroring to listen from another host * Using port mirroring to listen from another host
o The capabilities of data collection (and upload) networks vary o The capabilities of data collection (and upload) networks vary:
* Out-of-band networks with the same capacity as the in-band * Out-of-band networks with the same capacity as the in-band
network network
* Out-of-band networks with less capacity than the in-band * Out-of-band networks with less capacity than the in-band
network network
* Everything being on the in-band network * Everything being on the in-band network
Thus, there is a wide range of use cases from very limited data Thus, there is a wide range of use cases from very limited data
collection environments (third party hardware, servers that are under collection environments (third party hardware, servers that are under
attack, packet capture on the name server itself and no out-of-band attack, packet capture on the name server itself and no out-of-band
network) to "limitless" environments (self hosted, well provisioned network) to "limitless" environments (self hosted, well provisioned
servers, using a network tap or port mirroring with an out-of-band servers, using a network tap or port mirroring with an out-of-band
networks with the same capacity as the in-band network). In the networks with the same capacity as the in-band network). In the
former, it is infeasible to reliably collect full packet captures, former, it is infeasible to reliably collect full packet captures,
especially if the server is under attack. In the latter case, especially if the server is under attack. In the latter case,
collection of full packet captures may be reasonable. collection of full packet captures may be reasonable.
As a result of these restrictions, the C-DNS data format was designed As a result of these restrictions, the C-DNS data format is designed
with the most limited use case in mind such that: with the most limited use case in mind such that:
o data collection will occur on the same hardware as the name server o data collection will occur on the same hardware as the name server
itself itself
o collected data will be stored on the same hardware as the name o collected data will be stored on the same hardware as the name
server itself, at least temporarily server itself, at least temporarily
o collected data being returned to some central analysis system will o collected data being returned to some central analysis system will
use the same network interface as the DNS queries and responses use the same network interface as the DNS queries and responses
skipping to change at page 6, line 49 skipping to change at page 7, line 5
Another significant consideration for any application that records Another significant consideration for any application that records
DNS traffic is that the running of the name server software and the DNS traffic is that the running of the name server software and the
transmission of DNS queries and responses are the most important jobs transmission of DNS queries and responses are the most important jobs
of a name server; capturing data is not. Any data collection system of a name server; capturing data is not. Any data collection system
co-located with the name server needs to be intelligent enough to co-located with the name server needs to be intelligent enough to
carefully manage its CPU, disk, memory and network utilization. This carefully manage its CPU, disk, memory and network utilization. This
leads to designing a format that requires a relatively low overhead leads to designing a format that requires a relatively low overhead
to produce and minimizes the requirement for further potentially to produce and minimizes the requirement for further potentially
costly compression. costly compression.
However, it was also essential that interoperability with less However, it is also essential that interoperability with less
restricted infrastructure was maintained. In particular, it is restricted infrastructure is maintained. In particular, it is highly
highly desirable that the collection format should facilitate the re- desirable that the collection format should facilitate the re-
creation of common formats (such as PCAP) that are as close to the creation of common formats (such as PCAP) that are as close to the
original as is realistic given the restrictions above. original as is realistic given the restrictions above.
4. Design considerations 4. Design considerations
This section presents some of the major design considerations used in This section presents some of the major design considerations used in
the development of the C-DNS format. the development of the C-DNS format.
1. The basic unit of data is a combined DNS Query and the associated 1. The basic unit of data is a combined DNS Query and the associated
Response (a "Q/R data item"). The same structure will be used Response (a "Q/R data item"). The same structure will be used
for unmatched Queries and Responses. Queries without Responses for unmatched Queries and Responses. Queries without Responses
will be captured omitting the response data. Responses without will be captured omitting the response data. Responses without
queries will be captured omitting the Query data (but using the queries will be captured omitting the Query data (but using the
Question section from the response, if present, as an identifying Question section from the response, if present, as an identifying
QNAME). QNAME).
* Rationale: A Query and Response represents the basic level of * Rationale: A Query and Response represents the basic level of
a clients interaction with the server. Also, combining the a client's interaction with the server. Also, combining the
Query and Response into one item often reduces storage Query and Response into one item often reduces storage
requirements due to commonality in the data of the two requirements due to commonality in the data of the two
messages. messages.
In the context of generating a C-DNS file it is assumed that only
those DNS payloads which can be parsed to produce a well-formed
DNS message are stored in the C-DNS format and that all other
messages will be (optionally) recorded as malformed messages.
Parsing a well-formed message means as a minimum:
* The packet has a well-formed 12 byte DNS Header with a
recognised OPCODE.
* The section counts are consistent with the section contents.
* All of the resource records can be fully parsed.
2. All top level fields in each Q/R data item will be optional. 2. All top level fields in each Q/R data item will be optional.
* Rationale: Different users will have different requirements * Rationale: Different users will have different requirements
for data to be available for analysis. Users with minimal for data to be available for analysis. Users with minimal
requirements should not have to pay the cost of recording full requirements should not have to pay the cost of recording full
data, however this will limit the ability to perform certain data, though this will limit the ability to perform certain
kinds of data analysis and also reconstruct packet captures. kinds of data analysis and also to reconstruct packet
For example, omitting the resource records from a Response captures. For example, omitting the resource records from a
will reduce the C-DNS file size, and in principle responses Response will reduce the C-DNS file size; in principle
can be synthesized if there is enough context. responses can be synthesized if there is enough context.
3. Multiple Q/R data items will be collected into blocks in the 3. Multiple Q/R data items will be collected into blocks in the
format. Common data in a block will be abstracted and referenced format. Common data in a block will be abstracted and referenced
from individual Q/R data items by indexing. The maximum number from individual Q/R data items by indexing. The maximum number
of Q/R data items in a block will be configurable. of Q/R data items in a block will be configurable.
* Rationale: This blocking and indexing provides a significant * Rationale: This blocking and indexing provides a significant
reduction in the volume of file data generated. Although this reduction in the volume of file data generated. Although this
introduces complexity, it provides compression of the data introduces complexity, it provides compression of the data
that makes use of knowledge of the DNS message structure. that makes use of knowledge of the DNS message structure.
* It is anticipated that the files produced can be subject to * It is anticipated that the files produced can be subject to
further compression using general purpose compression tools. further compression using general purpose compression tools.
Measurements show that blocking significantly reduces the CPU Measurements show that blocking significantly reduces the CPU
required to perform such strong compression. See required to perform such strong compression. See
Appendix C.2. Appendix C.2.
* [TODO: Further discussion of commonality between DNS messages * Examples of commonality between DNS messages are that in most
e.g. common query signatures, a finite set of valid responses cases the QUESTION RR is the same in the query and response,
from authoritatives] and that there is a finite set of query signatures (based on a
subset of attributes). For many authoritative servers there
is very likely to be a finite set of responses that are
generated, of which a large number are NXDOMAIN.
4. Traffic metadata can optionally be included in each block. 4. Traffic metadata can optionally be included in each block.
Specifically, counts of some types of non-DNS packets (e.g. Specifically, counts of some types of non-DNS packets (e.g.
ICMP, TCP resets) sent to the server may be of interest. ICMP, TCP resets) sent to the server may be of interest.
5. The wire format content of malformed DNS messages can optionally 5. The wire format content of malformed DNS messages may optionally
be recorded. be recorded.
* Rationale: Any structured capture format that does not capture * Rationale: Any structured capture format that does not capture
the DNS payload byte for byte will be limited to some extent the DNS payload byte for byte will be limited to some extent
in that it cannot represent "malformed" DNS messages (see in that it cannot represent malformed DNS messages. Only
Section 8). Only those messages that can be fully parsed and those messages that can be fully parsed and transformed into
transformed into the structured format can be fully the structured format can be fully represented. Note,
represented. Therefore it can greatly aid downstream analysis however, this can result in rather misleading statistics. For
to have the wire format of the malformed DNS messages example, a malformed query which cannot be represented in the
available directly in the C-DNS file. Note, however, this can C-DNS format will lead to the (well formed) DNS responses with
result in rather misleading statistics. For example, a error code FORMERR appearing as 'unmatched'. Therefore it can
malformed query which cannot be represented in the C-DNS greatly aid downstream analysis to have the wire format of the
format will lead to the (well formed) DNS responses with error malformed DNS messages available directly in the C-DNS file.
code FORMERR appearing as 'unmatched'.
5. Choice of CBOR 5. Choice of CBOR
This document presents a detailed format description using CBOR, the This document presents a detailed format description using CBOR, the
Concise Binary Object Representation defined in [RFC7049]. Concise Binary Object Representation defined in [RFC7049].
The choice of CBOR was made taking a number of factors into account. The choice of CBOR was made taking a number of factors into account.
o CBOR is a binary representation, and thus is economical in storage o CBOR is a binary representation, and thus is economical in storage
space. space.
skipping to change at page 9, line 6 skipping to change at page 9, line 22
o CBOR is an IETF standard and familiar to IETF participants. It is o CBOR is an IETF standard and familiar to IETF participants. It is
based on the now-common ideas of lists and objects, and thus based on the now-common ideas of lists and objects, and thus
requires very little familiarization for those in the wider requires very little familiarization for those in the wider
industry. industry.
o CBOR is a simple format, and can easily be implemented from o CBOR is a simple format, and can easily be implemented from
scratch if necessary. More complex formats require library scratch if necessary. More complex formats require library
support which may present problems on unusual platforms. support which may present problems on unusual platforms.
o CBOR can also be easily converted to text formats such as JSON o CBOR can also be easily converted to text formats such as JSON
([RFC7159]) for debugging and other human inspection requirements. ([RFC8259]) for debugging and other human inspection requirements.
o CBOR data schemas can be described using CDDL o CBOR data schemas can be described using CDDL
[I-D.ietf-cbor-cddl]. [I-D.ietf-cbor-cddl].
6. C-DNS format conceptual overview 6. C-DNS format conceptual overview
The following figures show purely schematic representations of the The following figures show purely schematic representations of the
C-DNS format to convey the high-level structure of the C-DNS format. C-DNS format to convey the high-level structure of the C-DNS format.
Section 7 provides a detailed discussion of the CBOR representation Section 7 provides a detailed discussion of the CBOR representation
and individual elements. and individual elements.
skipping to change at page 10, line 30 skipping to change at page 10, line 45
6.2. Storage Parameters 6.2. Storage Parameters
The Block Parameters item includes a Storage Parameters item - this The Block Parameters item includes a Storage Parameters item - this
contains information about the specific data fields stored in the contains information about the specific data fields stored in the
C-DNS file. C-DNS file.
These parameters include: These parameters include:
o The sub-second timing resolution used by the data. o The sub-second timing resolution used by the data.
o Information (hints) on which optional data items can be expected o Information (hints) on which optional data are omitted. See
to appear in the data. See Section 6.2.1. Section 6.2.1.
o Recorded OPCODES and RR types. See Section 6.2.2. o Recorded OPCODES and RR types. See Section 6.2.2.
o Flags indicating, for example, whether the data is sampled or o Flags indicating, for example, whether the data is sampled or
anonymised. See Section 6.2.3. anonymised. See Section 6.2.3.
o Client and server IPv4 and IPv6 address prefixes. See o Client and server IPv4 and IPv6 address prefixes. See
Section 6.2.4 Section 6.2.4
6.2.1. Optional data items 6.2.1. Optional data items
To enable applications to store data to their precise requirements in To enable implementations to store data to their precise requirements
as space-efficient manner as possible, all fields in the following in as space-efficient manner as possible, all fields in the following
arrays are optional: arrays are optional:
o Query/Response o Query/Response
o Query Signature o Query Signature
o Malformed messages o Malformed messages
In other words, an application can choose to omit any data item that
is not required for its use case. In addition, implementations may In other words, an implementation can choose to omit any data item
be configured to not record all RRs, or only record messages with that is not required for its use case. In addition, implementations
may be configured to not record all RRs, or only record messages with
certain OPCODES. certain OPCODES.
This does, however, mean that a consumer of a C-DNS file faces two This does, however, mean that a consumer of a C-DNS file faces two
problems: problems:
1. How can it quickly determine whether a file contains the data 1. How can it quickly determine if a file definitely does not
items it requires to complete a particular task (e.g. contain the data items it requires to complete a particular task
reconstructing query traffic or performing a specific piece of (e.g. reconstructing query traffic or performing a specific piece
data analysis)? of data analysis)?
2. How can it determine if a data item is not present because it was 2. How can it determine if a data item is not present because it
was:
* explicitly not recorded, or * explicitly not recorded or
* either was not present in the original data stream or the data * the data item was not available/present.
item was not available to the collecting application?
For example, an application capturing C-DNS data from within a For example, capturing C-DNS data from within a nameserver
nameserver implementation is unlikely to be able to record the Client implementation makes it unlikely that the Client Hoplimit can be
Hoplimit. Or, if there is no query ARCount recorded and no query OPT recorded. Or, if there is no query ARCount recorded and no query OPT
RDATA recorded, is that because no query contained an OPT RR, or RDATA recorded, is that because no query contained an OPT RR, or
because that data was not stored? because that data was not stored?
The Storage Parameters therefore also contains a Storage Hints item The Storage Parameters therefore also contains a Storage Hints item
which specifies whether the encoder of the file recorded each data which specifies which items the encoder of the file omits from the
item if it was present. An application decoding that file can then stored data. An implementation decoding that file can then use these
use these to quickly determine whether the input data is rich enough to quickly determine whether the input data is rich enough for its
for its needs. needs.
QUESTION: Should the items within certain tables also be optional 6.2.2. Optional RRs and OPCODEs
e.g. within the RR table should all of Name index, ClassType, TTL and
RDATA be optional?
6.2.2. Optional RRs and OPCODES Also included in the Storage Parameters are explicit arrays listing
the RR types and the OPCODEs to be recorded. These remove any
ambiguity over whether messages containing particular OPCODEs or RR
types are not present because they did not occur, or because the
implementation is not configured to record them.
Also included in the Storage Parameters is an explicit array of the In the case of OPCODEs, for a message to be fully parsable, the
RR types and OPCODES that were recorded. Using an explicit array OPCODE must be known to the collecting implementation. Any message
removes any ambiguity about whether the OPCODE/RR type was not with an OPCODE unknown to the collecting implementation cannot be
recognised by the collecting implementation or whether it was validated as correctly formed, and so must be treated as malformed.
specifically configured not to record it. Messages with OPCODES known to the recording application but not
listed in the Storage Parameters are discarded (regardless of whether
they are malformed or not).
In the case of RR records, each record must be parsable, including In the case of RR records, each record in a message must be fully
parsing the record RDATA, to determine whether it is correctly parsable, including parsing the record RDATA, as otherwise the
formed. Otherwise it has to be regarded as at least potentially message cannot be validated as correctly formed. Any RR record with
partially malformed. See Section 8 for further discussion of storing an RR type not known to the collecting implementation cannot be
partially parsed messages. validated as correctly formed, and so must be treated as malformed.
For the case of unrecognised OPCODES the message may be parsable (for Once a message is correctly parsed, an implementation is free to
example, if it has a format similar enough to the one described in record only a subset of the RR records present.
[RFC1035]) or it may not. See Section 8 for further discussion of
storing partially parsed messages.
6.2.3. Storage flags 6.2.3. Storage flags
The Storage Parameters contains flags that can be used to indicate The Storage Parameters contains flags that can be used to indicate
if: if:
o the data is anonymised, o the data is anonymised,
o the data is produced from sample data, or o the data is produced from sample data, or
skipping to change at page 14, line 42 skipping to change at page 15, line 15
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
| major-format-version | M | U | Unsigned integer '1'. The major | | major-format-version | M | U | Unsigned integer '1'. The major |
| | | | version of format used in file. | | | | | version of format used in file. |
| | | | | | | | | |
| minor-format-version | M | U | Unsigned integer '0'. The minor | | minor-format-version | M | U | Unsigned integer '0'. The minor |
| | | | version of format used in file. | | | | | version of format used in file. |
| | | | | | | | | |
| private-version | O | U | Version indicator available for | | private-version | O | U | Version indicator available for |
| | | | private use by applications. | | | | | private use by implementations. |
| | | | | | | | | |
| block-parameters | M | A | Array of items of type | | block-parameters | M | A | Array of items of type |
| | | | "BlockParameters", see Section | | | | | "BlockParameters", see Section |
| | | | 7.4.1. The array must contain at | | | | | 7.4.1. The array must contain at |
| | | | least one entry. (The "block- | | | | | least one entry. (The "block- |
| | | | parameters-index" item in each | | | | | parameters-index" item in each |
| | | | "BlockPreamble" indicates which | | | | | "BlockPreamble" indicates which |
| | | | array entry applies to that | | | | | array entry applies to that |
| | | | "Block".) | | | | | "Block".) |
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
skipping to change at page 15, line 45 skipping to change at page 16, line 18
| | | | ticks in a second. | | | | | ticks in a second. |
| | | | | | | | | |
| max-block-items | M | U | The maximum number of items stored in | | max-block-items | M | U | The maximum number of items stored in |
| | | | any of the arrays in a "Block" item | | | | | any of the arrays in a "Block" item |
| | | | (Q/R items, address event counts or | | | | | (Q/R items, address event counts or |
| | | | malformed messages). An indication to | | | | | malformed messages). An indication to |
| | | | a decoder of the resources needed to | | | | | a decoder of the resources needed to |
| | | | process the file. | | | | | process the file. |
| | | | | | | | | |
| storage-hints | M | M | Collection of hints as to which fields | | storage-hints | M | M | Collection of hints as to which fields |
| | | | are present in the arrays that have | | | | | are omitted in the arrays that have |
| | | | optional fields. Map of type | | | | | optional fields. Map of type |
| | | | "StorageHints", see Section 7.4.1.1.1. | | | | | "StorageHints", see Section 7.4.1.1.1. |
| | | | | | | | | |
| opcodes | M | A | Array of OPCODES (unsigned integers) | | opcodes | M | A | Array of OPCODES (unsigned integers) |
| | | | recorded by the collection | | | | | recorded by the collection |
| | | | application. See Section 6.2.2. | | | | | implementation. See Section 6.2.2. |
| | | | | | | | | |
| rr-types | M | A | Array of RR types (unsigned integers) | | rr-types | M | A | Array of RR types (unsigned integers) |
| | | | recorded by the collection | | | | | recorded by the collection |
| | | | application. See Section 6.2.2. | | | | | implementation. See Section 6.2.2. |
| | | | | | | | | |
| storage-flags | O | U | Bit flags indicating attributes of | | storage-flags | O | U | Bit flags indicating attributes of |
| | | | stored data. | | | | | stored data. |
| | | | Bit 0. The data has been anonymised. | | | | | Bit 0. 1 if the data has been |
| | | | Bit 1. The data is sampled data. | | | | | anonymised. |
| | | | Bit 2. Names have been normalised | | | | | Bit 1. 1 if the data is sampled data. |
| | | | (converted to uniform case). | | | | | Bit 2. 1 if the names have been |
| | | | normalised (converted to uniform |
| | | | case). |
| | | | | | | | | |
| client-address | O | U | IPv4 client address prefix length. If | | client-address | O | U | IPv4 client address prefix length. If |
| -prefix-ipv4 | | | specified, only the address prefix | | -prefix-ipv4 | | | specified, only the address prefix |
| | | | bits are stored. | | | | | bits are stored. |
| | | | | | | | | |
| client-address | O | U | IPv6 client address prefix length. If | | client-address | O | U | IPv6 client address prefix length. If |
| -prefix-ipv6 | | | specified, only the address prefix | | -prefix-ipv6 | | | specified, only the address prefix |
| | | | bits are stored. | | | | | bits are stored. |
| | | | | | | | | |
| server-address | O | U | IPv4 server address prefix length. If | | server-address | O | U | IPv4 server address prefix length. If |
skipping to change at page 16, line 40 skipping to change at page 17, line 16
| | | | | | | | | |
| sampling-method | O | T | Information on the sampling method | | sampling-method | O | T | Information on the sampling method |
| | | | used. See Section 6.2.3. | | | | | used. See Section 6.2.3. |
| | | | | | | | | |
| anonymisation | O | T | Information on the anonymisation | | anonymisation | O | T | Information on the anonymisation |
| -method | | | method used. See Section 6.2.3. | | -method | | | method used. See Section 6.2.3. |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
7.4.1.1.1. "StorageHints" 7.4.1.1.1. "StorageHints"
An indicator of which fields the collecting application stores in the An indicator of which fields the collecting implementation omits in
arrays with optional fields. A map containing the following: the arrays with optional fields. A map containing the following:
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
| query-response | M | U | Hints indicating which "QueryResponse" | | query-response | M | U | Hints indicating which "QueryResponse" |
| -hints | | | fields are stored, see section Section | | -hints | | | fields are omitted, see section |
| | | | 7.6. If the field is stored the bit is | | | | | Section 7.6. If the field is omitted |
| | | | set. | | | | | the bit is unset. |
| | | | Bit 0. time-offset | | | | | Bit 0. time-offset |
| | | | Bit 1. client-address-index | | | | | Bit 1. client-address-index |
| | | | Bit 2. client-port | | | | | Bit 2. client-port |
| | | | Bit 3. transaction-id | | | | | Bit 3. transaction-id |
| | | | Bit 4. qr-signature-index | | | | | Bit 4. qr-signature-index |
| | | | Bit 5. client-hoplimit | | | | | Bit 5. client-hoplimit |
| | | | Bit 6. response-delay | | | | | Bit 6. response-delay |
| | | | Bit 7. query-name-index | | | | | Bit 7. query-name-index |
| | | | Bit 8. query-size | | | | | Bit 8. query-size |
| | | | Bit 9. response-size | | | | | Bit 9. response-size |
skipping to change at page 17, line 23 skipping to change at page 17, line 47
| | | | Bit 11. query-question-sections | | | | | Bit 11. query-question-sections |
| | | | Bit 12. query-answer-sections | | | | | Bit 12. query-answer-sections |
| | | | Bit 13. query-authority-sections | | | | | Bit 13. query-authority-sections |
| | | | Bit 14. query-additional-sections | | | | | Bit 14. query-additional-sections |
| | | | Bit 15. response-answer-sections | | | | | Bit 15. response-answer-sections |
| | | | Bit 16. response-authority-sections | | | | | Bit 16. response-authority-sections |
| | | | Bit 17. response-additional-sections | | | | | Bit 17. response-additional-sections |
| | | | | | | | | |
| query-response | M | U | Hints indicating which | | query-response | M | U | Hints indicating which |
| -signature-hints | | | "QueryResponseSignature" fields are | | -signature-hints | | | "QueryResponseSignature" fields are |
| | | | stored, see section Section 7.5.3.2. | | | | | omitted, see section Section 7.5.3.2. |
| | | | If the field is stored the bit is set. | | | | | If the field is omitted the bit is |
| | | | unset. |
| | | | Bit 0. server-address | | | | | Bit 0. server-address |
| | | | Bit 1. server-port | | | | | Bit 1. server-port |
| | | | Bit 2. qr-transport-flags | | | | | Bit 2. qr-transport-flags |
| | | | Bit 3. qr-type | | | | | Bit 3. qr-type |
| | | | Bit 4. qr-sig-flags | | | | | Bit 4. qr-sig-flags |
| | | | Bit 5. query-opcode | | | | | Bit 5. query-opcode |
| | | | Bit 6. dns-flags | | | | | Bit 6. dns-flags |
| | | | Bit 7. query-rcode | | | | | Bit 7. query-rcode |
| | | | Bit 8. query-class-type | | | | | Bit 8. query-class-type |
| | | | Bit 9. query-qdcount | | | | | Bit 9. query-qdcount |
| | | | Bit 10. query-ancount | | | | | Bit 10. query-ancount |
| | | | Bit 11. query-nscount | | | | | Bit 11. query-nscount |
| | | | Bit 12. query-arcount | | | | | Bit 12. query-arcount |
| | | | Bit 13. query-edns-version | | | | | Bit 13. query-edns-version |
| | | | Bit 14. query-udp-size | | | | | Bit 14. query-udp-size |
| | | | Bit 15. query-opt-rdata | | | | | Bit 15. query-opt-rdata |
| | | | Bit 16. response-rcode | | | | | Bit 16. response-rcode |
| | | | | | | | | |
| rr-hints | M | U | Hints indicating which optional "RR" | | rr-hints | M | U | Hints indicating which optional "RR" |
| | | | fields are stored, see Section | | | | | fields are omitted, see Section |
| | | | 7.5.3.4. If the data type is stored | | | | | 7.5.3.4. If the field is omitted the |
| | | | the bit is set. | | | | | bit is unset. |
| | | | Bit 0. ttl | | | | | Bit 0. ttl |
| | | | Bit 1. rdata-index |
| other-data-hints | M | U | Hints indicating which other data | | other-data-hints | M | U | Hints indicating which other data |
| | | | types are stored. If the data type is | | | | | types are are omitted. If the data |
| | | | stored the bit is set. | | | | | type is are omitted the bit is unset. |
| | | | Bit 0. malformed-messages | | | | | Bit 0. malformed-messages |
| | | | Bit 1. address-event-counts | | | | | Bit 1. address-event-counts |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
TODO: Revise non-QueryResponse hints to cover optional fields in
malformed message data maps.
7.4.2. "CollectionParameters" 7.4.2. "CollectionParameters"
Parameters relating to how data in the file was collected. Parameters relating to how data in the file was collected.
These parameters have no default. If they do not appear, nothing can These parameters have no default. If they do not appear, nothing can
be inferred about their value. be inferred about their value.
A map containing the following items: A map containing the following items:
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
skipping to change at page 21, line 18 skipping to change at page 21, line 18
| earliest-time | O | A | A timestamp (2 unsigned integers, | | earliest-time | O | A | A timestamp (2 unsigned integers, |
| | | | "Timestamp") for the earliest record | | | | | "Timestamp") for the earliest record |
| | | | in the "Block" item. The first integer | | | | | in the "Block" item. The first integer |
| | | | is the number of seconds since the | | | | | is the number of seconds since the |
| | | | Posix epoch ("time_t"). The second | | | | | Posix epoch ("time_t"). The second |
| | | | integer is the number of ticks since | | | | | integer is the number of ticks since |
| | | | the start of the second. This | | | | | the start of the second. This |
| | | | timestamp can only be omitted if all | | | | | timestamp can only be omitted if all |
| | | | block items containing a time offset | | | | | block items containing a time offset |
| | | | from the start of the block also omit | | | | | from the start of the block also omit |
| | | | the timestamp. | | | | | that time offset. |
| | | | | | | | | |
| block-parameters | O | U | The index of the item in the "block- | | block-parameters | O | U | The index of the item in the "block- |
| -index | | | parameters" array (in the "file- | | -index | | | parameters" array (in the "file- |
| | | | premable" item) applicable to this | | | | | premable" item) applicable to this |
| | | | block. If not present, index 0 is | | | | | block. If not present, index 0 is |
| | | | used. See Section 7.4.1. | | | | | used. See Section 7.4.1. |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
7.5.2. "BlockStatistics" 7.5.2. "BlockStatistics"
Basic statistical information about a "Block" item. A map containing Basic statistical information about a "Block" item. A map containing
the following: the following:
+---------------------+---+---+-------------------------------------+ +---------------------+---+---+-------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+---------------------+---+---+-------------------------------------+ +---------------------+---+---+-------------------------------------+
| total-messages | O | U | Total number of DNS messages | | processed-messages | O | U | Total number of DNS messages |
| | | | processed from the input traffic | | | | | processed from the input traffic |
| | | | stream during collection of data in | | | | | stream during collection of data in |
| | | | this "Block" item. | | | | | this "Block" item. |
| | | | | | | | | |
| total-pairs | O | U | Total number of Q/R data items in | | qr-data-items | O | U | Total number of Q/R data items in |
| | | | this "Block" item. | | | | | this "Block" item. |
| | | | | | | | | |
| unmatched-queries | O | U | Number of unmatched queries in this | | unmatched-queries | O | U | Number of unmatched queries in this |
| | | | "Block" item. | | | | | "Block" item. |
| | | | | | | | | |
| unmatched-responses | O | U | Number of unmatched responses in | | unmatched-responses | O | U | Number of unmatched responses in |
| | | | this "Block" item. | | | | | this "Block" item. |
| | | | | | | | | |
| malformed-messages | O | U | Number of malformed messages found | | discarded-opcode | O | U | Number of DNS messages processed |
| | | | from the input traffic stream |
| | | | during collection of data in this |
| | | | "Block" item but not recorded |
| | | | because their OPCODE is not in the |
| | | | list to be collected. |
| | | | |
| malformed-items | O | U | Number of malformed messages found |
| | | | in input for this "Block" item. | | | | | in input for this "Block" item. |
+---------------------+---+---+-------------------------------------+ +---------------------+---+---+-------------------------------------+
7.5.3. "BlockTables" 7.5.3. "BlockTables"
Arrays containing data referenced by individual "QueryResponse" or Arrays containing data referenced by individual "QueryResponse" or
"MalformedMessage" items in this "Block". Each element is an array "MalformedMessage" items in this "Block". Each element is an array
which, if present, must not be empty. which, if present, must not be empty.
An item in the "qlist" array contains indexes to values in the "qrr" An item in the "qlist" array contains indexes to values in the "qrr"
skipping to change at page 23, line 51 skipping to change at page 24, line 33
| Field | O | T | Description | | Field | O | T | Description |
+--------------------+---+---+--------------------------------------+ +--------------------+---+---+--------------------------------------+
| server-address | O | U | The index in the item in the "ip- | | server-address | O | U | The index in the item in the "ip- |
| -index | | | address" array of the server IP | | -index | | | address" array of the server IP |
| | | | address. See Section 7.5.3. | | | | | address. See Section 7.5.3. |
| | | | | | | | | |
| server-port | O | U | The server port. | | server-port | O | U | The server port. |
| | | | | | | | | |
| qr-transport-flags | O | U | Bit flags describing the transport | | qr-transport-flags | O | U | Bit flags describing the transport |
| | | | used to service the query. | | | | | used to service the query. |
| | | | Bit 0. IP version. 0 = IPv4, 1 = | | | | | Bit 0. IP version. 0 if IPv4, 1 if |
| | | | IPv6 | | | | | IPv6 |
| | | | Bit 1-4. Transport. 0 = UDP, 1 = | | | | | Bit 1-4. Transport. 4 bit unsigned |
| | | | TCP, 2 = TLS, 3 = DTLS. | | | | | value where 0 = UDP, 1 = TCP, 2 = |
| | | | Bit 5. Trailing bytes in query | | | | | TLS, 3 = DTLS. Values 4-15 are |
| | | | payload. The DNS query message in | | | | | reserved for future use. |
| | | | the UDP or TCP payload was followed | | | | | Bit 5. 1 if trailing bytes in query |
| | | | by some additional bytes, which were | | | | | packet. See Section 10.2. |
| | | | discarded. |
| | | | | | | | | |
| qr-type | O | U | Type of Query/Response transaction. | | qr-type | O | U | Type of Query/Response transaction. |
| | | | 0 = Stub. A query from a stub | | | | | 0 = Stub. A query from a stub |
| | | | resolver. | | | | | resolver. |
| | | | 1 = Client. An incoming query to a | | | | | 1 = Client. An incoming query to a |
| | | | recursive resolver. | | | | | recursive resolver. |
| | | | 2 = Resolver. A query sent from a | | | | | 2 = Resolver. A query sent from a |
| | | | recursive resolver to an authorative | | | | | recursive resolver to an authorative |
| | | | resolver. | | | | | resolver. |
| | | | 3 = Authorative. A query to an | | | | | 3 = Authorative. A query to an |
| | | | authorative resolver. | | | | | authorative resolver. |
| | | | 4 = Forwarder. A query sent from a | | | | | 4 = Forwarder. A query sent from a |
| | | | recursive resolver to an upstream | | | | | recursive resolver to an upstream |
| | | | recursive resolver. | | | | | recursive resolver. |
| | | | 5 = Tool. A query sent to a server | | | | | 5 = Tool. A query sent to a server |
| | | | by a server tool. | | | | | by a server tool. |
| | | | | | | | | |
| qr-sig-flags | O | U | Bit flags indicating information | | qr-sig-flags | O | U | Bit flags explicitly indicating |
| | | | present in this Q/R data item. | | | | | attributes of the message pair |
| | | | Bit 0. 1 if a Query is present. | | | | | represented by this Q/R data item |
| | | | Bit 1. 1 if a Response is present. | | | | | (not all attributes may be recorded |
| | | | Bit 2. 1 if one or more Question is | | | | | or deducible). |
| | | | present. | | | | | Bit 0. 1 if a Query was present. |
| | | | Bit 3. 1 if a Query is present and | | | | | Bit 1. 1 if a Response was present. |
| | | | it has an OPT Resource Record. | | | | | Bit 2. 1 if a Query was present and |
| | | | Bit 4. 1 if a Response is present | | | | | it had an OPT Resource Record. |
| | | | and it has an OPT Resource Record. | | | | | Bit 3. 1 if a Response was present |
| | | | Bit 5. 1 if a Response is present | | | | | and it had an OPT Resource Record. |
| | | | but has no Question. | | | | | Bit 4. 1 if a Query was present but |
| | | | had no Question. |
| | | | Bit 5. 1 if a Response was present |
| | | | but had no Question (only one query- |
| | | | name-index is stored per Q/R item). |
| | | | | | | | | |
| query-opcode | O | U | Query OPCODE. | | query-opcode | O | U | Query OPCODE. |
| | | | | | | | | |
| qr-dns-flags | O | U | Bit flags with values from the Query | | qr-dns-flags | O | U | Bit flags with values from the Query |
| | | | and Response DNS flags. Flag values | | | | | and Response DNS flags. Flag values |
| | | | are 0 if the Query or Response is | | | | | are 0 if the Query or Response is |
| | | | not present. | | | | | not present. |
| | | | Bit 0. Query Checking Disabled (CD). | | | | | Bit 0. Query Checking Disabled (CD). |
| | | | Bit 1. Query Authenticated Data | | | | | Bit 1. Query Authenticated Data |
| | | | (AD). | | | | | (AD). |
skipping to change at page 26, line 8 skipping to change at page 26, line 40
| | | | | | | | | |
| opt-rdata-index | O | U | The index in the "name-rdata" array | | opt-rdata-index | O | U | The index in the "name-rdata" array |
| | | | of the OPT RDATA. See Section 7.5.3. | | | | | of the OPT RDATA. See Section 7.5.3. |
| | | | | | | | | |
| response-rcode | O | U | Response RCODE. If the Response | | response-rcode | O | U | Response RCODE. If the Response |
| | | | contains OPT, this value | | | | | contains OPT, this value |
| | | | incorporates any | | | | | incorporates any |
| | | | EXTENDED_RCODE_VALUE. | | | | | EXTENDED_RCODE_VALUE. |
+--------------------+---+---+--------------------------------------+ +--------------------+---+---+--------------------------------------+
QUESTION: Currently we collect OPT RDATA as a blob as this is
consistent with and re-uses the generic mechanism for RDATA storage.
Should we break individual EDNS(0) options into Option code and data
and store the data separately in a new array within the Block type?
This would potentially allow exploitation of option data commonality.
7.5.3.3. "Question" 7.5.3.3. "Question"
Details on individual Questions in a Question section. A map Details on individual Questions in a Question section. A map
containing the following: containing the following:
+-----------------+---+---+-----------------------------------------+ +-----------------+---+---+-----------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+-----------------+---+---+-----------------------------------------+ +-----------------+---+---+-----------------------------------------+
| name-index | M | U | The index in the "name-rdata" array of | | name-index | M | U | The index in the "name-rdata" array of |
| | | | the QNAME. See Section 7.5.3. | | | | | the QNAME. See Section 7.5.3. |
skipping to change at page 26, line 47 skipping to change at page 27, line 33
+-----------------+---+---+-----------------------------------------+ +-----------------+---+---+-----------------------------------------+
| name-index | M | U | The index in the "name-rdata" array of | | name-index | M | U | The index in the "name-rdata" array of |
| | | | the NAME. See Section 7.5.3. | | | | | the NAME. See Section 7.5.3. |
| | | | | | | | | |
| classtype-index | M | U | The index in the "classtype" array of | | classtype-index | M | U | The index in the "classtype" array of |
| | | | the CLASS and TYPE of the RR. See | | | | | the CLASS and TYPE of the RR. See |
| | | | Section 7.5.3. | | | | | Section 7.5.3. |
| | | | | | | | | |
| ttl | O | U | The RR Time to Live. | | ttl | O | U | The RR Time to Live. |
| | | | | | | | | |
| rdata-index | M | U | The index in the "name-rdata" array of | | rdata-index | O | U | The index in the "name-rdata" array of |
| | | | the RR RDATA. See Section 7.5.3. | | | | | the RR RDATA. See Section 7.5.3. |
+-----------------+---+---+-----------------------------------------+ +-----------------+---+---+-----------------------------------------+
7.5.3.5. "MalformedMessageData" 7.5.3.5. "MalformedMessageData"
Details on malformed message items in this "Block" item. A map Details on malformed message items in this "Block" item. A map
containing the following: containing the following:
+--------------------+---+---+--------------------------------------+ +--------------------+---+---+--------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+--------------------+---+---+--------------------------------------+ +--------------------+---+---+--------------------------------------+
| server-address | O | U | The index in the "ip-address" array | | server-address | O | U | The index in the "ip-address" array |
| -index | | | of the server IP address. See | | -index | | | of the server IP address. See |
| | | | Section 7.5.3. | | | | | Section 7.5.3. |
| | | | | | | | | |
| server-port | O | U | The server port. | | server-port | O | U | The server port. |
| | | | | | | | | |
| mm-transport-flags | O | U | Bit flags describing the transport | | mm-transport-flags | O | U | Bit flags describing the transport |
| | | | used to service the query. Bit 0 is | | | | | used to service the query. Bit 0 is |
| | | | the least significant bit. | | | | | the least significant bit. |
| | | | Bit 0. IP version. 0 = IPv4, 1 = | | | | | Bit 0. IP version. 0 if IPv4, 1 if |
| | | | IPv6 | | | | | IPv6 |
| | | | Bit 1-4. Transport. 0 = UDP, 1 = | | | | | Bit 1-4. Transport. 4 bit unsigned |
| | | | TCP, 2 = TLS, 3 = DTLS. | | | | | value where 0 = UDP, 1 = TCP, 2 = |
| | | | TLS, 3 = DTLS. Values 4-15 are |
| | | | reserved for future use. |
| | | | | | | | | |
| mm-payload | O | B | The payload (raw bytes) of the DNS | | mm-payload | O | B | The payload (raw bytes) of the DNS |
| | | | message. | | | | | message. |
+--------------------+---+---+--------------------------------------+ +--------------------+---+---+--------------------------------------+
7.6. "QueryResponse" 7.6. "QueryResponse"
Details on individual Q/R data items. Details on individual Q/R data items.
Note that there is no requirement that the elements of the "query- Note that there is no requirement that the elements of the "query-
skipping to change at page 28, line 50 skipping to change at page 29, line 47
| | | | Section 7.6.2. | | | | | Section 7.6.2. |
| | | | | | | | | |
| response-extended | O | M | Extended Response data. Map of | | response-extended | O | M | Extended Response data. Map of |
| | | | type "QueryResponseExtended", see | | | | | type "QueryResponseExtended", see |
| | | | Section 7.6.2. | | | | | Section 7.6.2. |
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
The "query-size" and "response-size" fields hold the DNS message The "query-size" and "response-size" fields hold the DNS message
size. For UDP this is the size of the UDP payload that contained the size. For UDP this is the size of the UDP payload that contained the
DNS message. For TCP it is the size of the DNS message as specified DNS message. For TCP it is the size of the DNS message as specified
in the two-byte message length header. Trailing bytes with queries in the two-byte message length header. Trailing bytes in UDP queries
are routinely observed in traffic to authoritative servers and this are routinely observed in traffic to authoritative servers and this
value allows a calculation of how many trailing bytes were present. value allows a calculation of how many trailing bytes were present.
7.6.1. "ResponseProcessingData" 7.6.1. "ResponseProcessingData"
Information on the server processing that produced the response. A Information on the server processing that produced the response. A
map containing the following: map containing the following:
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
| bailiwick-index | O | U | The index in the "name-rdata" array of | | bailiwick-index | O | U | The index in the "name-rdata" array of |
| | | | the owner name for the response | | | | | the owner name for the response |
| | | | bailiwick. See Section 7.5.3. | | | | | bailiwick. See Section 7.5.3. |
| | | | | | | | | |
| processing-flags | O | U | Flags relating to response processing. | | processing-flags | O | U | Flags relating to response processing. |
| | | | Bit 0. 1 if the response came from | | | | | Bit 0. 1 if the response came from |
| | | | cache. | | | | | cache. |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
QUESTION: Should this be an item in the "QueryResponseSignature"?
7.6.2. "QueryResponseExtended" 7.6.2. "QueryResponseExtended"
Extended data on the Q/R data item. Extended data on the Q/R data item.
Each item in the map is present only if collection of the relevant Each item in the map is present only if collection of the relevant
details is configured. details is configured.
A map containing the following items: A map containing the following items:
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
skipping to change at page 30, line 27 skipping to change at page 31, line 27
| | | | Response. See Section 7.5.3. | | | | | Response. See Section 7.5.3. |
| | | | | | | | | |
| authority-index | O | U | The index in the "rrlist" array of the | | authority-index | O | U | The index in the "rrlist" array of the |
| | | | entry listing the Authority Resource | | | | | entry listing the Authority Resource |
| | | | Record sections for the Query or | | | | | Record sections for the Query or |
| | | | Response. See Section 7.5.3. | | | | | Response. See Section 7.5.3. |
| | | | | | | | | |
| additional-index | O | U | The index in the "rrlist" array of the | | additional-index | O | U | The index in the "rrlist" array of the |
| | | | entry listing the Additional Resource | | | | | entry listing the Additional Resource |
| | | | Record sections for the Query or | | | | | Record sections for the Query or |
| | | | Response. See Section 7.5.3. | | | | | Response. See Section 7.5.3. Note that |
| | | | Query OPT RR data can be optionally |
| | | | stored in the QuerySignature. |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
7.7. "AddressEventCount" 7.7. "AddressEventCount"
Counts of various IP related events relating to traffic with Counts of various IP related events relating to traffic with
individual client addresses. A map containing the following: individual client addresses. A map containing the following:
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
skipping to change at page 31, line 29 skipping to change at page 32, line 29
| ae-address-index | M | U | The index in the "ip-address" array of | | ae-address-index | M | U | The index in the "ip-address" array of |
| | | | the client address. See Section 7.5.3. | | | | | the client address. See Section 7.5.3. |
| | | | | | | | | |
| ae-count | M | U | The number of occurrences of this | | ae-count | M | U | The number of occurrences of this |
| | | | event during the block collection | | | | | event during the block collection |
| | | | period. | | | | | period. |
+------------------+---+---+----------------------------------------+ +------------------+---+---+----------------------------------------+
7.8. "MalformedMessage" 7.8. "MalformedMessage"
Details of malformed messages. See Section 8. A map containing the Details of malformed messages. A map containing the following:
following:
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
| Field | O | T | Description | | Field | O | T | Description |
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
| time-offset | O | U | Message timestamp as an offset in | | time-offset | O | U | Message timestamp as an offset in |
| | | | ticks from "earliest-time". | | | | | ticks from "earliest-time". |
| | | | | | | | | |
| client-address-index | O | U | The index in the "ip-address" | | client-address-index | O | U | The index in the "ip-address" |
| | | | array of the client IP address. | | | | | array of the client IP address. |
| | | | See Section 7.5.3. | | | | | See Section 7.5.3. |
| | | | | | | | | |
| client-port | O | U | The client port. | | client-port | O | U | The client port. |
| | | | | | | | | |
| message-data-index | O | U | The index in the "malformed- | | message-data-index | O | U | The index in the "malformed- |
| | | | message-data" array of the message | | | | | message-data" array of the message |
| | | | data for this message. See Section | | | | | data for this message. See Section |
| | | | 7.5.3. | | | | | 7.5.3. |
+----------------------+---+---+------------------------------------+ +----------------------+---+---+------------------------------------+
8. Malformed messages 8. C-DNS to PCAP
In the context of generating a C-DNS file it is assumed that only
those DNS messages which can be parsed to produce a well-formed DNS
message are stored in the C-DNS format and that all other messages
will be recorded (if at all) as malformed messages.
Parsing a well-formed message means as a minimum:
o The packet has a well-formed 12 byte DNS Header
o The section counts are consistent with the section contents
o All of the resource records can be parsed
In principle, packets that do not meet these criteria could be
classified into two categories:
o Partially malformed: those packets which can be decoded
sufficiently to extract
* a well-formed 12 byte DNS header (and therefore a DNS
transaction ID)
* the first Question in the Question section if QDCOUNT is
greater than 0
but suffer other issues while parsing. This is the minimum
information required to attempt Query/Response matching as described
in Section 10.1.
o Completely malformed: those packets that cannot be decoded to this
extent.
An open question is whether there is value in attempting to process
partially malformed messages in an analogous manner to well formed
messages in terms of attempting to match them with the corresponding
query or response. This could be done by creating 'placeholder'
records during Query/Response matching with just the information
extracted as above. If the packet were then matched the resulting
C-DNS Q/R data item would include flags to indicate a malformed query
or response or both record (in addition to capturing the wire format
of the packet).
An advantage of this would be that it would result in more meaningful
statistics about matched packets because, for example, some partially
malformed queries could be matched to responses. However it would
only apply to those queries where the first Question is well formed.
It could also simplify the downstream analysis of C-DNS files and the
reconstruction of packet streams from C-DNS.
A disadvantage is that this adds complexity to the Query/Response
matching and data representation, could potentially lead to false
matches and some additional statistics would be required (e.g. counts
for matched-partially-malformed, unmatched-partially-malformed,
completely-malformed).
NOTE: Note that within these definitions a message that contained an
unrecognised OPCODE or RR code would be treated as malformed. It may
be the case that the OPCODE/RR is not recognised just because the
implementation does not support it yet, rather than it not being
standardized. For the case of unrecognised OPCODES the message may
be parsable (for example, if it has a format similar enough to the
one described in [RFC1035]) or it may not. Similarly for
unrecognised RR types the RDATA can still be stored, but the
collector will not be able to process it to remove, for example, name
compression pointers.
QUESTION: There has been no feedback to date requesting further work
on the processing partially malformed messages. The editors are
inclined not to include it in this version. It could be the subject
of a future extension.
9. C-DNS to PCAP
It is possible to re-construct PCAP files from the C-DNS format in a It is possible to re-construct PCAP files from the C-DNS format in a
lossy fashion. Some of the issues with reconstructing both the DNS lossy fashion. Some of the issues with reconstructing both the DNS
payload and the full packet stream are outlined here. payload and the full packet stream are outlined here.
The reconstruction depends on whether or not all the optional The reconstruction depends on whether or not all the optional
sections of both the query and response were captured in the C-DNS sections of both the query and response were captured in the C-DNS
file. Clearly, if they were not all captured, the reconstruction file. Clearly, if they were not all captured, the reconstruction
will be imperfect. will be imperfect.
Even if all sections of the response were captured, one cannot Even if all sections of the response were captured, one cannot
reconstruct the DNS response payload exactly due to the fact that reconstruct the DNS response payload exactly due to the fact that
some DNS names in the message on the wire may have been compressed. some DNS names in the message on the wire may have been compressed.
Section 9.1 discusses this is more detail. Section 8.1 discusses this is more detail.
Some transport information is not captured in the C-DNS format. For Some transport information is not captured in the C-DNS format. For
example, the following aspects of the original packet stream cannot example, the following aspects of the original packet stream cannot
be re-constructed from the C-DNS format: be re-constructed from the C-DNS format:
o IP fragmentation o IP fragmentation
o TCP stream information: o TCP stream information:
* Multiple DNS messages may have been sent in a single TCP * Multiple DNS messages may have been sent in a single TCP
skipping to change at page 34, line 25 skipping to change at page 34, line 5
ICMP ICMP
Simple assumptions can be made on the reconstruction: fragmented and Simple assumptions can be made on the reconstruction: fragmented and
DNS-over-TCP messages can be reconstructed into single packets and a DNS-over-TCP messages can be reconstructed into single packets and a
single TCP session can be constructed for each TCP packet. single TCP session can be constructed for each TCP packet.
Additionally, if malformed messages and Non-DNS packets are captured Additionally, if malformed messages and Non-DNS packets are captured
separately, they can be merged with packet captures reconstructed separately, they can be merged with packet captures reconstructed
from C-DNS to produce a more complete packet stream. from C-DNS to produce a more complete packet stream.
9.1. Name compression 8.1. Name compression
All the names stored in the C-DNS format are full domain names; no All the names stored in the C-DNS format are full domain names; no
DNS style name compression is used on the individual names within the DNS style name compression is used on the individual names within the
format. Therefore when reconstructing a packet, name compression format. Therefore when reconstructing a packet, name compression
must be used in order to reproduce the on the wire representation of must be used in order to reproduce the on the wire representation of
the packet. the packet.
[RFC1035] name compression works by substituting trailing sections of [RFC1035] name compression works by substituting trailing sections of
a name with a reference back to the occurrence of those sections a name with a reference back to the occurrence of those sections
earlier in the message. Not all name server software uses the same earlier in the message. Not all name server software uses the same
skipping to change at page 35, line 13 skipping to change at page 34, line 42
algorithm in turn to see if it reproduces the original length, algorithm in turn to see if it reproduces the original length,
stopping at the first match. This would not guarantee the correct stopping at the first match. This would not guarantee the correct
algorithm has been used as it is possible to match the length whilst algorithm has been used as it is possible to match the length whilst
still not matching the on the wire bytes but, without further still not matching the on the wire bytes but, without further
information added to the C-DNS data, this is the best that can be information added to the C-DNS data, this is the best that can be
achieved. achieved.
Appendix B presents an example of two different compression Appendix B presents an example of two different compression
algorithms used by well-known name server software. algorithms used by well-known name server software.
10. Data collection 9. Data collection
This section describes a non-normative proposed algorithm for the This section describes a non-normative proposed algorithm for the
processing of a captured stream of DNS queries and responses and processing of a captured stream of DNS queries and responses and
matching queries/responses where possible. production of a stream of query/response items, matching queries/
responses where possible.
For the purposes of this discussion, it is assumed that the input has For the purposes of this discussion, it is assumed that the input has
been pre-processed such that: been pre-processed such that:
1. All IP fragmentation reassembly, TCP stream reassembly, and so 1. All IP fragmentation reassembly, TCP stream reassembly, and so
on, has already been performed on, has already been performed.
2. Each message is associated with transport metadata required to 2. Each message is associated with transport metadata required to
generate the Primary ID (see Section 10.2.1) generate the Primary ID (see Section 9.2.1).
3. Each message has a well-formed DNS header of 12 bytes and (if 3. Each message has a well-formed DNS header of 12 bytes and (if
present) the first Question in the Question section can be parsed present) the first Question in the Question section can be parsed
to generate the Secondary ID (see below). As noted earlier, this to generate the Secondary ID (see below). As noted earlier, this
requirement can result in a malformed query being removed in the requirement can result in a malformed query being removed in the
pre-processing stage, but the correctly formed response with pre-processing stage, but the correctly formed response with
RCODE of FORMERR being present. RCODE of FORMERR being present.
DNS messages are processed in the order they are delivered to the DNS messages are processed in the order they are delivered to the
application. It should be noted that packet capture libraries do not implementation.
necessary provide packets in strict chronological order.
TODO: Discuss the corner cases resulting from this in more detail. It should be noted that packet capture libraries do not necessarily
provide packets in strict chronological order. This can, for
example, arise on multi-core platforms where packets arriving at a
network device are processed by different cores. On systems where
this behaviour has been observed, the timestamps associated with each
packet are consistent; queries always have a timestamp prior to the
response timestamp. However, the order in which these packets appear
in the packet capture stream is not necessarily strictly
choronological; a response can appear in the capture stream before
the query that provoked the response. For this discussion, this non-
chronological delivery is termed "skew".
10.1. Matching algorithm In the presence of skew, a response packets can arrive for matching
before the corresponding query. To avoid generating false instances
of responses without a matching query, and queries without a matching
response, the matching algorithm must take account of the possibility
of skew.
9.1. Matching algorithm
A schematic representation of the algorithm for matching Q/R data A schematic representation of the algorithm for matching Q/R data
items is shown in the following diagram: items is shown in the following diagram:
Figure showing the Query/Response matching algorithm format (PNG) [5] Figure showing the Query/Response matching algorithm format (PNG) [5]
Figure showing the Query/Response matching algorithm format (SVG) [6] Figure showing the Query/Response matching algorithm format (SVG) [6]
Further details of the algorithm are given in the following sections. Further details of the algorithm are given in the following sections.
10.2. Message identifiers 9.2. Message identifiers
10.2.1. Primary ID (required) 9.2.1. Primary ID (required)
A Primary ID is constructed for each message. It is composed of the A Primary ID is constructed for each message. It is composed of the
following data: following data:
1. Source IP Address 1. Source IP Address
2. Destination IP Address 2. Destination IP Address
3. Source Port 3. Source Port
4. Destination Port 4. Destination Port
5. Transport 5. Transport
6. DNS Message ID 6. DNS Message ID
10.2.2. Secondary ID (optional) 9.2.2. Secondary ID (optional)
If present, the first Question in the Question section is used as a If present, the first Question in the Question section is used as a
secondary ID for each message. Note that there may be well formed secondary ID for each message. Note that there may be well formed
DNS queries that have a QDCOUNT of 0, and some responses may have a DNS queries that have a QDCOUNT of 0, and some responses may have a
QDCOUNT of 0 (for example, responses with RCODE=FORMERR or NOTIMP). QDCOUNT of 0 (for example, responses with RCODE=FORMERR or NOTIMP).
In this case the secondary ID is not used in matching. In this case the secondary ID is not used in matching.
10.3. Algorithm parameters 9.3. Algorithm parameters
1. Query timeout 1. Query timeout, QT. A query arrives with timestamp t1. If no
response matching that query has arrived before other input
arrives timestamped later than (t1 + QT), a query/response item
containing only a query item is recorded. The query timeout
value is typically of the order of 5 seconds.
2. Skew timeout 2. Skew timeout, ST. A response arrives with timestamp t2. If a
response has not been matched by a query before input arrives
timestamped later than (t2 + ST), a query/response item
containing only a response is recorded. The skew timeout value
is typically a few microseconds.
10.4. Algorithm requirements 9.4. Algorithm requirements
The algorithm is designed to handle the following input data: The algorithm is designed to handle the following input data:
1. Multiple queries with the same Primary ID (but different 1. Multiple queries with the same Primary ID (but different
Secondary ID) arriving before any responses for these queries are Secondary ID) arriving before any responses for these queries are
seen. seen.
2. Multiple queries with the same Primary and Secondary ID arriving 2. Multiple queries with the same Primary and Secondary ID arriving
before any responses for these queries are seen. before any responses for these queries are seen.
3. Queries for which no later response can be found within the 3. Queries for which no later response can be found within the
specified timeout. specified timeout.
4. Responses for which no previous query can be found within the 4. Responses for which no previous query can be found within the
specified timeout. specified timeout.
10.5. Algorithm limitations 9.5. Algorithm limitations
For cases 1 and 2 listed in the above requirements, it is not For cases 1 and 2 listed in the above requirements, it is not
possible to unambiguously match queries with responses. This possible to unambiguously match queries with responses. This
algorithm chooses to match to the earliest query with the correct algorithm chooses to match to the earliest query with the correct
Primary and Secondary ID. Primary and Secondary ID.
10.6. Workspace 9.6. Workspace
A FIFO structure is used to hold the Q/R data items during A FIFO structure is used to hold the Q/R data items during
processing. processing. A secondary responses FIFO holds responses awaiting
matching queries.
10.7. Output 9.7. Output
The output is a list of Q/R data items. Both the Query and Response The output is a list of Q/R data items. Both the Query and Response
elements are optional in these items, therefore Q/R data items have elements are optional in these items, therefore Q/R data items have
one of three types of content: one of three types of content:
1. A matched pair of query and response messages 1. A matched pair of query and response messages
2. A query message with no response 2. A query message with no response
3. A response message with no query 3. A response message with no query
The timestamp of a list item is that of the query for cases 1 and 2 The timestamp of a list item is that of the query for cases 1 and 2
and that of the response for case 3. and that of the response for case 3.
10.8. Post processing 9.8. Post processing
When ending capture, all remaining entries in the Q/R data item FIFO When ending capture, all items in the responses FIFO are timed out
should be treated as timed out queries. immediately, generating response-only entries to the Q/R data item
FIFO. These and all other remaining entries in the Q/R data item
FIFO should be treated as timed out queries.
11. Implementation guidance 10. Implementation guidance
Whilst this document makes no specific recommendations with respect Whilst this document makes no specific recommendations with respect
to Canonical CBOR (see Section 3.9 of [RFC7049]) the following to Canonical CBOR (see Section 3.9 of [RFC7049]) the following
guidance may be of use to implementors. guidance may be of use to implementors.
Adherence to the first two rules given in Section 3.9 of [RFC7049] Adherence to the first two rules given in Section 3.9 of [RFC7049]
will minimise file sizes. will minimise file sizes.
Adherence to the last two rules given in Section 3.9 of [RFC7049] for Adherence to the last two rules given in Section 3.9 of [RFC7049] for
all maps and arrays would unacceptably constrain implementations, for all maps and arrays would unacceptably constrain implementations, for
example, in the use case of real-time data collection in constrained example, in the use case of real-time data collection in constrained
environments. environments.
NOTE: With this clarification to the use of Canonical CBOR, we could 10.1. Optional data
consider re-ordering fields in maps to improve readability.
11.1. Optional data When decoding C-DNS data some of the items required for a particular
function that the consumer wishes to perform may be missing.
Consumers should consider providing configurable default values to be
used in place of the missing values in their output.
When decoding data some items required for a particular function the 10.2. Trailing bytes
consumer wishes to perform may be missing. Consumers should consider
providing configurable default values to be used in place of the
missing values in their output.
11.2. Trailing data in TCP A DNS query message in a UDP or TCP payload can be followed by some
additional (spurious) bytes, which are not stored in C-DNS.
TODO: Clarify the impact of processing wire captures which includes When DNS traffic is sent over TCP, each message is prefixed with a
trailing data in TCP. What will appear as trailing data, what will two byte length field which gives the message length, excluding the
appear as malformed messages? two byte length field. In this context, trailing bytes can occur in
two circumstances with different results:
11.3. Limiting collection of RDATA 1. The number of bytes consumed by fully parsing the message is less
than the number of bytes given in the length field (i.e. the
length field is incorrect and too large). In this case, the
surplus bytes are considered trailing bytes in an analogous
manner to UDP and recorded as such. If only this case occurs it
is possible to process a packet containing multiple DNS messages
where one or more has trailing bytes.
2. There are surplus bytes between the end of a well-formed message
and the start of the length field for the next message. In this
case the first of the surplus bytes will be processed as the
first byte of the next length field, and parsing will proceed
from there, almost certainly leading to the next and any
subsequent messages in the packet being considered malformed.
This will not generate a trailing bytes record for the processed
well-formed message.
10.3. Limiting collection of RDATA
Implementations should consider providing a configurable maximum Implementations should consider providing a configurable maximum
RDATA size for capture , for example, to avoid memory issues when RDATA size for capture, for example, to avoid memory issues when
confronted with large XFR records. confronted with large XFR records.
12. Implementation status 11. Implementation status
[Note to RFC Editor: please remove this section and reference to [Note to RFC Editor: please remove this section and reference to
[RFC7942] prior to publication.] [RFC7942] prior to publication.]
This section records the status of known implementations of the This section records the status of known implementations of the
protocol defined by this specification at the time of posting of this protocol defined by this specification at the time of posting of this
Internet-Draft, and is based on a proposal described in [RFC7942]. Internet-Draft, and is based on a proposal described in [RFC7942].
The description of implementations in this section is intended to The description of implementations in this section is intended to
assist the IETF in its decision processes in progressing drafts to assist the IETF in its decision processes in progressing drafts to
RFCs. Please note that the listing of any individual implementation RFCs. Please note that the listing of any individual implementation
skipping to change at page 39, line 5 skipping to change at page 39, line 36
features. Readers are advised to note that other implementations may features. Readers are advised to note that other implementations may
exist. exist.
According to [RFC7942], "this will allow reviewers and working groups According to [RFC7942], "this will allow reviewers and working groups
to assign due consideration to documents that have the benefit of to assign due consideration to documents that have the benefit of
running code, which may serve as evidence of valuable experimentation running code, which may serve as evidence of valuable experimentation
and feedback that have made the implemented protocols more mature. and feedback that have made the implemented protocols more mature.
It is up to the individual working groups to use this information as It is up to the individual working groups to use this information as
they see fit". they see fit".
12.1. DNS-STATS Compactor 11.1. DNS-STATS Compactor
ICANN/Sinodun IT have developed an open source implementation called ICANN/Sinodun IT have developed an open source implementation called
DNS-STATS Compactor. The Compactor is a suite of tools which can DNS-STATS Compactor. The Compactor is a suite of tools which can
capture DNS traffic (from either a network interface or a PCAP file) capture DNS traffic (from either a network interface or a PCAP file)
and store it in the Compacted-DNS (C-DNS) file format. PCAP files and store it in the Compacted-DNS (C-DNS) file format. PCAP files
for the captured traffic can also be reconstructed. See Compactor for the captured traffic can also be reconstructed. See Compactor
[7]. [7].
This implementation: This implementation:
o is mature but has only been deployed for testing in a single
environment so is not yet classified as production ready.
o covers the whole of the specification described in the -03 draft o covers the whole of the specification described in the -03 draft
with the exception of support for malformed messages (Section 8) with the exception of support for malformed messages and pico
and pico second time resolution. (Note: this implementation does second time resolution. (Note: this implementation does allow
allow malformed messages to be dumped to a PCAP file). malformed messages to be recorded separately in a PCAP file).
o is released under the Mozilla Public License Version 2.0. o is released under the Mozilla Public License Version 2.0.
o has a users mailing list available, see dns-stats-users [8]. o has a users mailing list available, see dns-stats-users [8].
There is also some discussion of issues encountered during There is also some discussion of issues encountered during
development available at Compressing Pcap Files [9] and Packet development available at Compressing Pcap Files [9] and Packet
Capture [10]. Capture [10].
This information was last updated on 29th of June 2017. This information was last updated on 3rd of May 2018.
13. IANA considerations 12. IANA considerations
None None
14. Security considerations 13. Security considerations
Any control interface MUST perform authentication and encryption. Any control interface MUST perform authentication and encryption.
Any data upload MUST be authenticated and encrypted. Any data upload MUST be authenticated and encrypted.
15. Acknowledgements 14. Acknowledgements
The authors wish to thank CZ.NIC, in particular Tomas Gavenciak, for The authors wish to thank CZ.NIC, in particular Tomas Gavenciak, for
many useful discussions on binary formats, compression and packet many useful discussions on binary formats, compression and packet
matching. Also Jan Vcelak and Wouter Wijngaards for discussions on matching. Also Jan Vcelak and Wouter Wijngaards for discussions on
name compression and Paul Hoffman for a detailed review of the name compression and Paul Hoffman for a detailed review of the
document and the C-DNS CDDL. document and the C-DNS CDDL.
Thanks also to Robert Edmonds, Jerry Lundstroem, Richard Gibson, Thanks also to Robert Edmonds, Jerry Lundstroem, Richard Gibson,
Stephane Bortzmeyer and many other members of DNSOP for review. Stephane Bortzmeyer and many other members of DNSOP for review.
Also, Miek Gieben for mmark [11] Also, Miek Gieben for mmark [11]
16. Changelog 15. Changelog
draft-ietf-dnsop-dns-capture-format-07
o Resolve outstanding questions and TODOs
o Make RR RDATA optional
o Update matching diagram and explain skew
o Add count of discarded messages to block statistics
o Editorial clarifications and improvements
draft-ietf-dnsop-dns-capture-format-06 draft-ietf-dnsop-dns-capture-format-06
o Correct BlockParameters type to map o Correct BlockParameters type to map
o Make RR ttl optional o Make RR ttl optional
o Add storage flag indicating name normalisation o Add storage flag indicating name normalisation
o Add storage parameter fields for sampling and anonymisation o Add storage parameter fields for sampling and anonymisation
methods methods
o Editorial clarifications and improvements o Editorial clarifications and improvements
draft-ietf-dnsop-dns-capture-format-05 draft-ietf-dnsop-dns-capture-format-05
skipping to change at page 41, line 4 skipping to change at page 41, line 41
o Add specifics of how to record malformed messages o Add specifics of how to record malformed messages
o Add implementation guidance o Add implementation guidance
o Improve terminology and naming consistency o Improve terminology and naming consistency
draft-ietf-dnsop-dns-capture-format-04 draft-ietf-dnsop-dns-capture-format-04
o Correct query-d0 to query-do in CDDL o Correct query-d0 to query-do in CDDL
o Clarify that map keys are unsigned integers o Clarify that map keys are unsigned integers
o Add Type to Class/Type table o Add Type to Class/Type table
o Clarify storage format in section 7.12 o Clarify storage format in section 7.12
draft-ietf-dnsop-dns-capture-format-03 draft-ietf-dnsop-dns-capture-format-03
o Added an Implementation Status section o Added an Implementation Status section
draft-ietf-dnsop-dns-capture-format-02
o Update qr_data_format.png to match CDDL o Update qr_data_format.png to match CDDL
o Editorial clarifications and improvements o Editorial clarifications and improvements
draft-ietf-dnsop-dns-capture-format-01 draft-ietf-dnsop-dns-capture-format-01
o Many editorial improvements by Paul Hoffman o Many editorial improvements by Paul Hoffman
o Included discussion of malformed message handling o Included discussion of malformed message handling
skipping to change at page 42, line 4 skipping to change at page 42, line 39
o Update authors address o Update authors address
o Improve wording in Abstract o Improve wording in Abstract
o Changed DNS-STAT to C-DNS in CDDL o Changed DNS-STAT to C-DNS in CDDL
o Set the format version in the CDDL o Set the format version in the CDDL
o Added a TODO: Add block statistics o Added a TODO: Add block statistics
o Added a TODO: Add extend to support pico/nano. Also do this for o Added a TODO: Add extend to support pico/nano. Also do this for
Time offset and Response delay Time offset and Response delay
o Added a TODO: Need to develop optional representation of malformed o Added a TODO: Need to develop optional representation of malformed
messages within C-DNS and what this means for packet matching. messages within C-DNS and what this means for packet matching.
This may influence which fields are optional in the rest of the This may influence which fields are optional in the rest of the
representation. representation.
o Added section on design goals to Introduction o Added section on design goals to Introduction
o Added a TODO: Can Class be optimised? Should a class of IN be o Added a TODO: Can Class be optimised? Should a class of IN be
inferred if not present? inferred if not present?
draft-dickinson-dnsop-dns-capture-format-00 draft-dickinson-dnsop-dns-capture-format-00
o Initial commit o Initial commit
17. References 16. References
17.1. Normative References 16.1. Normative References
[RFC1035] Mockapetris, P., "Domain names - implementation and [RFC1035] Mockapetris, P., "Domain names - implementation and
specification", STD 13, RFC 1035, DOI 10.17487/RFC1035, specification", STD 13, RFC 1035, DOI 10.17487/RFC1035,
November 1987, <https://www.rfc-editor.org/info/rfc1035>. November 1987, <https://www.rfc-editor.org/info/rfc1035>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, <https://www.rfc- DOI 10.17487/RFC2119, March 1997, <https://www.rfc-
editor.org/info/rfc2119>. editor.org/info/rfc2119>.
[RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object [RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object
Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049, Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049,
October 2013, <https://www.rfc-editor.org/info/rfc7049>. October 2013, <https://www.rfc-editor.org/info/rfc7049>.
17.2. Informative References 16.2. Informative References
[ditl] DNS-OARC, "DITL", 2016, <https://www.dns- [ditl] DNS-OARC, "DITL", 2016, <https://www.dns-
oarc.net/oarc/data/ditl>. oarc.net/oarc/data/ditl>.
[dnscap] DNS-OARC, "DNSCAP", 2016, <https://www.dns-oarc.net/tools/ [dnscap] DNS-OARC, "DNSCAP", 2016, <https://www.dns-oarc.net/tools/
dnscap>. dnscap>.
[dnstap] dnstap.info, "dnstap", 2016, <http://dnstap.info/>. [dnstap] dnstap.info, "dnstap", 2016, <http://dnstap.info/>.
[dsc] Wessels, D. and J. Lundstrom, "DSC", 2016, [dsc] Wessels, D. and J. Lundstrom, "DSC", 2016,
<https://www.dns-oarc.net/tools/dsc>. <https://www.dns-oarc.net/tools/dsc>.
[I-D.daley-dnsxml] [I-D.daley-dnsxml]
Daley, J., Morris, S., and J. Dickinson, "dnsxml - A Daley, J., Morris, S., and J. Dickinson, "dnsxml - A
standard XML representation of DNS data", draft-daley- standard XML representation of DNS data", draft-daley-
dnsxml-00 (work in progress), July 2013. dnsxml-00 (work in progress), July 2013.
[I-D.hoffman-dns-in-json] [I-D.hoffman-dns-in-json]
Hoffman, P., "Representing DNS Messages in JSON", draft- Hoffman, P., "Representing DNS Messages in JSON", draft-
hoffman-dns-in-json-13 (work in progress), October 2017. hoffman-dns-in-json-14 (work in progress), April 2018.
[I-D.ietf-cbor-cddl] [I-D.ietf-cbor-cddl]
Birkholz, H., Vigano, C., and C. Bormann, "Concise data Birkholz, H., Vigano, C., and C. Bormann, "Concise data
definition language (CDDL): a notational convention to definition language (CDDL): a notational convention to
express CBOR data structures", draft-ietf-cbor-cddl-02 express CBOR data structures", draft-ietf-cbor-cddl-02
(work in progress), February 2018. (work in progress), February 2018.
[packetq] .SE - The Internet Infrastructure Foundation, "PacketQ", [packetq] .SE - The Internet Infrastructure Foundation, "PacketQ",
2014, <https://github.com/dotse/PacketQ>. 2014, <https://github.com/dotse/PacketQ>.
[pcap] tcpdump.org, "PCAP", 2016, <http://www.tcpdump.org/>. [pcap] tcpdump.org, "PCAP", 2016, <http://www.tcpdump.org/>.
[pcapng] Tuexen, M., Risso, F., Bongertz, J., Combs, G., and G. [pcapng] Tuexen, M., Risso, F., Bongertz, J., Combs, G., and G.
Harris, "pcap-ng", 2016, <https://github.com/pcapng/ Harris, "pcap-ng", 2016, <https://github.com/pcapng/
pcapng>. pcapng>.
[RFC7159] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March
2014, <https://www.rfc-editor.org/info/rfc7159>.
[RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running [RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running
Code: The Implementation Status Section", BCP 205, Code: The Implementation Status Section", BCP 205,
RFC 7942, DOI 10.17487/RFC7942, July 2016, RFC 7942, DOI 10.17487/RFC7942, July 2016,
<https://www.rfc-editor.org/info/rfc7942>. <https://www.rfc-editor.org/info/rfc7942>.
17.3. URIs [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017, <https://www.rfc-
editor.org/info/rfc8259>.
16.3. URIs
[1] https://github.com/dns-stats/draft-dns-capture- [1] https://github.com/dns-stats/draft-dns-capture-
format/blob/master/draft-06/cdns_format.png format/blob/master/draft-07/cdns_format.png
[2] https://github.com/dns-stats/draft-dns-capture- [2] https://github.com/dns-stats/draft-dns-capture-
format/blob/master/draft-06/cdns_format.svg format/blob/master/draft-07/cdns_format.svg
[3] https://github.com/dns-stats/draft-dns-capture- [3] https://github.com/dns-stats/draft-dns-capture-
format/blob/master/draft-06/qr_data_format.png format/blob/master/draft-07/qr_data_format.png
[4] https://github.com/dns-stats/draft-dns-capture- [4] https://github.com/dns-stats/draft-dns-capture-
format/blob/master/draft-06/qr_data_format.svg format/blob/master/draft-07/qr_data_format.svg
[5] https://github.com/dns-stats/draft-dns-capture- [5] https://github.com/dns-stats/draft-dns-capture-
format/blob/master/draft-06/packet_matching.png format/blob/master/draft-07/packet_matching.png
[6] https://github.com/dns-stats/draft-dns-capture- [6] https://github.com/dns-stats/draft-dns-capture-
format/blob/master/draft-06/packet_matching.svg format/blob/master/draft-07/packet_matching.svg
[7] https://github.com/dns-stats/compactor/wiki [7] https://github.com/dns-stats/compactor/wiki
[8] https://mm.dns-stats.org/mailman/listinfo/dns-stats-users [8] https://mm.dns-stats.org/mailman/listinfo/dns-stats-users
[9] https://www.sinodun.com/2017/06/compressing-pcap-files/ [9] https://www.sinodun.com/2017/06/compressing-pcap-files/
[10] https://www.sinodun.com/2017/06/more-on-debian-jessieubuntu- [10] https://www.sinodun.com/2017/06/more-on-debian-jessieubuntu-
trusty-packet-capture-woes/ trusty-packet-capture-woes/
skipping to change at page 47, line 6 skipping to change at page 47, line 41
query-question-sections : 11, ; Second & subsequent questions query-question-sections : 11, ; Second & subsequent questions
query-answer-sections : 12, query-answer-sections : 12,
query-authority-sections : 13, query-authority-sections : 13,
query-additional-sections : 14, query-additional-sections : 14,
response-answer-sections : 15, response-answer-sections : 15,
response-authority-sections : 16, response-authority-sections : 16,
response-additional-sections : 17, response-additional-sections : 17,
) )
QueryResponseHints = uint .bits QueryResponseHintValues QueryResponseHints = uint .bits QueryResponseHintValues
QueryResponseSignatureHintValues =&( QueryResponseSignatureHintValues = &(
server-address : 0, server-address : 0,
server-port : 1, server-port : 1,
qr-transport-flags : 2, qr-transport-flags : 2,
qr-type : 3, qr-type : 3,
qr-sig-flags : 4, qr-sig-flags : 4,
query-opcode : 5, query-opcode : 5,
dns-flags : 6, dns-flags : 6,
query-rcode : 7, query-rcode : 7,
query-class-type : 8, query-class-type : 8,
query-qdcount : 9, query-qdcount : 9,
skipping to change at page 47, line 28 skipping to change at page 48, line 15
query-arcount : 11, query-arcount : 11,
query-nscount : 12, query-nscount : 12,
query-edns-version : 13, query-edns-version : 13,
query-udp-size : 14, query-udp-size : 14,
query-opt-rdata : 15, query-opt-rdata : 15,
response-rcode : 16, response-rcode : 16,
) )
QueryResponseSignatureHints = uint .bits QueryResponseSignatureHintValues QueryResponseSignatureHints = uint .bits QueryResponseSignatureHintValues
RRHintValues = &( RRHintValues = &(
ttl : 0, ttl : 0,
rdata-index : 1,
) )
RRHints = uint .bits RRHintValues RRHints = uint .bits RRHintValues
OtherDataHintValues = &( OtherDataHintValues = &(
malformed-messages : 0, malformed-messages : 0,
address-event-counts : 1, address-event-counts : 1,
) )
OtherDataHints = uint .bits OtherDataHintValues OtherDataHints = uint .bits OtherDataHintValues
StorageFlagValues = &( StorageFlagValues = &(
skipping to change at page 49, line 4 skipping to change at page 49, line 40
? earliest-time => Timestamp, ? earliest-time => Timestamp,
? block-parameters-index => uint .default 0, ? block-parameters-index => uint .default 0,
} }
earliest-time = 0 earliest-time = 0
block-parameters-index = 1 block-parameters-index = 1
; Ticks are subsecond intervals. The number of ticks in a second is file/block ; Ticks are subsecond intervals. The number of ticks in a second is file/block
; metadata. Signed and unsigned tick types are defined. ; metadata. Signed and unsigned tick types are defined.
ticks = int ticks = int
uticks = uint uticks = uint
Timestamp = [ Timestamp = [
timestamp-secs : uint, timestamp-secs : uint,
timestamp-uticks : uticks, timestamp-uticks : uticks,
] ]
; ;
; Statistics about the block contents. ; Statistics about the block contents.
; ;
BlockStatistics = { BlockStatistics = {
? total-messages => uint, ? processed-messages => uint,
? total-pairs => uint, ? qr-data-items => uint,
? total-unmatched-queries => uint, ? unmatched-queries => uint,
? total-unmatched-responses => uint, ? unmatched-responses => uint,
? total-malformed-messages => uint, ? discarded-opcode => uint,
? malformed-items => uint,
} }
total-messages = 0 processed-messages = 0
total-pairs = 1 qr-data-items = 1
total-unmatched-queries = 2 unmatched-queries = 2
total-unmatched-responses = 3 unmatched-responses = 3
total-malformed-messages = 4 discarded-opcode = 4
malformed-items = 5
; ;
; Tables of common data referenced from records in a block. ; Tables of common data referenced from records in a block.
; ;
BlockTables = { BlockTables = {
? ip-address => [+ IPAddress], ? ip-address => [+ IPAddress],
? classtype => [+ ClassType], ? classtype => [+ ClassType],
? name-rdata => [+ bstr], ; Holds both Name RDATA and RDATA ? name-rdata => [+ bstr], ; Holds both Name RDATA and RDATA
? qr-sig => [+ QueryResponseSignature], ? qr-sig => [+ QueryResponseSignature],
? QuestionTables, ? QuestionTables,
skipping to change at page 51, line 30 skipping to change at page 52, line 20
client : 1, client : 1,
resolver : 2, resolver : 2,
auth : 3, auth : 3,
forwarder : 4, forwarder : 4,
tool : 5, tool : 5,
) )
QueryResponseFlagValues = &( QueryResponseFlagValues = &(
has-query : 0, has-query : 0,
has-reponse : 1, has-reponse : 1,
query-has-question : 2, query-has-opt : 2,
query-has-opt : 3, response-has-opt : 3,
response-has-opt : 4, query-has-no-question : 4,
response-has-no-question: 5, response-has-no-question: 5,
) )
QueryResponseFlags = uint .bits QueryResponseFlagValues QueryResponseFlags = uint .bits QueryResponseFlagValues
DNSFlagValues = &( DNSFlagValues = &(
query-cd : 0, query-cd : 0,
query-ad : 1, query-ad : 1,
query-z : 2, query-z : 2,
query-ra : 3, query-ra : 3,
query-rd : 4, query-rd : 4,
skipping to change at page 52, line 33 skipping to change at page 53, line 22
rrlist => [+ RRList], rrlist => [+ RRList],
rr => [+ RR] rr => [+ RR]
) )
RRList = [+ uint] ; Index of RR RRList = [+ uint] ; Index of RR
RR = { RR = {
name-index => uint, ; Index to a name in the name-rdata table name-index => uint, ; Index to a name in the name-rdata table
classtype-index => uint, classtype-index => uint,
? ttl => uint, ? ttl => uint,
rdata-index => uint, ; Index to RDATA in the name-rdata table ? rdata-index => uint, ; Index to RDATA in the name-rdata table
} }
; Other map key values already defined above. ; Other map key values already defined above.
ttl = 2 ttl = 2
rdata-index = 3 rdata-index = 3
MalformedMessageData = { MalformedMessageData = {
? server-address-index => uint, ? server-address-index => uint,
? server-port => uint, ? server-port => uint,
? mm-transport-flags => TransportFlags, ? mm-transport-flags => TransportFlags,
? mm-payload => bstr, ? mm-payload => bstr,
 End of changes. 141 change blocks. 
364 lines changed or deleted 372 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/