DNS telemetry—the logs of every domain a user or device resolves—is a privacy and compliance balancing act. The core answer: collect only what you need for security and operations (query name, client IP, timestamp, response code), mask or hash the client identifier and the query name’s sensitive labels, and retain the data for the shortest period your use case justifies, typically 24 hours to 30 days unless a legal obligation requires longer. This article explains what to collect, what to mask, how long to keep it, and how to stay compliant without losing visibility.
Formula
Total storage (GB) = (queries per day × storage per query in bytes × retention days) ÷ (1024³). Per-day storage = (queries per day × storage per query in bytes) ÷ (1024³). Variables: q = average DNS queries per day, s = average bytes stored per query (including metadata, not just the raw packet), t = retention period in days. The result is an estimate; actual compression and indexing will change it.
Worked Example
Using the defaults: q = 1,000,000 queries/day, s = 150 bytes/query, t = 30 days. Per-day storage = (1,000,000 × 150) ÷ 1,073,741,824 = 0.14 GB. Total = 0.14 × 30 = 4.19 GB. So a small network generating a million queries a day with 150 bytes per log entry would use about 4.2 GB for a month of logs—before any indexes or backups.
Why DNS Telemetry Is Sensitive
DNS queries reveal a surprising amount: every website visited, every app that phones home, every device that checks for updates, and even the domains used by malware command-and-control. Unlike HTTP, DNS is rarely encrypted end-to-end (even with DoH/DoT, the resolver sees everything). This makes DNS logs a high-value target for attackers, a privacy liability for users, and a compliance headache for operators under GDPR, CCPA, or sector-specific rules like HIPAA or PCI DSS.
What to Collect
Minimize to the fields that directly support your stated purpose. For security monitoring and troubleshooting, the essential set is:
- Query name (the full domain, e.g.,
www.example.com) - Query type (A, AAAA, MX, etc.)
- Response code (NOERROR, NXDOMAIN, SERVFAIL)
- Timestamp (with timezone or UTC)
- Client IP or a hashed identifier
- Resolver IP (if you operate recursive resolvers)
- Response size and query size (for anomaly detection)
Do not collect the full DNS packet payload, EDNS client subnet (ECS) details beyond what your network needs, or the contents of the answer section unless you have a specific security reason. The less you store, the less you can lose.
What to Mask or Hash
Masking is about reducing identifiability while preserving utility. Common techniques:
- Client IP: Hash with a salted key (e.g., HMAC-SHA256) or truncate to a /24 IPv4 or /48 IPv6 prefix. Full IPs are personal data under GDPR.
- Query name: For analytics, strip subdomains and keep only the registrable domain (e.g.,
a.b.example.com→example.com). For security, you may need the full name, but consider hashing the labels after the TLD. - ECS data: Drop or round to a coarse geographic region.
- User identifiers: If you tie queries to accounts, replace with a random ID that cannot be reversed without a key.
Important: hashing is not anonymization if the hash is deterministic and the key is weak. Use a per-organization secret key and rotate it periodically. If you need to retain the ability to re-identify a user for a security investigation, store the key separately and limit access.
Retention: How Long Is Too Long?
There is no universal answer, but the principle is: retain only as long as the purpose requires. Typical retention windows:
- 24 hours: Enough for most incident response triage and immediate troubleshooting.
- 7–30 days: Common for security monitoring and threat hunting; aligns with many breach notification timelines.
- 90 days or more: Only if you have a documented legal or regulatory obligation (e.g., certain financial or government requirements).
Shorter retention reduces breach impact, lowers storage costs, and simplifies compliance. If you must keep data longer, consider aggregating or anonymizing after the initial period—e.g., keep only daily counts of unique domains, not per-client logs.
Compliance Frameworks and What They Require
No law specifically mentions DNS logs, but general principles apply:
- GDPR: DNS logs are personal data when they can be linked to an individual. You need a lawful basis (legitimate interest for security is possible), a data protection impact assessment (DPIA) if processing is large-scale, and you must honor access and deletion requests.
- CCPA/CPRA: Similar rights for California residents; you must disclose what you collect and allow opt-out of sale/sharing. DNS logs are not typically “sold,� but sharing with third parties for advertising would trigger requirements.
- PCI DSS: If you are a payment processor, DNS logs may fall under logging requirements, but they also must be protected and retained per the standard (usually 12 months for audit trails).
- ePrivacy (EU): May apply to the use of DNS data for marketing; security use is generally exempt.
Practical steps: document your purpose, map data flows, implement access controls, and set up a deletion schedule. Anonymize or pseudonymize as early as possible.
Common Pitfalls and Misconceptions
One common mistake is treating DNS logs as “just metadata� and ignoring them. Another is over-retaining “just in case,� which increases risk. A third is hashing client IPs but leaving the query names in clear, which still allows profiling. Also, beware of log aggregation: if you combine DNS logs with other data (e.g., DHCP logs), the combined dataset may re-identify users even if each source is masked. Finally, don’t forget backups and archives—they extend retention beyond your policy unless you actively delete.
How to Choose a Retention Policy
Start by listing your use cases: security monitoring, troubleshooting, capacity planning, or compliance. For each, determine the maximum age of data that is still useful. For example, a security analyst may need 30 days to correlate with a malware campaign that was discovered weeks later. Then set a policy that is the minimum of those needs. Automate deletion—manual deletion fails. And document the policy so you can prove compliance.
Frequently Asked Questions
Is hashing client IPs enough to make DNS logs anonymous?
No. If the hash is deterministic and the key is weak, it can be reversed with a dictionary attack. Even with a strong key, hashing is pseudonymization, not anonymization—the data can still be linked to an individual if the key is compromised. For true anonymization, you must remove the ability to re-identify, which usually means aggregating or adding noise.
Do I need to get consent to collect DNS logs?
Under GDPR, you need a lawful basis. Legitimate interest for network security is often valid, but you must balance it against user privacy and offer an opt-out where possible. Consent is not usually required for security monitoring, but it is required if you use the data for marketing. Check your local guidance.
What is the difference between DoH, DoT, and traditional DNS for logging?
With traditional DNS, the query is visible to your network and ISP. With DNS over HTTPS (DoH) or DNS over TLS (DoT), the query is encrypted between the client and the resolver, so your network cannot see the content—but the resolver (e.g., a public DoH provider) can. That shifts the telemetry to the resolver, which has its own privacy policy.
Can I use DNS logs for threat hunting without storing client IPs?
Yes. You can hash client IPs with a key and still correlate queries from the same client. For hunting, the full query name is often more important than the IP. You can also store only the domain and the timestamp, and only resolve the IP when an alert triggers.
How do I handle DNS logs for minors or sensitive categories?
Under GDPR, data about children may require extra protections. DNS logs are unlikely to reveal age, but if you know a user is a minor (e.g., from an account), you should apply stricter retention and access controls. Avoid using DNS data for profiling or advertising to minors.
What should I do if I already have years of DNS logs?
Audit them: determine what you actually need, then delete or anonymize the rest. If you cannot delete immediately, restrict access and encrypt the data. Moving forward, implement a retention schedule. Keeping historical data without a purpose is a liability.
Related
Formula used
in t | Retention (days) | days | 30 | 1 | 3650 in q | Queries per day | queries | 1000000 | 1 | 1000000000000 in s | Storage per query (bytes) | bytes | 150 | 1 | 10000 out total | Total storage (GB) | GB | (q * s * t) / (1024^3) | 2 out per_day | Storage per day (GB) | GB | (q * s) / (1024^3) | 2
Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.
More cards in this drawer
- Poco X3 GT Dimensity 1100 67W Fast Charging Launched?5.00 min/km
- Flash Converter: What It Is & How to Use It
- Choosing a Pipe Flow Meter for Swirling Flow
- Motorola Phone Recovery: How to Recover Data
- Convert 2D Videos to Oculus Rift VR: How-To
- DRAM Light on Motherboard: Meaning & Fixes5.00 min/km