Observability guide · Data protection in telemetry
Keeping personal data out of logs and traces, before it leaves your network
Personal data reaches observability tools through log messages, span attributes, URLs, request headers and exception text. To remove it before it leaves your network, use a processing tier inside your own infrastructure that applies one policy to every language, with the observability platform's own scanner as a final check. This guide sets out that design, with patterns for Saudi mobile numbers and national ID and Iqama formats, and shows where those patterns over-match or miss.
Published 8 min read
In short
- Where it leaks in: log bodies, span attributes, query strings, authorisation headers and cookies, exception messages and captured database statements.
- Three layers: do not collect it in code; enforce masking centrally in your network; scan again in the platform as a last line.
- Why the middle layer matters: Datadog's Sensitive Data Scanner, run in Datadog's cloud, scans data after it is submitted, so the data has already left your environment. Masking inside your network happens before export.
- Test before production: simple patterns both over-match (timestamps, order numbers) and miss (numbers written with spaces or in Arabic-Indic digits).
This guide covers telemetry leaving your applications. Copies of production databases used for development and testing need a different process: Data masking for test environments
Where personal data gets into telemetry
| Source | Typical example | Control |
|---|---|---|
| Log messages | A developer logs the full request body, including a customer's mobile number | Never log request or response bodies by default; mask patterns in log text |
| Span attributes | user.email or a national ID added to a span to "help debugging" | Never put e-mail, phone, national ID, card number, address or free-text input in span attributes; delete known keys centrally |
| URLs | A token or ID in the query string of url.full | Strip query strings centrally |
| Headers | Authorization and Cookie headers captured by instrumentation | Delete these attributes centrally |
| Exceptions | A validation error message that repeats the invalid card number | Make masking rules cover exception messages, not only attributes |
| Database spans | The SQL text of a query with literal values | Use parameterised queries so values are not recorded |
| Identifiers you need | A customer ID used to group errors per user | Replace it with a salted hash (pseudonymised ID) |
Three layers of control
| Layer | Where | Technique | What it catches |
|---|---|---|---|
| 1. Do not collect | Application code | Developer rules; pseudonymised identifiers | Leaks at the source |
| 2. Enforce centrally | A processing tier inside your network, such as the OpenTelemetry Collector | Delete, hash or mask attributes; pattern-replace in log text and exception messages; allow-lists for regulated services | What developers miss, for every language with one policy |
| 3. Last line of defence | Datadog | Sensitive Data Scanner rules; restricted log indexes and role-based access | What slipped through, after ingestion |
Layer 2 is the one that keeps data in your network. Datadog documents that when Sensitive Data Scanner runs in its cloud, data is submitted first and "leaves your environment before it gets redacted". Layer 3 still matters: it catches patterns your own rules did not anticipate, and its predefined rule library covers common items such as e-mail addresses, card numbers, API keys and tokens.
Two ways to run the in-network layer
| OpenTelemetry Collector | Datadog Observability Pipelines Worker | |
|---|---|---|
| What it is | The open-source Collector you run as a gateway | Datadog's worker that runs in your infrastructure to aggregate, process and route data |
| Signals | Traces, metrics and logs | Logs and metrics, as documented |
| How masking is done | Attribute, transform and redaction processors with your own rules | A Sensitive Data Redaction template with built-in or custom rules; Datadog also documents running the Sensitive Data Scanner processor here, before data leaves your premises |
| Fits when | Applications already send OpenTelemetry data, especially traces | Large log volumes need processing and routing before they reach Datadog or other destinations |
The two are not exclusive: trace masking in the Collector and log processing in Observability Pipelines can run side by side, provided one policy document defines what both must remove.
Choosing a Collector processor
| Processor | Suited to | Behaviour |
|---|---|---|
| attributes | Known keys | Delete, hash, update or insert named attributes, for example hash user.id, delete user.email |
| transform | Pattern-based masking anywhere: attributes, span names, log bodies | Regular-expression replacement and conditional logic per signal |
| redaction | Strict allow-list governance for regulated services such as payments and health | Removes every attribute not explicitly allowed, and masks values that match blocked patterns |
For a payments service, an allow-list is the safer default: name the attributes that may leave (service, version, environment, HTTP method, route, status code, order ID, amount, currency, payment method, error type) and let everything else be removed.
Patterns for Saudi identifiers, and where they fail
Check whether your scanning rule library covers Saudi formats; where it does not, add your own. These are the patterns in Interkey's reference design, with the results of testing them on sample values.
| Data | Pattern (regular expression) | What it matches |
|---|---|---|
| Saudi mobile number | \b(?:\+?966|0)5\d{8}\b | 05XXXXXXXX and 9665XXXXXXXX written without spaces |
| National ID / Iqama | \b[12]\d{9}\b | Any 10-digit number starting with 1 or 2 |
| Payment card number | \b(?:\d[ -]?){12,18}\d\b | 13 to 19 digits, with optional spaces or dashes |
| E-mail address | [A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,} | Standard e-mail formats |
What our test run of these patterns showed:
| Sample value | Result | What to do |
|---|---|---|
| 0512345678 | Masked | — |
| +966512345678 | Digits masked, the leading "+" remains | Harmless. To mask the "+" too, remove the word boundary in front of it: (?:\+966|\b966|\b0)5\d{8}\b |
| 055 123 4567 | Not masked: the pattern does not allow spaces | Add a variant that tolerates spaces and dashes if users type numbers that way |
| ٠٥١٢٣٤٥٦٧٨ (the same mobile number in Arabic-Indic digits) | Not masked: no pattern matches it | If forms or apps accept Arabic-Indic digits, normalise them to Western digits before logging, or add patterns written for them |
| 1793609689 (a Unix timestamp in seconds) | Masked as a national ID | Apply the ID rule only to fields that can hold IDs, or require a nearby keyword; every Unix timestamp in seconds between 2001 and 2065 is 10 digits long and starts with 1 or 2 |
| 1793609689123 (the same timestamp in milliseconds) | Masked as a card number | Exempt known timestamp fields from the card rule |
| 12345678901234 (a 14-digit order number) | Masked as a card number | Exempt known order-number fields, or validate candidates further before masking |
We ran these tests with two regular-expression engines, Python's and Go's, and both gave the same results. Repeat them inside your own Collector before production: in the Collector's YAML configuration, a pattern inside double quotes needs every backslash doubled (\\b, \\d), and a missed escape changes what the rule matches or stops the configuration loading.
Rules for developers
Publish these in your engineering handbook; they stop leaks at the source, before any pipeline sees them.
- Never log request or response bodies by default.
- Never put e-mail, phone, national ID, card number, address or free-text user input in span attributes.
- Use pseudonymised IDs, such as a salted hash of the user ID, when you need per-user analysis.
- Assume exception messages can contain personal data; masking rules must cover them.
- Capture database statements only from parameterised queries, so values are never recorded.
Testing masking rules before production
- Build a payload set: real-looking samples of every format you mask, plus values that look similar but must survive (timestamps, order numbers, reference codes).
- Run it through a non-production Collector with the debug exporter at detailed verbosity, and compare what comes out with what you expected.
- Record over-matches and misses, fix the rules and keep the payload set as a regression test.
- Re-run the set whenever rules or Collector versions change, and before each new service is onboarded.
- Watch for side effects in production: dashboards that suddenly group "[redacted]" values can mean a rule is masking a field it should not.
What masking does not settle
- Where data is stored. Masking reduces what leaves; it does not change where the rest lands. As of November 2026 Datadog's sites are in the United States, Germany, Japan, Australia and the United Kingdom. See Observability data residency in Saudi Arabia.
- Who can see what is kept. Use separate log indexes for sensitive services and restrict access by role.
- Your legal position. Whether a dataset is personal data, and what Saudi Arabia's Personal Data Protection Law requires of you, is a question for your data protection officer and legal team. Masking is one technical control within that answer.
How Interkey helps
Interkey is an official Datadog partner and implements and operates Datadog for organisations in Saudi Arabia. In our OpenTelemetry-to-Datadog reference design, masking is part of the pipeline from day one: the developer rules above, central rules in the Collector covering traces and logs, an allow-list for regulated services, and Datadog's Sensitive Data Scanner as the last line. Rule review with your security team and data protection officer is a named step in our implementation checklist.
For the full pipeline, see Sending OpenTelemetry data to Datadog.
Questions buyers ask
Doesn't Datadog already redact sensitive data?
Datadog's Sensitive Data Scanner can discover, redact, partially redact or hash sensitive data in logs, APM spans, RUM events and events. In Datadog's cloud it works after the data is submitted. To redact logs before they leave your premises, Datadog documents running it in Observability Pipelines inside your infrastructure; for traces, the OpenTelemetry Collector does that job.
Is a hashed ID still personal data?
A salted hash hides the raw value but still links records to the same person, which is why it is called pseudonymisation. Agree with your data protection officer how pseudonymised IDs are classified and who may hold the salt.
Will masking make troubleshooting harder?
It need not. Hashing user IDs keeps per-user grouping, and allow-lists keep the technical attributes engineers need: route, status code, error type, service and version.
Does masking in the Collector slow the pipeline?
Every rule adds processing work, especially pattern rules applied to large log bodies. Measure Collector CPU and memory with your payload set, and scale the gateway accordingly.
Can you review rules we already have?
Yes. Send us your current rules and a description of the data your services handle; we test them against Saudi formats and the over-match cases above.
Next
Related guides
Next step
Review your telemetry masking rules
Tell us which services handle personal data, the languages they are written in, and where their telemetry goes today. A specialist from the Riyadh team replies on Saudi working days, in Arabic or English.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia