Database DevOps · Test data
Data masking for test environments: methods, pitfalls and a process that holds up
Data masking replaces sensitive values in a copy of a database — names, national ID and Iqama numbers, mobile numbers, account details — with realistic substitutes, so developers and testers work with data that behaves like production without exposing the people in it. This page compares the methods, explains the failure that makes masked data useless, and sets out a process that turns test-data refreshes into a routine pipeline step.
Published 7 min read
In short
- What it is: transforming a dataset so values can no longer be linked to the real people they describe, while keeping the format and behaviour applications expect.
- Where it is used: copies of production made for development, testing, user acceptance, training, analytics and outside contractors.
- Two main kinds: static masking changes a copy permanently; dynamic masking hides values at the moment they are read and leaves the stored data unchanged.
- The hard part: finding every sensitive column, and masking the same person the same way everywhere, so joins and tests still work.
This page covers copies of databases. Personal data in application logs and traces needs different controls: Keeping personal data out of logs and traces
Why a test copy needs the same care as production
Non-production environments usually have more people with access, looser controls and more copies than production does. A full restore of production into test therefore multiplies the number of places where real customer data lives — and personal data does not stop being personal data because it sits on a test server. Ask your data-protection or legal function which obligations apply to your non-production copies; this page deals with the engineering.
Static, dynamic, synthetic: which approach fits
Masking is one of several ways to give teams data they can safely use. They solve different problems, and many estates end up combining two of them.
| Approach | What it does | Fits when | Watch out for |
|---|---|---|---|
| Static masking | Permanently replaces sensitive values in a copy before anyone uses it | Development, testing and UAT databases refreshed from production | Must run before access is granted; a copy masked after the fact has already been exposed |
| Dynamic masking | Hides values at read time based on who is asking; stored data is unchanged | Restricting what some users see in a live or reporting database | The real values still exist in that database; not a substitute for masking test copies |
| Synthetic data | Generates artificial records that follow the schema and rules | New features with no production data yet, or edge cases production rarely contains | Misses the odd combinations real data contains; good as a supplement |
| Subsetting | Copies a smaller, related slice of production instead of everything | Tests that need representative data, not every row | Not masking on its own — the subset still needs masking |
| Encryption | Makes data unreadable without a key | Protecting data at rest and in transit | Reversible by anyone with the key, so it does not make data safe to hand to a test team |
What goes inside a masking rule
Each sensitive column gets a rule. The common techniques are:
- Substitution: replace the value with a realistic one from a list — a different name, a different address.
- Shuffling: reorder values within a column so they no longer belong to the original row.
- Nulling or suppression: remove the value where tests do not need it.
- Generalisation: reduce precision — a birth date becomes a birth year, a street address becomes a city.
- Format-preserving replacement: generate a new value with the same length and structure as the original.
Read more
Identifiers deserve particular care in Saudi datasets. National ID and Iqama numbers, mobile numbers in local format and bank account numbers are checked by applications for length and structure; replacements must pass the same validation or tests fail for the wrong reason. Name fields often exist in both Arabic and English; masking one and not the other leaves the person identifiable.
The failure that makes masked data useless: broken relationships
A customer appears in the customers table, in orders, in payments, in a support system and often in a second database entirely. If each place masks that customer differently, joins break, totals stop reconciling and integration tests fail — and teams quietly go back to asking for unmasked copies.
Three rules that keep masked data usable
- Mask deterministically. The same original value must always become the same masked value, across every table and every database in scope.
- Search free text, not only obvious columns. Notes, comments, addresses typed into a single field and log tables often contain names and numbers.
- Leave reference data alone. Product codes, branch lists and currency tables are not personal and masking them only breaks tests.
A masking process that holds up
-
Classify.
Run automated discovery to find columns that look like personal data, then have the data owners confirm or correct the list. Discovery proposes; people decide.
-
Write a rule per column.
Choose the technique, the format it must keep, and whether the value must match across databases.
-
Decide how much data each environment needs.
Where a full copy is not required, subset first — smaller copies refresh faster and expose less.
-
Mask during provisioning.
Apply the rules as the copy is created, before any developer, tester or contractor can connect to it.
-
Validate.
Spot-check that original values no longer appear, and run the application's test suite against the masked copy.
-
Automate.
Put the refresh into the delivery pipeline so a masked environment is a routine job rather than a quarterly favour from a DBA.
-
Keep the record.
Store the classification and the rules with the pipeline, so you can show what was masked, how and when.
How Redgate Test Data Manager covers these steps
Interkey implements Redgate's tooling for database delivery. For test data, Redgate Test Data Manager maps onto the process above as follows, according to Redgate's own documentation:
| Step | What the tool does | What you still decide |
|---|---|---|
| Classify | Automatically finds and categorises data to identify personally identifiable information | Which proposed columns are truly sensitive, and what was missed in free text |
| Mask | Deterministic masking that replaces values with realistic substitutes and keeps referential integrity | The rule per column and the formats applications validate |
| Subset | Builds a smaller, representative slice that keeps referential integrity, starting from a chosen table with filter clauses | How much data each environment really needs |
| Provision | Creates a clone of an entire database instance (using Redgate Clone), with masking during provisioning as an option | Who gets which environment, and the refresh schedule |
| Automate | A command line designed for CI/CD pipelines, alongside a graphical interface; runs on Windows and Linux | Where in the pipeline the refresh runs |
The tool is documented for four mainstream relational database engines. Supported engines and features vary by version and edition, so confirm your exact engine and edition during scoping. Redgate licensing is quoted at Redgate's terms alongside Interkey's work.
What Interkey does
Interkey's database DevOps practice treats non-production data with the same care as production data and plans masking or generated test data from the start, not after an audit finding. The engineering is done by Interkey's own engineers in Riyadh. Interkey is certified to ISO/IEC 27001:2022.
A test-data engagement with Interkey is scoped around:
- Assessment: how test data is produced today — who copies what, to where, how often, and who can access it.
- Classification with your data owners: the confirmed list of sensitive columns, including free-text fields.
- Rules and subsets: masking rules per column and subset definitions per environment.
- Pipeline integration: masked refreshes built into your database delivery process.
- Handover: runbooks and standards your team owns; ongoing operation under a managed-services scope if you want it.
Read more
When this is not the right engagement: if you only need to hide a few columns from some users of a single reporting database, your database platform may already offer a built-in option — check that first. If your estate is mostly non-relational, tell us; scope is agreed per platform.
Questions buyers ask
Is data masking the same as encryption?
No. Encryption makes data unreadable without a key and is reversible by anyone who holds the key. Masking replaces the values and is usually irreversible, which is why it suits copies handed to development and test teams.
Is masked data anonymous?
Not automatically. If enough original details are left untouched — a rare job title, a small town, an exact date — a person can sometimes be re-identified by combining them. Good rules generalise or replace those combinations as well as the obvious identifiers.
Will masking break our tests?
It breaks tests when relationships or formats break. Deterministic masking keeps the same person consistent across tables and databases, and format-preserving replacements keep identifiers valid. Keep a small set of synthetic records for edge cases that masking removes.
Can you mask non-relational databases?
Redgate Test Data Manager is documented for relational database engines. For other platforms, the approach is agreed per platform during scoping, and we will say so if the tooling is not a fit.
Do we have to use Redgate to work with Interkey on test data?
No. The process on this page applies whatever the tooling. Where your estate and licensing make Redgate the right fit, Interkey implements it; where they don't, we say so.
Does masking settle our data-protection obligations?
No tool settles that. Which obligations apply to your non-production copies, and what they require, is for your legal or data-protection function to determine. Masking reduces the number of places where real personal data lives.
Next
Read next
Next step
Review how your test data is produced
Tell us how your non-production databases are refreshed today and roughly how many hold customer data. The Riyadh team reviews your request and a specialist replies on Saudi working days, in Arabic or English.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia