Database DevOps · Masking
Data masking techniques: which one fits each column, and how to keep masked data usable
Data masking techniques are the individual rules that turn a real value into a safe one: substitution, shuffling, nulling, number and date variance, scrambling, format-preserving masking, tokenisation and pseudonymisation. Each keeps something and gives something away. This guide compares them, shows which suits a name, a national ID, a salary or a free-text note in a Saudi database, and explains how to keep relationships intact across tables.
Published 11 min read
Orientation
At a glance
The whole page in one table, before the detail.
- What it covers
- The rule applied to each sensitive column when a copy of a database is masked
- Main techniques
- Substitution, shuffling, nulling, number and date variance, scrambling, format-preserving masking
- Reversible methods
- Tokenisation and pseudonymisation keep a way back to the real value; masking proper does not
- The hard part
- The same person masked the same way in every table and every database
- Saudi angle
- National ID, Iqama, mobile and IBAN formats; Arabic and English names; personal data under the PDPL
- Where Redgate fits
- Test Data Manager: classification, deterministic masking and subsetting of non-production copies
The problem in plain words
- The situation: test copies must be masked, and someone has to choose a rule for each of hundreds of columns.
- Why it matters: a weak rule leaves people identifiable; a heavy one breaks the application.
- What decides it: how tests use the column, whether it is a key, what format the application checks, and whether anyone must get the real value back.
- The short answer: substitution and format-preserving masking for identifiers, variance for amounts and dates, nulling for what tests never read, and tokenisation only where the original must be recoverable.
What data masking techniques are, and where this guide fits
A masking technique is the transformation applied to one column: replace it, reorder it, blank it, shift it, or swap it for a token. Data obfuscation is the wider term for any of these; masking usually means the change cannot be undone. The choice is made column by column, and it decides whether developers trust the masked copy.
For the overall approach and a step-by-step process for masked refreshes, see data masking for test environments. For hiding values at query time in a live database, see dynamic data masking.
Relateddata masking for test environmentsdynamic data masking
Why the choice of technique matters for data in Saudi Arabia
Personal data in the Kingdom falls under the Personal Data Protection Law, overseen by the Saudi Data and Artificial Intelligence Authority (SDAIA). Our overview of the PDPL gives the general picture, and PDPL and test data covers what it means for non-production copies. Your compliance owner confirms which obligations apply.
The technique decides whether the copy still identifies people. Saudi datasets add their own details: national ID and Iqama numbers with a fixed length and structure, mobile numbers in local and international form, IBANs with a check value, names held in Arabic and English, and dates recorded in both Hijri and Gregorian calendars.
RelatedSaudi Data and Artificial Intelligence Authority (SDAIA)PDPL and test data
Data masking techniques compared: what each keeps and what it gives away
Most masked databases use several of these at once, chosen per column.
| Technique | What it does | Example | Reversible | Keeps for testing |
|---|---|---|---|---|
| Substitution | Replaces the value with a realistic one from a list | A name becomes another plausible name | No | Realistic look and length |
| Shuffling | Moves values between rows of the same column | Salaries moved between employees | No | The real distribution |
| Nulling or redaction | Removes the value or puts a fixed mark in its place | A notes field emptied; a card shown as XXXX | No | Nothing but the column |
| Number and date variance | Shifts a value by a random amount within a set range | Salary moved by up to 10 percent | No | Approximate totals and order |
| Character scrambling | Reorders characters inside the value | A reference code with letters reordered | No | Length only |
| Format-preserving masking | Generates a new value with the same structure | A new 10-digit ID that passes validation | No, when keyed and one-way | Format, validation and joins |
| Tokenisation | Swaps the value for a token; the original sits in a protected store | A card number replaced by a token | Yes, through the token store | Joins on the token |
| Pseudonymisation | Replaces identifiers with a consistent alias; a key is held separately | Customer ID replaced by a keyed hash | Yes, with the key or mapping | Links across systems |
Tokenisation and pseudonymisation are reversible by design, so a copy that uses them is only as safe as the store or key behind it.
Substitution, shuffling, variance and format-preserving masking: the data obfuscation techniques column by column
Substitution replaces a name, address or email with one drawn from a list, so the copy still reads like real data. Arabic and English name fields must be substituted together, from matching lists, or a record ends up with names belonging to two different people. Dependent columns must follow: if first and last names change, the full name and the email built from them change the same way.
Read more
Shuffling keeps the column's real values and moves them between rows, which suits amounts or categories where tests need the true spread. The real values are still there, though: in a small table, or where a value is rare, someone who knows the organisation can match it to a person. Shuffle within groups, such as a department, and never shuffle a column that identifies someone by itself.
Nulling, number variance and date shifting
Nulling or redaction is the safest and the least useful. Apply it to columns tests never read, such as free-text notes and staff comments, after checking that the application accepts an empty value.
Number variance moves each amount by a random percentage within a range you set. Totals stay roughly right and no individual value is real. Keep the range wide enough that a value cannot be guessed back, and keep the rounding rules.
Date variance shifts dates by a random number of days. Shift all of one person's dates by the same offset, or the order breaks: a hire date can land before a birth date. Where a system stores both Hijri and Gregorian forms, derive one from the other after shifting rather than masking each separately.
Format-preserving masking and scrambling for Saudi identifiers
Identifiers break tests most often. Applications often validate the format of a national ID or Iqama number, check that a mobile number has the local structure, and that an IBAN carries a valid check value. A replacement that fails these checks makes tests fail for the wrong reason.
Format-preserving masking generates a new value with the same structure, built to pass the same validation rules your application applies to the original, so the value is accepted and still belongs to no one. Done deterministically with a secret key kept out of the test environment, the same original always gives the same replacement, so joins hold.
Character scrambling, reordering the characters inside a value, is cheaper but rarely right for identifiers: a scrambled ID keeps the original digits and usually fails format validation. Keep it for internal codes no application validates.
Tokenization vs masking vs pseudonymisation
The three are often confused. The question that separates them is whether anyone can get the real value back.
| Masking | Tokenisation | Pseudonymisation | |
|---|---|---|---|
| Can the original be recovered | No, if done properly | Yes, from the token store | Yes, with the key or the mapping table |
| Where the link to the person lives | Nowhere | In a protected token store | In a key or table held separately |
| Typical place | Development, test and vendor copies | Production systems that pass a value on without holding it | Analysis that follows one person across datasets |
| Still personal data | Often not, if no one can be re-identified | Usually yes, for whoever can reach the store | Usually yes; treat it as personal data |
| Main risk | Leftover details that let someone be recognised | The token store becomes the target | An unsalted hash or a leaked key undoes it |
For test copies, masking is normally the right tool: testers need data that behaves correctly, not a way back to the customer. Tokenisation belongs in production architecture; pseudonymisation suits analytics that must link one person's records across sources.
Pseudonymisation and hashing: a common shortcut that does not hold
Hashing a national ID is often presented as masking. Without a secret, it is not. Three points to agree before using a hash:
- Small number spaces can be searched. A 10-digit identifier has at most ten billion possible values. Anyone can hash every one of them and match the results, so a plain hash of an ID or mobile number is reversible in practice.
- Use a keyed hash, and guard the key. With a secret key kept outside the test environment, that search fails. Whoever holds the key can still link records, so the result is pseudonymised.
- Keep the format if the application checks it. Where the column is validated, use keyed format-preserving masking instead of a raw hash.
Keeping referential integrity across tables and databases
Masking one column is easy. Masking a customer who appears in ten tables and three databases, the same way everywhere, is the real work. Beyond the basic rules, these cases catch teams out.
Read more
- Personal data used as a key. A national ID or mobile number used as a key must be masked deterministically, keeping its format, with the same rule in every table that refers to it.
- Relationships the schema does not declare. Links kept in application code, with no foreign key, must be listed by hand or no tool can keep them.
- Derived columns. Age from date of birth, a full name built from its parts, a search column copying the phone number. Each must be masked from the same source.
- Several systems. If the CRM and billing both hold the customer, both run the same rule with the same key.
Which technique fits which column: a decision table
A starting point for the rule per column. Your data owners confirm the list; tests confirm the rules.
| Column | Technique that usually fits | Why | Avoid |
|---|---|---|---|
| Names, Arabic and English | Substitution from matching lists, deterministic | Readable, consistent across tables | Masking one language and not the other |
| National ID or Iqama number | Format-preserving, keyed and deterministic | Passes validation; joins hold | Plain hashing; scrambling |
| Mobile number | Format-preserving in the local structure | Forms and SMS logic still accept it | Real numbers that could receive a message |
| Email address | Substitution on a test-only domain | No message can reach a real person | Keeping the real domain and changing the name only |
| IBAN or card number | Format-preserving with a valid check value | Payment logic still validates | Real account numbers in any copy |
| Salary and amounts | Number variance, or shuffling within groups | Totals and spreads stay realistic | Variance so small the value can be guessed |
| Dates of birth and other personal dates | Date variance, one offset per person | Order and age bands survive | Shifting each date independently |
| Free-text notes | Nulling, or replacement with neutral text | Names and numbers hide in free text | Assuming the field holds nothing personal |
| Reference and lookup data | Leave unchanged | Not personal; masking it breaks tests | Masking codes the application needs |
Where Redgate Test Data Manager fits, and where it does not
Redgate is a technology partner whose tooling Interkey implements for database delivery. According to Redgate's documentation, Test Data Manager classifies data to find personally identifiable information, masks it deterministically so the same person stays consistent across tables, builds subsets that keep referential integrity and runs from a command line built for CI/CD pipelines. That turns the decision table above into rules that run on every refresh. Our test data management page covers the tool and the engines Redgate documents it for.
Read more
Where it does not fit: it is a tool for non-production copies, not a token store for production payment data and not read-time masking in a live database. Which masking functions and formats it offers for a given column type, such as a Saudi identifier your application validates, we confirm against Redgate's current documentation for your engine and edition during scoping, and say so before you commit to licences if a custom rule is needed.
Relatedtest data management
How Interkey helps
Interkey's database DevOps practice turns your list of sensitive columns into masking rules: a technique per column, the keys behind them, the relationships that must hold, and checks that prove the masked copy works. Rules run in your delivery pipeline, with Redgate where it fits. Support comes from Riyadh, in Arabic and English, and masking runs on servers you choose inside your network. Interkey's work is quoted per scope; Redgate licensing is quoted at Redgate's terms.
Relateddatabase DevOps
Questions buyers ask about data masking techniques
What is the difference between tokenization and data masking?
Masking replaces a value so it cannot be turned back into the original. Tokenisation swaps it for a token and keeps the original in a protected store, so it can be recovered. Masking suits test copies; tokenisation suits production systems that must pass a value on without holding it.
Which masking technique should we use for national ID and Iqama numbers?
Format-preserving masking, keyed and deterministic. The new number keeps the format your application validates, so the application accepts it, and the same original always gives the same replacement, so joins across tables hold.
Is hashing a national ID number enough to protect it in a test database?
No. A 10-digit number has at most ten billion values, so anyone can hash them all and match the results. Use a keyed method with the key kept outside the test environment.
Is pseudonymised data still personal data in Saudi Arabia?
Usually it should be treated as such, because whoever holds the key or mapping can link it back to a person. How the PDPL treats your specific case is for your compliance owner to confirm from SDAIA's texts. Properly masked test data, with no way back, carries far less risk.
Can masking keep the same customer consistent across different databases?
Yes, if every database runs the same deterministic rule with the same key. The customer then has the same masked name and ID in the CRM and billing. Relationships kept only in application code must be listed by hand.
How long does it take to write masking rules for our databases, and what drives the cost?
It is estimated per project, from the number of databases and sensitive columns, identifiers used as keys, undeclared relationships and free-text fields. Interkey's work is quoted per scope; Redgate licensing is quoted at Redgate's terms.
Next
Related pages
Next step
Agree a masking rule for every sensitive column
Tell us the databases that hold personal data, the identifiers you use as keys, and how your test copies are made today. We reply with a proposed technique per column type and a first scope.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia