Data platform · Replication
Change data capture: how to keep two databases in step without stopping either
Change data capture (CDC) detects the inserts, updates and deletes made in a source database and applies them to another system shortly after they happen. It is the usual mechanism behind continuous replication, phased migrations with a parallel run, and two-way synchronisation between systems that both stay live. This page explains the three common ways changes are captured, what each asks of the source system, what goes wrong in practice, and what to settle before you rely on it.
Published 8 min read
In short
- What it is: a way of finding only what changed in a database, so only the changes move — not the whole table every night.
- What it is used for: bridging old and new systems during a migration, feeding a reporting copy, keeping two live applications in step, and maintaining a standby copy.
- Three common capture methods: polling with timestamps or version columns, database triggers, and reading the transaction log.
- What decides success: access to the source, change order, what happens when the target is down, and how schema changes are handled.
Where change data capture is used
The same mechanism serves different jobs. The difference that matters for planning is whether the replication has an end date.
| Use | What CDC does | Ends when |
|---|---|---|
| Phased migration | Keeps the new system current while the old one still takes transactions | The cutover is complete and the old system is retired |
| Reporting or analytics copy | Feeds a separate copy so heavy queries do not run on production | It does not; it runs for as long as the copy is needed |
| Two live applications | Keeps shared records consistent between systems that both stay in service, sometimes in both directions | It does not; it becomes part of operations |
| Standby copy | Maintains a second copy of a critical database in another location | It does not; it is part of continuity planning |
If the replication has no end date, treat it as an integration that needs an owner, monitoring and change management — not as a project task.
Data migration or data integration: the difference (in Arabic)
Three common ways to capture changes
How changes are found decides the load on the source system, whether deletes are seen, and what you need from the database owner.
| Method | How it works | Effect on the source | What to watch |
|---|---|---|---|
| Polling (timestamps or version columns) | A job queries rows whose "last changed" value is newer than the last run | Extra queries on a schedule; heavier as tables grow | Rows deleted outright leave nothing to find unless deletes are recorded; changes between polls are seen only as their final state |
| Database triggers | Triggers on each table write the change into a change table that is then read | Adds work to every insert, update and delete | Triggers are part of the schema, so they must be deployed, versioned and tested with it |
| Transaction-log reading | Reads the database's own log of committed changes as it is written | Minimal; no application or schema change | Needs log access and enough log retention, coordinated with log archiving; log formats differ by database and version |
Which method applies depends on the source system and the tool. Ask any provider, including us, which method will be used for each of your sources and why.
What CDC asks of the source system owner
The source database usually belongs to a team that is not running the migration or the integration. Agree these points with them before building anything:
Read more
- Permissions: the account the capture process uses and what it is allowed to read.
- Log retention: if changes are read from the log, how long log records are kept and how that fits with archiving and backups.
- Initial load: the existing data is copied first, then the stream of changes catches up from a known point. Decide when the initial copy runs and how much load it may place on production.
- Load limits: when the source is busiest, and how much capacity replication may take during those hours.
- Schema changes: what happens when a column is added, renamed or dropped on the source — and who tells the replication team.
- Network path: where the capture process runs and how it reaches the source and the target.
What goes wrong in practice
Capturing changes is half the job. The other half is delivering them reliably, and that is where these problems appear:
Read more
- The target is unavailable. A design that fails when the target is down loses changes or stops the pipeline. Changes should be held and delivered when the target returns.
- Order matters. An update applied before the insert it depends on fails or corrupts data. Changes must be applied in the order they were committed.
- Duplicates happen. After a restart, some changes may be delivered twice. Applying the same change twice must not create two records.
- Deletes are invisible to some methods. Polling on a timestamp column cannot see a row that no longer exists.
- Replication competes with production. Without limits, catching up after an outage can load the source at the worst moment.
- Lag is not watched. If nobody measures how far the target is behind the source, the first sign of trouble is a user reporting missing data.
One-way replication or two-way synchronisation
One-way replication has one source of truth: changes flow from the source to the target, and the target is read-only for that data. Two-way synchronisation lets both systems accept writes and sends each side's changes to the other — which is useful when old and new systems must both be used during a transition, and risky when the same record can be changed on both sides at once.
Before choosing two-way synchronisation
Decide which system owns each type of record, or how a conflict is resolved when the same record changes on both sides. This is a design decision to settle during scoping, not something a tool resolves on its own. Where you can, keep ownership one-way per record type and use two-way only where the business genuinely needs both systems to write.
Change data capture as the bridge in a phased migration
In a phased migration, CDC is what lets the old system keep running while the new one is proven:
- Copy the existing data to the new system from a known point.
- Stream changes continuously from the old system so the new one stays current.
- Move reads first. Point reporting and read-only functions at the new system while writes still go to the old one.
- Rehearse the cutover, including the switch back, before doing it for real.
- Cut over, then keep the way back open for the agreed period, with changes flowing in the reverse direction if the plan requires it.
Read more
The length of the parallel run and the condition that triggers cutover are agreed before the work starts. For how Interkey sequences this end to end: Database migration services. For the decisions to settle first: The migration decision checklist
Where iblync fits
iblync is Interkey's own data migration and replication platform, built in Saudi Arabia, for moving data between systems that were not designed to exchange it. What it does:
- Continuous replication, including two-way synchronisation where both sides accept writes.
- Changes held in a queue when the target is unavailable, then delivered when it returns, rather than failing.
- Throttling, so replication does not compete with production for capacity.
- Low-code workflows configured rather than hand-scripted per project, so your own database team can see, review and repeat the migration.
- A connector set that is actively extended, with custom connectors built for systems it does not yet cover.
- Switching back stays possible during the parallel run; its length and the cutover trigger are agreed in scoping.
Read more
Where iblync runs is decided per project, and it is priced by quotation based on the source systems, target systems and data volume. Throughput on your estate depends on row volume, object count, transaction rate and how different the source and target data models are, so it is measured on your own systems during scoping rather than quoted from a benchmark.
When iblync is the wrong tool: if both systems come from one vendor that provides a supported migration or replication path, that path is usually cheaper and better supported. iblync fits when no such path exists, when the data models differ enough to need real transformation, or when systems must stay in sync.
iblync: data migration and replication
Questions to settle before you rely on CDC
- Which capture method will be used for each source, and what access does it need?
- How long are source log records kept, and who changes that setting if needed?
- How is the initial copy taken, and how is the catch-up point recorded?
- What happens to changes while the target is unavailable, and for how long can they be held?
- Are changes applied in commit order, and is applying the same change twice safe?
- How are schema changes on the source detected and handled?
- How is lag between source and target measured, and who is alerted?
- If two-way: who owns each record type, and how are conflicts resolved?
Questions buyers ask
Is change data capture the same as ETL?
No. ETL (extract, transform, load) describes moving and reshaping data. CDC is a way of finding what changed so that only the changes move. A pipeline can use CDC as its extract step, run continuously instead of in nightly batches, and still transform data on the way.
Does replication replace backups?
No. Replication copies mistakes as faithfully as it copies good data: a wrongly deleted table is deleted on the target too. Keep backups, and treat a replicated copy as a way to stay available, not as a way to go back in time.
How close to real time is it?
It depends on the capture method, the network, the target's write speed and the volume of changes. Measure it on your own systems before you depend on it, rather than relying on a published figure.
Can a migration run with no downtime at all?
With a parallel run and continuous replication, downtime can be reduced to the cutover step itself. How short that step is depends on application changes and acceptance testing, so it is set during assessment, not promised in advance.
We moved to a new database and replication stopped working. What usually causes that?
Common causes are a capture process still pointing at the old server or an old log position, permissions that were not recreated, log retention set lower on the new server, schema differences between the old and new databases, and triggers or identity settings that behave differently after the move. Check each of these before rebuilding the pipeline.
Does the target have to be the same kind of database as the source?
No. CDC is often used to move data into a different kind of system, for example from a relational database to a document database. The data model then needs real transformation, which is where most of the design effort goes. Couchbase is one target Interkey implements, but iblync does not require it.
Next
Read next
Next step
Scope a replication or phased migration
Tell us the source and target systems, roughly how much data changes each day, and whether replication is temporary (for a migration) or permanent. The Riyadh team reviews your request and a specialist replies on Saudi working days, in Arabic or English.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia