Data platform · ETL vs CDC
ETL vs CDC: when ETL data migration is enough and when continuous CDC is safer
ETL data migration extracts data from the old system, transforms it to fit the new one and loads it, usually in batches inside a planned window. Change data capture (CDC) loads the data once and then carries every later change across continuously, so the old system keeps working until the switch. This guide explains how to choose, when to combine them, and where iblync, Interkey's own migration and replication platform, fits.
Published 9 min read
Orientation
At a glance
The whole page in one table, before the detail.
- ETL in a migration
- Extract, transform and load in batches; the source stops changing during the final run
- CDC in a migration
- One initial load, then every change applied continuously until the switch
- ETL is usually enough when
- The data fits one window, changes slowly, or the old system can be frozen
- CDC is usually safer when
- The data is large or busy, the window is short, or a way back is required
- The common combination
- Transform during the initial load, then keep the target current with CDC
- Where iblync fits
- The initial load and the continuous sync, when native tools do not cover the path
The choice in plain words
- The question: do we copy the data once in a planned outage, or keep copying it while the business works and switch later?
- Batch ETL is enough when the whole load, its checks and the switch fit inside the downtime the business accepts.
- Continuous CDC is safer when they do not fit, when the source keeps changing during a long load, or when the old system must stay ready as the way back.
- The cost of getting it wrong: an outage that overruns, or a target already stale on go-live day.
ETL data migration and CDC: two answers to the same question
Every data migration must answer one question: how do today's data, and the data written tomorrow, reach the new system intact?
In ETL data migration, a job reads the source, reshapes the records for the target (types, codes, keys, merged or split tables) and writes them, in batches. For a migration that means rehearsals on copies, then a final run during an outage while the source is frozen.
Read more
With CDC, the bulk of the data moves once in an initial load; after that only the changes travel, each insert, update and delete applied to the target continuously. The source is frozen only for the final switch. How changes are captured is covered on our change data capture page.
The two are not opposites: CDC finds what changed, ETL moves and reshapes data, and many of the safest migrations use both.
Relatedchange data capture
Why the choice matters for migrations in Saudi Arabia
Two local factors weigh on the decision. The first is the calendar: many organisations in the Kingdom get short, fixed windows, away from month-end closing, Ramadan, Hajj and peak seasons. A twelve-hour final ETL run may wait months for its window; a short switch after CDC can be booked much sooner.
Read more
The second is where the data sits while it moves. Extract files, staging areas, change streams and tool logs all hold real data and have a location. If the data is personal or classified, each place should be approved before the first run, a question of data residency in Saudi Arabia. The rules include the Personal Data Protection Law, overseen by the Saudi Data and Artificial Intelligence Authority (SDAIA), and the controls of the National Cybersecurity Authority (NCA), alongside your sector regulator's. Which apply depends on your sector and your data, so read the authorities' own texts with your compliance owner.
Relateddata residency in Saudi ArabiaSaudi Data and Artificial Intelligence Authority (SDAIA)National Cybersecurity Authority (NCA)
Batch vs real-time data migration: the options and their trade-offs
Four approaches, each right somewhere.
| Approach | How it works | Usually fits when | Trade-off |
|---|---|---|---|
| One-time batch ETL | Rehearsals on copies, then one final extract, transform and load while the source is frozen | The full run, its checks and the switch fit the downtime window | Simple and well understood; the outage is the whole final run |
| Batch ETL in waves | Data is moved in slices (by year, region or module), each in its own window | Data splits cleanly and the slices are independent of each other | Several shorter outages; slices that share records are hard to cut |
| Initial load plus continuous CDC | Bulk load once, then every change applied until the switch | Large or busy data, a short window, or a way back is required | More set-up and monitoring; the outage is only the switch |
| The engine's own replication | The target replicates the source natively and is promoted at the switch | Same engine, compatible versions, little transformation | Short outage and well supported, but limited to what the engine allows |
CDC earns its extra set-up when the data is too large or busy for one window, or when the old system must stay live as the way back.
When batch ETL is enough, and when continuous CDC is safer
When batch ETL is enough
Batch ETL is the right answer more often than it may seem. It is usually enough when:
- The data fits the window. A rehearsal shows the final run and its checks finish with time left for the switch.
- The source can be frozen. The old system can stop accepting writes for the window, for example after a financial close.
- Transformation is heavy. Records are merged, cleaned or re-coded; a batch job is easier to test and sign off than the same logic on a live stream.
- The source is being retired. Archive data that no one writes has no changes to capture.
Read more
In these cases CDC adds moving parts without removing a real risk.
When continuous CDC is safer
CDC becomes the safer route when any of these is true:
- The final run does not fit. A full load would overrun the window.
- The source keeps changing. A busy system cannot be frozen for long, and any batch copy is stale before it finishes.
- A way back is required. The old system stays current until the switch, so going back is a decision, not a restore.
- Both systems run in parallel. Users check the new system for weeks while the old one remains the record.
In return CDC needs access to the source's change log, enough log retention to survive a pause, lag monitoring and a plan for schema changes. Our CDC page lists what it asks of the source owner. For a large commercial source engine, our log-based capture guide for that engine covers the specifics.
The common combination: ETL for the initial load, CDC for the changes
Many migrations use both. The transformation is designed once and applied in two modes.
-
Design the mapping.
Agree with the data owners how each table, code and key becomes its target form.
-
Initial load with transformation.
Move the bulk once, transformed and throttled. Mark the point in the change log where the load began.
-
Apply changes from that point.
Apply every later change with the same mapping, and agree the acceptable lag.
-
Reconcile.
Row counts per table, checksums on money, identity and date columns, and one business report run on both sides.
-
Switch, and keep the way back.
Stop writes, let the last changes arrive, check and repoint applications. Keep the old system until the new one has carried a real peak.
ETL vs CDC decision table: questions to answer first
| Question | If the answer is yes | Who answers it |
|---|---|---|
| Does a rehearsed full load, plus checks, fit inside the downtime window? | Batch ETL is likely enough | Database administrator |
| Can the source stop accepting writes for that whole window? | Batch ETL is likely enough | Business owner |
| Does the data change heavily during working hours? | Lean towards CDC | Application owner |
| Must the old system stay current as the way back after the switch? | CDC, or the engine's own replication | Project lead |
| Will both systems run in parallel for weeks? | CDC | Business owner |
| Is the transformation heavy (merges, re-coding, model change)? | Batch ETL for the load; CDC on top if the window is short | Architect |
| Can the capture method read the source's changes without new load on it? | CDC is practical; if not, test the impact first | Database administrator |
For the wider decisions, see our database migration decision checklist and data migration framework.
ETL tools for data migration: what to ask of any tool
Whichever approach you choose, ask any tool before selecting it:
- Can a stopped run resume where it stopped?
- Can it throttle itself so the source keeps serving users?
- How does it prove completeness: row counts, checksums, per table?
- Where do its files, queues and logs live, and can they stay in the Kingdom?
- If the target is down for an hour, do changes queue or get lost?
- Who supports it, in which language, on the night of the switch?
If the migration starts a lasting flow of data between systems, the question becomes integration; our data integration services page covers that case.
Relateddata integration services
Where iblync fits, and where it does not
iblync is Interkey's own platform for data migration and replication between systems that were not designed to exchange data. In a migration it carries the initial load and the continuous sync, with real-time replication and flow control: throttling protects production, and changes queue if the target is down. iblync's own site states that an interrupted migration resumes where it stopped, and that validation uses checksums and row counts. Interkey confirms where it runs for each engagement.
Where it does not fit: if a single batch run or the engine's own replication does the job inside your window, that path is usually cheaper, and we will say so. Your mapping, downtime budget and data locations remain your owners' decisions.
Relatedreal-time replication
How Interkey helps you choose and run the migration
Interkey plans and runs data migrations for organisations in Saudi Arabia: the assessment, the choice between batch ETL, CDC or both, the mapping with your data owners, load and sync with iblync where it fits, reconciliation, the rehearsed switch and support after go-live. If your team has the ETL jobs, we can add only the sync and the switch.
Support comes from Riyadh, with Arabic-speaking engineers from the first call to the night of the switch. Before work starts we confirm where iblync runs and whether any data leaves your environment.
Read more
What drives the cost and the timeline
iblync is priced by quotation, against the source systems, the target systems and the data volume in scope; there is no published price list. Services are scoped by the steps you want us to run. A migration done with batch ETL and native tools alone costs only the work.
The timeline depends on the data more than the tool: size, peak change rate, transformation, network path, rehearsals and how long both systems stay live. We give an estimated range after the assessment.
Questions buyers ask about ETL and CDC for migration
Is a batch ETL job enough to migrate our database over one weekend?
It is if a rehearsal shows the final extract, transform, load and checks finish inside the weekend with time left to switch or go back. If the rehearsal runs close to the limit, or the data keeps growing, plan for CDC.
When should a Saudi organisation choose CDC instead of ETL for a migration?
When the data is too large or busy for the window you can get, when the old system must stay current as the way back, or when both systems will run in parallel. Short, fixed windows make this common in the Kingdom.
Can we transform the data in the initial load and then use CDC for later changes?
Yes, and many migrations do. The mapping is designed once, applied to the bulk load, then applied to every change read from the point where the load began, until the switch.
Does reading changes for CDC slow down our production database?
Log-based capture usually adds little load because it reads the change log, not the tables; trigger-based capture adds more. Measure it on a copy of your system first, and throttle the initial load.
How are an ETL or CDC migration with iblync priced and scheduled?
iblync is priced by quotation, against source systems, target systems and data volume; the services around it by the steps you want us to run. The timeline is estimated after the assessment, from size, change rate, transformation and rehearsals.
Can extract files, staging data and change logs stay inside Saudi Arabia during the migration?
Yes, if the plan places them there. Every file, staging area, queue and log is named and agreed with your compliance owner first; Where iblync runs is confirmed per engagement.
Next
Related pages
Next step
Choose the right migration method
Tell us the source and target systems, the data size, the peak change rate and your downtime window. The team that would run the migration replies.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia