Data platform
ETL tools: the kinds that exist, and how to choose one
The four kinds of ETL tools, the criteria that separate them, and the questions to ask any vendor before you sign.
Guide Published 6 min read
- Saudi technology company, in Riyadh since 1999 CR 1010156897
- ISO 9001:2015 and ISO/IEC 27001:2022 Certified by AQC
- Implements and operates, after go-live too The same team from design to operation
ETL tools extract data from source systems, transform it, and load it into a target for reporting or analytics. They fall into four kinds: hand-written scripts, open-source frameworks, cloud provider services, and commercial platforms. The right kind depends on your sources, your team's skills, where the data may be processed, and how often it must refresh.
Four kinds of ETL tools
| Kind | Who it suits | What it really costs you | What to watch |
|---|---|---|---|
| Hand-written scripts | A small team with one or two well-known sources | Engineer time to write each job, and to fix it when a source changes | They depend on their author; logging, retries and alerts are often missing |
| Open-source frameworks | Teams with engineers who will build and run jobs in production | No licence fee, but servers, upgrades, monitoring and the people who run it | Who answers a failed job at night, and whether your connectors are maintained |
| Cloud provider services | Organisations whose sources and warehouse sit with one provider | Use of the provider's services, and work to connect sources outside it | The processing region, and the effort for each source outside that cloud |
| Commercial platforms | Many sources and jobs, and a team that prefers configuration to code | A supplier contract, on top of designing and running the jobs | Connectors for your exact systems, local support, and how hard it is to leave |
Most estates end up with more than one kind. The table describes kinds and names no product.
Eight criteria for choosing
Ask the same eight things of every tool, whatever its kind.
- Connectors for your sources. A ready connector for each of your systems, at the version you run.
- Where processing runs and where data stays. Extracts, staging tables and run logs hold real data; each place should be approved before the first run.
- Pricing basis. The basis of the price, in writing, for your sources and volumes.
- Skills required. Code, visual job design or configuration only, and who on your side has those skills.
- Incremental loading. Loading only the rows changed since the last run, not every table each time.
- Monitoring and alerting. A view of every run, and an alert to a named person when a job fails.
- Local support. Who answers, and in which language, when a job stops before the morning reports.
- Ease of leaving. Jobs and mappings that export in a readable form.
ETL tool or synchronisation tool
An ETL tool works in runs. It reads the source on a schedule, reshapes the data and loads it into a warehouse or a reporting database, and between runs the target is as old as the last load. That suits reports and analytics, where yesterday's figures or this morning's are enough. A synchronisation tool has a different job: it reads each change in the source as it happens and applies it to the target, so two systems stay the same while both are in use. If you need an application database kept in step with another, or a live system moved with a short switch, you are looking for synchronisation, not ETL. For migrations, see ETL vs CDC for data migration.
Where iblync fits
When the job is keeping systems in step
iblync is Interkey's platform for migration and continuous sync between different systems, set up through low-code workflows and delivered with the team that operates it. It serves the second job above.
Relatediblync
Full integration suite or a focused tool
Some data integration platforms are suites: one product for extraction and loading, data quality rules, a catalogue of the data, lineage that shows where each figure came from, and governance over who may use what. Others are focused tools that do one job, such as moving and syncing data between systems.
Read more
A focused replication and sync tool is usually enough when the task is defined: move these databases, keep these two systems in step, feed this report. The data owners know what the data means, and quality is checked at the source. A suite earns its place when many teams share the same data, when each record needs one agreed definition, when quality must be measured and reported across systems, or when auditors ask where a number came from. A suite bought for one migration adds work; dozens of pipelines with no quality or governance layer leave gaps.
Questions to ask any vendor
Six short questions to copy into an email to every supplier.
- Which of our systems have a ready connector, and who maintains it after a source upgrade?
- Where do staging files and run logs sit, and who can read them?
- Can a job load only what changed since the last run, and how?
- When a job fails at night, who is alerted, and can it rerun from the failed step?
- Can our analysts read and change a transformation themselves?
- If we leave, can we export our jobs and mappings in a readable form?
Processing location and support in Saudi Arabia
In the Kingdom, two criteria often settle the choice. The first is where processing runs: an ETL tool copies data on the way into staging areas, temporary files and logs, and a cloud service processes it in the region its settings choose. For personal or classified data, each place should be approved before the first run; our guide to data residency in Saudi Arabia lists the questions.
The second is who answers when a nightly job fails before the morning reports, in which language, and how it is escalated.
Scope
What this page covers
This page covers ETL tools for reporting and analytics. To move a database to a new system, see database migration tools.
Relateddatabase migration tools
Questions about ETL tools
What are ETL tools?
ETL tools extract data from source systems, transform it, and load it into a target for reporting or analytics. They run on a schedule or on demand and record each run. The four kinds are hand-written scripts, open-source frameworks, cloud provider services and commercial platforms.
Which ETL tool is best?
No ETL tool is best for every organisation. The right one passes the eight criteria above on your own sources: a tool with every connector but no local support, or one easy to run that processes data where it may not go, is the wrong choice. Test the shortlist on your systems.
Are open-source ETL tools enough?
They are enough when your team can run them: deploy, upgrade, monitor and fix jobs in production. The licence is free; the operation is not. Count the servers, the monitoring and the engineers on call before comparing open source with other kinds.
Can ETL tools keep two systems in sync?
Not as designed. ETL tools work in batches, so the target is only as current as the last run; running them every few minutes adds load on the source and still leaves a gap. Continuous sync needs a tool that captures or replicates changes; see ETL vs CDC.
Where should ETL processing run for Saudi data?
Where your data residency requirements allow, which for personal or classified data may mean inside the Kingdom. That covers the processing and every staging file, temporary table and log the tool writes. Settle it before choosing; see data residency in Saudi Arabia.
Next
Read next
Next step
Ask an engineer one question
Tell us your sources, your target and how often the data must refresh. Ask one question about the choice in front of you.
What happens next
- We read your request
- An engineer contacts you
- You get an estimate or a demo
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia
or What is ETL?