Data platform

ETL tools: the kinds that exist, and how to choose one

The four kinds of ETL tools, the criteria that separate them, and the questions to ask any vendor before you sign.

  • Saudi technology company, in Riyadh since 1999 CR 1010156897
  • ISO 9001:2015 and ISO/IEC 27001:2022 Certified by AQC
  • Implements and operates, after go-live too The same team from design to operation

ETL tools extract data from source systems, transform it, and load it into a target for reporting or analytics. They fall into four kinds: hand-written scripts, open-source frameworks, cloud provider services, and commercial platforms. The right kind depends on your sources, your team's skills, where the data may be processed, and how often it must refresh.

Four kinds of ETL tools

The four kinds of ETL tools compared
KindWho it suitsWhat it really costs youWhat to watch
Hand-written scriptsA small team with one or two well-known sourcesEngineer time to write each job, and to fix it when a source changesThey depend on their author; logging, retries and alerts are often missing
Open-source frameworksTeams with engineers who will build and run jobs in productionNo licence fee, but servers, upgrades, monitoring and the people who run itWho answers a failed job at night, and whether your connectors are maintained
Cloud provider servicesOrganisations whose sources and warehouse sit with one providerUse of the provider's services, and work to connect sources outside itThe processing region, and the effort for each source outside that cloud
Commercial platformsMany sources and jobs, and a team that prefers configuration to codeA supplier contract, on top of designing and running the jobsConnectors for your exact systems, local support, and how hard it is to leave

Most estates end up with more than one kind. The table describes kinds and names no product.

Eight criteria for choosing

Ask the same eight things of every tool, whatever its kind.

  • Connectors for your sources. A ready connector for each of your systems, at the version you run.
  • Where processing runs and where data stays. Extracts, staging tables and run logs hold real data; each place should be approved before the first run.
  • Pricing basis. The basis of the price, in writing, for your sources and volumes.
  • Skills required. Code, visual job design or configuration only, and who on your side has those skills.
  • Incremental loading. Loading only the rows changed since the last run, not every table each time.
  • Monitoring and alerting. A view of every run, and an alert to a named person when a job fails.
  • Local support. Who answers, and in which language, when a job stops before the morning reports.
  • Ease of leaving. Jobs and mappings that export in a readable form.

ETL tool or synchronisation tool

An ETL tool works in runs. It reads the source on a schedule, reshapes the data and loads it into a warehouse or a reporting database, and between runs the target is as old as the last load. That suits reports and analytics, where yesterday's figures or this morning's are enough. A synchronisation tool has a different job: it reads each change in the source as it happens and applies it to the target, so two systems stay the same while both are in use. If you need an application database kept in step with another, or a live system moved with a short switch, you are looking for synchronisation, not ETL. For migrations, see ETL vs CDC for data migration.

Where iblync fits

When the job is keeping systems in step

iblync is Interkey's platform for migration and continuous sync between different systems, set up through low-code workflows and delivered with the team that operates it. It serves the second job above.

Full integration suite or a focused tool

Some data integration platforms are suites: one product for extraction and loading, data quality rules, a catalogue of the data, lineage that shows where each figure came from, and governance over who may use what. Others are focused tools that do one job, such as moving and syncing data between systems.

Read more

A focused replication and sync tool is usually enough when the task is defined: move these databases, keep these two systems in step, feed this report. The data owners know what the data means, and quality is checked at the source. A suite earns its place when many teams share the same data, when each record needs one agreed definition, when quality must be measured and reported across systems, or when auditors ask where a number came from. A suite bought for one migration adds work; dozens of pipelines with no quality or governance layer leave gaps.

Questions to ask any vendor

Six short questions to copy into an email to every supplier.

  • Which of our systems have a ready connector, and who maintains it after a source upgrade?
  • Where do staging files and run logs sit, and who can read them?
  • Can a job load only what changed since the last run, and how?
  • When a job fails at night, who is alerted, and can it rerun from the failed step?
  • Can our analysts read and change a transformation themselves?
  • If we leave, can we export our jobs and mappings in a readable form?

Processing location and support in Saudi Arabia

In the Kingdom, two criteria often settle the choice. The first is where processing runs: an ETL tool copies data on the way into staging areas, temporary files and logs, and a cloud service processes it in the region its settings choose. For personal or classified data, each place should be approved before the first run; our guide to data residency in Saudi Arabia lists the questions.

The second is who answers when a nightly job fails before the morning reports, in which language, and how it is escalated.

Scope

What this page covers

This page covers ETL tools for reporting and analytics. To move a database to a new system, see database migration tools.

Questions about ETL tools

What are ETL tools?

ETL tools extract data from source systems, transform it, and load it into a target for reporting or analytics. They run on a schedule or on demand and record each run. The four kinds are hand-written scripts, open-source frameworks, cloud provider services and commercial platforms.

Which ETL tool is best?

No ETL tool is best for every organisation. The right one passes the eight criteria above on your own sources: a tool with every connector but no local support, or one easy to run that processes data where it may not go, is the wrong choice. Test the shortlist on your systems.

Are open-source ETL tools enough?

They are enough when your team can run them: deploy, upgrade, monitor and fix jobs in production. The licence is free; the operation is not. Count the servers, the monitoring and the engineers on call before comparing open source with other kinds.

Can ETL tools keep two systems in sync?

Not as designed. ETL tools work in batches, so the target is only as current as the last run; running them every few minutes adds load on the source and still leaves a gap. Continuous sync needs a tool that captures or replicates changes; see ETL vs CDC.

Where should ETL processing run for Saudi data?

Where your data residency requirements allow, which for personal or classified data may mean inside the Kingdom. That covers the processing and every staging file, temporary table and log the tool writes. Settle it before choosing; see data residency in Saudi Arabia.

Next step

Ask an engineer one question

Tell us your sources, your target and how often the data must refresh. Ask one question about the choice in front of you.

Required field

By sending this form, you agree to our Privacy Notice.

What happens next

  1. We read your request
  2. An engineer contacts you
  3. You get an estimate or a demo

Or directly

+966-11-2180999 [email protected]

Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia

or What is ETL?

Published by Interkey. Last updated . Interkey is registered in Riyadh, Saudi Arabia under commercial registration 1010156897.