Datadog CI Visibility · pipelines and tests
CI Visibility with Datadog: slow pipelines and flaky tests, measured
CI Visibility in Datadog shows every pipeline run as a trace: which job waited in the queue, which step failed, which test fails one run and passes the next. For an engineering manager it answers a plain question: where does the time between a commit and a release go? Interkey is an official Datadog partner in Saudi Arabia: we sell the licences, connect your CI tools, and provide technical support from Riyadh.
In short
- Two views, one product family: CI Visibility shows pipelines, stages, jobs and steps; Test Optimization shows the tests inside them. Datadog groups both as CI/CD Optimization.
- Where it connects: GitHub Actions, GitLab, Jenkins and Azure Pipelines, among the CI tools Datadog lists, each through its own integration.
- What Interkey does: licences, connection of each CI tool, one tagging standard, dashboards and alerts on pipeline health, and technical support.
At a glance: how CI Visibility connects to each CI tool
The table summarises what Datadog documents for each CI tool as of 11 November 2026. We check it against your versions before rollout.
| CI tool | How it connects | What Datadog shows | Watch before you start |
|---|---|---|---|
| GitHub Actions | A private GitHub App with read-only access to Actions | Workflows as pipelines, jobs and steps, queue and run time | Runner host metrics need the Datadog Agent on the runner |
| GitLab | The Datadog integration in GitLab settings, per project or group | Pipelines, stages and jobs, queue time, approval waits, downstream pipelines | GitLab.com or self-managed 14.1 and later; older versions need a feature flag or webhooks |
| Jenkins | The Datadog Jenkins plugin, through the Datadog Agent or agentless with an API key | Pipelines, stages and jobs, queue time, failure reasons | Jenkins 2.346.1 or later; the Agent is needed for host correlation |
| Azure Pipelines | Datadog's Azure app on an Entra-connected Azure DevOps organisation, or service hooks | Runs, stages and jobs; steps only with the app method | Azure DevOps Server is not officially supported |
Why pipelines become slow and flaky
In Saudi banks, telcos and government entities, a release often passes through several pipelines: a build in GitHub Actions or GitLab, integration tests on self-hosted runners, a Jenkins job that still deploys the older systems, and a database change applied before the application. Each pipeline was fast when it was written. Over time, more tests are added, runners are shared, a cache stops working, and a few tests start to fail at random.
Read more
The cost is quiet. Developers press "retry" until a flaky test passes, wait for a busy runner, or stop trusting red builds. The CI tool shows one run at a time, so nobody sees the trend: which job grew from four minutes to twelve, or which test failed on twenty commits that did not touch it.
CI pipeline monitoring: what CI Visibility shows
A pipeline-first view. Datadog describes CI Visibility as a pipeline-first view of CI health. Each run arrives as a trace, with the pipeline at the top and stages, jobs and, where the CI tool allows it, steps underneath. Queue time and execution time are measured separately, so a job that waits for a free runner is not mistaken for a job that runs slowly.
Read more
The critical path. For every CI tool in the table, Datadog identifies the jobs on the critical path of the pipeline: the ones that decide how long the whole run takes. That is where an hour of optimisation work saves the most time.
Failures with their cause. Datadog links failures to the commits that caused them. When job logs are sent to Datadog, its Logs Analysis groups failures by likely cause. Log storage is billed separately from CI Visibility.
Monitors and dashboards. Pipeline duration, failure rate and queue time become metrics like any other in Datadog, so a monitor can alert the owning team when the main branch build slows down or starts failing.
Test visibility and flaky test detection
Tests, not only jobs. Test Optimization gives a test-first view: each test case with its duration, result and history, across branches and commits. Datadog lists set-up for .NET, Java and JVM languages, JavaScript, Python, Ruby, Swift and Go, and accepts JUnit XML reports from other frameworks.
Flaky test detection. Datadog defines a flaky test as one that both passes and fails across several runs for the same commit. Such tests are tagged as flaky, and a failure of a test already known to be flaky is marked so, which tells the developer the failure may come from the test rather than the change.
Read more
Optional features. Three features are off until enabled in Datadog's CI/CD Optimization settings: Early Flake Detection retries new tests to catch flakiness before they merge; Auto Test Retries retries a failed test a set number of times so a flaky test does not fail the build; and Test Impact Analysis skips tests that a change cannot affect, using code coverage and git data. Datadog's PR Gates can also block a pull request that introduces a new flaky test.
DevOps observability: from the pipeline to production
CI Visibility is most useful when the pipeline and production speak the same language. With unified service tagging, the pipeline sets the same service, env and version tags that Datadog APM reads in production, so a slower endpoint can be traced back to the release, and the release to the pipeline run that built it.
Read more
Database deploys. A database change is often the step that fails late or holds a release. When migrations run as their own job, for example with Redgate Flyway, CI Visibility times and reports that job like any other, so a slow or failed migration is visible next to the build and tests. Our database DevOps page covers how Interkey puts schema changes under version control, and the database CI/CD guide how they move through the pipeline.
How a CI Visibility rollout runs with Interkey
-
List the pipelines.
Each CI tool, its version, its runners and the teams that own the main pipelines.
-
Count the committers.
Active contributors per month, the basis of the licence quote, with bots and excluded accounts set aside.
-
Connect one CI tool.
One project or organisation first, with the tagging standard written down.
-
Add the tests.
Test Optimization on the main test suites, with flaky test detection reviewed with the developers.
-
Build dashboards and monitors.
Duration, failure rate and queue time per pipeline, each alert naming the owning team.
-
Operate or hand over.
Interkey operates the set-up from Riyadh, or hands it over with documentation and stays on as technical support.
Data residency, cost and timeline
Pipeline data carries repository names, branch names, commit messages and commit author emails, and job logs may hold internal host names or a secret printed by mistake. As of November 2026 Datadog lists no site in the Middle East, so this data is stored outside the Kingdom. Interkey agrees with your security team which logs are sent, which repositories are included, and how secrets are kept out of logs. Our Datadog data residency guide covers the wider question. Support comes from Interkey's engineers in Riyadh, in Arabic and English.
Read more
Datadog bills CI Visibility by committer: in its billing documentation, an active git contributor identified by author email who makes at least three commits in a month. Commits by verified bots are not billed. Our Datadog pricing guide sets out how each unit is billed, and Interkey quotes the licences with the set-up and support.
The timeline is an estimate, not a promise. A first CI tool usually reports within days once access is agreed. A full estate typically takes a few weeks, depending on the number of CI tools and versions, self-hosted runners, and how many test suites need a test library added.
RelatedDatadog data residency guide
Questions buyers ask
Which CI tools can Datadog CI Visibility monitor?
Datadog lists GitHub Actions, GitLab, Jenkins, Azure Pipelines, AWS CodePipeline, Buildkite, CircleCI, Codefresh and TeamCity, and other CI tools through its datadog-ci command line. The detail collected differs by tool.
Can Datadog tell us which tests are flaky in our pipelines?
Yes, with Test Optimization. Datadog marks a test as flaky when it both passes and fails for the same commit, and can retry new or failed tests so a flaky test does not block the build.
Does CI Visibility work with self-hosted GitLab and Jenkins inside our network?
Yes, within Datadog's limits: self-managed GitLab 14.1 or later, and Jenkins 2.346.1 or later with the Datadog plugin. The servers need outbound access to Datadog, directly or through the Datadog Agent.
How is Datadog CI Visibility licensed for a team in KSA?
By committer: Datadog counts a git author who makes at least three commits in a month. Verified bots are not billed. Interkey sells Datadog licences in Saudi Arabia and quotes them with set-up and support.
Does our pipeline and test data leave Saudi Arabia?
Yes. As of November 2026 Datadog lists no Middle East site. Interkey agrees with your security team which repositories and logs are sent, and how secrets are kept out of job logs.
How long does a CI Visibility rollout take?
A first CI tool usually reports within days once access is agreed. A full estate typically takes a few weeks, depending on CI tools and versions, self-hosted runners and the number of test suites.
Next
Read next
Next step
Measure where your pipeline time goes
Tell us which CI tools you run (GitHub Actions, GitLab, Jenkins, Azure Pipelines), whether runners are self-hosted, roughly how many developers commit each month, and whether Datadog is in use. We can quote licences, set-up and support together.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia