API management · AI gateway
LLM gateway in Saudi Arabia: one control point for every model your teams call
An LLM gateway is a single layer between your applications and the large language models they use. Every prompt, response and agent tool call passes through it, so model access, cost limits, personal-data removal and logging are decided once, not in each application. This page sets out what an LLM gateway must do, what to ask before you choose one, and how Interkey implements it on Kong AI Gateway in Saudi Arabia.
In short, for executives
- What it is: one control point for model traffic: which models each team may use, what it may spend, what data may leave.
- Why now: once several teams call several models with their own keys, nobody can say what was sent or what it cost.
- What Interkey implements: Kong AI Gateway. Kong Inc. is one of Interkey's technology partners.
- What it does not do: change where a hosted model processes a prompt, or make an unsafe application safe.
At a glance
| Question | Answer |
|---|---|
| What it controls | Model access, routing, token and cost limits, prompt rules, personal data, caching, MCP tools, usage records |
| Product Interkey implements | Kong AI Gateway, managed from Kong Konnect or configured on-premises |
| Where prompts are handled | A data plane in your environment: your servers, a cloud account you control, or Kubernetes |
| How it is licensed | At Kong's terms; the AI policies are part of Kong's AI Gateway Enterprise offering |
| From Interkey | Inventory, policy, deployment, Arabic testing, support from Riyadh and operation |
What an LLM gateway is, and why teams add one
An API gateway decides who may call which service and how often; our API gateway page covers it. An LLM gateway does the same for calls to language models, and adds controls that only make sense there: limits counted in tokens, rules that read the prompt, removal of personal data from free text, and a cost on every request. Vendors also call it an AI gateway; it is the same layer.
Read more
The need appears in a familiar order. One team builds an assistant with a provider key, a second picks another model, an agent project starts calling internal tools. Soon there are many keys, no record of what left the network, and an invoice nobody can split by team. The gateway puts those decisions back in one place without rewriting each application.
What an LLM gateway must do
Use this list to compare any proposal, including ours.
| Requirement | Why it matters | Ask the supplier |
|---|---|---|
| Multi-LLM routing and failover | Applications call one endpoint; each request goes to the model approved for its data, and moves when that model is down | Which fallback applies to each use case, and who approved it? |
| Token and cost limits | One team's experiment cannot spend another team's budget | Are limits in tokens or cost, per team, model or provider? |
| PII sanitisation | Names, phone and ID numbers are removed before a prompt leaves | How are Arabic text and Saudi number formats tested? |
| Prompt guards | Disallowed topics and injection attempts are blocked first | Do the rules work on Arabic and mixed prompts? |
| Semantic caching | Repeated questions are answered without a new model call | Which use cases may reuse an answer? |
| MCP tool governance | Agents call only allowed tools, with tighter access for writes | How are tool calls scoped and logged? |
| Usage records | Security sees what was sent; finance sees who spent what | What is logged, for how long, and where? |
Multi-LLM routing and LLM cost control
Routing is a policy decision first. A customer assistant, an internal summariser and a code helper rarely need the same model or carry the same data. We write a routing table with you: for each use case, the data class, the primary model, the acceptable fallback (or none) and whether simple requests may go to a cheaper model. Changing a model then becomes a reviewed gateway change.
Read more
Budgets are counted in tokens. Providers charge per token, so a request count says little about spend. The gateway counts tokens or cost per team, application or model, and blocks, slows or alerts at the limit. Usage records carry cost per consumer, so spend can be charged back.
Semantic caching compares the meaning of a new prompt with earlier ones, through an embeddings model and a vector store, and returns the stored answer when they are close. It suits stable questions, such as policy FAQs, not answers that depend on the user, the date or live data, so we enable it per use case. The embeddings call is one more place a prompt goes.
PII sanitisation, prompt guards and the MCP gateway
A sanitiser removes names, phone, card and ID numbers before the prompt leaves, and can restore them in the response. Prompt guards block requests that match disallowed topics or resemble known attacks. Both need Arabic work: English rules miss Arabic phrasing, and Saudi mobile, national ID and Iqama numbers can arrive in Arabic-Indic digits. We run an Arabic, English and mixed test set through the gateway before go-live. Our prompt injection guide shows which controls belong in the gateway and which in the application.
Read more
The MCP gateway. Agents call tools through the Model Context Protocol (MCP). An MCP gateway applies the same control point to those calls: which tools each agent may discover, how each call is scoped, and a record of what was done. Writes should need tighter access than reads. Where your internal APIs already sit behind a gateway, agent traffic is governed in the same place.
Relatedprompt injection guide
How Kong AI Gateway meets each requirement, and what drives cost
A summary, as Kong documents it. The policy-by-policy detail is on our Kong AI Gateway page.
| Requirement | How Kong AI Gateway handles it |
|---|---|
| Routing and failover | One consistent API to the supported LLM providers, load balancing, and failover to another provider or model |
| Token and cost limits | AI Rate Limiting Advanced, per consumer, consumer group, model or provider |
| PII sanitisation | AI Sanitizer with a separate anonymiser service; custom patterns for formats such as Arabic and Saudi IDs |
| Prompt guards | AI Prompt Guard with allow and deny patterns; AI Semantic Prompt Guard by similarity to examples |
| Semantic caching | AI Semantic Cache for near-duplicate prompts |
| MCP tools | Tools from MCP servers, existing APIs turned into MCP tools, calls scoped with OAuth2 |
| Usage records | Tokens, latency and cost through audit logs, metrics exporters and OpenTelemetry |
Kong states that several of these policies, including AI Rate Limiting Advanced, AI Sanitizer and AI Semantic Cache, are only available in its AI Gateway Enterprise offering. Names and licensing change between releases, so we confirm both for your version before quoting. See our Kong AI Gateway page for the full walk-through.
What drives cost. Kong licensing is quoted at Kong's terms; this page prints no prices. The total depends on the AI policies you need, Konnect or an on-premises control plane, the number of environments, use cases and teams to onboard, the Arabic test work, and whether Interkey operates the gateway. Our quote shows licence, implementation and operation on separate lines.
Where an LLM gateway runs in Saudi Arabia
- The data plane reads every prompt. Kong documents it running in your environment, so it can stay on infrastructure you operate in the Kingdom.
- The control plane holds configuration and usage metadata. Kong documents Konnect in several regions, including one labelled Middle East, and on-premises configuration. Choose by your classification.
- The model receives the prompt after sanitisation, and a hosted model processes it where its provider runs it. A model you host in the Kingdom keeps the whole path local.
Personal data in prompts falls under Saudi Arabia's Personal Data Protection Law, overseen by SDAIA. Our data residency guide covers classifying data first.
Relateddata residency guide
Where organisations in the Kingdom use an LLM gateway, and who supports it
- Internal assistants over policy documents, with a budget per department.
- Customer service in Arabic and English, with names and numbers removed first.
- Agents that update records through MCP tools, with writes scoped tightly.
Support from Riyadh. Interkey's engineers work from Riyadh, in Arabic and English, Sunday to Thursday. After go-live we back your platform team or operate the gateway under a written managed-services scope. To see inside the application around each call, pair the gateway with LLM observability; for the wider programme, see enterprise AI.
Relatedenterprise AI
How Interkey delivers an LLM gateway
-
Inventory the AI use.
Which teams and agents call which models, with which data and keys.
-
Write the policy.
Routing, budgets, sanitiser and guard rules, caching and logging, agreed with security.
-
Deploy.
Data plane in your environment, control plane to fit your classification, as configuration.
-
Test in Arabic and English.
Attack, personal-data and normal prompts, before production.
-
Move teams and operate.
One team at a time, retiring direct keys; then updates and cost reports.
Questions buyers ask
What is an LLM gateway, and do we need one if we use a single model?
It is one control point between your applications and the models they call, applying access, cost, personal-data and logging rules to every request. With one model and one team, provider settings may do at first; the need usually comes with the second team, model or agent.
Can an LLM gateway keep our prompts and personal data inside Saudi Arabia?
The data plane that reads prompts can run on infrastructure you operate in the Kingdom and remove personal data before a prompt leaves. Where a hosted model then processes it depends on its provider. Prompts that must not leave go to a model hosted in the Kingdom, or are blocked.
How does an LLM gateway control what each team spends on models?
It counts tokens or cost per team, application or model, and blocks, slows or alerts at your limit. Usage records carry cost per consumer for chargeback, and semantic caching and cheaper routes reduce the total.
When is semantic caching safe to switch on?
For stable questions whose answer does not depend on who asks, such as policy FAQs. Keep it off where answers depend on the user, the date or live data, and check where the embeddings model runs.
Is an MCP gateway separate from an LLM gateway?
On Kong it is the same gateway: Kong AI Gateway governs model traffic and MCP tool calls together, with calls scoped through OAuth2. Design them as one, because an agent sends prompts and calls tools in the same task.
How long does an LLM gateway rollout take in KSA, and what does it cost?
It depends on the use cases and teams, the Arabic test work and your security review. A first use case is the short part; moving every team takes longer. Kong licensing is quoted at Kong's terms and Interkey's work beside it, with an estimate after the inventory.
Next
Read next
Next step
Plan your LLM gateway
Tell us which teams use language models, which models they call, what data the prompts carry and what must stay in the Kingdom. We will propose a gateway design and a first use case.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia