API management · AI gateway
AI gateway in Saudi Arabia: one control point for LLM and agent traffic
An AI gateway sits between your applications and the language models and AI agents they call. It applies identity, token budgets, prompt and data rules, routing and logging in one place, so each new AI feature does not arrive with its own keys and its own rules. Interkey designs, implements and operates it with Kong AI Gateway, from Riyadh.
Published 8 min read
In short
- What it is: a gateway for AI traffic: calls to hosted LLM providers, calls to models you run yourself, and the tool calls AI agents make to your systems.
- When you need one: when a second team or a second model provider appears, or when someone asks what data has been sent to a model and what it cost.
- What it controls: who may call which model, token budgets per team, prompt and response rules, removal of personal data before a request leaves, routing and fallback between models, and one usage log.
- What it does not decide: where a model processes your data. That follows from the model you route to. The gateway enforces the routing your data policy allows.
Why AI traffic needs its own control point
The first LLM feature usually goes live with one provider key in an application's configuration. By the third or fourth, several teams hold keys to different providers, nobody can say which data classes are being sent where, and the model bill arrives as one number. An AI gateway puts that traffic under the same discipline you already expect from your APIs: one entry point, one policy, one log.
AI gateway or the API gateway you already run?
Both sit in the same place in the architecture. The difference is the unit of control: an API gateway counts requests and checks who is calling; an AI gateway also has to understand prompts, tokens and models.
| API gateway | AI gateway | |
|---|---|---|
| Limits are counted in | Requests per period | Tokens per period, per consumer or team |
| What is inspected | Caller identity, paths, headers, payload size and schema | Prompt and response content, against topic, prompt-injection and personal-data rules |
| Routing | To a service, by path or host | To a model or provider, by policy, with retry and fallback; optionally by the meaning of the prompt |
| Caching | Exact-match responses | Semantic: a near-duplicate prompt can be answered from cache |
| Cost reporting | Request counts, with no cost per call | Token spend attributed to the team or application that caused it |
| AI agents | Agent calls are treated as ordinary API calls | Governs agent tool calls to MCP servers, and can publish existing APIs as tools agents may call |
If you already run Kong for your APIs, this is an extension rather than a second product: Kong AI Gateway is built on the same core runtime as Kong Gateway, and both can be managed from the same Kong platform.
What Kong AI Gateway can enforce, and what to decide first
Each capability below is documented by Kong. None of them works well without a decision your organisation has to make first, so each row names that decision.
| Capability | What it does | Decide before you switch it on |
|---|---|---|
| Model access through one API | Applications call one gateway endpoint; the gateway holds the provider credentials and reaches the approved models, whether hosted (Kong documents the major providers, including Amazon Bedrock and Google's models) or run by you | Which providers and models are approved, and for which data classes |
| Token budgets | Limits token consumption per consumer and attributes cost per token to teams or applications | Who owns each budget, and what happens when it runs out |
| Prompt rules | Blocks requests that match disallowed topics or keywords; a semantic guard catches jailbreak and prompt-injection attempts written in natural language | The topic policy, and who reviews blocked requests |
| Personal-data removal | Redacts personal data before the request reaches the provider, and can put it back in the response | Which data types must never leave. Kong's documentation does not list Arabic among its bundled language models, so test on your own Arabic and English samples |
| Reliability | Balances load across model instances, retries, and falls back to another model when one fails | Which models are acceptable substitutes for each use |
| Semantic cache | Answers near-duplicate prompts from cache instead of calling the model again | Which use cases may receive a cached answer |
| Agent tool calls (MCP) | Puts agent traffic to MCP servers behind the same authentication and logging, and can turn existing APIs into tools an agent can discover and call | Which internal APIs an agent may use at all |
| Usage visibility | Token volumes and usage per consumer, in Kong's own analytics or exported to the monitoring tools you already use | Where AI usage logs are kept and for how long |
| Configuration as code | Managed as code with Kong's declarative tools (decK for self-hosted, kongctl for Konnect) or through Kong's API | Who approves policy changes |
The prompt rules reduce risk; they do not remove it. What an agent is allowed to do inside your systems still has to be limited by the permissions you give it. Kong documents several of these policies, including token budgets, personal-data removal, semantic caching and load balancing across models, as part of its AI Gateway Enterprise offering, which needs an AI licence. Kong licensing is quoted on Kong's terms.
Where the gateway runs, and where your prompts go
These are two separate questions. The gateway can run inside the Kingdom. Whether a prompt leaves the Kingdom depends on the model it is sent to.
Read more
- Where traffic passes (data plane): Kong documents data plane nodes running self-hosted, in a cloud, or on Kubernetes. Interkey deploys Kong self-managed in the Kingdom, or in a hybrid shape where only configuration is managed outside your estate.
- Where configuration lives (control plane): either self-managed alongside the data plane, or in Kong Konnect. Konnect offers several geographic regions, including a Middle East region. Kong documents that objects such as gateway services, routes and consumers are specific to the region you select, and that authentication, billing and usage are shared across regions. Check that arrangement against your own data policy before choosing it.
- Where the model processes the data: a hosted model API means the input leaves your estate. The gateway can enforce "this data class may only go to this model", including a model you run yourself. It cannot change where a provider processes what it receives.
Example: routing by data class
| Your data class (example) | Models it may reach | Gateway rule |
|---|---|---|
| Public content, such as marketing drafts | Any approved hosted provider | Token budget and logging |
| Internal documents | Approved providers only | Personal-data removal and topic rules |
| Customer personal data | Only models running where your policy allows | Restricted to named applications; everything else blocked |
| Restricted data | None, unless explicitly approved | Denied by default |
This is an illustration. Your own classification policy, and the rules that apply to your sector, decide the actual rows.
How Interkey sets up an AI gateway
The order is the same one Interkey uses for any gateway: inventory and policy first, the gateway third.
-
Inventory AI use.
Which applications and teams call which models, with which keys, carrying which classes of data.
-
Write the policy.
Approved models per data class, token budgets, prompt and personal-data rules, and how long usage logs are kept.
-
Deploy in the shape your policy allows.
Self-managed in the Kingdom, or hybrid with configuration managed outside your estate.
-
Move consumers and close the back doors.
Provider keys move into the gateway; direct calls from applications to providers are switched off.
-
Operate.
Watch usage and blocked requests, tune rules, and upgrade. Interkey can operate the gateway as a managed service on an agreed scope; coverage hours and response targets are written into each agreement.
What to ask any AI gateway provider, including us
- Can the data plane run where our policy requires, and where exactly is configuration stored?
- Can we set token budgets per team and see cost per team?
- How are provider keys stored and rotated?
- Which personal-data types does redaction cover, and how does it perform on our Arabic and English samples?
- What happens when a model provider is down or rate-limits us?
- Do agent tool calls go through the same authentication and logs as everything else?
- Which features need a paid licence in the deployment we are choosing?
- Who operates and upgrades the gateway after go-live?
Questions buyers ask
Do we need an AI gateway if we only use one model provider?
Not always on day one. It becomes necessary when a second team, a second provider or a data-protection question arrives. Even with one provider, keeping the key, the token budget and the usage log in one place saves rework later.
We already run Kong for our APIs. Is this a separate product?
Kong AI Gateway is built on the same core runtime as Kong Gateway and is managed with the same Kong tooling, so it extends the gateway you already operate. Several AI policies, such as token budgets and personal-data removal, are part of Kong's AI Gateway Enterprise offering and need an AI licence, so licensing depends on the features you use.
Does an AI gateway stop prompt injection?
It detects and blocks attempts that match the rules you configure, including attempts phrased in natural language. That lowers the risk; it does not replace limiting what an agent is allowed to do.
Can everything stay inside the Kingdom?
The gateway can. Whether prompts leave depends on the model they are routed to. If a data class must be processed in the Kingdom, route that class only to a model running in an environment that meets your policy, and block it everywhere else.
Is Kong an Interkey product?
No. Kong is a product of Kong Inc. Interkey designs, implements and operates it for organisations in Saudi Arabia.
Next
Related pages
Next step
Discuss an AI gateway design
Tell us which applications call which models today and what kind of data they carry. We will reply with a proposed gateway shape and the questions your security team is likely to ask.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia