Kong · AI Gateway
Kong AI Gateway: govern LLM traffic, cost and personal data
Kong AI Gateway sits between your applications and the large language models (LLMs) they call. Every prompt, response, agent tool call and token passes through one control point, where you set which models each team may use, how much they may spend, and what personal data must be removed before a prompt leaves your network. Interkey designs, implements and operates Kong AI Gateway for organisations in Saudi Arabia, from Riyadh, in Arabic and English.
Published 7 min read
In short
- What it is: Kong's gateway for LLM, MCP and agent-to-agent (A2A) traffic, with one endpoint for each traffic type.
- What it controls: model access, token and cost limits, prompt rules, personal-data redaction, caching, failover and usage logging.
- Where it runs: the data plane, which carries every prompt, runs in your environment: self-hosted, in a cloud you choose or on Kubernetes.
- How it is licensed: the AI policies covered on this page are part of Kong's AI Gateway Enterprise offering, quoted at Kong's terms.
- What Interkey does: design, deployment, policy configuration, team onboarding and operation after go-live.
Why an AI gateway, and how it differs from an API gateway
An API gateway controls who may call which service and how often. An AI gateway does the same for model traffic, and adds controls that only make sense for LLMs: limits counted in tokens rather than requests, rules that read the content of a prompt, redaction of personal data inside free text, and a cost figure for every request.
Without one, each team connects to model providers directly, with its own keys and no shared record of what was sent. Security cannot see which data left, and finance cannot see who spent what. A gateway puts those decisions in one place.
What Kong AI Gateway controls
| Control | What Kong documents | What your organisation decides |
|---|---|---|
| One endpoint for many models | Applications call one consistent API; the gateway connects to the supported LLM providers | Which providers and models are approved, and for which data classes |
| Failover and load balancing | Automatic failover to a different provider or model when one is slow or unavailable | Which fallback is acceptable for each use case |
| Token and cost limits | Limits by total, prompt or completion tokens, or by cost, per consumer, consumer group, model or provider | Budgets per team and what happens when a budget is reached |
| Prompt rules | Block requests that match disallowed topics or keywords; a semantic guard catches jailbreak and prompt-injection attempts phrased in natural language | Which topics are out of bounds, and who maintains the lists |
| Personal-data redaction | Redact personal data before it reaches the model provider, with optional restoration in the response | Which data types must never leave, and how matches are tested |
| Semantic caching | Serve repeated or near-duplicate prompts from cache instead of calling the model again | Which use cases may reuse answers |
| Prompt templates | Standard prompts and default instructions shared across teams | Who owns each template |
| Usage records | Token usage, latency and cost through audit logs, metrics exporters and OpenTelemetry | Where the records are stored and who can read them |
Feature names and behaviour change between Kong releases. Kong's documentation is the authority, and we confirm the current behaviour for your version during design.
Keeping personal data out of prompts
The AI Sanitizer policy sends each request to a separate anonymiser service, which can run as a container alongside the gateway. It detects categories such as names, phone numbers, email addresses, card numbers, passport numbers, national ID formats for several countries, bank details and credentials, and replaces them with placeholders or synthetic values before the prompt goes to the model. It can put the original values back into the response, so the user still sees a complete answer.
Read more
One point matters for Saudi services. Kong's documentation lists language images for English, Spanish, French, German, Italian, Japanese, Korean, Portuguese and Turkish. Arabic is not in that list. For Arabic free text and Saudi-specific formats, such as mobile numbers and national ID or Iqama numbers, plan custom pattern rules, which the sanitizer supports, and test them on real examples before go-live. Patterns can miss variants or mask harmless values, so we test both.
Controlling LLM cost per team
Kong's AI Rate Limiting Advanced policy counts tokens, or cost, rather than requests. Cost is calculated as prompt tokens multiplied by the input price, plus completion tokens multiplied by the output price, divided by one million. Limits can apply per consumer, consumer group, model or provider, so one team's experiment cannot use up another team's budget.
Two other policies reduce spend without changing applications: semantic caching returns stored answers to near-duplicate prompts, and routing can send work to a cheaper model where that is acceptable. Usage records carry token counts and cost per consumer, so AI spend can be charged back to the teams that create it.
Agents and MCP tools
AI agents do not only send prompts; they call tools. Kong AI Gateway can expose and govern tools from external Model Context Protocol (MCP) servers, and turn existing APIs into MCP tools that an agent can discover and call. Tool calls can be scoped with OAuth2, so actions that write data can require tighter access than actions that only read. Agent-to-agent traffic can be routed and secured through the same gateway. If your internal APIs already sit behind Kong Gateway, agent traffic can be governed in the same place.
Where it runs, and what that decides
Kong documents AI Gateway as managed through Konnect, Kong's control plane, with data plane nodes running in your environment: self-hosted, in a cloud or on Kubernetes. Kong also documents on-premises configuration. Konnect is offered in several geographic regions, including one labelled Middle East.
| Question | Why it matters |
|---|---|
| Where does the data plane run? | It reads every prompt and response in order to apply policy. Running it in-Kingdom keeps that traffic on infrastructure you control until it is sent to the model. |
| Where does the model run? | A hosted model receives the prompt after redaction. The gateway decides what leaves; it does not change where the provider processes it. |
| Where are logs and usage records kept? | Prompt logging is a policy choice. Decide what is logged, in which form, and for how long, by data class. |
| Which control plane fits your classification? | Configuration and usage metadata go to the control plane. Settle this with your security team before design. |
Kong AI Gateway pricing and licensing
Kong labels several AI policies "AI License Required" and states that they are only available as part of its AI Gateway Enterprise offering. These include AI Rate Limiting Advanced, AI Sanitizer, AI Semantic Cache, AI Proxy Advanced and AI Semantic Response Guard. Licensing therefore depends on the policies and deployment you choose.
Kong licensing is quoted at the vendor's terms. Interkey's quote shows the implementation and any operation scope separately from the licence, so you can see what each part costs.
How Interkey delivers an AI gateway
-
Inventory the AI use.
Which teams call which models, with which data, and through which keys.
-
Write the policy.
Approved models per data class, budgets per team, prompt rules, redaction rules and logging rules, agreed with security and data protection.
-
Deploy in the right shape.
Data plane in your environment, control plane chosen to fit your classification, built as repeatable configuration.
-
Move teams onto the gateway.
One team at a time, retiring direct provider keys as each one moves.
-
Operate.
Policy updates, upgrades, cost reports and redaction tests, by your team or under an Interkey managed-services scope.
Questions buyers ask
Is Kong AI Gateway a separate product from Kong Gateway?
Kong documents AI Gateway as its connectivity and governance layer for LLM, MCP and agent traffic, applied through AI-specific policies with their own licensing: the AI policies on this page are part of Kong's AI Gateway Enterprise offering. If your APIs already run on Kong Gateway, the design shows how these policies fit your existing deployment.
Can prompts stay inside Saudi Arabia?
The data plane, which carries the prompts, can run on infrastructure you operate in-Kingdom. Whether a prompt then leaves depends on where the model you call is hosted. The gateway lets you redact data before it leaves and block requests that should not leave at all.
Does the sanitizer understand Arabic?
Arabic is not among the language images Kong lists. Saudi formats and Arabic text are handled with custom pattern rules, which we design and test with your data protection team.
Can we start with one use case?
Yes. A common start is one internal assistant, with token limits, redaction and logging switched on, before other teams move onto the gateway.
Who operates the gateway after go-live?
Your team, with the configuration repository and runbooks handed over, or Interkey under a written managed-services scope.
How is it priced?
Per programme. Kong licensing is quoted at Kong's terms, and Interkey's implementation and operation are quoted beside it.
Next
Read next
Next step
Discuss an AI gateway
Tell us which teams use LLMs today, which models they call, and which data classes must not leave the Kingdom. That is enough to propose a gateway design and a first use case.
Or directly
+966-11-2180999 [email protected]Tawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia