AI Gateways for Marketing Teams: What They Do and How to Choose One

This information is correct as of August 2026. Growth Method has made every attempt to ensure the accuracy and reliability of the information provided.

What is an AI gateway?

An AI gateway (you will also see “LLM router” and “LLM proxy”, which in practice describe the same layer) is a service that sits between your tools and the model providers. Your software calls the gateway instead of OpenAI or Anthropic directly, usually by changing one base URL, and the gateway forwards each request to whichever provider you asked for, or whichever its routing rules choose.

The case for:

• Every new model becomes a configuration change rather than an integration project: it appears in the catalogue and you tick a box.
• When a provider has an outage, requests fail over to a backup automatically.
• Finance gets one dashboard and one bill, with spend limits per team or per person.
• Safety nets (PII redaction, prompt injection screening, request logs) are configured once, centrally, instead of rebuilt inside every tool.
• Keys live in one place, so access ends when someone leaves.

The case against:

• Another company sits in the traffic and can see every prompt, so logging defaults and data agreements need checking.
• It is one more hop that can fail, and it adds a little latency (small in practice, as the benchmark later in this article shows).
• The economics have edges: credit fees of around 5%, or a platform subscription, which start to matter at volume.
• In router mode, provider-specific features such as prompt caching can quietly degrade (the caching trap covered below).
• Plenty of production apps run happily with no gateway at all: direct provider SDKs, failover and cost tracking inside the application. A gateway earns its place when the team, the spend or the risk outgrows what that covers.

This is no longer a niche pattern. Gartner has formally named the category AI gateways, and its October 2025 market guide predicts that by 2028, 70% of software engineering teams building multi-model applications will use one, up from 25% in 2025.

The two kinds of gateway

“Gateway” covers two different plumbing arrangements, and the difference decides how much changes when you adopt one.

1. Unified router

A unified router becomes your provider: one key and one bill cover the whole catalogue, models live in one namespace such as anthropic/claude-sonnet-5, and requests are normalised into a common format so the gateway can route and fail over freely. It is how OpenRouter works by default.

Pros:

• One key and one bill cover every model from every provider.
• Routing and failover across providers with no per-provider setup, which makes the route-by-job habit cheap to adopt.

Cons:

• Provider-specific features travel less well: prompt caching can need different settings or quietly stop working, and niche provider tools get flakier through the normalised format.
• Pricing needs re-checking once router fees are added.
• Prompts flow through a second company by design; bringing your own keys changes the billing, not the flow.
• The caching trap covered later in this article bites hardest in router mode.

2. Pass-through proxy

A pass-through proxy leaves your provider relationships alone. Your tools keep speaking each provider’s native API (a request to Claude still looks exactly like an Anthropic request) and the gateway sits in the middle, adding logs, metering, rate limits, retries and guardrails.

Pros:

• Adoption is a one-line change per provider: point the base URL at the gateway.
• Everything provider-specific keeps working untouched, including prompt caching, extended reasoning modes and provider-side tools.
• One dashboard, one place for keys, one place for spend limits.

Cons:

• The gateway sees every prompt, so check the logging defaults (Cloudflare, for instance, logs prompts and responses unless you turn that off) and make sure your data processing agreement covers it.
• You still hold an account and contract per provider; the proxy adds visibility, not consolidation.

Which kind to start with

Start with pass-through if you already hold direct provider accounts and mainly want visibility and controls; start with router mode when speed matters more, since one key and a catalogue is the fastest first step. In practice Cloudflare, Vercel, Portkey and LiteLLM all support pass-through use and offer a unified endpoint too, while OpenRouter is router-first.

The common mature setup combines the two: a pass-through gateway for observability, direct contracts with the major providers, and a router such as OpenRouter kept as one provider among several for the long tail of models. The arrangement even composes: Cloudflare lists OpenRouter as one of its supported providers, so the long-tail router can sit behind the same gateway as everything else. Gateway founders describe the same split. Concentrate’s Zach Moskow calls it the “Router Wars”: OpenRouter gave individual developers their gateway, and enterprises now want theirs, governed and with AI spend tied to business outcomes (conversion, revenue, time saved) rather than token counts.

Why marketing teams need one now

The short answer is model sprawl. a16z’s survey of enterprise CIOs found 37% of enterprises already use five or more models, and Menlo Ventures measured enterprise model API spend at $8.4B by mid-2025, more than doubling in six months. No single provider wins every job, which is exactly what we found when we compared the major AI models for marketing work.

Hyperbolic co-founder and CTO Yuchen Jin put the practical problem well:

“We desperately need a smart model router. 1. We’re seeing a model explosion: GPT-5.6, Grok 4.5, Muse Spark 1.1, GLM-5.2, and Fable 5 all launched within the past month. 2. Even for a single model family like GPT-5.6, there’re 3 (Sol, Terra, Luna) and 5 reasoning-effort levels.”

Yuchen Jin, co-founder and CTO, Hyperbolic

For marketing teams the sprawl has a second dimension: cost. The right habit is routing by job. Use premium proprietary models where prose quality or agent reliability earns the premium, and hosted open-weight models (via Groq, Together AI, Fireworks and similar) for the high-volume work: classification, enrichment, summarising, first drafts of variants. Vercel’s own gateway traffic data shows how far this has gone: in July 2026, open-weight models handled 36% of token volume for 8.6% of spend across its customers.

“Most businesses will not run on one model or one provider. Different workloads require different considerations: a low-cost model for routine classification, a stronger model for reasoning, a fast endpoint for user-facing interactions, and approved providers for sensitive data.”

Zach Moskow, founding GTM at Concentrate

Growth Method is built around exactly this split. You can use an expensive OpenAI or Anthropic reasoning model to reason over your live data and create a campaign rooted in real insight, then a cheaper non-reasoning model for planning, and an even cheaper open-weight model for execution. The heavy thinking happens once, at the start; every later stage runs on the cheapest model that does the job well.

Standard gateway features

The standard feature set and what each does in practice, in roughly descending order of importance for a typical marketing team (your weighting will differ). The checklist below shows who covers what, in the same order.

1. Every model from every provider. The core feature. One key covers the proprietary majors, the open-weight hosts, and usually images, video and audio too.
2. Open-weight models on equal terms. Hosted open-weight models sit in the same catalogue as proprietary ones, so routing high-volume jobs to models that cost a few percent as much is a dropdown change.
3. Spend limits. Cloudflare shipped dollar-denominated spend limits in June 2026: budgets per model, provider, team or user, with a choice between blocking requests and quietly rerouting to a cheaper model when the budget runs out. OpenRouter, Vercel, Portkey and LiteLLM all offer per-key or per-team budgets too.
4. Resilience during outages. Every gateway on the list can retry a failed request and fall back to another provider serving the same model, and most let you define explicit fallback chains (if Anthropic errors, try the same model on Bedrock, then fall back to a different model).
5. PII guards. Scans prompts (and sometimes responses) for emails, phone numbers, card numbers, names and addresses, then flags, blocks or redacts them before anything leaves for the model provider.
6. Prompt injection protection. Screens inputs for instructions designed to hijack the model, which matters once agents read web pages, emails or documents. Detection is mostly pattern- and classifier-based, and most gateways offer a flag-only mode so you can measure hits before enforcing.
7. Hallucination guards. Almost no gateway ships one natively. The partner checks that exist grade whether a response is grounded in the context you supplied, and are best treated as flags, not filters: recent research found many published detectors barely beat trivial baselines. The dependable defences remain retrieval with citations, constrained tasks, and evals of your AI workflows.
8. DLP (data loss prevention). The enterprise superset of a PII guard: prompts and responses are scanned against configurable profiles of sensitive data (financial records, credentials, customer identifiers), not just common PII patterns, and matches are flagged or blocked.
9. EU hosting and data residency. Approaches range from EU inference pinning to enterprise-gated EU processing to self-hosting. Whichever gateway you pick, it can only route to EU-resident inference that exists: Azure OpenAI’s EU data zones, OpenAI’s EU projects, Claude in Bedrock’s EU regions including Frankfurt, Mistral’s Paris-hosted platform, and Groq’s Helsinki data centre for open-weight models.
10. Unified billing. You buy credits from the gateway and it pays the providers, so every model lands on one invoice. Fees are charged on credit purchases (typically around 5%) rather than as token markups.
11. BYOK (bring your own key). The opposite arrangement: you keep your own provider accounts and store their keys in the gateway, keeping negotiated rates and data agreements while the gateway keeps the logs and controls. BYOK traffic can carry its own fees, and can bypass gateway budget limits unless explicitly included.
12. Per-customer BYOK. The same idea one level down. An AI tool you buy (or build) lets each customer plug in their own provider key, so their usage runs on their contract, data agreement and region while the vendor’s gateway still handles the logs, guardrails and limits. Enterprise buyers increasingly ask for it, and it only works if the gateway can hold several keys per provider and pick one per request: Cloudflare stores multiple keys per provider and selects one with a request header, Vercel accepts request-scoped credentials but retries on its own keys if yours fail, and OpenRouter holds keys per workspace and can restrict each to specific API keys. Note what it does not change: the customer’s prompts still pass through the vendor’s gateway, so the subprocessor list stays the same.
13. Dynamic routing. Traffic rules without code changes: send 10% of requests to a cheaper model and compare results, give one team a different default, or switch to a fallback when a budget runs out.

The rest of the standard kit

Caching (identical requests answered free, and prompt caching passed through correctly), usage logs your ops team can actually query, zero-data-retention and no-training routing for sensitive work, identity-based access so keys stop working when people leave (Cloudflare’s identity-aware gateway does this through your SSO), and an exit path: an OpenAI-compatible API means leaving the gateway is the same one-line change as adopting it. If you run a SaaS product, look for per-customer cost attribution too: LiteLLM sets budgets down to the end-customer, and Cloudflare’s spend limits split budgets by custom metadata such as a customer ID.

The evaluation checklist

Here is the tick-box view across the three goliaths most teams will shortlist first, against the features above. Portkey and LiteLLM, the two other big names, are covered in the rest of the field below.

Maturity figures are vendor-reported or from deal coverage: OpenRouter’s Stripe acquisition and token volume, and Vercel’s Production Index.

The evaluation checklist, summarised

We don’t rank gateways: the right one depends on your requirements, and the checklist above is the tool for scoring them. What follows is a summary of how each of the three profiles against that checklist, in alphabetical order.

Cloudflare AI Gateway

Strongest on controls without engineering. Content guardrails enabled with a dashboard toggle, PII scanning via DLP, spend limits that can downgrade traffic to cheaper models instead of blocking it, caching, and per-user budgets tied to your SSO through the identity-aware gateway. Core features are free, and unified billing adds a 5% fee on purchased credits with no markup on the tokens themselves. Trade-offs: guardrails add around 500ms and disable streaming, there is no self-hosted option, and EU log residency is not there yet, with no published timeline for it. Fits teams that want governance switched on from a dashboard, especially ones already running on Cloudflare.

OpenRouter

Strongest on catalogue breadth with the least setup. The category leader, with 500+ models across 80+ providers, no markup on inference (it earns a roughly 5.5% fee when you buy credits), and since May 2026 a proper guardrails suite: PII redaction, prompt injection defence, budgets and zero-data-retention routing, all configured from the dashboard with no code. Trade-offs: EU residency is gated to enterprise, the credit fee compounds at volume, and it trailed ComputeSDK’s latency table, by a margin too small to matter for marketing workloads. Fits teams whose first requirement is every model behind one key, and teams keeping a router for the long tail alongside direct contracts. Stripe’s $7B+ acquisition of OpenRouter, finalised in August 2026, is worth watching for pricing and roadmap changes.

Vercel AI Gateway

Strongest on speed, data posture and marketer-adjacent surfaces. It led ComputeSDK’s latest benchmark run, charges zero markup on tokens, retains no data by default, and has the clearest EU story of the hosted gateways with regional inference pinning. The AI Gateway plugin for WordPress lets any plugin built on the WordPress AI Client call text, image and video models with one API key, which puts a gateway inside the CMS a large share of marketing sites already run. Trade-offs: no PII guard, no injection protection, no moderation; Vercel’s answer is to build those at the application layer. Fits teams on Vercel or WordPress, teams with EU inference requirements, and teams with engineers to own their own guardrails.

The rest of the field

The big three are not the whole market. Others worth knowing:

• Portkey: the enterprise security pick, with the deepest guardrails of any gateway (60+ checks, native PII redaction, and hallucination checks via Patronus and Qualifire partners). Palo Alto Networks acquired it in May 2026 and made it the AI gateway inside Prisma AIRS; it bills a subscription on top of your model spend, and the gateway itself went fully open source in March 2026. Fits organisations where the security team leads the decision.
• LiteLLM: the self-hosted open-source workhorse (56k GitHub stars, used by Netflix and Stripe), with the most granular controls anywhere: budgets per key, user, team, model and end-customer, 100+ providers, and every guardrail as a config entry. It is an engineering project (you run the proxy, Postgres and Redis), and its 2026 security record, a backdoored PyPI release in March and a privilege-escalation chain disclosed in June (patched before disclosure), demands eyes-open adoption. Fits teams with platform engineers and strict residency requirements.
• Pydantic AI Gateway: from the team behind Pydantic, now part of its Logfire observability platform; cost caps per project, user and key. Second in ComputeSDK’s latest benchmark run.
• LLM Gateway: open source and self-hostable in one Docker image; 200+ models across 40+ providers. Third in ComputeSDK’s latest run.
• Concentrate: no-fee gateway pitched at enterprises wanting governed multi-model routing. Fourth in ComputeSDK’s latest run.
• Helicone: fast Rust gateway paired with strong observability tooling.
• Kong AI Gateway: semantic caching and routing on Kong’s API platform; the natural choice where Kong already runs the API estate.
• Requesty: PII redaction and EU residency, but charges a 5% markup on tokens.
• Bifrost: open-source, single-binary Go gateway from Maxim, built for low overhead.
• Cloud equivalents: Amazon Bedrock’s intelligent prompt routing and Azure API Management’s AI gateway features, worth a look only if you are already deep in that cloud.

Benchmarks

Independent benchmark data is thin, and most published numbers come from vendors measuring themselves against rivals. Pull the latest data at decision time rather than trusting anyone’s editorial (including ours). What exists as of August 2026:

• ComputeSDK’s AI gateway benchmark is the only independent, regularly repeated comparison: a weekly latency and reliability test of six gateways against calling Anthropic directly, measuring cold and warm latency, time to first token, throughput and success rate, with raw results on GitHub and updates posted on X. The mid-August 2026 run scored the direct-to-Anthropic baseline 89.3, then Vercel 89.0, Pydantic AI Gateway 88.5, LLM Gateway 88.4, Concentrate AI 88.1, Cloudflare 87.6 and OpenRouter 87.1. Read it for what it is: speed and reliability only, with a spread so small (2.1 points across the whole field) that latency should rarely drive the decision. The method has fair critics too: one practitioner notes that it is a single model tested once a week from GitHub’s shared runners, without enough controls to separate gateway variance from network noise.
• Vendor-run comparisons exist from Kong (against Portkey and LiteLLM) and Maxim’s Bifrost (against LiteLLM, claiming roughly 40x lower overhead); an independent reproduction of the Bifrost comparison found the direction right but the gap far smaller. LLM Gateway’s own write-up at least states the caveat that applies to all of these: “results are a property of where you measure from, not a global ranking.”
• Vendor self-claims include Helicone’s roughly 8ms median overhead and Vercel’s sub-20ms routing (routing overhead, not end-to-end response time).
• No independent uptime comparison exists across gateways as of August 2026. If reliability is your requirement, ask vendors for SLAs and status-page history; benchmarks will not answer it.

The catch: auto-routing can break your caching

One gateway feature deserves scepticism: automatic model selection, where a classifier picks a model per request. It is having a moment. Cursor launched Cursor Router in July 2026, claiming frontier-quality results at 60% lower cost in production A/B tests, and OpenRouter relaunched its Auto Router in August, choosing models based on what its users collectively spend on similar tasks.

The pushback is worth hearing. Developer Kevin Gray’s reply to the Cursor launch travelled far beyond his follower count:

“Don’t use this. Don’t use openrouter’s router either.

The largest cost is actually input tokens in coding agents. And if you switch providers you lose context caching and your cost balloons.”

Kevin Gray, replying to the Cursor Router launch

He is right about the mechanics. Prompt caches live with a single provider and do not transfer: cached input costs as little as a tenth of the uncached price, and in agent workloads input tokens dominate the bill. A router that hops providers mid-conversation throws those warm caches away. Coinbase reported the same mechanic from the other side: optimising prompt caching raised one internal tool’s cache hit rate from 5% to 60%. The router vendors know it too. OpenRouter added sticky provider routing to keep follow-up requests on warm caches, and Cursor says its savings figures already account for cache misses caused by switching models.

The practical rule for marketing workloads: set deliberate defaults per job (a cheap open-weight model for enrichment, a frontier model for customer-facing prose), and let auto-routing prove itself in a low-stakes corner first. Cost optimisation order: caching first, defaults second, auto-routing last.

Gateways are becoming ambient infrastructure

The clearest signal that this layer matters is how many platforms now ship one. Vercel put a gateway inside WordPress. Cursor built a router into its editor. Coinbase runs its own internally, defaulting engineers to open-weight models through it. And the money agrees: Stripe agreed to pay more than $7B for OpenRouter in August 2026, five times its valuation three months earlier. Even niche developer platforms bundle one: VM platform exe.dev gives every workspace managed model access with no provider keys stored on the machine. And the same pattern is spreading from models to tools: Firecrawl teased an open-source MCP gateway to consolidate every MCP server behind one endpoint (it never shipped, though Docker, Microsoft and AWS all offer one), and LiteLLM and Portkey both include MCP gateways today. One governed door for models, and increasingly one for tool connections too.

The takeaway for marketing leaders: gateways are the AI equivalent of single sign-on. They will be everywhere within two years, and teams that adopt deliberately get the spend controls and safety nets now.

Where to start

A practical sequence for a marketing team this quarter:

1. Pick a hosted gateway. OpenRouter for maximum coverage, Cloudflare for maximum control, Vercel if you live in its ecosystem. All three are free to try and none marks up tokens.
2. Turn on the safety nets in flag mode. PII detection and prompt injection guards, observing rather than blocking for two weeks. You will learn a lot from what gets flagged.
3. Set spend limits per person or per project, sized generously. The goal is a circuit breaker, not rationing.
4. Set defaults by job. Open-weight models for volume work (enrichment, classification, summaries), premium models where quality is the product. Review the split monthly against your actual outputs.
5. Keep your evals outside the gateway. A gateway controls access, cost and safety; it cannot tell you whether the output was any good. That is what evals are for.

None of this is a campaign in itself. It makes every AI-assisted campaign after it cheaper, safer and faster to run, so the team spends its time running more campaigns against your goal, not managing vendor dashboards.

About Growth Method

Growth Method is the agentic marketing platform for B2B teams. Pre-built AI agents plan, launch and analyse campaigns from your live marketing data, with your martech stack connected in minutes. The routing-by-job this article describes is built in: use an expensive OpenAI or Anthropic reasoning model to create campaigns rooted in your real data, a cheaper non-reasoning model to plan them, and an even cheaper open-weight model for execution. Growth Method is completely provider and model agnostic, so your marketing operations are entirely free from vendor lock-in. None of it needs a gateway login, a spend limit or a pricing table from you, because the platform carries that plumbing as part of its agentic design stack. If you would rather run more campaigns than manage model infrastructure, get started here.

Frequently asked questions

What is an AI gateway?

An AI gateway (also called an LLM router or LLM proxy) is a service that sits between your tools and AI model providers. You integrate once, against a single API and key, and the gateway gives you access to hundreds of models from OpenAI, Anthropic, Google and open-weight hosts, plus centralised controls: automatic failover, spend limits, PII redaction, prompt injection protection and usage logs.

What is the difference between an AI gateway and an LLM router?

In practice the terms overlap. A gateway is the broad control layer: one API for many providers, with governance, observability and guardrails. A router is the narrower decision engine that picks which model should handle each request based on cost, quality or task type. Most gateways include routing, and Gartner has settled on “AI gateways” as the category name.

What is the difference between a pass-through proxy and a unified router?

A pass-through proxy keeps your existing provider accounts. Requests still use each provider’s native API, and the gateway sits in the middle adding logs, spend limits and guardrails, so provider-specific features such as prompt caching keep working. A unified router becomes your provider: one key and one bill for every model, with requests normalised to a common format. That buys breadth and failover, but provider-specific features can degrade and pricing needs re-checking. Cloudflare, Vercel, Portkey and LiteLLM support both modes; OpenRouter is router-first.

How should a marketing team choose an AI gateway?

Evaluate against your own requirements rather than a league table. If the priority is the widest model catalogue with dashboard-level guardrails, look at OpenRouter. If it is spend limits and content guardrails on a free plan, look at Cloudflare AI Gateway. If your stack runs on Vercel or WordPress, or EU inference pinning matters, look at Vercel AI Gateway. If a security team leads the decision, look at Portkey. If you have platform engineers and strict residency needs, look at self-hosted LiteLLM. For latency, check the benchmarking providers’ latest published data; the differences between serious gateways are small.

Do AI gateways add cost?

Mostly no. OpenRouter, Cloudflare and Vercel all pass through provider token prices with no markup, making money instead on credit purchase fees (around 5 percent), paid tiers or adjacent products. Portkey charges a platform subscription on top of your model spend, and self-hosting LiteLLM costs engineering time rather than fees. At high volume, small percentage fees are worth comparing against going direct.

Can an AI gateway stop PII reaching model providers?

Yes, within limits. OpenRouter, Cloudflare, Portkey and LiteLLM can all detect emails, phone numbers, card numbers and similar patterns in prompts, then flag, block or redact them before the request reaches the model. Detection is pattern- and classifier-based, so it is a safety net rather than a guarantee. Vercel AI Gateway has no PII guard; you build that at the application layer.

Can an AI gateway detect hallucinations?

Not natively, in almost every case. None of OpenRouter, Cloudflare or Vercel ship a hallucination guard. Portkey comes closest by plugging in partner checks such as Patronus and Qualifire that grade whether an answer is grounded in the provided context. Independent research suggests hallucination detectors remain unreliable, so citations, retrieval and human review are still the real defence.