Every request goes to the right Bedrock model.
AI Cloud Gateway is an OpenAI-compatible router for Amazon Bedrock. Each call goes to the lowest-cost model that can answer it well. Inference runs in your own AWS account, at your Bedrock prices.
extra model calls. A rules engine decides in milliseconds.
endpoint across Bedrock's leading models from Amazon, Anthropic, Meta and OpenAI
markup on tokens. AWS bills Bedrock to you directly.
US regions for inference in your account
Your app calls one endpoint. Bedrock runs in your account.
The gateway picks the model and makes the call through a role you create. AWS bills the Bedrock tokens to you at your own prices. We bill only for routed requests.
- STEP 1
Subscribe on AWS Marketplace
Pick Starter, Team or Business. Billing runs through your AWS invoice.
~2 min - STEP 2
Connect your AWS account
Launch our CloudFormation stack. It creates a role that can only invoke Bedrock models and apply your guardrails.
~5 min - STEP 3
Enable models and create a key
Turn on the models you want in the Bedrock console, then create an acg_ API key in the dashboard.
~5 min - STEP 4
Point your SDK at the gateway
Change the base URL and set model="auto". The rest of your code keeps working.
1 line
Most requests don't need your strongest model.
Send everything to one model and every request pays for the hardest one. Routed, each request goes to the cheapest model expected to answer it well, and on Bedrock the price gap between models is more than a hundredfold. Our plan is a flat fee next to that.
of requests go to Claude Haiku 5.5 instead of Claude Sonnet 5.5. Everything routed beyond that is money kept.
The ceiling assumes every request could go to the cheapest model, which real traffic never allows: hard prompts still need strong models, and the router sends them there. Bedrock on-demand list prices, us-east-1, for the visible tokens shown; Claude 5.5 also thinks before answering, which adds output tokens. Bedrock bills you directly; every response names the model that answered.
Rules you can read, results you can repeat.
The router reads the task type, difficulty, length, tools and images. The same request always picks the same model, and every response says which model answered.
autoThe default. Picks the cheapest model expected to answer the request well.
auto:costLeans further toward lower-cost models.
auto:qualityLeans toward stronger models when the task is hard.
anthropic.claude-…Pin one model by its ID when you need to.
task: · effort:Tags in the prompt steer the choice. They are removed before the model sees it.
import os from openai import OpenAI client = OpenAI( base_url="https://api.aicloudgateway.com/v1", api_key=os.environ["ACG_API_KEY"], ) reply = client.chat.completions.create( model="auto", # or "auto:cost", "auto:quality" messages=[{"role": "user", "content": "Hello"}], stream=True, )
curl https://api.aicloudgateway.com/v1/chat/completions \ -H "Authorization: Bearer $ACG_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "auto", "messages": [{"role": "user", "content": "What is a sports car?"}]}'
# every response says what happened x-acg-model: amazon.nova-micro x-acg-strategy: routed x-acg-mode: auto x-acg-catalog-version: bedrock-2026-10-18-r2 x-request-id: req_7f3c…e21 # throttled or not enabled? the request moves to # the next suitable model before the stream opens.
Each request gets one that can do the job.
Before cost comes capability. The router checks what a request needs, rules out the models that can't do it, then picks the cheapest of the rest.
- 1
Can it do this?
A request with tools only goes to models that call tools; one with images only to models that read them; one asking for JSON only to models that return it.
- 2
Does it fit?
The model's context window must hold the input plus the expected answer, so long documents never reach a model that would cut them off.
- 3
Is it allowed and enabled?
Only models enabled in your AWS account, in your region, and allowed by your policy are considered.
- 4
Cheapest that answers well
Of what is left, the cheapest model expected to answer at this difficulty wins. Hard prompts go to stronger models.
Spend limits and guardrails you set.
Monthly budgets
Cap estimated spend, request count, or both, for the whole subscription or for one API key. When a cap is reached, further requests are refused, and refused calls are not billed.
acg_…prod · spend$312.40 of $400Example values. Spend is estimated from Bedrock on-demand list prices.
Your own Bedrock Guardrails
Name a guardrail from your AWS account and the gateway applies it to every request. You keep the policies and filters you already manage in Bedrock.
Opt-in. Works with cross-Region inference.
See every request, model and dollar.
Usage by day and by model, your plan allowance, budgets, keys and the AWS connection, in one place.
AI Cloud GatewayUsage
Oct 7 – Nov 5, 2026| Model | Requests | Input tok | Output tok |
|---|---|---|---|
| amazon.nova-micro | 288,140 | 352.1M | 61.0M |
| anthropic.claude-haiku-5-5 | 141,920 | 198.4M | 39.7M |
| amazon.nova-lite | 96,310 | 201.8M | 34.2M |
| openai.gpt-oss-120b | 58,700 | 121.5M | 27.9M |
| anthropic.claude-sonnet-5-5 | 27,410 | 67.4M | 25.8M |
Example data. Use the sidebar to look around.
Clear about what we keep, and what we don't.
What we store
- Hourly request and token counts per model
- The routing signals behind each decision, with no prompt text
- API keys, as a one-way hash
- Your role ARN and External ID
What we never store
- Prompts or completions
- Images or tool results you send
- AWS credentials. Calls use a short-lived STS session.
Requests pass through the router in AWS us-east-1, in memory only, then go to Bedrock in your account. Inference regions: us-east-1, us-east-2 and us-west-2. Read the data-handling guide →
Pay per routed request. Tokens stay at your AWS price.
Starter
then $0.30 per 1,000
- Full routing, budgets and guardrails
- Email support, next business day
Team
Most teamsthen $0.25 per 1,000
- Everything in Starter
- Faster email support
Business
then $0.22 per 1,000
- Everything in Team
- Priority support
AWS Marketplace
Subscribe with your AWS account and pay on your AWS bill. Bedrock runs in your account in us-east-1, us-east-2 or us-west-2.
View the listingAzure Marketplace
An edition for Azure AI Foundry is planned, with the same routing and controls. Tell us if you want early access.
Ask for early accessShort guides here. Full detail on GitHub.
Each topic has a one-page summary on this site and the complete guide in our public docs repository.
Quickstart→
Subscribe, connect your AWS account, create a key and make your first call.
routing.mdRouting→
Modes, prompt tags, determinism, response headers and fallback.
examples.mdExamples→
Python, JavaScript, streaming, tools, images and agents.
controls.mdBudgets and guardrails→
Spend and request caps, and your own Bedrock Guardrail.
aws-account.mdYour AWS account→
What the CloudFormation role can do, enabling models, disconnecting.
api-reference.mdAPI reference→
Request fields, headers, limits, key rotation and error codes.
Before you subscribe
Do I pay for tokens through you?
No. Bedrock runs in your AWS account, and AWS bills the tokens to you at your own prices. AI Cloud Gateway bills only per routed request, against your plan's allowance.
Do I need to change my code?
Usually only the base URL and API key. The API follows the OpenAI chat completions format, with streaming, tool calls, images and JSON output. Embeddings and the Responses API are not supported yet.
Which models can it route to?
Amazon Nova Micro, Lite and Pro; Claude Haiku, Sonnet and Opus 5.5; Llama 3.3 70B; and gpt-oss-120b. You enable each one in your own Bedrock console.
What happens when a model is throttled?
The request moves to the next suitable model before the response stream opens, and the x-acg-model header tells you which model answered.
Do you store my prompts?
No. Requests pass through the router in memory and go to Bedrock in your account. We keep hourly request and token counts per model, not prompt or completion text.
Is there a free trial?
Not at the moment. Starter is $19 a month for 50,000 requests, and you can upgrade at any time.
Route your first request in about 15 minutes.
Subscribe on AWS Marketplace, connect your account, and change one line of code.