How routing works
Pick a mode
model | Use it for |
|---|---|
auto (default) | Most work. The cheapest model expected to answer well |
auto:cost | High volume, simple work: classification, extraction, short replies |
auto:quality | Work where a wrong answer costs more than a stronger model |
a model id, e.g. anthropic.claude-sonnet-5-5 | One fixed model, e.g. for a whole conversation |
With auto, the gateway reads the request (task, difficulty, length, tools, images) and picks
from the models enabled in your AWS account. GET /models lists the ids you can pin.
Deterministic and explainable
The choice is made by a rules engine, not by another language model.
- The same request, mode and catalog version always choose the same model.
- It takes milliseconds and makes no extra model call, so routing adds no tokens to your Bedrock bill.
- You can audit each decision from the response headers:
| Header | Meaning |
|---|---|
x-acg-model | The model that answered |
x-acg-strategy | routed (gateway chose) or pinned (you named a model) |
x-acg-mode | cost, balanced or quality |
x-acg-catalog-version | Model catalog version used for the choice |
Fallback
If the chosen model is throttled or unavailable, the gateway moves to the next suitable one before any output is sent. Requests Bedrock rejects as invalid are not retried.
Tags
Add a tag to the system prompt or the latest user message. The gateway removes it before the model sees the prompt.
| Tag | Effect |
|---|---|
task:simple | Allows a cheaper model |
task:complex | Chooses a stronger model |
effort:low, effort:medium, effort:high | As task:complex, and the model reasons at that effort |
Write tags exactly, with no space after the colon. A tag in the user message wins over one in the system prompt, and pinned models ignore tags.
If your end users write prompts, they can write tags too, and move requests to more expensive models.
Conversations
The gateway is stateless: send the full messages on every call. Each turn is routed on its
own, so turns can go to different models. Pin a model to keep one.
Full guide: routing.md on GitHub