One Model Was 41% of Our Users' Spend

Model routing is usually argued in the abstract — "route easy requests to cheap models". This is the same argument with the actual numbers from 30 days of traffic, and an honest accounting of where the idea breaks down.

The setup

For every paid text call we recorded the model, the token counts and the cost. We then recomputed every call at the price of the cheapest text model in the catalog (deepseek-v4-flash: $0.08 per 1M input, $0.17 per 1M output). Same tokens, same requests, different price.

Where the money went

The concentration

146 calls — about 1% of traffic — carried 41% of the bill. That is what a flagship does to a cost distribution: the model is not 36x better at everything, but it is 36x more expensive on the same tokens. The remaining 90%+ of calls ran on models whose price multiple is under 4x.

The routing map

Costs are provider list prices from the live feed; the cheapest-per-task page recomputes the rankings every time prices change.

How to route without rewriting your code

Every model sits behind the same OpenAI-compatible endpoint, so routing is a per-request model string:

# cheap path: classification / extraction
curl https://aiapi-pro.com/v1/chat/completions \
  -H "Authorization: Bearer $NOVAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"deepseek-v4-flash","messages":[...]}'

# escalate only when the task shape says so
curl ... -d '{"model":"kimi-k3","messages":[...]}'

There is also a router MCP tool (pick_model) that takes a task shape and returns the cheapest viable models — see the MCP page, including the hosted endpoint at https://aiapi-pro.com/api/mcp (no install, no key).

What this does not say

• It does not say cheap models are as good. They are not, on hard tasks.
• It does not say every request should be routed. Adding a classifier has its own latency and failure cost.
• It does not say the mix above is typical. This is one gateway's traffic, weighted toward agent and retrieval workloads.

It does say one thing clearly: on the same tokens, the spread between the cheapest and the most expensive model here is 36x, and a single model choice moved 41% of the bill.

FAQ

How much can model routing save?

In 30 days of real traffic, routing every text call to the cheapest model would have cut the bill from $188.74 to $45.53, a 75.9% reduction. That is an upper bound, not a recommendation: quality-sensitive calls would suffer.

Which model dominated spend?

Kimi K3: 146 calls (about 1% of traffic) cost $76.65, which is 41% of all text spend. The same tokens on the cheapest model would cost $2.10.

Should I just use the cheapest model for everything?

No. Route by task: cheap models for classification, extraction and short answers; mid-tier for coding and longer reasoning; flagship only when the task genuinely needs it. The 75.9% figure shows the ceiling of the lever, not a target.

How do I route between models without rewriting my code?

Every model is reachable through the same OpenAI-compatible endpoint, so routing is a per-request model id change. NovAI also exposes a router MCP tool (pick_model) and a cheapest-per-task page that rank models by comparable cost.

Analysis window: 30 days ending 2026-10-22. 58 text models with published list prices; media and free-tier calls excluded. "At cheapest" recomputes each call's input/output tokens at deepseek-v4-flash list price. Feed: /china-ai-leaderboard.json.