# Latency by model, measured at the gateway (markdown mirror)

Source: https://aiapi-pro.com/blog/model-latency-measured-2026

30 days of gateway traffic, 63 models with >=25 latency samples. Latency is end-to-end (queue + prefill + generation), not TTFT.

## Tiers (p50)

| Class | p50 |
|---|---|
| OCR / translation | 0.3-1.0s |
| Mid-tier chat | 1.1-2.6s |
| Flagship text | 3-16s |
| Image generation | 10-68s |

## Measured p50 latency (models with median prompt <2K tokens)

| Model | n | median input | p50 | p90 |
|---|---|---|---|---|
| qwen-vl-ocr | 33 | 13 | 324ms | 1,052ms |
| qwen-mt-plus | 28 | 13 | 333ms | 1,156ms |
| qwen3-omni-flash | 32 | 19 | 355ms | 944ms |
| qwen3.5-ocr | 31 | 17 | 370ms | 3,870ms |
| qwen3-coder-flash | 44 | 13 | 403ms | 869ms |
| qwen3-vl-plus | 42 | 78 | 796ms | 1,811ms |
| hy-mt2-plus | 34 | 17 | 975ms | 2,500ms |
| minimax-text-01 | 80 | 559 | 1,115ms | 28,856ms |
| glm-5v-turbo | 42 | 15 | 1,174ms | 5,352ms |
| kimi-k2.7-code-highspeed | 37 | 16 | 1,359ms | 1,952ms |
| kimi-k2.8-preview | 289 | 89 | 1,624ms | 4,788ms |
| minimax-m2.5 | 317 | 44 | 1,860ms | 3,090ms |
| minimax-m2.7 | 370 | 46 | 1,911ms | 7,268ms |
| glm-5.3 | 146 | 146 | 2,156ms | 58,450ms |
| kimi-k2.6 | 47 | 48 | 2,623ms | 11,849ms |
| kimi-k2.5 | 61 | 56 | 2,963ms | 42,784ms |
| glm-5 | 128 | 17 | 1,585ms | 3,769ms |
| step-5-preview | 88 | 75 | 2,665ms | 71,785ms |
| qwen3-max | 166 | 1,774 | 6,132ms | 10,000ms |
| qwen3.7-max | 112 | 490 | 16,228ms | 43,910ms |
| doubao-seedream-5.0-pro (image) | 40 | - | 68,115ms | 110,651ms |

## Three observations

1. **Cheap and fast coincide at the small-model end.** Qwen3.5-OCR: 370ms at $0.70/1M. Hunyuan MT2-plus: 975ms at $0.072/1M. Flagships are slower, not faster.
2. **Output length drives the tail.** p90 ranges from 869ms to 71s - that spread is generation, not prefill.
3. **Generation tasks are a different shape.** Image models take 10-68s at p50: queue it, do not request/response it.

Comparability caveat: filtered to models whose median prompt is under 2K tokens, because prompt size correlates with model choice in real traffic.

## How to use this

- Interactive work: sub-2s tier.
- Batch work: latency irrelevant, pick on cost.
- Generation: design async from day one.

## FAQ

**What is a typical latency for a Chinese LLM API?**
From 30 days of gateway traffic: OCR and translation models respond in 324-975ms at p50, mid-tier chat models in 1.1-2.6s, flagship text models in 3-16s depending on size, and image generation takes 10-68s. The median p50 across 63 models was 1.86s.

**Why are the cheapest models often the fastest?**
Small models are cheaper per token and need less compute per request. Qwen3.5-OCR responds in 370ms at $0.70 per 1M tokens, while a flagship 1M-context model can take 16s.

**Does latency include generation time?**
Yes. This is end-to-end time as observed at the gateway: upstream queueing plus generation. It is not time-to-first-token, so models producing long outputs appear slower even when their prefill is fast.

**Do larger prompts make responses slower?**
Not in a simple way in this data, because prompt size correlates with which model a workload chooses. Comparing only models whose median prompt is under 2K tokens removes that confound.