Schema.aivLLM main-2026-07-21 · Version scoped
max_num_seqs
Maximum number of sequences to process in a scheduler iteration.
Settingverified Aug 20
Metric knowledge (1)
| Claim | Semantic layer | Verified |
|---|---|---|
The latest optimization docs snapshot says that decreasing max_num_seqs can reduce concurrent requests in a batch and require less KV cache space when preemptions are frequent. | Interpretation & composition | Aug 20 |