Schema.ai#40696: Full-block eligibility creates prefix-cache cliffs
A reusable prefix shorter than one effective block can still produce zero hits
Reported environment
Qwen3.5 hybrid/Mamba on A100 and H20 with a reported 528-token effective block.
The nuance
Full-block matching can create sharp eligibility boundaries; zero hits need not mean the counter is broken.
Expected behavior
Explain eligibility, connect cached/computed tokens to hit/query counters, and distinguish proposals from released behavior.
What Schema establishes
Semantics explain the mechanism; model geometry and net impact require version-matched measurement.
Runtime evidence still needed
- Effective block geometry
- Boundary sweep of cached and computed tokens plus QPS and latency
Try the questions (2)
Why are prefix hits zero below a prompt boundaryrun →Why can prefix caching hurt around hybrid block sizerun →Grounding claims
- In vLLM v0.23.0,
vllm:prefix_cache_hitsis incremented fromSchedulerStats.prefix_cache_stats.hitson each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separatevllm:external_prefix_cache_*counters). - The vLLM v0.23.0 metrics design docs state the metric of interest is the prefix cache hit rate (hits per query): the counters are exposed raw so operators compute the rate over an interval of their choosing with PromQL (
rate(hits)/rate(queries)); vLLM's own logging equivalent aggregateshit_rateover the most recent queries. - In vLLM v0.23.0,
vllm:request_prefill_kv_computed_tokensis registered as a Prometheus histogram.