vllm:prefix_cache_hits

Names: vllm:prefix_cache_hits, vllm:gpu_prefix_cache_hits

Aliases: vllm:gpu_prefix_cache_hits → vllm:prefix_cache_hits

Counts prefix-cache activity in tokens.

Changed in v0.7.3: has_prometheus_metric_type claim set changed

Changed in v0.9.0: has_metric_semantics claim set changed

Changed in v0.9.2: has_name claim set changed

Changed in v0.9.2: alias_of claim set changed

        self.counter_gpu_prefix_cache_hits = self._counter_cls(
            name="vllm:gpu_prefix_cache_hits",
            documentation=(
                "GPU prefix cache hits, in terms of number of cached tokens."
                "DEPRECATED: Use vllm:prefix_cache_hits instead."),
            labelnames=labelnames).labels(*labelvalues)

        self.counter_prefix_cache_hits = self._counter_cls(
            name="vllm:prefix_cache_hits",
            documentation=(
                "Prefix cache hits, in terms of number of cached tokens."),
            labelnames=labelnames).labels(*labelvalues)

# means the number of tokens that were queried from the cache.