Latency SLO Violation on Mixed Endpoints
warningFastAPI applications grouping multiple endpoints into a single latency SLO may violate targets when one slow endpoint drags down the aggregate percentile. The 99th percentile latency objective (e.g., P99 < 250ms) can fail even when most endpoints perform well.
Monitor per-function latency percentiles within SLO groups. Alert when the aggregate P99 latency for an SLO objective exceeds the defined threshold (e.g., 250ms for ObjectiveLatency.Ms250), while investigating which specific functions contribute disproportionately to tail latency.
Use autometrics function-level metrics to identify which endpoints within an SLO group contribute most to tail latency violations. Consider splitting read-heavy query endpoints from write-heavy ingestion endpoints into separate SLO objectives with distinct latency targets. Optimize or offload the slowest functions identified.