Technologies/Trino/trino.execution.running_queries
TrinoTrinoMetric

trino.execution.running_queries

Queries currently executing
Dimensions:None
Available on:Native (1)PrometheusPrometheus (1)DatadogDatadog (1)
Interface Metrics (3)
Native
Number of currently running queries in the cluster
Dimensions:None
PrometheusPrometheus
Number of queries currently executing
Dimensions:None
DatadogDatadog
Number of queries currently executing
Dimensions:None
Sources
trino.execution.QueryManager.RunningQueriestrino.io
trino.execution.running_queriesdocs.datadoghq.com

Technical Annotations (42)

Configuration Parameters (10)
task.concurrencyrecommended: 4
Controls number of concurrent tasks per worker, set in reported configuration
task.max-drivers-per-taskrecommended: 8
Limits drivers per task, may affect parallelism when threads are blocked
task.http-timeout-threadsrecommended: 15-20
HTTP timeout thread pool size, exhaustion can cause worker communication issues
task.http-response-threadsrecommended: 300
HTTP response thread pool, insufficient threads can block worker communication
query.remote-task.max-error-durationrecommended: 1m
Maximum time to wait before abandoning a remote task, may cause premature failures
retry-policyrecommended: QUERY
Query-level retry policy, may interact with intermittent failures
query-retry-attemptsrecommended: 2
Number of retry attempts, could contribute to retry loops if failures are consistent
exchange.compression-enabledrecommended: true
Enables compression for exchange manager intermediate data
query.low-memory-killer.delayrecommended: 0s
Delay before killing queries due to low memory
query.max-concurrent-queriesrecommended: lower limit under high concurrency
controls query admission to prevent thread pool exhaustion
Error Signatures (7)
No nodes available to run queryexception
io.trino.spi.TrinoException: No nodes available to run queryexception
INTERNAL ERROR - NO_NODES_AVAILABLEerror code
Previously active node is missinglog pattern
Error getting info for tasklog pattern
Server refused connectionlog pattern
Failed communicating with serverlog pattern
CLI Commands (4)
kubectl get pods -wmonitoring
kubectl scale deployment my-trino-cluster-worker --replicas=Nremediation
kubectl exec -it <pod-name> -- cat /etc/trino/config.propertiesdiagnostic
SELECT max(running_queries) AS max_concurrency FROM (SELECT count(*) AS running_queries, create_time FROM trino_events.trino_queries WHERE create_time >= now() - interval '1' day AND state = 'RUNNING' GROUP BY create_time) t;diagnostic
Technical References (21)
coordinatorcomponentworker parallelismconceptquery splitsconceptWAITINGconceptTIMED_WAITINGconceptMetabase v0.51.12componentStarburst Metabase driver v5.0componentTrino v457componenttrino_events.trino_queriescomponentresource groupconceptBinPackingNodeAllocatorServicecomponentEventDrivenFaultTolerantQuerySchedulercomponentexchange managercomponentresource groupsconceptadmission controlconceptTaskInfoFetchercomponentRequestErrorTrackercomponentJFRcomponentJprofilercomponentfault-tolerant executionconceptbatch operationsconcept
Related Insights (11)
Coordinator capacity insufficient for concurrent query loadwarning
▸
Worker parallelism drops to zero with threads stuck in WAITING statecritical
▸
Cluster intermittently enters idle state after initial query executioncritical
▸
High query concurrency causes resource contentionwarning
▸
Fault-tolerant query fails with NO_NODES_AVAILABLE on worker pod terminationcritical
▸
High query concurrency exhausts worker thread pool resourceswarning
▸
Coordinator CPU exhausted polling dead tasks from terminated queriescritical
▸
Query hangs with blocked threads when scanning large tablescritical
▸
Task concurrency misconfiguration causes performance degradationwarning
▸
High volume of short queries on fault-tolerant cluster causes latencywarning
▸
High query concurrency indicates capacity constraintswarning
▸