Technologies/Trino/trino.jvm.process_cpu_load
TrinoTrinoMetric

trino.jvm.process_cpu_load

Process CPU utilization
Dimensions:None
Available on:Native (1)
Interface Metrics (1)
Native
Recent CPU usage for the Trino JVM process
Dimensions:None
Sources
java.lang.OperatingSystem.ProcessCpuLoadtrino.io

Technical Annotations (41)

Configuration Parameters (11)
query.remote-task.max-error-durationrecommended: 1m
Maximum time to wait before abandoning a remote task, may cause premature failures
retry-policyrecommended: QUERY
Query-level retry policy, may interact with intermittent failures
query-retry-attemptsrecommended: 2
Number of retry attempts, could contribute to retry loops if failures are consistent
-Xmx110Grecommended: 110G
max heap on 125G worker; may need tuning for off-heap memory pressure
-XX:ReservedCodeCacheSizerecommended: 2G
code cache for JIT compiled PageFilter classes; may fill under sustained load
-XX:PerMethodRecompilationCutoffrecommended: 10000
controls how many times a method is recompiled before stopping
-XX:PerBytecodeRecompilationCutoffrecommended: 10000
controls bytecode-level recompilation frequency
task.max-worker-threadsrecommended: lower than CPU count
prevents worker threads from starving communication threads at high CPU utilization
task.concurrency
default is physical CPUs (min 2, max 32); lower for many concurrent queries, higher for few; must be power of two
task.min-drivers
default is task.max-worker-threads * 2; increase if resources underutilized
task.info-update-intervalrecommended: 3s
default; reduce if scheduling suboptimal, increase to reduce coordinator CPU
Error Signatures (11)
io.trino.operator.PageTransportTimeoutExceptionexception
java.util.concurrent.TimeoutException: Total timeout 10000 ms elapsedexception
io.trino.server.IoExceptionSuppressingWriterInterceptor Could not write to output: EofException(null)log pattern
SERVICE_UNAVAILABLEhttp status
io.trino.server.remotetask.SimpleHttpResponseHandler$ServiceUnavailableException: Server returned SERVICE_UNAVAILABLEexception
Memory info update request to http://.*v1/memory has not returnedlog pattern
Node state update request to http://.*v1/info/state has not returnedlog pattern
Error getting info for tasklog pattern
Server refused connectionlog pattern
Failed communicating with serverlog pattern
QUERY_CANCELEDerror code
CLI Commands (3)
top -H -p <pid>diagnostic
jstack -l <pid>diagnostic
EXPLAIN ANALYZE VERBOSEdiagnostic
Technical References (16)
Metabase v0.51.12componentStarburst Metabase driver v5.0componentTrino v457componentHttpPageBufferClientcomponent/v1/task/{task-id}/resultscomponentio.trino.$gen.PageFiltercomponentC2 CompilerThreadcomponentAzul Zulu JDK 17.0.4componenttask.max-worker-threadscomponentcoordinator-worker communicationconcept/v1/memoryfile path/v1/info/statefile path/v1/taskfile pathcoordinatorcomponentTaskInfoFetchercomponentRequestErrorTrackercomponent
Related Insights (11)
Cluster intermittently enters idle state after initial query executioncritical
▸
PageTransportTimeoutException occurs without worker failure or obvious resource exhaustionwarning
▸
Single worker CPU saturation causes cluster-wide query timeoutscritical
▸
C2 JIT compiler continuously recompiles PageFilter classeswarning
▸
Worker thread oversubscription starves coordinator-worker communication causing 503 errorscritical
▸
Coordinator bottleneck from hundreds of concurrent analystswarning
▸
Coordinator CPU exhausted polling dead tasks from terminated queriescritical
▸
Task concurrency misconfiguration causes performance degradationwarning
▸
Underutilized worker resources from low task.min-drivers settinginfo
▸
Suboptimal split scheduling from stale task informationinfo
▸
Queries canceled due to memory or CPU limit exceededwarning
▸