LlamaIndex Agent Tool Execution Bottleneck

Monitor tool execution overhead by comparing llama_index.agent.step.duration against llama_index.agent.tool.calls. Alert when tool calls dominate agent step latency (e.g., >70% of step duration) or when tool call frequency is abnormally high (>5 calls per agent step on average). Track P95 of agent.step.duration specifically for tool-heavy steps.

1. Investigate: Identify which tools are contributing most to latency. Check if tools are being called sequentially when they could be parallelized. Review tool call traces to find slow external APIs. 2. Diagnose: Profile individual tool implementations to find inefficiencies (unnecessary network calls, blocking I/O, inefficient algorithms). Check if agent is making redundant tool calls for the same information. Verify tool timeout configurations are reasonable. 3. Remediate: Parallelize independent tool calls using async execution. Implement caching for idempotent tool results (e.g., search results, API lookups with TTL). Optimize slow tools (add indexes, cache external API calls, reduce payload sizes). Set reasonable timeouts for tool execution (e.g., 3-5s) with graceful handling. Consider providing agent with more context upfront to reduce tool dependency. 4. Prevent: Dashboard tool call latency per tool type to identify consistently slow tools. Set alerts on average tool calls per step > threshold. Implement tool execution budgets (max time per step, max calls per step). Monitor tool cache hit rates to verify caching effectiveness.