LangChain insights

LangSmith dashboards track user feedback and online evaluator scores separately. If user scores trend negative while evaluator scores remain stable (or vice versa), evaluation criteria may be misaligned with real user needs.

ClickHouse Disk Space Exhaustion in Self-Hosted Observabilitycritical

Self-hosted LangSmith observability stacks crash when ClickHouse runs out of disk space during async trace insertion, manifesting as NOT_ENOUGH_SPACE errors that prevent trace ingestion.

LLM Time-to-First-Token Latency Spikewarning

High time-to-first-token from LLM providers indicates queuing, rate limiting, or model cold starts, causing user-perceived delays even when total generation time is acceptable.

Time-to-First-Token (TTFT) Spikes Under Loadcritical

TTFT combines scheduling delay and prompt processing time, making it highly sensitive to system load and prompt length. Spikes indicate resource contention (GPU memory, queuing) or unexpectedly large prompts, directly degrading user-perceived responsiveness.

High langchain_request_error or langchain_chain_error rates can suppress latency metrics (fast-failing requests skew averages downward), hiding underlying performance issues that affect successful requests.

langchain_langgraph_recursion_limit_hit fires when graph execution exceeds max depth, often signaling infinite loops, poor graph design, or adversarial inputs. Left unchecked, this consumes resources and inflates costs without useful output.

langchain_agent_intermediate_steps counts reasoning/tool-use iterations. Unbounded growth indicates agents spinning on complex tasks, inefficient tool selection, or poor stopping criteria, driving up latency and cost.

Dynatrace AI Observability dashboards use span queries that consume DPS trace capacity even when no data is shown. Without sampling controls, exploratory dashboards can inflate query costs unexpectedly.

langchain_tool_error spikes often reflect downstream API failures, rate limits, or network issues—not application bugs. Treating tool errors in isolation misses the root cause: external service degradation.

Agent Tool Call Failure Cascadecritical

AI agents making sequential tool calls can experience cascading failures when early tools fail. Tracking tool usage success rates and agent operation errors reveals brittle integration points.

Deep step-level observability instrumentation introduces 12-15% latency overhead in multi-step agent workflows, especially when serializing detailed prompts, outputs, and token traces synchronously on the execution path.

Agents can enter infinite tool-calling loops when tool selection logic fails or tools return results that trigger repeated invocations, consuming tokens and delaying responses.

Tool executions that timeout continue chain processing with missing data, causing downstream errors or poor quality outputs without clear failure signals.

Agent CPU Throttling During High-Volume Trace Collectionwarning

AI agents and observability sidecars consume excessive CPU when processing high trace volumes, leading to throttling that impacts agent decision-making latency and reliability.

Runaway Token Consumption Cost Spikecritical

Recursive chains, agent loops, or unbounded context windows can generate thousands of tokens in seconds, causing unexpected cost explosions (e.g., $12k-$30k bills).

Prompt Cache Metrics Misreportingwarning

LangChain's usage_metadata for Anthropic prompt caching incorrectly aggregates input_tokens (includes cached reads/writes), requiring manual reconstruction. This breaks cost and token analysis in observability dashboards and alerts.

Multi-Turn Session Context Driftwarning

In conversational AI applications, agent performance may degrade across multiple turns as context drifts, memory becomes incoherent, or earlier information is forgotten. Individual traces appear healthy, but the full conversation reveals deteriorating quality.

LangGraph agents executing async tasks or parallel workflows create orphaned spans when OpenTelemetry context doesn't propagate across async boundaries, resulting in disconnected traces that make debugging multi-step agent workflows nearly impossible.

Multiple OpenTelemetry exporters (Jaeger, Last9, console) processing the same LangGraph spans independently cause lag, memory pressure from overloaded buffers, and missing data when one exporter blocks the shared processing pipeline.

Invisible Agent Tool Call Failureswarning

Multi-step agent workflows fail at specific tool invocation stages, but lack of granular tracing makes debugging difficult and prolongs incident resolution time.

Token Usage Forecast Drift from Model Changeswarning

LangSmith's token usage forecasting assumes stable model behavior. Untagged model version changes (e.g., GPT-4 → GPT-4-turbo, Claude updates) can shift token distributions, invalidating forecasts and triggering false cost alerts.

LLM Request Timeout After Inactivitywarning

LangChain LLM requests timeout after periods of inactivity due to connection pool staleness or dropped connections, requiring server restart to resolve.

LLM Rate Limiting Without Backoffwarning

LLM provider rate limits cause request failures that aren't retried with appropriate backoff, leading to cascading failures during usage spikes.

Embedding Latency from Token Encoding Downloadwarning

First embedding requests can timeout waiting for tiktoken encoding files to download from external CDNs (e.g., openaipublic.blob.core.windows.net), causing initial request failures.