Monitor LangGraph Agent Runs in Production

This guide shows you how to monitor LangGraph agent runs in production, covering tracing, metrics, logging, and alerting. You’ll learn to set up structured observability for your agents so you can debug failures, track performance, and understand user behavior.

What You’ll Need

• A LangGraph application deployed (e.g., on a server, container, or serverless function).
• Access to your deployment environment (terminal, cloud console).
• An observability stack: at minimum, a logging system (e.g., ELK, Loki, CloudWatch) and a metrics system (e.g., Prometheus, Datadog). For tracing, consider LangSmith or OpenTelemetry.
• Basic familiarity with Python, LangGraph, and your chosen monitoring tools.

Step-by-Step

1. Enable LangSmith tracing (if using LangSmith). Set the environment variables LANGCHAIN_TRACING_V2=true and LANGCHAIN_API_KEY=<your-key>. This automatically captures traces for every run, including node inputs/outputs, latency, and token usage. For self-hosted options, use Langfuse or Helicone.
2. Add structured logging to your agent. Use Python’s logging module with a JSON formatter. Log key events: run start, each node entry/exit, tool calls, and final response. Include a unique run ID (e.g., from LangSmith or a UUID) in every log line to correlate logs with traces.
3. Instrument with OpenTelemetry (if not using LangSmith). Install the OpenTelemetry SDK and the opentelemetry-instrumentation-langchain package (if available) to automatically create spans for LangGraph nodes. Alternatively, manually create spans in your graph’s nodes using @tracer.start_as_current_span.
4. Export metrics to your monitoring system. Use Prometheus client library to expose metrics like: langgraph_run_total (counter), langgraph_run_duration_seconds (histogram), langgraph_node_duration_seconds, and langgraph_tool_calls_total. Use the Python client to increment counters and observe histograms.
5. Set up dashboards. In Grafana or your cloud monitoring, create dashboards showing: run volume over time, p50/p95/p99 latency, error rate by node, and token usage. Filter by user ID or session ID if applicable.
6. Implement alerting. Create alerts for: error rate exceeding 5% over 5 minutes, p95 latency above a threshold (e.g., 10s), or a spike in retries. Use Prometheus Alertmanager or cloud alerts.
7. Add run-level metadata. In your agent’s code, attach metadata like user ID, session ID, or input type to the run state. This helps in filtering and debugging.
8. Log full inputs and outputs (with caution). For debugging, log the full state at each node, but be mindful of sensitive data. Redact PII or use data masking.
9. Create a run inspection process. For each failed run, have a way to replay it (e.g., using LangSmith’s playground or a custom replay script). Save the run ID and input for analysis.

Common Mistakes

• Not correlating logs with traces. If you don’t include the run ID in logs, you can’t connect a log line to a specific trace.
• Over-logging sensitive data. Logging full prompts or API keys can leak secrets. Always sanitize.
• Ignoring asynchronous runs. If you use ainvoke, ensure your tracing and logging are async-compatible.
• Not setting up alerting. Monitoring without alerts means you only find out about issues when users complain.
• Using default log levels. Debug logs in production can overwhelm your system; set appropriate levels.

Tips

• Use langgraph.checkpoint to persist state; this also helps with debugging and replay.
• For token usage, use LangSmith’s token tracking or manually count tokens from the model response.
• Set up a separate environment for testing monitoring before deploying to production.
• Use structured logging from the start; retrofitting is painful.
• Consider using a service like Langfuse for a full LLM observability platform if you prefer not to use LangSmith.

Troubleshooting

No traces appear in LangSmith. Check that LANGCHAIN_TRACING_V2 is set to true and your API key is valid. Also, ensure you’re using a recent version of langchain and langgraph.

Metrics not showing in Prometheus. Verify that the Prometheus client is initialized and the endpoint is being scraped. Check the port and path.

Logs are missing. Ensure your logging handler is attached to the root logger and that the level is set appropriately. Also, check for exceptions that might prevent logging.

Alerts firing too often. Adjust thresholds based on baseline data. Start with wider thresholds and tighten gradually.

Can’t correlate logs and traces. Double-check that the run ID is being generated at the start of the run and passed to all logging calls.

Frequently Asked Questions

Do I need LangSmith to monitor LangGraph?

No. You can use OpenTelemetry and your own metrics/logging stack. LangSmith is convenient but not required.

Can I monitor LangGraph on serverless platforms?

Yes, but be careful with cold starts and timeouts. Use cloud-native monitoring like CloudWatch or Azure Monitor, and export metrics accordingly.

How do I track token usage?

LangSmith automatically tracks tokens if you use LangChain’s model wrappers. Otherwise, manually count tokens from the response’s usage_metadata field.

What is the best way to log tool calls?

Log the tool name, input, output, and duration. Use a structured format and include the run ID.

How do I handle sensitive data in logs?

Implement a redaction function that masks emails, phone numbers, and API keys before logging. Alternatively, log only hashed values.

Can I monitor multiple agents with one setup?

Yes, use a common tracing and metrics setup. Add an agent_name label to metrics and a field in logs to differentiate.

Related

• Outlook “Message Has Been Changed” Error: How to Fix It
• Cybersecurity Facts and Stats: What the Numbers Mean
• Is Thumbtack Safe? How to Hire With Confidence