Observability changing

What it is

An agent run is a tree of decisions: a session, the model calls inside it, the tool calls each model call triggered, and the sub-agents it spawned. Observability is capturing that tree, with the inputs, outputs, token counts, cost, latency and errors at every node, so you can answer “what did the agent do and why” after the fact. It is the raw material for evals, debugging and cost control.

How it works

The emerging common format is the OpenTelemetry semantic conventions for generative AI. They define:

• A root span for the agent session, with child spans for each model invocation and each tool execution, so the trace matches the run’s structure.
• Attribute names under gen_ai.* for the model, the provider, prompt and completion token counts, and the operation type, so traces from different frameworks look the same.
• Metrics for token usage and latency, including streaming.
• Conventions for MCP client and server spans.

In June 2026 the conventions moved out of the core OpenTelemetry repository into a dedicated one so they can iterate faster. Every attribute is still marked Development, which means names can change.

Langfuse, Braintrust, Arize, PostHog and most other LLM observability products either emit or ingest this format. Instrument once against the conventions and you can switch backends.

When it matters

Before the first user. You cannot evaluate, debug or bill for what you did not record, and adding tracing after an incident is too late for that incident.

Watch out for

• Traces contain prompts and tool outputs, which means user data. Redact before export and set retention.
• The conventions are not stable. Pin your instrumentation library and read its changelog.
• Sampling that works for web requests loses the one run you needed. Sample by session, not by span.