Skip to content
sinc-LLMBook a 15-minute call

OpenTelemetry for LLM Tracing: The GenAI Semantic Conventions in Practice

How to trace LLM calls and agents with OpenTelemetry's GenAI conventions: span names, required attributes, token counts, prompt capture and the metrics.

On this page

By Mario Alexandre · November 19, 2026 · 5 min read

The short answer: OpenTelemetry traces an LLM call as a client span named after the operation and the model, for example chat gpt-4, with two required attributes, gen_ai.operation.name and gen_ai.provider.name, and recommended attributes for the model, token counts and finish reasons. Agents get their own spans for creating an agent, invoking it and planning. Prompts and responses are not recorded by default: the conventions treat them as sensitive and recommend storing them outside the trace in production. All of these GenAI conventions still carry Development status, so pin the version you implement against.

Where the conventions live and what status they have

The generative AI conventions are maintained in OpenTelemetry's semantic-conventions-genai repository, under docs/gen-ai. The index page lists separate documents for client inference, embeddings, tool execution, retrieval, remote agent invocation, events, exceptions, metrics and agent spans, plus provider-specific pages for Anthropic, Azure AI Inference, AWS Bedrock and OpenAI, and pages for the Model Context Protocol and Agent2Agent.

The index and each GenAI document cited in this guide are marked Development. In practice that means attribute names and span shapes can still change between versions. Write down the version your instrumentation follows, and expect to migrate.

The inference span

A call to a model that generates a response or requests a tool call is the gen_ai.client.inference span. The rules that matter most:

  • Span name: {gen_ai.operation.name} {gen_ai.request.model}, for example chat claude-sonnet or generate_content gemini-pro.
  • Span kind: CLIENT, or INTERNAL when the model runs in the same process.
  • Duration: the logical operation, from the request to the full response, including automatic retries.
  • Errors: set error.type when the operation ends in an error, matching the provider's or client library's error code where there is one.
Attribute Requirement What it holds
gen_ai.operation.name Required The operation, such as chat, generate_content or text_completion
gen_ai.provider.name Required The provider, using a well-known value such as openai, anthropic, aws.bedrock or gcp.vertex_ai when one applies
gen_ai.request.model Conditionally required, if available The exact model name requested
gen_ai.conversation.id Conditionally required, when available The session or thread identifier, to correlate turns
gen_ai.response.model Recommended The model that actually answered
gen_ai.usage.input_tokens Recommended Input tokens, including cached tokens and all modalities
gen_ai.usage.output_tokens Recommended Output tokens, including reasoning tokens and all modalities
gen_ai.response.finish_reasons Recommended Why the model stopped, per generated choice
gen_ai.request.temperature, gen_ai.request.max_tokens Recommended The sampling settings sent

Two details in the token attributes save arguments later. Input tokens include cached tokens, with gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_write.input_tokens as breakdowns. Output tokens include reasoning tokens, with gen_ai.usage.reasoning.output_tokens as the breakdown. If your provider prices cached or reasoning tokens differently, compute cost from the breakdowns rather than from the totals alone.

Prompts and responses: three recording patterns

The conventions say instrumentations should not capture instructions, inputs or outputs by default, because they are sensitive and often large, and should offer an opt-in. They describe three patterns:

  1. Do not record them. This is the default.
  2. Record them on the span, in gen_ai.system_instructions, gen_ai.input.messages and gen_ai.output.messages. Suited to pre-production, or where volume is manageable and the telemetry store meets your privacy obligations.
  3. Store them externally and record a reference on the span, using the _ref attributes such as gen_ai.input.messages_ref. This is the pattern recommended for production when volume is a concern or sensitive data must be handled securely, because the external store can have its own access controls.

The third pattern is usually the right one for customer data: engineers can see that a call happened, how long it took and what it cost, while the content sits behind a separate permission.

Agents, tools and plans

The agent spans document defines spans for the parts of an agent run:

  • Create agent: operation create_agent, span name create_agent {gen_ai.agent.name}.
  • Invoke agent: span name {gen_ai.operation.name} {gen_ai.agent.name}, or just the operation name when the agent has no name. The span is for user-facing entry points; an internal runner that a framework starts under the hood should not get its own invoke span.
  • Plan: operation plan, span name plan {gen_ai.agent.name}, for the planning or task decomposition phase.
  • Execute tool: operation execute_tool, defined in its own document.

The result is a trace that reads like the run: one invoke span, with planning, inference and tool spans nested inside it. That is the structure you want when you ask why an agent took an action. Our checklist for LLM observability metrics in production covers which of these signals to put on a dashboard.

The metrics

The conventions also define metrics, so counts and latencies do not have to be derived from spans:

Metric What it measures
gen_ai.client.inference.duration Client-side duration of inference calls
gen_ai.client.inference.time_to_first_chunk Time to the first streamed chunk
gen_ai.client.inference.time_per_output_chunk Time between streamed chunks
gen_ai.client.inference.usage.input_tokens and .output_tokens Token usage counters
gen_ai.server.time_to_first_token, gen_ai.server.time_per_output_token The same latencies measured on a model server you run
gen_ai.invoke_agent.duration, .inference_calls, .tool_calls Agent run duration and how many model and tool calls it made

When counting operations or tokens from spans, the inference document asks consumers to group by service.name, because one client request can involve nested inference operations recorded by the provider.

A worked example: one traced support summary

The example is fictional. A support tool calls a hosted model to summarize a ticket thread, with prompts stored in an internal bucket. Its inference span carries:

span name:   chat claude-sonnet
span kind:   CLIENT
gen_ai.operation.name:            chat
gen_ai.provider.name:             anthropic
gen_ai.request.model:             claude-sonnet
gen_ai.conversation.id:           ticket-48213
gen_ai.request.max_tokens:        600
gen_ai.usage.input_tokens:        3120
gen_ai.usage.cache_read.input_tokens: 2400
gen_ai.usage.output_tokens:       412
gen_ai.response.finish_reasons:   ["stop"]
gen_ai.input.messages_ref:        s3://support-llm-content/48213/in.json
gen_ai.output.messages_ref:       s3://support-llm-content/48213/out.json

From this one span an engineer can see that 2,400 of the 3,120 input tokens were read from the cache, that the answer finished normally rather than hitting the 600 token limit, and where the content is if a reviewer with access needs it. Nobody without access to the bucket sees the customer's text.

Getting this set up on an existing pipeline

AI Observability Setup is a scoped project that instruments an existing AI pipeline with structured logs, drift detection and alerts, so your team can see where failures or output changes happen. The signals and alert destinations are agreed during scoping. The published price is $12,500 for 5 to 7 days, with final scope and timing confirmed before work begins. To keep alerts useful once they exist, see our guide to AI monitoring alerts and signal to noise.

Book a 15-minute call about LLM tracing · Request an estimate by email

Frequently asked questions about OpenTelemetry for LLMs

Are the OpenTelemetry GenAI semantic conventions stable?

No. The GenAI documents carry Development status, so names and structures can change. Pin a version and plan for migrations.

Should I record prompts in traces?

Not by default. The conventions recommend leaving content out unless you opt in, recording it on spans only where volume and privacy allow, and storing it externally with a reference on the span in production.

What is the span name for an LLM call?

{gen_ai.operation.name} {gen_ai.request.model}, such as chat gpt-4. Provider or framework conventions may define a different format, within OpenTelemetry's general span naming rules.

Sources

Span names, attributes, requirement levels, recording patterns and metric names come from the OpenTelemetry GenAI semantic conventions documents in the semantic-conventions-genai repository: the index, client inference, model spans, agent spans and metrics pages. The traced support summary and its values are a fictional example. OpenTelemetry GenAI semantic conventions (GitHub) · Client inference conventions · Agent span conventions