The short answer: OpenTelemetry traces an LLM call as a client span named after the operation and the model, for example chat gpt-4, with two required attributes, gen_ai.operation.name and gen_ai.provider.name, and recommended attributes for the model, token counts and finish reasons. Agents get their own spans for creating an agent, invoking it and planning. Prompts and responses are not recorded by default: the conventions treat them as sensitive and recommend storing them outside the trace in production. All of these GenAI conventions still carry Development status, so pin the version you implement against.
Where the conventions live and what status they have
The generative AI conventions are maintained in OpenTelemetry's semantic-conventions-genai repository, under docs/gen-ai. The index page lists separate documents for client inference, embeddings, tool execution, retrieval, remote agent invocation, events, exceptions, metrics and agent spans, plus provider-specific pages for Anthropic, Azure AI Inference, AWS Bedrock and OpenAI, and pages for the Model Context Protocol and Agent2Agent.
The index and each GenAI document cited in this guide are marked Development. In practice that means attribute names and span shapes can still change between versions. Write down the version your instrumentation follows, and expect to migrate.
The inference span
A call to a model that generates a response or requests a tool call is the gen_ai.client.inference span. The rules that matter most:
- Span name:
{gen_ai.operation.name} {gen_ai.request.model}, for examplechat claude-sonnetorgenerate_content gemini-pro. - Span kind:
CLIENT, orINTERNALwhen the model runs in the same process. - Duration: the logical operation, from the request to the full response, including automatic retries.
- Errors: set
error.typewhen the operation ends in an error, matching the provider's or client library's error code where there is one.
| Attribute | Requirement | What it holds |
|---|---|---|
gen_ai.operation.name |
Required | The operation, such as chat, generate_content or text_completion |
gen_ai.provider.name |
Required | The provider, using a well-known value such as openai, anthropic, aws.bedrock or gcp.vertex_ai when one applies |
gen_ai.request.model |
Conditionally required, if available | The exact model name requested |
gen_ai.conversation.id |
Conditionally required, when available | The session or thread identifier, to correlate turns |
gen_ai.response.model |
Recommended | The model that actually answered |
gen_ai.usage.input_tokens |
Recommended | Input tokens, including cached tokens and all modalities |
gen_ai.usage.output_tokens |
Recommended | Output tokens, including reasoning tokens and all modalities |
gen_ai.response.finish_reasons |
Recommended | Why the model stopped, per generated choice |
gen_ai.request.temperature, gen_ai.request.max_tokens |
Recommended | The sampling settings sent |
Two details in the token attributes save arguments later. Input tokens include cached tokens, with gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_write.input_tokens as breakdowns. Output tokens include reasoning tokens, with gen_ai.usage.reasoning.output_tokens as the breakdown. If your provider prices cached or reasoning tokens differently, compute cost from the breakdowns rather than from the totals alone.
Prompts and responses: three recording patterns
The conventions say instrumentations should not capture instructions, inputs or outputs by default, because they are sensitive and often large, and should offer an opt-in. They describe three patterns:
- Do not record them. This is the default.
- Record them on the span, in
gen_ai.system_instructions,gen_ai.input.messagesandgen_ai.output.messages. Suited to pre-production, or where volume is manageable and the telemetry store meets your privacy obligations. - Store them externally and record a reference on the span, using the
_refattributes such asgen_ai.input.messages_ref. This is the pattern recommended for production when volume is a concern or sensitive data must be handled securely, because the external store can have its own access controls.
The third pattern is usually the right one for customer data: engineers can see that a call happened, how long it took and what it cost, while the content sits behind a separate permission.
Agents, tools and plans
The agent spans document defines spans for the parts of an agent run:
- Create agent: operation
create_agent, span namecreate_agent {gen_ai.agent.name}. - Invoke agent: span name
{gen_ai.operation.name} {gen_ai.agent.name}, or just the operation name when the agent has no name. The span is for user-facing entry points; an internal runner that a framework starts under the hood should not get its own invoke span. - Plan: operation
plan, span nameplan {gen_ai.agent.name}, for the planning or task decomposition phase. - Execute tool: operation
execute_tool, defined in its own document.
The result is a trace that reads like the run: one invoke span, with planning, inference and tool spans nested inside it. That is the structure you want when you ask why an agent took an action. Our checklist for LLM observability metrics in production covers which of these signals to put on a dashboard.
The metrics
The conventions also define metrics, so counts and latencies do not have to be derived from spans:
| Metric | What it measures |
|---|---|
gen_ai.client.inference.duration |
Client-side duration of inference calls |
gen_ai.client.inference.time_to_first_chunk |
Time to the first streamed chunk |
gen_ai.client.inference.time_per_output_chunk |
Time between streamed chunks |
gen_ai.client.inference.usage.input_tokens and .output_tokens |
Token usage counters |
gen_ai.server.time_to_first_token, gen_ai.server.time_per_output_token |
The same latencies measured on a model server you run |
gen_ai.invoke_agent.duration, .inference_calls, .tool_calls |
Agent run duration and how many model and tool calls it made |
When counting operations or tokens from spans, the inference document asks consumers to group by service.name, because one client request can involve nested inference operations recorded by the provider.
A worked example: one traced support summary
The example is fictional. A support tool calls a hosted model to summarize a ticket thread, with prompts stored in an internal bucket. Its inference span carries:
span name: chat claude-sonnet
span kind: CLIENT
gen_ai.operation.name: chat
gen_ai.provider.name: anthropic
gen_ai.request.model: claude-sonnet
gen_ai.conversation.id: ticket-48213
gen_ai.request.max_tokens: 600
gen_ai.usage.input_tokens: 3120
gen_ai.usage.cache_read.input_tokens: 2400
gen_ai.usage.output_tokens: 412
gen_ai.response.finish_reasons: ["stop"]
gen_ai.input.messages_ref: s3://support-llm-content/48213/in.json
gen_ai.output.messages_ref: s3://support-llm-content/48213/out.json
From this one span an engineer can see that 2,400 of the 3,120 input tokens were read from the cache, that the answer finished normally rather than hitting the 600 token limit, and where the content is if a reviewer with access needs it. Nobody without access to the bucket sees the customer's text.
Getting this set up on an existing pipeline
AI Observability Setup is a scoped project that instruments an existing AI pipeline with structured logs, drift detection and alerts, so your team can see where failures or output changes happen. The signals and alert destinations are agreed during scoping. The published price is $12,500 for 5 to 7 days, with final scope and timing confirmed before work begins. To keep alerts useful once they exist, see our guide to AI monitoring alerts and signal to noise.
Book a 15-minute call about LLM tracing · Request an estimate by email
Frequently asked questions about OpenTelemetry for LLMs
Are the OpenTelemetry GenAI semantic conventions stable?
No. The GenAI documents carry Development status, so names and structures can change. Pin a version and plan for migrations.
Should I record prompts in traces?
Not by default. The conventions recommend leaving content out unless you opt in, recording it on spans only where volume and privacy allow, and storing it externally with a reference on the span in production.
What is the span name for an LLM call?
{gen_ai.operation.name} {gen_ai.request.model}, such as chat gpt-4. Provider or framework conventions may define a different format, within OpenTelemetry's general span naming rules.
Sources
Span names, attributes, requirement levels, recording patterns and metric names come from the OpenTelemetry GenAI semantic conventions documents in the semantic-conventions-genai repository: the index, client inference, model spans, agent spans and metrics pages. The traced support summary and its values are a fictional example. OpenTelemetry GenAI semantic conventions (GitHub) · Client inference conventions · Agent span conventions
sinc-LLM