LiveKit Agents 1.8.0, released 5 September 2026, moved its spans, metrics and logs onto the OpenTelemetry GenAI semantic conventions, so Langfuse and Datadog read them with the stock OTLP exporter and no custom code. Set two environment variables, call set_tracer_provider(), and the first trace lands on the next call.
That is the good news. The rest of this page is the part that costs people a day: the GenAI conventions describe an LLM call, and the spans that decide whether a phone call feels human — time to first audio, the endpointing decision, the barge-in — are not in that convention at all. They exist, but under vendor-prefixed names, and at least one major backend drops spans carrying no gen_ai.* attribute. Our companion page on what AI agent observability means and what to demand from a vendor covers the three layers a buyer should ask about. This is the implementation page against a named standard, verified at source on 16 September 2026.
How do I get my voice agent’s traces into Datadog or Langfuse without writing a custom exporter?
You do not write an exporter. You point the standard OpenTelemetry Python SDK at the backend and hand the provider to the framework. Day one, in order:
- Upgrade to LiveKit Agents 1.8.2 or later (published 15 September 2026). 1.8.0 is the release that adopted the conventions; 1.8.2 is the one that added the voice-specific spans covered below. If you are on 1.7.1 and your offline test suite hangs, that was a real exporter bug — issue 7029, opened 28 August 2026, closed as completed on 1 September 2026 with the maintainer pointing at PR 7033, which shipped in 1.8.0.
- Install the OTLP HTTP exporter:
pip install opentelemetry-exporter-otlp-proto-http. Not the gRPC one, for reasons in step 3. - Set the endpoint and the auth header. Langfuse receives traces on
/api/public/otel; its docs state it supports OTLP over HTTP with both HTTP/JSON and HTTP/protobuf, and that gRPC is not supported yet. Datadog Agent Observability wantsdd-api-keyplusdd-otlp-source=llmobsinOTEL_EXPORTER_OTLP_TRACES_HEADERS, and documentsOTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf. Grafana Cloud takes a base endpoint of the formhttps://otlp-gateway-prod-us-east-0.grafana.net/otlpand lets the SDK append/v1/traces— and carries a Python note worth reading before you debug a 401: in Python, replaceBasicin the generated connection variables withBasic%20. - Hand the provider to LiveKit. Build a normal
TracerProviderwith aBatchSpanProcessorand anOTLPSpanExporter, then calllivekit.agents.telemetry.set_tracer_provider(provider)in your prewarm or entrypoint. There is no LiveKit-specific exporter to write; the framework attaches its PII processor to whatever provider you pass. - Decide the content question before the first production call, not after. See the PII section below. This is the step people skip.
Finish state for day one: one test call produces a single trace rooted at job_entrypoint, containing agent_session, then a user_turn with an eou_wait child, then an agent_turn with llm_node, tts_node and agent_speaking under it. See that shape in the backend and the plumbing is done; everything after is dashboards. Do it in the same pass as the rest of your pre-launch voice agent testing, because a trace nobody has looked at is not instrumentation.
What changed in LiveKit Agents 1.8.0, and what it broke
LiveKit published the migration as a before-and-after table in its own release notes, and the breaking half is unusually blunt: dashboards and processors reading the events see nothing. The five GenAI content events — gen_ai.system.message, gen_ai.user.message, gen_ai.assistant.message, gen_ai.tool.message and gen_ai.choice — are gone, replaced by JSON part-based attributes: gen_ai.system_instructions, gen_ai.input.messages, gen_ai.output.messages and gen_ai.response.finish_reasons.
Three more changes will silently empty a panel rather than error:
- Realtime inference moved off
agent_turnonto a newrealtime_inferencechild span, so any query reading realtime token usage fromagent_turnnow returns nothing. - That span’s
gen_ai.operation.namechanged fromchattogenerate_content. gen_ai.provider.nameis normalised to the convention registry spelling:api.openai.combecomesopenai,AWS Bedrockbecomesaws.bedrock,googlebecomesgcp.gen_ai. Every group-by on the old strings splits.
One trap is worth calling out because it inflates a number rather than zeroing it: reasoning tokens now go out under both gen_ai.usage.reasoning.output_tokens and gen_ai.usage.reasoning_tokens, and LiveKit says plainly that a backend summing both spellings double-counts. If you are reconciling telemetry against an invoice, that is one of the ways a voice agent bill stops matching minutes times rate.
Which spans and attributes matter for a voice SLO
This is the table the convention does not give you. Column two is what the OpenTelemetry GenAI semantic conventions define as at 16 September 2026; columns three and four are what each framework actually emits, read from source at the released tag.
| The voice question | In the GenAI convention? | LiveKit Agents 1.8.2 | Pipecat |
|---|---|---|---|
| Time to first audio out of the agent | No. gen_ai.response.time_to_first_chunk covers the model stream only |
lk.response.ttfb on tts_node |
metrics.ttfb on the TTS span |
| How long the agent waited before deciding the caller had finished | No | eou_wait span: lk.eou.wait_duration, lk.eou.outcome (committed / user_resumed / dropped), lk.eou.endpointing_delay, lk.eou.rearm_count |
Not a span; infer from turn boundaries |
| Why the agent was cut off, and how much audio had played | No | lk.interrupted, lk.interruption.source (audio_activity / user_turn / programmatic), lk.playout.position on agent_turn |
turn.was_interrupted on the turn span: a boolean, no cause |
| End-to-end caller-to-agent latency | No | lk.e2e_latency |
turn.user_bot_latency_seconds |
| Which provider actually served the request after a failover | Partly: gen_ai.request.model versus gen_ai.response.model |
lk.fallback.label, lk.fallback.index on the attempt span |
Provider on the service span |
| Token usage, including audio tokens | Yes: gen_ai.usage.audio.input_tokens, .audio.output_tokens, .cache_read.input_tokens |
Emitted, plus non-registry aliases such as gen_ai.usage.input_audio_tokens for Langfuse |
gen_ai.usage.input_tokens, .output_tokens on the LLM span |
| A tool call that stalled the conversation | Yes: execute_tool span with gen_ai.tool.* |
function_tool span carrying gen_ai.tool.name, .call.id, .call.arguments, .call.result |
llm_tool_call and llm_tool_result on the realtime path |
| Why the first hello took four seconds | No | job_entrypoint back-dated to the availability request, with lk.job.dispatch_latency and its four components (accept, assignment, launch, entrypoint) |
Not emitted |
Read the first four rows together and the shape of the problem is obvious: every attribute that determines whether a call sounds human is vendor-prefixed, and every attribute the convention standardises is about the model. The convention enumerates eighteen well-known values for gen_ai.operation.name — chat, generate_content, embeddings, execute_tool, invoke_agent, plan, the memory operations and so on. None is speech-to-text, text-to-speech, turn detection or endpointing, and speech appears only as a value of gen_ai.output.type. That is a scope boundary, not a criticism; knowing where it sits stops you waiting for a standard that is not coming. For what the numbers in those spans should look like, read why a voice agent interrupts callers and the sub-second latency budget.
Why half your voice spans never appear in the backend
Here is the rule that explains most of the confused support tickets, and it is worth naming because it is the one thing to check first:
The marker rule: to a GenAI backend, a span with no gen_ai.* attribute is not a GenAI span, however voice-critical it is.
Datadog states it directly on its OpenTelemetry instrumentation page for Agent Observability: Spans without any gen_ai.* attribute are also dropped individually, even in a trace that qualifies. The same page says Agent Observability supports spans that follow the OpenTelemetry 1.37+ semantic conventions for generative AI, and that a span must carry at least one recognised marker — gen_ai.operation.name, gen_ai.provider.name, openinference.span.kind, langfuse.observation.type or the deprecated gen_ai.system — for the trace to qualify at all.
Now apply it. In LiveKit 1.8.2, agent_turn carries gen_ai.agent.name and the session carries gen_ai.workflow.name, so those survive; eou_wait, job_entrypoint and the event-loop blocking span carry only lk.* attributes, and rpc_call carries lk.* plus rpc.method from the OpenTelemetry RPC conventions, which is not a GenAI marker either. In Pipecat, the conversation and turn spans carry conversation.* and turn.* attributes only (conversation.id, conversation.type, turn.number, turn.type, turn.duration_seconds, turn.was_interrupted and turn.user_bot_latency_seconds) — no gen_ai.* attribute at all — while its STT and TTS spans do set gen_ai.provider.name, gen_ai.request.model and gen_ai.operation.name, and the TTS span adds gen_ai.output.type.
The fix is not a custom exporter either. It is an OpenTelemetry Collector with an attribute processor that stamps a marker attribute onto the voice spans you want kept, or a second destination for them. Your agent code does not change. If you have not picked a framework yet, our LiveKit Agents versus Pipecat comparison is the place to start, and this table is a reason to weigh telemetry in that decision rather than treat it as a later problem.
Where PII filtering sits now, and the thing it does not do
In 1.8.0 the stripping moved in-process. LiveKit describes the before-and-after in one line: PII was stripped at LiveKit Cloud’s collector; it is now stripped in-process, and under project redaction content never reaches any exporter, Cloud included. Two controls matter:
set_tracer_provider(..., allow_pii=False), orLIVEKIT_TELEMETRY_ALLOW_PII=0, strips conversational content before every third-party exporter. The source docstring for that argument states the default isTrue— the reasoning given is that a GenAI backend can only render the conversation if it receives it.OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=0omits message payloads, tool definitions and tool arguments and results. LiveKit says content capture is on by default.
That default is worth sitting with, because the convention takes the opposite position. The OpenTelemetry GenAI specification marks gen_ai.input.messages, gen_ai.output.messages and gen_ai.system_instructions as Opt-In and says instrumentations SHOULD NOT capture them by default but SHOULD provide an option to opt in. LiveKit ships that opt-in already flipped. Neither position is wrong, but if nobody on your team chose, the choice was made for you, and every caller utterance is now in a third-party system.
The mechanism has a boundary LiveKit documents in the module itself: this is field-level filtering, not entity-level redaction — a matching attribute is dropped whole, never scanned and masked. Attributes carrying user data are marked with a dot-delimited pii segment (lk.pii.user_transcript, lk.pii.sip.phoneNumber, lk.pii.rpc.payload), and twelve fixed GenAI content attribute names are filtered by an explicit list, because a standard name cannot carry the marker (twelve is our own count of that list in telemetry/pii.py at tag 1.8.2, not a figure LiveKit publishes). A card number spoken into an unmarked custom attribute you added yourself is therefore not detected and not stripped: redaction protects the fields LiveKit knows about, and that is the whole promise. Where the traces are then allowed to land is a data residency question, hop by hop. Teams that cannot let transcripts leave their own network are why Zian supports private model deployment on customer infrastructure; the telemetry decision and the inference decision are the same decision.
The metrics that are silently absent
The dangerous failure in telemetry is not an error. It is a panel that is green because the counter never fired. Four documented cases, each verified at source on 16 September 2026, with the status as at that date:
| What is missing | Where | Status on 16 Sep 2026 |
|---|---|---|
LangChain LLMAdapter never populates usage on the chunks it emits, so llm_node_tps is always absent and there is no token usage to aggregate. Time-to-first-token and TTS TTFB report normally, which is what hides it. |
livekit/agents issue 6948 | Open, raised 22 August 2026, confirmed on main by a contributor on 8 September 2026 |
| In token-streaming mode, TTS usage is lost for text queued as a spoken line rather than generated by the LLM: an interruption clears the accumulator with no flush and no discard metric. Greetings, transfer announcements and goodbyes take exactly that path. | pipecat issue 5611 | Open, raised 3 September 2026 against Pipecat 1.8.1 |
| A per-job Prometheus teardown call was dropped in a 1.3.0 refactor, so multiprocess metric files accumulate on long-lived workers. | livekit/agents issue 7006 | Open, raised 27 August 2026, a contributor reported a focused restoration ready the same day |
Telemetry export intermittently deadlocked offline processes on 1.7.1, with OTEL_SDK_DISABLED=true as the workaround. |
livekit/agents issue 7029 | Closed as completed 1 September 2026; the maintainer pointed to PR 7033, which shipped in 1.8.0 |
The lesson generalises past these four. Before you trust a voice dashboard, prove each panel can go red. Run one call with the tool deliberately failing, one where you interrupt the agent mid-greeting, and one on the fallback provider. A panel that does not move under any of the three is measuring nothing, and otherwise you find that out during an incident. The same applies to dead air during tool calls, where the span exists but nobody alerted on its duration.
Doing this yourself versus handing it over
The first case is genuinely a same-day job and you should not pay anyone for it.
| If this is your situation | Do it yourself? | What it actually takes |
|---|---|---|
| One framework, one backend, content may leave your network | Yes | Two environment variables and one set_tracer_provider() call. No exporter code, no collector. |
| Two or more services in one trace (the agent plus your own tool backend) | Yes, with a collector | An OpenTelemetry Collector you run and pay for, plus context propagation across your own services. Application code still unchanged. |
You need the lk.* voice spans in a GenAI-only backend |
Yes, with a transform | A collector transform that stamps a marker attribute, or accept that those spans are dropped and keep a second destination for them. |
| Transcripts must never leave your infrastructure | Yes, with a trade | allow_pii=False plus a self-hosted backend. You keep every timing and lose the conversation text that makes a trace readable. |
| Ten or more providers, a compliance retention policy, and the convention still at Development status | Consider not | Someone owns the mapping permanently. LiveKit shipped a breaking attribute rename inside one minor version; that will happen again. |
That last row is the real cost, and it is the standing job rather than the setup. The GenAI semantic conventions are still marked Development status and now live in their own repository, open-telemetry/semantic-conventions-genai, split out of the main semantic conventions repository. Development status means the definitions can change, and 1.8.0 is proof that they do. Budget for a person who re-reads the migration table every minor release, or accept that your dashboards will drift out of alignment with your traces. And remember that timings tell you the call was fast; only attribution tells you it worked, which is why Zian ties every script variant through PrecisionPitch AI™ to the outcome it produced rather than to its latency.
FAQ
Do I need a custom exporter to send LiveKit Agents traces to Langfuse?
No. Langfuse receives traces on its own OTLP endpoint at /api/public/otel, and the Langfuse OpenTelemetry documentation states that Langfuse supports OTLP over HTTP with both HTTP/JSON and HTTP/protobuf, and that gRPC is not supported yet. So install the OTLP HTTP exporter rather than the gRPC one, set the endpoint and the basic auth header, and pass the tracer provider to LiveKit. The exporter is the stock OpenTelemetry one.
What broke in LiveKit Agents 1.8.0 if I already had dashboards?
Three things, and all of them empty a panel rather than raising an error. Conversation content is no longer emitted as span events, so anything reading gen_ai.user.message or gen_ai.choice sees nothing. Realtime inference moved onto a new realtime_inference child span, so queries reading realtime token usage from agent_turn return nothing. And gen_ai.provider.name is normalised to the registry spelling, so a group-by on the old values splits. LiveKit publishes a before-and-after migration table in the 1.8.0 release notes.
Does the OpenTelemetry GenAI convention define a span for endpointing or barge-in?
No. As at 16 September 2026 the convention enumerates eighteen well-known values for gen_ai.operation.name, covering chat, content generation, embeddings, retrieval, tool execution, agent and workflow invocation, planning and memory operations. None of them describes speech-to-text, text-to-speech, turn detection or interruption, and searching the span documents for speech, voice, barge-in, endpointing and interrupt returns only speech as a value of gen_ai.output.type. Frameworks emit those signals under their own prefixes instead: LiveKit under lk. and Pipecat under turn. and metrics.
Why do some of my voice spans not show up in Datadog?
Because they carry no attribute the backend recognises as generative AI. Datadog documents, for Agent Observability, that spans without any gen_ai attribute are dropped individually even inside a trace that qualifies. LiveKit endpointing and job dispatch spans carry only lk. attributes, the RPC span adds only rpc.method, and Pipecat conversation and turn spans carry only conversation. and turn. attributes, so all of those meet that description. The remedy is a collector attribute processor that stamps a recognised marker, not a change to your agent.
Does turning on tracing send call transcripts to my observability vendor?
By default in LiveKit Agents, yes. The release notes say content capture is on by default, and the source documentation for the allow_pii argument states that it defaults to true, on the reasoning that a GenAI backend can only render the conversation if it receives it. The OpenTelemetry specification takes the opposite default and marks the message attributes as opt-in. Set LIVEKIT_TELEMETRY_ALLOW_PII to 0, or pass allow_pii as false, to strip content before every third-party exporter. Note that this is field-level filtering, so content in a custom attribute you added yourself is not detected.
Voice agent telemetry is worth doing properly because the alternative is guessing about the two seconds that lost you the call. If you want autonomous outbound where the traces, the guardrail events and the booked outcome all live in one record, Apply For Partnership.