mirror of
https://github.com/SigNoz/signoz.git
synced 2026-11-03 06:50:42 +00:00
a621fb2b49
Some checks failed
build-staging / staging (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description
- Langfuse SDK spans reached the thread with empty formatted messages:
the SDK writes them to `langfuse.observation.input/output`, which the
mapper never moved onto `gen_ai.*.messages`.
- `gen_ai.llm` v5 moves `langfuse.observation.input/output` onto
`gen_ai.input/output.messages` and copies `langfuse.observation.type`
onto `gen_ai.operation.name`, above every existing source. New spans
only.
- Formatter handles the shapes these spans carry: a superjson `{json}`
envelope, a single message object (incl. role-less `{tool_calls}` from
the LangChain handler), a bare list of Responses items (LiteLLM
`langfuse_otel` tool turns), and falls back to the raw output when the
OpenInference bucket is empty. Mixed-format warning dropped when one
side is generic or OpenInference.
#### Additional Information
- Verified with unit tests, the shipped definition through the
in-process collector pipeline.
- Why Langfuse keeps its own keys instead of emitting semconv, checked
in the Langfuse repos:
- Docs ([OpenTelemetry
integration](https://langfuse.com/integrations/native/opentelemetry)):
`langfuse.*` attributes "always take precedence over the generic
OpenTelemetry conventions" and are "recommended for all users that are
manually instrumenting"; semconv is "still evolving", so OTel traces are
mapped into the Langfuse model.
- [Discussion #2509](https://github.com/orgs/langfuse/discussions/2509),
Sep 2024, maintainer: Langfuse features "not yet covered by the
conventions" and the risk of "breaking backwards compatibility once
conventions are in place" are the reasons for an own namespace.
- Ingestion (`OtelIngestionProcessor.ts`, `extractInputAndOutput`): own
keys first, then vendor keys, then semconv `gen_ai.*.messages`, then
events. Semconv is one vendor format among many.
- SDK sources: input and output are any JSON on any observation type
(span, agent, tool, generation, …), looser than `gen_ai.input.messages`.
- Still unmapped: `usage_details`, `cost_details`, `model.parameters`
are JSON objects; the mapper cannot extract fields, so pure Langfuse
spans have no tokens or cost.
Closes https://github.com/SigNoz/nerve-pod/issues/579