feat(aiobservability): read the messages the Langfuse SDK writes (#13331)

#### Description

- Langfuse SDK spans reached the thread with empty formatted messages:
the SDK writes them to `langfuse.observation.input/output`, which the
mapper never moved onto `gen_ai.*.messages`.
- `gen_ai.llm` v5 moves `langfuse.observation.input/output` onto
`gen_ai.input/output.messages` and copies `langfuse.observation.type`
onto `gen_ai.operation.name`, above every existing source. New spans
only.
- Formatter handles the shapes these spans carry: a superjson `{json}`
envelope, a single message object (incl. role-less `{tool_calls}` from
the LangChain handler), a bare list of Responses items (LiteLLM
`langfuse_otel` tool turns), and falls back to the raw output when the
OpenInference bucket is empty. Mixed-format warning dropped when one
side is generic or OpenInference.

  #### Additional Information

- Verified with unit tests, the shipped definition through the
in-process collector pipeline.
- Why Langfuse keeps its own keys instead of emitting semconv, checked
in the Langfuse repos:
- Docs ([OpenTelemetry
integration](https://langfuse.com/integrations/native/opentelemetry)):
`langfuse.*` attributes "always take precedence over the generic
OpenTelemetry conventions" and are "recommended for all users that are
manually instrumenting"; semconv is "still evolving", so OTel traces are
mapped into the Langfuse model.
- [Discussion #2509](https://github.com/orgs/langfuse/discussions/2509),
Sep 2024, maintainer: Langfuse features "not yet covered by the
conventions" and the risk of "breaking backwards compatibility once
conventions are in place" are the reasons for an own namespace.
- Ingestion (`OtelIngestionProcessor.ts`, `extractInputAndOutput`): own
keys first, then vendor keys, then semconv `gen_ai.*.messages`, then
events. Semconv is one vendor format among many.
- SDK sources: input and output are any JSON on any observation type
(span, agent, tool, generation, …), looser than `gen_ai.input.messages`.
- Still unmapped: `usage_details`, `cost_details`, `model.parameters`
are JSON objects; the mapper cannot extract fields, so pure Langfuse
spans have no tokens or cost.


Closes https://github.com/SigNoz/nerve-pod/issues/579

docs/api/openapi.yml

@@ -1689,108 +1689,6 @@ components:
       required:
       - config
       type: object
-    AnnotationtypesAnnotation:
-      discriminator:
-        mapping:
-          deployment: '#/components/schemas/AnnotationtypesAnnotationDeployment'
-        propertyName: kind
-      oneOf:
-      - $ref: '#/components/schemas/AnnotationtypesAnnotationDeployment'
-      type: object
-    AnnotationtypesAnnotationDeployment:
-      properties:
-        kind:
-          $ref: '#/components/schemas/AnnotationtypesAnnotationKind'
-        spec:
-          $ref: '#/components/schemas/AnnotationtypesDeploymentSpec'
-      required:
-      - kind
-      - spec
-      type: object
-    AnnotationtypesAnnotationKind:
-      enum:
-      - deployment
-      type: string
-    AnnotationtypesAnnotations:
-      properties:
-        historyAvailableFrom:
-          format: int64
-          type: integer
-        records:
-          items:
-            $ref: '#/components/schemas/AnnotationtypesAnnotation'
-          nullable: true
-          type: array
-        total:
-          type: integer
-      required:
-      - records
-      - total
-      - historyAvailableFrom
-      type: object
-    AnnotationtypesDeploymentSpec:
-      properties:
-        environment:
-          type: string
-        firstSeen:
-          format: int64
-          type: integer
-        id:
-          type: string
-        logId:
-          type: string
-        resourceAttributes:
-          additionalProperties:
-            type: string
-          nullable: true
-          type: object
-        service:
-          type: string
-        signal:
-          $ref: '#/components/schemas/TelemetrytypesSignal'
-        spanId:
-          type: string
-        traceId:
-          type: string
-        version:
-          type: string
-      required:
-      - id
-      - signal
-      - service
-      - environment
-      - version
-      - firstSeen
-      - traceId
-      - spanId
-      - logId
-      - resourceAttributes
-      type: object
-    AnnotationtypesPostableAnnotations:
-      properties:
-        end:
-          format: int64
-          type: integer
-        filter:
-          $ref: '#/components/schemas/Querybuildertypesv5Filter'
-        kind:
-          $ref: '#/components/schemas/AnnotationtypesAnnotationKind'
-        limit:
-          type: integer
-        signals:
-          items:
-            $ref: '#/components/schemas/TelemetrytypesSignal'
-          nullable: true
-          type: array
-        start:
-          format: int64
-          type: integer
-      required:
-      - kind
-      - start
-      - end
-      - limit
-      type: object
     AuthtypesAttributeMapping:
       properties:
         email:
@@ -13652,73 +13550,6 @@ paths:
       tags:
       - alerts
       x-signoz-stability: alpha
-  /api/v1/annotations:
-    post:
-      deprecated: false
-      description: Returns annotation markers of the requested kind inside the time
-        range. For kind 'deployment', a marker is the first time a (service, environment,
-        version) tuple was seen in the requested signals (traces, logs, or both when
-        omitted). Each record carries the signal that produced it, the first seen
-        time in epoch milliseconds, the trace and span ids or the log id of the first
-        row, and the resource attributes of that row minus the identity keys. Supports
-        an optional resource-attribute filter expression. historyAvailableFrom is
-        the earliest time the result can be trusted, bounded by the signal retention.
-      operationId: ListAnnotations
-      requestBody:
-        content:
-          application/json:
-            schema:
-              $ref: '#/components/schemas/AnnotationtypesPostableAnnotations'
-      responses:
-        "200":
-          content:
-            application/json:
-              schema:
-                properties:
-                  data:
-                    $ref: '#/components/schemas/AnnotationtypesAnnotations'
-                  status:
-                    type: string
-                required:
-                - status
-                - data
-                type: object
-          description: OK
-        "400":
-          content:
-            application/json:
-              schema:
-                $ref: '#/components/schemas/RenderErrorResponse'
-          description: Bad Request
-        "401":
-          content:
-            application/json:
-              schema:
-                $ref: '#/components/schemas/RenderErrorResponse'
-          description: Unauthorized
-        "403":
-          content:
-            application/json:
-              schema:
-                $ref: '#/components/schemas/RenderErrorResponse'
-          description: Forbidden
-        "500":
-          content:
-            application/json:
-              schema:
-                $ref: '#/components/schemas/RenderErrorResponse'
-          description: Internal Server Error
-      security:
-      - api_key:
-        - traces:read
-        - logs:read
-      - tokenizer:
-        - traces:read
-        - logs:read
-      summary: List annotations
-      tags:
-      - annotations
-      x-signoz-stability: alpha
   /api/v1/authz/check:
     post:
       deprecated: false

docs/contributing/go/spanmapper.md

@@ -17,7 +17,7 @@ Condition: `model`, `llm.input_messages` (OpenInference sets `llm.model_name` fr
 | `gen_ai.response.model`, last source for the request model | vLLM, which writes only the response model |
 | `ai.model.id`, `ai.model.provider`, `ai.response.model`, `ai.response.finishReason`, `ai.prompt.messages`, `ai.response.text`, `ai.response.toolCalls`, `ai.usage.promptTokens`, `ai.usage.completionTokens` | Vercel AI SDK v4, Mastra |
 | `ai.usage.inputTokens`, `ai.usage.outputTokens`, `ai.usage.cachedInputTokens`, `ai.usage.reasoningTokens` | Vercel AI SDK v5 |
-| `langfuse.observation.model.name`, `langfuse.session.id` | Langfuse SDK v3 |
+| `langfuse.observation.model.name`, `langfuse.observation.input`, `langfuse.observation.output`, `langfuse.observation.type`, `langfuse.session.id` | Langfuse SDK v3 (Python) and v4 (JS), LiteLLM `langfuse_otel` callback |
 | `session.id` | OpenInference, Langfuse, Claude Code |
 | `gen_ai.client.session_id` | opentelemetry-hooks runner (Cursor, Copilot, Claude Code hooks) |
 | `model`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens` | Claude Code `claude_code.llm_request` spans ([docs](https://code.claude.com/docs/en/monitoring-usage#span-attributes)) |

frontend/package.json

@@ -70,7 +70,7 @@
     "antd": "5.11.0",
     "antd-table-saveas-excel": "2.2.1",
     "antlr4": "4.13.2",
-    "axios": "1.18.0",
+    "axios": "1.21.0",
     "babel-jest": "^29.6.4",
     "chart.js": "3.9.1",
     "chartjs-adapter-date-fns": "^2.0.0",

frontend/pnpm-lock.yaml

@@ -127,8 +127,8 @@ importers:
         specifier: 4.13.2
         version: 4.13.2
       axios:
-        specifier: 1.18.0
-        version: 1.18.0
+        specifier: 1.21.0
+        version: 1.21.0
       babel-jest:
         specifier: ^29.6.4
         version: 29.6.4(@babel/core@7.29.7)
@@ -4593,8 +4593,8 @@ packages:
     resolution: {integrity: sha512-UzGt8zg7Ny8djbYMhxl2zuEevVa7r2gJjYY5Lwr1xM7+XU2nd6CkIWFTVcCIbAP63vSz71NaVyyuSk9lHKcy0A==}
     engines: {node: '>=4'}
 
-  axios@1.18.0:
-    resolution: {integrity: sha512-E32NzpYKp++W7XRe52rHiXV2ehxmh3wbdgO7MHeFM+vqxLBYHzt0ElkiImtOBxtOmyp0yoC8C6uESVV84Y2/hw==}
+  axios@1.21.0:
+    resolution: {integrity: sha512-YCzXMB53Q9PZBsh7yN7wtapit4WepwcHS2fP8QhGsBxSz5YtgYobV7GGlWlAJi+RxfU77GQTQi2DSLUma42EYA==}
 
   babel-jest@29.6.4:
     resolution: {integrity: sha512-meLj23UlSLddj6PC+YTOFRgDAtjnZom8w/ACsrx0gtPtv5cJZk0A5Unk5bV4wixD7XaPCN1fQvpww8czkZURmw==}
@@ -14283,7 +14283,7 @@ snapshots:
 
   axe-core@4.13.0: {}
 
-  axios@1.18.0:
+  axios@1.21.0:
     dependencies:
       follow-redirects: 1.16.0
       form-data: 4.0.6
@@ -20310,7 +20310,7 @@ snapshots:
 
   wait-on@7.2.0:
     dependencies:
-      axios: 1.18.0
+      axios: 1.21.0
       joi: 17.13.7
       lodash: 4.18.1
       minimist: 1.2.8

frontend/src/api/generated/services/annotations/index.ts

@@ -1,105 +0,0 @@
-/**
- * ! Do not edit manually
- * * The file has been auto-generated using Orval for SigNoz
- * * regenerate with 'pnpm generate:api'
- * SigNoz
- */
-import { useMutation } from 'react-query';
-import type {
-	MutationFunction,
-	UseMutationOptions,
-	UseMutationResult,
-} from 'react-query';
-
-import type {
-	AnnotationtypesPostableAnnotationsDTO,
-	ListAnnotations200,
-	RenderErrorResponseDTO,
-} from '../sigNoz.schemas';
-
-import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
-import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
-
-/**
- * Returns annotation markers of the requested kind inside the time range. For kind 'deployment', a marker is the first time a (service, environment, version) tuple was seen in the requested signals (traces, logs, or both when omitted). Each record carries the signal that produced it, the first seen time in epoch milliseconds, the trace and span ids or the log id of the first row, and the resource attributes of that row minus the identity keys. Supports an optional resource-attribute filter expression. historyAvailableFrom is the earliest time the result can be trusted, bounded by the signal retention.
- * @summary List annotations
- */
-export const listAnnotations = (
-	annotationtypesPostableAnnotationsDTO?: BodyType<AnnotationtypesPostableAnnotationsDTO>,
-	signal?: AbortSignal,
-) => {
-	return GeneratedAPIInstance<ListAnnotations200>({
-		url: `/api/v1/annotations`,
-		method: 'POST',
-		headers: { 'Content-Type': 'application/json' },
-		data: annotationtypesPostableAnnotationsDTO,
-		signal,
-	});
-};
-
-export const getListAnnotationsMutationOptions = <
-	TError = ErrorType<RenderErrorResponseDTO>,
-	TContext = unknown,
->(options?: {
-	mutation?: UseMutationOptions<
-		Awaited<ReturnType<typeof listAnnotations>>,
-		TError,
-		{ data?: BodyType<AnnotationtypesPostableAnnotationsDTO> },
-		TContext
-	>;
-}): UseMutationOptions<
-	Awaited<ReturnType<typeof listAnnotations>>,
-	TError,
-	{ data?: BodyType<AnnotationtypesPostableAnnotationsDTO> },
-	TContext
-> => {
-	const mutationKey = ['listAnnotations'];
-	const { mutation: mutationOptions } = options
-		? options.mutation &&
-			'mutationKey' in options.mutation &&
-			options.mutation.mutationKey
-			? options
-			: { ...options, mutation: { ...options.mutation, mutationKey } }
-		: { mutation: { mutationKey } };
-
-	const mutationFn: MutationFunction<
-		Awaited<ReturnType<typeof listAnnotations>>,
-		{ data?: BodyType<AnnotationtypesPostableAnnotationsDTO> }
-	> = (props) => {
-		const { data } = props ?? {};
-
-		return listAnnotations(data);
-	};
-
-	return { mutationFn, ...mutationOptions };
-};
-
-export type ListAnnotationsMutationResult = NonNullable<
-	Awaited<ReturnType<typeof listAnnotations>>
->;
-export type ListAnnotationsMutationBody =
-	| BodyType<AnnotationtypesPostableAnnotationsDTO>
-	| undefined;
-export type ListAnnotationsMutationError = ErrorType<RenderErrorResponseDTO>;
-
-/**
- * @summary List annotations
- */
-export const useListAnnotations = <
-	TError = ErrorType<RenderErrorResponseDTO>,
-	TContext = unknown,
->(options?: {
-	mutation?: UseMutationOptions<
-		Awaited<ReturnType<typeof listAnnotations>>,
-		TError,
-		{ data?: BodyType<AnnotationtypesPostableAnnotationsDTO> },
-		TContext
-	>;
-}): UseMutationResult<
-	Awaited<ReturnType<typeof listAnnotations>>,
-	TError,
-	{ data?: BodyType<AnnotationtypesPostableAnnotationsDTO> },
-	TContext
-> => {
-	return useMutation(getListAnnotationsMutationOptions(options));
-};

frontend/src/api/generated/services/sigNoz.schemas.ts

@@ -3217,127 +3217,6 @@ export interface AlertmanagertypesUpdatableNotificationChannelDTO {
 	config: AlertmanagertypesChannelConfigDTO;
 }
 
-export enum AnnotationtypesAnnotationDeploymentDTOKind {
-	deployment = 'deployment',
-}
-export enum TelemetrytypesSignalDTO {
-	traces = 'traces',
-	logs = 'logs',
-	metrics = 'metrics',
-	'' = '',
-}
-export type AnnotationtypesDeploymentSpecDTOResourceAttributesAnyOf = {
-	[key: string]: string;
-};
-
-/**
- * @nullable
- */
-export type AnnotationtypesDeploymentSpecDTOResourceAttributes = AnnotationtypesDeploymentSpecDTOResourceAttributesAnyOf | null;
-
-export interface AnnotationtypesDeploymentSpecDTO {
-	/**
-	 * @type string
-	 */
-	environment: string;
-	/**
-	 * @type integer
-	 * @format int64
-	 */
-	firstSeen: number;
-	/**
-	 * @type string
-	 */
-	id: string;
-	/**
-	 * @type string
-	 */
-	logId: string;
-	/**
-	 * @type object,null
-	 */
-	resourceAttributes: AnnotationtypesDeploymentSpecDTOResourceAttributes;
-	/**
-	 * @type string
-	 */
-	service: string;
-	signal: TelemetrytypesSignalDTO;
-	/**
-	 * @type string
-	 */
-	spanId: string;
-	/**
-	 * @type string
-	 */
-	traceId: string;
-	/**
-	 * @type string
-	 */
-	version: string;
-}
-
-export interface AnnotationtypesAnnotationDeploymentDTO {
-	/**
-	 * @type string
-	 * @enum deployment
-	 */
-	kind: AnnotationtypesAnnotationDeploymentDTOKind;
-	spec: AnnotationtypesDeploymentSpecDTO;
-}
-
-export type AnnotationtypesAnnotationDTO =
-	AnnotationtypesAnnotationDeploymentDTO;
-
-export enum AnnotationtypesAnnotationKindDTO {
-	deployment = 'deployment',
-}
-export interface AnnotationtypesAnnotationsDTO {
-	/**
-	 * @type integer
-	 * @format int64
-	 */
-	historyAvailableFrom: number;
-	/**
-	 * @type array,null
-	 */
-	records: AnnotationtypesAnnotationDTO[] | null;
-	/**
-	 * @type integer
-	 */
-	total: number;
-}
-
-export interface Querybuildertypesv5FilterDTO {
-	/**
-	 * @type string
-	 * @description A filter in the query filter syntax: comparisons such as =, !=, <, >, IN, LIKE, CONTAINS and EXISTS on field names, combined with AND, OR and NOT, with string values in single quotes, for example key = 'value' AND other_key > 10. Which fields exist depends on the signal or endpoint the filter is used with.
-	 */
-	expression?: string;
-}
-
-export interface AnnotationtypesPostableAnnotationsDTO {
-	/**
-	 * @type integer
-	 * @format int64
-	 */
-	end: number;
-	filter?: Querybuildertypesv5FilterDTO;
-	kind: AnnotationtypesAnnotationKindDTO;
-	/**
-	 * @type integer
-	 */
-	limit: number;
-	/**
-	 * @type array,null
-	 */
-	signals?: TelemetrytypesSignalDTO[] | null;
-	/**
-	 * @type integer
-	 * @format int64
-	 */
-	start: number;
-}
-
 export interface AuthtypesAttributeMappingDTO {
 	/**
 	 * @type string
@@ -5133,6 +5012,14 @@ export type Querybuildertypesv5BucketOptionsDTO =
 	| Querybuildertypesv5BucketOptionsLinearDTO
 	| Querybuildertypesv5BucketOptionsLogDTO;
 
+export interface Querybuildertypesv5FilterDTO {
+	/**
+	 * @type string
+	 * @description A filter in the query filter syntax: comparisons such as =, !=, <, >, IN, LIKE, CONTAINS and EXISTS on field names, combined with AND, OR and NOT, with string values in single quotes, for example key = 'value' AND other_key > 10. Which fields exist depends on the signal or endpoint the filter is used with.
+	 */
+	expression?: string;
+}
+
 /**
  * The argument value as a number, string or boolean, for example the threshold for cutOffMin or the duration for timeShift.
  */
@@ -5201,6 +5088,12 @@ export enum TelemetrytypesFieldDataTypeDTO {
 	number = 'number',
 	'' = '',
 }
+export enum TelemetrytypesSignalDTO {
+	traces = 'traces',
+	logs = 'logs',
+	metrics = 'metrics',
+	'' = '',
+}
 export interface Querybuildertypesv5GroupByKeyDTO {
 	/**
 	 * @type string
@@ -14684,14 +14577,6 @@ export type GetAlerts200 = {
 	status: string;
 };
 
-export type ListAnnotations200 = {
-	data: AnnotationtypesAnnotationsDTO;
-	/**
-	 * @type string
-	 */
-	status: string;
-};
-
 export type AuthzCheck200 = {
 	/**
 	 * @type array

pkg/apiserver/signozapiserver/annotations.go

@@ -1,51 +0,0 @@
-package signozapiserver
-
-import (
-	"net/http"
-
-	"github.com/SigNoz/signoz/pkg/http/handler"
-	"github.com/SigNoz/signoz/pkg/querybuilder"
-	"github.com/SigNoz/signoz/pkg/types/annotationtypes"
-	"github.com/SigNoz/signoz/pkg/types/authtypes"
-	"github.com/SigNoz/signoz/pkg/types/coretypes"
-	"github.com/gorilla/mux"
-)
-
-func (provider *provider) addAnnotationsRoutes(router *mux.Router) error {
-	if err := router.Handle("/api/v1/annotations", handler.New(
-		provider.authzMiddleware.CheckResources(provider.annotationsHandler.ListAnnotations, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
-		handler.OpenAPIDef{
-			ID:                  "ListAnnotations",
-			Tags:                []string{"annotations"},
-			Summary:             "List annotations",
-			Description:         "Returns annotation markers of the requested kind inside the time range. For kind 'deployment', a marker is the first time a (service, environment, version) tuple was seen in the requested signals (traces, logs, or both when omitted). Each record carries the signal that produced it, the first seen time in epoch milliseconds, the trace and span ids or the log id of the first row, and the resource attributes of that row minus the identity keys. Supports an optional resource-attribute filter expression. historyAvailableFrom is the earliest time the result can be trusted, bounded by the signal retention.",
-			Request:             new(annotationtypes.PostableAnnotations),
-			RequestContentType:  "application/json",
-			Response:            new(annotationtypes.Annotations),
-			ResponseContentType: "application/json",
-			SuccessStatusCode:   http.StatusOK,
-			ErrorStatusCodes:    []int{http.StatusBadRequest, http.StatusUnauthorized},
-			Deprecated:          false,
-			SecuritySchemes: newScopedSecuritySchemes([]string{
-				coretypes.ResourceTelemetryResourceTraces.Scope(coretypes.VerbRead),
-				coretypes.ResourceTelemetryResourceLogs.Scope(coretypes.VerbRead),
-			}),
-		}, handler.WithResourceDefs(
-			handler.TelemetryResourceDef{
-				Verb:      coretypes.VerbRead,
-				Category:  coretypes.ActionCategoryDataAccess,
-				Selector:  querybuilder.TelemetrySelector,
-				Resources: querybuilder.BuilderQueryWithFilterExpressionResources(coretypes.ResourceTelemetryResourceTraces, "filter.expression"),
-			},
-			handler.TelemetryResourceDef{
-				Verb:      coretypes.VerbRead,
-				Category:  coretypes.ActionCategoryDataAccess,
-				Selector:  querybuilder.TelemetrySelector,
-				Resources: querybuilder.BuilderQueryWithFilterExpressionResources(coretypes.ResourceTelemetryResourceLogs, "filter.expression"),
-			},
-		))).Methods(http.MethodPost).GetError(); err != nil {
-		return err
-	}
-
-	return nil
-}

pkg/apiserver/signozapiserver/provider.go

@@ -18,7 +18,6 @@ import (
 	"github.com/SigNoz/signoz/pkg/identn"
 	"github.com/SigNoz/signoz/pkg/licensing"
 	"github.com/SigNoz/signoz/pkg/modules/aiobservability"
-	"github.com/SigNoz/signoz/pkg/modules/annotations"
 	"github.com/SigNoz/signoz/pkg/modules/authdomain"
 	"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
 	"github.com/SigNoz/signoz/pkg/modules/dashboard"
@@ -74,7 +73,6 @@ type provider struct {
 	metricsExplorerHandler     metricsexplorer.Handler
 	metricReductionRuleHandler metricreductionrule.Handler
 	infraMonitoringHandler     inframonitoring.Handler
-	annotationsHandler         annotations.Handler
 	gatewayHandler             gateway.Handler
 	gatewayService             gateway.Gateway
 	fieldsHandler              fields.Handler
@@ -121,7 +119,6 @@ func NewFactory(
 	metricsExplorerHandler metricsexplorer.Handler,
 	metricReductionRuleHandler metricreductionrule.Handler,
 	infraMonitoringHandler inframonitoring.Handler,
-	annotationsHandler annotations.Handler,
 	gatewayHandler gateway.Handler,
 	gatewayService gateway.Gateway,
 	fieldsHandler fields.Handler,
@@ -176,7 +173,6 @@ func NewFactory(
 			metricsExplorerHandler,
 			metricReductionRuleHandler,
 			infraMonitoringHandler,
-			annotationsHandler,
 			gatewayHandler,
 			gatewayService,
 			fieldsHandler,
@@ -233,7 +229,6 @@ func newProvider(
 	metricsExplorerHandler metricsexplorer.Handler,
 	metricReductionRuleHandler metricreductionrule.Handler,
 	infraMonitoringHandler inframonitoring.Handler,
-	annotationsHandler annotations.Handler,
 	gatewayHandler gateway.Handler,
 	gatewayService gateway.Gateway,
 	fieldsHandler fields.Handler,
@@ -290,7 +285,6 @@ func newProvider(
 		metricsExplorerHandler:     metricsExplorerHandler,
 		metricReductionRuleHandler: metricReductionRuleHandler,
 		infraMonitoringHandler:     infraMonitoringHandler,
-		annotationsHandler:         annotationsHandler,
 		gatewayHandler:             gatewayHandler,
 		gatewayService:             gatewayService,
 		fieldsHandler:              fieldsHandler,
@@ -437,10 +431,6 @@ func (provider *provider) AddToRouter(router *mux.Router) error {
 		return err
 	}
 
-	if err := provider.addAnnotationsRoutes(router); err != nil {
-		return err
-	}
-
 	if err := provider.addGatewayRoutes(router); err != nil {
 		return err
 	}

pkg/modules/annotations/annotations.go

@@ -1,19 +0,0 @@
-package annotations
-
-import (
-	"context"
-	"net/http"
-
-	"github.com/SigNoz/signoz/pkg/statsreporter"
-	"github.com/SigNoz/signoz/pkg/types/annotationtypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-)
-
-type Handler interface {
-	ListAnnotations(http.ResponseWriter, *http.Request)
-}
-
-type Module interface {
-	statsreporter.StatsCollector
-	ListAnnotations(ctx context.Context, orgID valuer.UUID, req *annotationtypes.PostableAnnotations) (*annotationtypes.Annotations, error)
-}

pkg/modules/annotations/implannotations/collect.go

@@ -1,27 +0,0 @@
-package implannotations
-
-import (
-	"context"
-	"fmt"
-
-	"github.com/SigNoz/signoz/pkg/errors"
-	"github.com/SigNoz/signoz/pkg/valuer"
-)
-
-func (m *module) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
-	stats := make(map[string]any)
-
-	query := fmt.Sprintf(
-		"SELECT (SELECT count() FROM (SELECT 1 FROM %s.%s WHERE labels LIKE '%%\"service.version\"%%' LIMIT 1)), (SELECT count() FROM (SELECT 1 FROM %s.%s WHERE labels LIKE '%%\"service.version\"%%' LIMIT 1))",
-		tracesTables.dbName, tracesTables.resourceTable, logsTables.dbName, logsTables.resourceTable,
-	)
-	var tracesCount, logsCount uint64
-	if err := m.telemetryStore.ClickhouseDB().QueryRow(ctx, query).Scan(&tracesCount, &logsCount); err == nil {
-		stats["telemetry.traces.service_version.exists"] = tracesCount > 0
-		stats["telemetry.logs.service_version.exists"] = logsCount > 0
-	} else {
-		m.logger.DebugContext(ctx, "failed to collect service.version existence stats", errors.Attr(err))
-	}
-
-	return stats, nil
-}

pkg/modules/annotations/implannotations/deployments.go

@@ -1,263 +0,0 @@
-package implannotations
-
-import (
-	"context"
-	"fmt"
-	"time"
-
-	"github.com/SigNoz/signoz/pkg/errors"
-	"github.com/SigNoz/signoz/pkg/querybuilder"
-	"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
-	"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
-	"github.com/SigNoz/signoz/pkg/types/annotationtypes"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-	"github.com/huandu/go-sqlbuilder"
-)
-
-const resourceBucketSeconds = 1800
-
-type signalTables struct {
-	signal        telemetrytypes.Signal
-	dbName        string
-	resourceTable string
-	dataTable     string
-	rawAlias      string
-}
-
-var (
-	tracesTables = signalTables{
-		signal:        telemetrytypes.SignalTraces,
-		dbName:        tracestelemetryschema.DBName,
-		resourceTable: tracestelemetryschema.TracesResourceV3TableName,
-		dataTable:     tracestelemetryschema.SpanIndexV3TableName,
-		rawAlias:      "raw_traces",
-	}
-	logsTables = signalTables{
-		signal:        telemetrytypes.SignalLogs,
-		dbName:        logstelemetryschema.DBName,
-		resourceTable: logstelemetryschema.LogsResourceV2TableName,
-		dataTable:     logstelemetryschema.LogsV2TableName,
-		rawAlias:      "raw_logs",
-	}
-)
-
-// queryTracesDeployments returns the marker rows and historyAvailableFrom in ms, 0 when there is no trace history.
-func (m *module) queryTracesDeployments(ctx context.Context, orgID valuer.UUID, req *annotationtypes.PostableAnnotations) ([]annotationtypes.DeploymentRow, int64, error) {
-	ctx = m.withAnnotationsContext(ctx, telemetrytypes.SignalTraces, "ListAnnotations")
-
-	filterClause, err := m.buildFilterClause(ctx, orgID, telemetrytypes.SignalTraces, req.Filter, req.Start, req.End)
-	if err != nil {
-		return nil, 0, err
-	}
-
-	rBucket, err := m.readHistoryEdge(ctx, tracesTables)
-	if err != nil {
-		return nil, 0, err
-	}
-	if rBucket == 0 {
-		return []annotationtypes.DeploymentRow{}, 0, nil
-	}
-
-	query, args := buildTracesDeploymentsQuery(filterClause, req.Start, req.End, rBucket)
-	rows, err := m.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
-	if err != nil {
-		return nil, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to query traces deployments")
-	}
-	defer rows.Close()
-
-	result := []annotationtypes.DeploymentRow{}
-	for rows.Next() {
-		var (
-			row       = annotationtypes.DeploymentRow{Signal: telemetrytypes.SignalTraces}
-			firstSeen time.Time
-			firstIDs  []string
-		)
-		if err := rows.Scan(&row.Service, &row.Environment, &row.Version, &firstSeen, &firstIDs, &row.Labels); err != nil {
-			return nil, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to scan traces deployment row")
-		}
-		row.FirstSeen = firstSeen.UnixMilli()
-		if len(firstIDs) == 2 {
-			row.TraceID = firstIDs[0]
-			row.SpanID = firstIDs[1]
-		}
-		result = append(result, row)
-	}
-	if err := rows.Err(); err != nil {
-		return nil, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to iterate traces deployment rows")
-	}
-
-	return result, historyAvailableFromMs(rBucket), nil
-}
-
-// queryLogsDeployments returns the marker rows and historyAvailableFrom in ms, 0 when there is no log history.
-func (m *module) queryLogsDeployments(ctx context.Context, orgID valuer.UUID, req *annotationtypes.PostableAnnotations) ([]annotationtypes.DeploymentRow, int64, error) {
-	ctx = m.withAnnotationsContext(ctx, telemetrytypes.SignalLogs, "ListAnnotations")
-
-	filterClause, err := m.buildFilterClause(ctx, orgID, telemetrytypes.SignalLogs, req.Filter, req.Start, req.End)
-	if err != nil {
-		return nil, 0, err
-	}
-
-	rBucket, err := m.readHistoryEdge(ctx, logsTables)
-	if err != nil {
-		return nil, 0, err
-	}
-	if rBucket == 0 {
-		return []annotationtypes.DeploymentRow{}, 0, nil
-	}
-
-	query, args := buildLogsDeploymentsQuery(filterClause, req.Start, req.End, rBucket)
-	rows, err := m.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
-	if err != nil {
-		return nil, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to query logs deployments")
-	}
-	defer rows.Close()
-
-	result := []annotationtypes.DeploymentRow{}
-	for rows.Next() {
-		var (
-			row         = annotationtypes.DeploymentRow{Signal: telemetrytypes.SignalLogs}
-			firstSeenNs uint64
-		)
-		if err := rows.Scan(&row.Service, &row.Environment, &row.Version, &firstSeenNs, &row.LogID, &row.Labels); err != nil {
-			return nil, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to scan logs deployment row")
-		}
-		row.FirstSeen = int64(firstSeenNs / 1000000)
-		result = append(result, row)
-	}
-	if err := rows.Err(); err != nil {
-		return nil, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to iterate logs deployment rows")
-	}
-
-	return result, historyAvailableFromMs(rBucket), nil
-}
-
-// readHistoryEdge returns the oldest bucket the resource table holds, 0 when it is empty.
-func (m *module) readHistoryEdge(ctx context.Context, tables signalTables) (int64, error) {
-	var rBucket int64
-	if err := m.telemetryStore.ClickhouseDB().QueryRow(ctx, buildHistoryEdgeQuery(tables)).Scan(&rBucket); err != nil {
-		return 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to read %s retention edge", tables.signal.StringValue())
-	}
-	return rBucket, nil
-}
-
-func buildTracesDeploymentsQuery(filterClause *sqlbuilder.WhereClause, startMs, endMs, rBucket int64) (string, []any) {
-	bounds := []string{
-		"fromUnixTimestamp64Milli(toInt64(?)) AS t1_dt",
-		"fromUnixTimestamp64Milli(toInt64(?)) AS t2_dt",
-	}
-	ctes, cteArgs := buildResourcePasses(tracesTables, filterClause, startMs, endMs, rBucket, bounds, []any{startMs, endMs})
-
-	final := buildFinalSelect(
-		tracesTables,
-		"argMin((raw_traces.trace_id, raw_traces.span_id), raw_traces.timestamp) AS first_ids",
-		"first_seen >= t1_dt AND first_seen <= t2_dt",
-	)
-
-	return querybuilder.CombineCTEs(ctes) + final, querybuilder.PrependArgs(cteArgs, nil)
-}
-
-func buildLogsDeploymentsQuery(filterClause *sqlbuilder.WhereClause, startMs, endMs, rBucket int64) (string, []any) {
-	bounds := []string{
-		"toUInt64(?) * 1000000 AS t1_ns",
-		"toUInt64(?) * 1000000 AS t2_ns",
-	}
-	ctes, cteArgs := buildResourcePasses(logsTables, filterClause, startMs, endMs, rBucket, bounds, []any{startMs, endMs})
-
-	final := buildFinalSelect(
-		logsTables,
-		"argMin(raw_logs.id, raw_logs.timestamp) AS first_id",
-		"first_seen >= t1_ns AND first_seen <= t2_ns",
-	)
-
-	return querybuilder.CombineCTEs(ctes) + final, querybuilder.PrependArgs(cteArgs, nil)
-}
-
-// buildResourcePasses returns the constants, the signal's time bounds and the two resource-table CTEs shared by both signals, with their args in order.
-func buildResourcePasses(tables signalTables, filterClause *sqlbuilder.WhereClause, startMs, endMs, rBucket int64, bounds []string, boundArgs []any) ([]string, [][]any) {
-	constants := []string{
-		"toInt64(intDiv(?, 1000)) AS t1",
-		"toInt64(intDiv(?, 1000)) AS t2",
-		fmt.Sprintf("toInt64(intDiv(t1, %d) * %d) AS bt1", resourceBucketSeconds, resourceBucketSeconds),
-		"toInt64(?) AS r_bucket",
-	}
-	constants = append(constants, bounds...)
-	constantArgs := append([]any{startMs, endMs, rBucket}, boundArgs...)
-
-	resourceTable := fmt.Sprintf("%s.%s", tables.dbName, tables.resourceTable)
-
-	firstBucketInner := sqlbuilder.NewSelectBuilder()
-	firstBucketInner.Select(
-		"simpleJSONExtractString(labels, 'service.name') AS service",
-		"simpleJSONExtractString(labels, 'deployment.environment') AS env",
-		"simpleJSONExtractString(labels, 'service.version') AS version",
-		"seen_at_ts_bucket_start AS bucket",
-	)
-	firstBucketInner.From(resourceTable)
-	firstBucketInner.Where("version != ''", "seen_at_ts_bucket_start <= t2")
-	firstBucketInner.AddWhereClause(sqlbuilder.CopyWhereClause(filterClause))
-	firstBucketInnerSQL, firstBucketArgs := firstBucketInner.BuildWithFlavor(sqlbuilder.ClickHouse)
-	firstBucketSQL := fmt.Sprintf(
-		"SELECT service, env, version, min(bucket) AS min_bucket FROM (%s) GROUP BY service, env, version HAVING min_bucket >= bt1 AND min_bucket > r_bucket",
-		firstBucketInnerSQL,
-	)
-
-	cand := sqlbuilder.NewSelectBuilder()
-	cand.Select(
-		"simpleJSONExtractString(labels, 'service.name') AS service",
-		"simpleJSONExtractString(labels, 'deployment.environment') AS env",
-		"simpleJSONExtractString(labels, 'service.version') AS version",
-		"seen_at_ts_bucket_start AS min_bucket",
-		"fingerprint AS fp",
-		"labels",
-	)
-	cand.From(resourceTable)
-	cand.Where(
-		"version != ''",
-		"(service, env, version, seen_at_ts_bucket_start) GLOBAL IN (SELECT service, env, version, min_bucket FROM first_bucket)",
-	)
-	cand.AddWhereClause(sqlbuilder.CopyWhereClause(filterClause))
-	candSQL, candArgs := cand.BuildWithFlavor(sqlbuilder.ClickHouse)
-
-	ctes := make([]string, 0, len(constants)+2)
-	ctes = append(ctes, constants...)
-	ctes = append(ctes, fmt.Sprintf("first_bucket AS (%s)", firstBucketSQL), fmt.Sprintf("cand AS (%s)", candSQL))
-
-	return ctes, [][]any{constantArgs, firstBucketArgs, candArgs}
-}
-
-// buildFinalSelect joins the raw table back to cand; idSelect and having carry the signal-specific columns and bounds.
-func buildFinalSelect(tables signalTables, idSelect, having string) string {
-	raw := tables.rawAlias
-	return fmt.Sprintf(
-		"SELECT cand.service, cand.env, cand.version, min(%s.timestamp) AS first_seen, %s, argMin(cand.labels, %s.timestamp) AS labels "+
-			"FROM %s.%s AS %s "+
-			"GLOBAL INNER JOIN cand ON %s.resource_fingerprint = cand.fp AND toInt64(%s.ts_bucket_start) = cand.min_bucket "+
-			"WHERE (%s.ts_bucket_start, %s.resource_fingerprint) GLOBAL IN (SELECT toUInt64(min_bucket), fp FROM cand) "+
-			"GROUP BY cand.service, cand.env, cand.version "+
-			"HAVING %s "+
-			"ORDER BY first_seen",
-		raw, idSelect, raw,
-		tables.dbName, tables.dataTable, raw,
-		raw, raw,
-		raw, raw,
-		having,
-	)
-}
-
-func buildHistoryEdgeQuery(tables signalTables) string {
-	return fmt.Sprintf("SELECT min(seen_at_ts_bucket_start) FROM %s.%s", tables.dbName, tables.resourceTable)
-}
-
-func historyAvailableFromMs(rBucket int64) int64 {
-	return (rBucket + resourceBucketSeconds) * 1000
-}
-
-// resolveSignals returns the requested signals, both when none were given.
-func resolveSignals(req *annotationtypes.PostableAnnotations) []telemetrytypes.Signal {
-	if len(req.Signals) == 0 {
-		return []telemetrytypes.Signal{telemetrytypes.SignalTraces, telemetrytypes.SignalLogs}
-	}
-	return req.Signals
-}

pkg/modules/annotations/implannotations/deployments_test.go

@@ -1,125 +0,0 @@
-package implannotations
-
-import (
-	"strings"
-	"testing"
-
-	"github.com/SigNoz/signoz/pkg/types/annotationtypes"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/huandu/go-sqlbuilder"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-)
-
-func serviceFilterClause(value string) *sqlbuilder.WhereClause {
-	clause := sqlbuilder.NewWhereClause()
-	cond := sqlbuilder.NewCond()
-	clause.AddWhereExpr(cond.Args, cond.Equal("simpleJSONExtractString(labels, 'service.name')", value))
-	return clause
-}
-
-// withServiceFilter inserts the service predicate into both resource passes of an unfiltered golden.
-func withServiceFilter(golden string) string {
-	golden = strings.Replace(golden, "seen_at_ts_bucket_start <= t2)", "seen_at_ts_bucket_start <= t2 AND simpleJSONExtractString(labels, 'service.name') = ?)", 1)
-	return strings.Replace(golden, "min_bucket FROM first_bucket))", "min_bucket FROM first_bucket) AND simpleJSONExtractString(labels, 'service.name') = ?)", 1)
-}
-
-func TestBuildTracesDeploymentsQuery(t *testing.T) {
-	noFilter := "WITH toInt64(intDiv(?, 1000)) AS t1, toInt64(intDiv(?, 1000)) AS t2, toInt64(intDiv(t1, 1800) * 1800) AS bt1, toInt64(?) AS r_bucket, fromUnixTimestamp64Milli(toInt64(?)) AS t1_dt, fromUnixTimestamp64Milli(toInt64(?)) AS t2_dt, " +
-		"first_bucket AS (SELECT service, env, version, min(bucket) AS min_bucket FROM (SELECT simpleJSONExtractString(labels, 'service.name') AS service, simpleJSONExtractString(labels, 'deployment.environment') AS env, simpleJSONExtractString(labels, 'service.version') AS version, seen_at_ts_bucket_start AS bucket FROM signoz_traces.distributed_traces_v3_resource WHERE version != '' AND seen_at_ts_bucket_start <= t2) GROUP BY service, env, version HAVING min_bucket >= bt1 AND min_bucket > r_bucket), " +
-		"cand AS (SELECT simpleJSONExtractString(labels, 'service.name') AS service, simpleJSONExtractString(labels, 'deployment.environment') AS env, simpleJSONExtractString(labels, 'service.version') AS version, seen_at_ts_bucket_start AS min_bucket, fingerprint AS fp, labels FROM signoz_traces.distributed_traces_v3_resource WHERE version != '' AND (service, env, version, seen_at_ts_bucket_start) GLOBAL IN (SELECT service, env, version, min_bucket FROM first_bucket)) " +
-		"SELECT cand.service, cand.env, cand.version, min(raw_traces.timestamp) AS first_seen, argMin((raw_traces.trace_id, raw_traces.span_id), raw_traces.timestamp) AS first_ids, argMin(cand.labels, raw_traces.timestamp) AS labels FROM signoz_traces.distributed_signoz_index_v3 AS raw_traces GLOBAL INNER JOIN cand ON raw_traces.resource_fingerprint = cand.fp AND toInt64(raw_traces.ts_bucket_start) = cand.min_bucket WHERE (raw_traces.ts_bucket_start, raw_traces.resource_fingerprint) GLOBAL IN (SELECT toUInt64(min_bucket), fp FROM cand) GROUP BY cand.service, cand.env, cand.version HAVING first_seen >= t1_dt AND first_seen <= t2_dt ORDER BY first_seen"
-
-	testCases := []struct {
-		name     string
-		filter   *sqlbuilder.WhereClause
-		wantSQL  string
-		wantArgs []any
-	}{
-		{
-			name:     "NoFilter_Golden",
-			filter:   sqlbuilder.NewWhereClause(),
-			wantSQL:  noFilter,
-			wantArgs: []any{int64(1700000000000), int64(1700003600000), int64(1699990000), int64(1700000000000), int64(1700003600000)},
-		},
-		{
-			name:     "Filtered_ArgsInBothPasses",
-			filter:   serviceFilterClause("cart"),
-			wantSQL:  withServiceFilter(noFilter),
-			wantArgs: []any{int64(1700000000000), int64(1700003600000), int64(1699990000), int64(1700000000000), int64(1700003600000), "cart", "cart"},
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			sql, args := buildTracesDeploymentsQuery(testCase.filter, 1700000000000, 1700003600000, 1699990000)
-			assert.Equal(t, testCase.wantSQL, sql)
-			assert.Equal(t, testCase.wantArgs, args)
-			assert.Equal(t, strings.Count(sql, "?"), len(args))
-		})
-	}
-}
-
-func TestBuildLogsDeploymentsQuery(t *testing.T) {
-	noFilter := "WITH toInt64(intDiv(?, 1000)) AS t1, toInt64(intDiv(?, 1000)) AS t2, toInt64(intDiv(t1, 1800) * 1800) AS bt1, toInt64(?) AS r_bucket, toUInt64(?) * 1000000 AS t1_ns, toUInt64(?) * 1000000 AS t2_ns, " +
-		"first_bucket AS (SELECT service, env, version, min(bucket) AS min_bucket FROM (SELECT simpleJSONExtractString(labels, 'service.name') AS service, simpleJSONExtractString(labels, 'deployment.environment') AS env, simpleJSONExtractString(labels, 'service.version') AS version, seen_at_ts_bucket_start AS bucket FROM signoz_logs.distributed_logs_v2_resource WHERE version != '' AND seen_at_ts_bucket_start <= t2) GROUP BY service, env, version HAVING min_bucket >= bt1 AND min_bucket > r_bucket), " +
-		"cand AS (SELECT simpleJSONExtractString(labels, 'service.name') AS service, simpleJSONExtractString(labels, 'deployment.environment') AS env, simpleJSONExtractString(labels, 'service.version') AS version, seen_at_ts_bucket_start AS min_bucket, fingerprint AS fp, labels FROM signoz_logs.distributed_logs_v2_resource WHERE version != '' AND (service, env, version, seen_at_ts_bucket_start) GLOBAL IN (SELECT service, env, version, min_bucket FROM first_bucket)) " +
-		"SELECT cand.service, cand.env, cand.version, min(raw_logs.timestamp) AS first_seen, argMin(raw_logs.id, raw_logs.timestamp) AS first_id, argMin(cand.labels, raw_logs.timestamp) AS labels FROM signoz_logs.distributed_logs_v2 AS raw_logs GLOBAL INNER JOIN cand ON raw_logs.resource_fingerprint = cand.fp AND toInt64(raw_logs.ts_bucket_start) = cand.min_bucket WHERE (raw_logs.ts_bucket_start, raw_logs.resource_fingerprint) GLOBAL IN (SELECT toUInt64(min_bucket), fp FROM cand) GROUP BY cand.service, cand.env, cand.version HAVING first_seen >= t1_ns AND first_seen <= t2_ns ORDER BY first_seen"
-
-	testCases := []struct {
-		name     string
-		filter   *sqlbuilder.WhereClause
-		wantSQL  string
-		wantArgs []any
-	}{
-		{
-			name:     "NoFilter_Golden",
-			filter:   sqlbuilder.NewWhereClause(),
-			wantSQL:  noFilter,
-			wantArgs: []any{int64(1700000000000), int64(1700003600000), int64(1699990000), int64(1700000000000), int64(1700003600000)},
-		},
-		{
-			name:     "Filtered_ArgsInBothPasses",
-			filter:   serviceFilterClause("cart"),
-			wantSQL:  withServiceFilter(noFilter),
-			wantArgs: []any{int64(1700000000000), int64(1700003600000), int64(1699990000), int64(1700000000000), int64(1700003600000), "cart", "cart"},
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			sql, args := buildLogsDeploymentsQuery(testCase.filter, 1700000000000, 1700003600000, 1699990000)
-			assert.Equal(t, testCase.wantSQL, sql)
-			assert.Equal(t, testCase.wantArgs, args)
-			assert.Equal(t, strings.Count(sql, "?"), len(args))
-		})
-	}
-}
-
-func TestBuildHistoryEdgeQuery(t *testing.T) {
-	assert.Equal(t, "SELECT min(seen_at_ts_bucket_start) FROM signoz_traces.distributed_traces_v3_resource", buildHistoryEdgeQuery(tracesTables))
-	assert.Equal(t, "SELECT min(seen_at_ts_bucket_start) FROM signoz_logs.distributed_logs_v2_resource", buildHistoryEdgeQuery(logsTables))
-}
-
-func TestHistoryAvailableFromMs(t *testing.T) {
-	assert.Equal(t, int64(1700001800000), historyAvailableFromMs(1700000000))
-}
-
-func TestResolveSignals(t *testing.T) {
-	testCases := []struct {
-		name string
-		in   []telemetrytypes.Signal
-		want []telemetrytypes.Signal
-	}{
-		{name: "Empty_Both", in: nil, want: []telemetrytypes.Signal{telemetrytypes.SignalTraces, telemetrytypes.SignalLogs}},
-		{name: "LogsOnly_Logs", in: []telemetrytypes.Signal{telemetrytypes.SignalLogs}, want: []telemetrytypes.Signal{telemetrytypes.SignalLogs}},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			got := resolveSignals(&annotationtypes.PostableAnnotations{Signals: testCase.in})
-			require.Len(t, got, len(testCase.want))
-			assert.Equal(t, testCase.want, got)
-		})
-	}
-}

pkg/modules/annotations/implannotations/handler.go

@@ -1,46 +0,0 @@
-package implannotations
-
-import (
-	"net/http"
-
-	"github.com/SigNoz/signoz/pkg/http/binding"
-	"github.com/SigNoz/signoz/pkg/http/render"
-	"github.com/SigNoz/signoz/pkg/modules/annotations"
-	"github.com/SigNoz/signoz/pkg/types/annotationtypes"
-	"github.com/SigNoz/signoz/pkg/types/authtypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-)
-
-type handler struct {
-	module annotations.Module
-}
-
-func NewHandler(m annotations.Module) annotations.Handler {
-	return &handler{
-		module: m,
-	}
-}
-
-func (h *handler) ListAnnotations(rw http.ResponseWriter, req *http.Request) {
-	claims, err := authtypes.ClaimsFromContext(req.Context())
-	if err != nil {
-		render.Error(rw, err)
-		return
-	}
-
-	orgID := valuer.MustNewUUID(claims.OrgID)
-
-	var parsedReq annotationtypes.PostableAnnotations
-	if err := binding.JSON.BindBody(req.Body, &parsedReq); err != nil {
-		render.Error(rw, err)
-		return
-	}
-
-	result, err := h.module.ListAnnotations(req.Context(), orgID, &parsedReq)
-	if err != nil {
-		render.Error(rw, err)
-		return
-	}
-
-	render.Success(rw, http.StatusOK, result)
-}

pkg/modules/annotations/implannotations/helpers.go

@@ -1,55 +0,0 @@
-package implannotations
-
-import (
-	"context"
-	"strings"
-
-	"github.com/SigNoz/signoz/pkg/querybuilder"
-	qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-	"github.com/huandu/go-sqlbuilder"
-)
-
-// Returns an empty clause when the expression is empty or holds no resource condition.
-func (m *module) buildFilterClause(ctx context.Context, orgID valuer.UUID, signal telemetrytypes.Signal, filter *qbtypes.Filter, startMillis, endMillis int64) (*sqlbuilder.WhereClause, error) {
-	expression := ""
-	if filter != nil {
-		expression = strings.TrimSpace(filter.Expression)
-	}
-	if expression == "" {
-		return sqlbuilder.NewWhereClause(), nil
-	}
-
-	whereClauseSelectors := querybuilder.QueryStringToKeysSelectors(expression)
-	for idx := range whereClauseSelectors {
-		whereClauseSelectors[idx].Signal = signal
-		whereClauseSelectors[idx].SelectorMatchType = telemetrytypes.FieldSelectorMatchTypeExact
-	}
-
-	whereClauseSelectors = querybuilder.ExpandKeySelectorsForFamilies(ctx, orgID, m.fl, whereClauseSelectors)
-	keys, _, err := m.telemetryMetadataStore.GetKeysMulti(ctx, orgID, whereClauseSelectors)
-	if err != nil {
-		return nil, err
-	}
-
-	opts := querybuilder.FilterExprVisitorOpts{
-		Context:            ctx,
-		Query:              querybuilder.NewQueryInfo(ctx, orgID, m.fl, signal, nil, querybuilder.ToNanoSecs(uint64(startMillis)), querybuilder.ToNanoSecs(uint64(endMillis))),
-		Storage:            m.storage,
-		Logger:             m.logger,
-		FieldKeys:          keys,
-		SkipFullTextFilter: true,
-	}
-
-	whereClause, err := querybuilder.PrepareWhereClause(expression, opts)
-	if err != nil {
-		return nil, err
-	}
-
-	if whereClause.IsEmpty() {
-		return sqlbuilder.NewWhereClause(), nil
-	}
-
-	return whereClause.WhereClause, nil
-}

pkg/modules/annotations/implannotations/helpers_test.go

@@ -1,165 +0,0 @@
-package implannotations
-
-import (
-	"context"
-	"strings"
-	"testing"
-
-	"github.com/SigNoz/signoz/pkg/flagger"
-	"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
-	"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
-	"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
-	qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
-	"github.com/SigNoz/signoz/pkg/valuer"
-	"github.com/huandu/go-sqlbuilder"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-)
-
-func newTestModule(t *testing.T, fl flagger.Flagger, signal telemetrytypes.Signal) *module {
-	t.Helper()
-	mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
-	mockMetadataStore.KeysMap = map[string][]*telemetrytypes.TelemetryFieldKey{
-		"service.name": {{
-			Name:          "service.name",
-			FieldContext:  telemetrytypes.FieldContextResource,
-			FieldDataType: telemetrytypes.FieldDataTypeString,
-		}},
-		"k8s.namespace.name": {{
-			Name:          "k8s.namespace.name",
-			FieldContext:  telemetrytypes.FieldContextResource,
-			FieldDataType: telemetrytypes.FieldDataTypeString,
-		}},
-		"deployment.environment": {{
-			Name:          "deployment.environment",
-			FieldContext:  telemetrytypes.FieldContextResource,
-			FieldDataType: telemetrytypes.FieldDataTypeString,
-		}},
-		"deployment.environment.name": {{
-			Name:          "deployment.environment.name",
-			FieldContext:  telemetrytypes.FieldContextResource,
-			FieldDataType: telemetrytypes.FieldDataTypeString,
-		}},
-		"http.request.method": {{
-			Name:          "http.request.method",
-			FieldContext:  telemetrytypes.FieldContextAttribute,
-			FieldDataType: telemetrytypes.FieldDataTypeString,
-		}},
-	}
-	for _, keys := range mockMetadataStore.KeysMap {
-		for _, key := range keys {
-			key.Signal = signal
-		}
-	}
-	return &module{
-		telemetryMetadataStore: mockMetadataStore,
-		storage:                resourcefilter.NewStorage(),
-		fl:                     fl,
-		logger:                 instrumentationtest.New().ToProviderSettings().Logger,
-	}
-}
-
-func buildSQL(clause *sqlbuilder.WhereClause) (string, []any) {
-	sb := sqlbuilder.NewSelectBuilder()
-	sb.Select("fingerprint")
-	sb.From("t")
-	sb.AddWhereClause(clause)
-	sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
-	return strings.TrimSpace(sql), args
-}
-
-func TestBuildFilterClause(t *testing.T) {
-	testCases := []struct {
-		name     string
-		signal   telemetrytypes.Signal
-		filter   *qbtypes.Filter
-		familyOn bool
-		wantSQL  string
-		wantArgs []any
-	}{
-		{
-			name:     "NilFilter_EmptyClause",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   nil,
-			wantSQL:  "SELECT fingerprint FROM t",
-			wantArgs: nil,
-		},
-		{
-			name:     "WhitespaceExpression_EmptyClause",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   &qbtypes.Filter{Expression: "   "},
-			wantSQL:  "SELECT fingerprint FROM t",
-			wantArgs: nil,
-		},
-		{
-			name:     "ResourceKey_Traces_Compiled",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   &qbtypes.Filter{Expression: "service.name = 'cartservice'"},
-			wantSQL:  "SELECT fingerprint FROM t WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?)",
-			wantArgs: []any{"cartservice", "%service.name%", "%service.name\":\"cartservice%"},
-		},
-		{
-			name:     "ResourceKey_Logs_Compiled",
-			signal:   telemetrytypes.SignalLogs,
-			filter:   &qbtypes.Filter{Expression: "k8s.namespace.name = 'shop'"},
-			wantSQL:  "SELECT fingerprint FROM t WHERE (simpleJSONExtractString(labels, 'k8s.namespace.name') = ? AND labels LIKE ? AND labels LIKE ?)",
-			wantArgs: []any{"shop", "%k8s.namespace.name%", "%k8s.namespace.name\":\"shop%"},
-		},
-		{
-			name:     "InList_Compiled",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   &qbtypes.Filter{Expression: "service.name IN ('frontend', 'checkout')"},
-			wantSQL:  "SELECT fingerprint FROM t WHERE ((simpleJSONExtractString(labels, 'service.name') = ? OR simpleJSONExtractString(labels, 'service.name') = ?) AND labels LIKE ? AND (labels LIKE ? OR labels LIKE ?))",
-			wantArgs: []any{"frontend", "checkout", "%service.name%", "%service.name\":\"frontend%", "%service.name\":\"checkout%"},
-		},
-		{
-			name:     "SpanAttributeOnly_EmptyClause",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   &qbtypes.Filter{Expression: "http.request.method = 'GET'"},
-			wantSQL:  "SELECT fingerprint FROM t",
-			wantArgs: nil,
-		},
-		{
-			name:     "SpecialCharacters_BoundAsArgs",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   &qbtypes.Filter{Expression: "service.name = 'ca`rt$0'"},
-			wantSQL:  "SELECT fingerprint FROM t WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?)",
-			wantArgs: []any{"ca`rt$0", "%service.name%", "%service.name\":\"ca`rt$0%"},
-		},
-		{
-			name:     "EnvironmentFamily_FlagOn_Widened",
-			signal:   telemetrytypes.SignalTraces,
-			filter:   &qbtypes.Filter{Expression: "deployment.environment = 'production'"},
-			familyOn: true,
-			wantSQL:  "SELECT fingerprint FROM t WHERE (COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') = ? AND (labels LIKE ? OR labels LIKE ?) AND (labels LIKE ? OR labels LIKE ?))",
-			wantArgs: []any{"production", "%deployment.environment.name%", "%deployment.environment%", "%deployment.environment.name\":\"production%", "%deployment.environment\":\"production%"},
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			fl := flaggertest.New(t)
-			if testCase.familyOn {
-				fl = flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true})
-			}
-			m := newTestModule(t, fl, testCase.signal)
-
-			clause, err := m.buildFilterClause(context.Background(), valuer.UUID{}, testCase.signal, testCase.filter, 1000, 2000)
-			require.NoError(t, err)
-			require.NotNil(t, clause)
-
-			sql, args := buildSQL(clause)
-			assert.Equal(t, testCase.wantSQL, sql)
-			assert.Equal(t, testCase.wantArgs, args)
-		})
-	}
-}
-
-func TestBuildFilterClause_ParseError(t *testing.T) {
-	m := newTestModule(t, flaggertest.New(t), telemetrytypes.SignalTraces)
-
-	_, err := m.buildFilterClause(context.Background(), valuer.UUID{}, telemetrytypes.SignalTraces, &qbtypes.Filter{Expression: "service.name = "}, 1000, 2000)
-	assert.Error(t, err)
-}

pkg/modules/annotations/implannotations/module.go

@@ -1,102 +0,0 @@
-package implannotations
-
-import (
-	"context"
-	"log/slog"
-	"slices"
-
-	"github.com/SigNoz/signoz/pkg/factory"
-	"github.com/SigNoz/signoz/pkg/flagger"
-	"github.com/SigNoz/signoz/pkg/modules/annotations"
-	"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
-	"github.com/SigNoz/signoz/pkg/telemetrystore"
-	"github.com/SigNoz/signoz/pkg/types/annotationtypes"
-	"github.com/SigNoz/signoz/pkg/types/ctxtypes"
-	"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
-	qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-	"golang.org/x/sync/errgroup"
-)
-
-type module struct {
-	telemetryStore         telemetrystore.TelemetryStore
-	telemetryMetadataStore telemetrytypes.MetadataStore
-	storage                qbtypes.Storage
-	fl                     flagger.Flagger
-	logger                 *slog.Logger
-}
-
-func NewModule(
-	telemetryStore telemetrystore.TelemetryStore,
-	telemetryMetadataStore telemetrytypes.MetadataStore,
-	fl flagger.Flagger,
-	providerSettings factory.ProviderSettings,
-) annotations.Module {
-	return &module{
-		telemetryStore:         telemetryStore,
-		telemetryMetadataStore: telemetryMetadataStore,
-		storage:                resourcefilter.NewStorage(),
-		fl:                     fl,
-		logger:                 providerSettings.Logger,
-	}
-}
-
-func (m *module) ListAnnotations(ctx context.Context, orgID valuer.UUID, req *annotationtypes.PostableAnnotations) (*annotationtypes.Annotations, error) {
-	if err := req.Validate(); err != nil {
-		return nil, err
-	}
-
-	var (
-		tracesSpecs, logsSpecs             []annotationtypes.DeploymentSpec
-		tracesHistoryEdge, logsHistoryEdge int64
-	)
-	signals := resolveSignals(req)
-	errGroup, gCtx := errgroup.WithContext(ctx)
-
-	if slices.Contains(signals, telemetrytypes.SignalTraces) {
-		errGroup.Go(func() error {
-			rows, edge, err := m.queryTracesDeployments(gCtx, orgID, req)
-			if err != nil {
-				return err
-			}
-			tracesSpecs, err = annotationtypes.NewDeploymentSpecsFromRows(rows)
-			if err != nil {
-				return err
-			}
-			tracesHistoryEdge = edge
-			return nil
-		})
-	}
-
-	if slices.Contains(signals, telemetrytypes.SignalLogs) {
-		errGroup.Go(func() error {
-			rows, edge, err := m.queryLogsDeployments(gCtx, orgID, req)
-			if err != nil {
-				return err
-			}
-			logsSpecs, err = annotationtypes.NewDeploymentSpecsFromRows(rows)
-			if err != nil {
-				return err
-			}
-			logsHistoryEdge = edge
-			return nil
-		})
-	}
-
-	if err := errGroup.Wait(); err != nil {
-		return nil, err
-	}
-
-	page, total := annotationtypes.ApplyLimit(annotationtypes.MergeDeployments(tracesSpecs, logsSpecs), req.Limit)
-	return annotationtypes.NewAnnotationsFromDeployments(page, total, annotationtypes.EarliestHistoryEdge(tracesHistoryEdge, logsHistoryEdge)), nil
-}
-
-func (m *module) withAnnotationsContext(ctx context.Context, signal telemetrytypes.Signal, functionName string) context.Context {
-	comments := map[string]string{
-		instrumentationtypes.TelemetrySignal:  signal.StringValue(),
-		instrumentationtypes.CodeNamespace:    "annotations",
-		instrumentationtypes.CodeFunctionName: functionName,
-	}
-	return ctxtypes.NewContextWithCommentVals(ctx, comments)
-}

pkg/modules/spanmapper/implspanmapper/fs/definitions/gen_ai.llm.json

@@ -1,5 +1,5 @@
 {
-  "version": 4,
+  "version": 5,
   "definition": {
     "name": "gen_ai.llm",
     "condition": {
@@ -134,6 +134,12 @@
         "fieldContext": "attribute",
         "config": {
           "sources": [
+            {
+              "key": "langfuse.observation.type",
+              "context": "attribute",
+              "operation": "copy",
+              "priority": 40
+            },
             {
               "key": "openinference.span.kind",
               "context": "attribute",
@@ -356,6 +362,12 @@
         "fieldContext": "attribute",
         "config": {
           "sources": [
+            {
+              "key": "langfuse.observation.input",
+              "context": "attribute",
+              "operation": "move",
+              "priority": 40
+            },
             {
               "key": "gen_ai.prompt",
               "context": "attribute",
@@ -382,6 +394,12 @@
         "fieldContext": "attribute",
         "config": {
           "sources": [
+            {
+              "key": "langfuse.observation.output",
+              "context": "attribute",
+              "operation": "move",
+              "priority": 40
+            },
             {
               "key": "gen_ai.completion",
               "context": "attribute",

pkg/signoz/handler.go

@@ -14,8 +14,6 @@ import (
 	"github.com/SigNoz/signoz/pkg/licensing"
 	"github.com/SigNoz/signoz/pkg/modules/aiobservability"
 	"github.com/SigNoz/signoz/pkg/modules/aiobservability/implaiobservability"
-	"github.com/SigNoz/signoz/pkg/modules/annotations"
-	"github.com/SigNoz/signoz/pkg/modules/annotations/implannotations"
 	"github.com/SigNoz/signoz/pkg/modules/apdex"
 	"github.com/SigNoz/signoz/pkg/modules/apdex/implapdex"
 	"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
@@ -74,7 +72,6 @@ type Handlers struct {
 	MetricsExplorer         metricsexplorer.Handler
 	MetricReductionRule     metricreductionrule.Handler
 	InfraMonitoring         inframonitoring.Handler
-	Annotations             annotations.Handler
 	Global                  global.Handler
 	FlaggerHandler          flagger.Handler
 	GatewayHandler          gateway.Handler
@@ -128,7 +125,6 @@ func NewHandlers(
 		MetricsExplorer:         implmetricsexplorer.NewHandler(modules.MetricsExplorer),
 		MetricReductionRule:     implmetricreductionrule.NewHandler(modules.MetricReductionRule),
 		InfraMonitoring:         implinframonitoring.NewHandler(modules.InfraMonitoring),
-		Annotations:             implannotations.NewHandler(modules.Annotations),
 		SpanPercentile:          implspanpercentile.NewHandler(modules.SpanPercentile),
 		Global:                  signozglobal.NewHandler(global),
 		FlaggerHandler:          flagger.NewHandler(flaggerService),

pkg/signoz/module.go

@@ -9,8 +9,6 @@ import (
 	"github.com/SigNoz/signoz/pkg/emailing"
 	"github.com/SigNoz/signoz/pkg/factory"
 	"github.com/SigNoz/signoz/pkg/flagger"
-	"github.com/SigNoz/signoz/pkg/modules/annotations"
-	"github.com/SigNoz/signoz/pkg/modules/annotations/implannotations"
 	"github.com/SigNoz/signoz/pkg/modules/apdex"
 	"github.com/SigNoz/signoz/pkg/modules/apdex/implapdex"
 	"github.com/SigNoz/signoz/pkg/modules/authdomain"
@@ -87,7 +85,6 @@ type Modules struct {
 	MetricsExplorer      metricsexplorer.Module
 	MetricReductionRule  metricreductionrule.Module
 	InfraMonitoring      inframonitoring.Module
-	Annotations          annotations.Module
 	Promote              promote.Module
 	ServiceAccount       serviceaccount.Module
 	ServiceAccountGetter serviceaccount.Getter
@@ -158,7 +155,6 @@ func NewModules(
 		MetricsExplorer:      implmetricsexplorer.NewModule(telemetryStore, telemetryMetadataStore, cache, ruleStore, dashboard, fl, providerSettings, config.MetricsExplorer),
 		MetricReductionRule:  metricReductionRule,
 		InfraMonitoring:      implinframonitoring.NewModule(telemetryStore, telemetryMetadataStore, querier, fl, providerSettings, config.InfraMonitoring),
-		Annotations:          implannotations.NewModule(telemetryStore, telemetryMetadataStore, fl, providerSettings),
 		Promote:              implpromote.NewModule(telemetryMetadataStore, telemetryStore, providerSettings),
 		ServiceAccount:       serviceAccount,
 		ServiceAccountGetter: serviceAccountGetter,

pkg/signoz/openapi.go

@@ -21,7 +21,6 @@ import (
 	"github.com/SigNoz/signoz/pkg/instrumentation"
 	"github.com/SigNoz/signoz/pkg/licensing"
 	"github.com/SigNoz/signoz/pkg/modules/aiobservability"
-	"github.com/SigNoz/signoz/pkg/modules/annotations"
 	"github.com/SigNoz/signoz/pkg/modules/authdomain"
 	"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
 	"github.com/SigNoz/signoz/pkg/modules/dashboard"
@@ -83,7 +82,6 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
 		struct{ metricsexplorer.Handler }{},
 		struct{ metricreductionrule.Handler }{},
 		struct{ inframonitoring.Handler }{},
-		struct{ annotations.Handler }{},
 		struct{ gateway.Handler }{},
 		struct{ gateway.Gateway }{},
 		struct{ fields.Handler }{},

pkg/signoz/provider.go

@@ -350,7 +350,6 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
 			handlers.MetricsExplorer,
 			handlers.MetricReductionRule,
 			handlers.InfraMonitoring,
-			handlers.Annotations,
 			handlers.GatewayHandler,
 			gatewayService,
 			handlers.Fields,

pkg/signoz/signoz.go

@@ -570,7 +570,6 @@ func New(
 		cloudIntegrationModule,
 		modules.LogsPipeline,
 		modules.InfraMonitoring,
-		modules.Annotations,
 		querier,
 		authz,
 	}

pkg/statementbuilder/resourcefilter/compile_test.go

@@ -200,7 +200,7 @@ func TestConditionBuilder(t *testing.T) {
 		},
 	}
 
-	storage := NewStorage()
+	storage := newStorage()
 
 	for _, tc := range testCases {
 		sb := sqlbuilder.NewSelectBuilder()

pkg/statementbuilder/resourcefilter/family_test.go

@@ -20,7 +20,7 @@ import (
 
 func TestFamilyEqualWidensIndexHintsToAnyMember(t *testing.T) {
 	fl := flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true})
-	storage := NewStorage()
+	storage := newStorage()
 	fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
 		"deployment.environment.name": {{
 			Name:          "deployment.environment.name",
@@ -48,7 +48,7 @@ func TestFamilyEqualWidensIndexHintsToAnyMember(t *testing.T) {
 
 func TestFamilyNotEqualDropsNegatedValueHint(t *testing.T) {
 	fl := flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true})
-	storage := NewStorage()
+	storage := newStorage()
 	fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
 		"deployment.environment.name": {{
 			Name:          "deployment.environment.name",
@@ -76,7 +76,7 @@ func TestFamilyNotEqualDropsNegatedValueHint(t *testing.T) {
 
 func TestFamilyExistsIsAnyMemberPresence(t *testing.T) {
 	fl := flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true})
-	storage := NewStorage()
+	storage := newStorage()
 	fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
 		"deployment.environment.name": {{
 			Name:          "deployment.environment.name",
@@ -115,7 +115,7 @@ func TestFamilyExistsIsAnyMemberPresence(t *testing.T) {
 // including the negated value hint on !=.
 func TestSingleMemberShapesUnchanged(t *testing.T) {
 	fl := flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true})
-	storage := NewStorage()
+	storage := newStorage()
 	fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
 		"deployment.environment.name": {{
 			Name:          "deployment.environment.name",

pkg/statementbuilder/resourcefilter/statement_builder.go

@@ -49,7 +49,7 @@ func New[T any](
 		logger:         set.Logger(),
 		dbName:         dbName,
 		tableName:      tableName,
-		storage:        NewStorage(),
+		storage:        newStorage(),
 		metadataStore:  metadataStore,
 		signal:         signal,
 		source:         source,

pkg/statementbuilder/resourcefilter/storage.go

@@ -22,7 +22,7 @@ type storage struct{}
 
 var _ qbtypes.Storage = (*storage)(nil)
 
-func NewStorage() qbtypes.Storage {
+func newStorage() *storage {
 	return &storage{}
 }
 

pkg/types/aiobservabilitytypes/message_format.go

@@ -121,10 +121,13 @@ func (f *formatting) messages() {
 
 // OpenRouter's answer object or an Anthropic response beside the OpenAI chat request it answers is one
 // format, not a mix: an Anthropic request without system and without Anthropic-only blocks reads as an
-// OpenAI chat request.
+// OpenAI chat request. A side kept as generic already warned, and a side read from OpenInference's
+// buckets beside the raw payload of the other is the documented fallback.
 func (f *formatting) warnMixedFormats(inputLabel, outputLabel string) {
 	switch {
 	case inputLabel == "" || outputLabel == "" || outputLabel == inputLabel:
+	case inputLabel == FormatterGeneric || outputLabel == FormatterGeneric:
+	case inputLabel == FormatterOpenInference || outputLabel == FormatterOpenInference:
 	case inputLabel == FormatterOpenAIChat && (outputLabel == FormatterOpenRouter || outputLabel == FormatterAnthropic):
 	default:
 		f.warn("output formatted as %s, input as %s", outputLabel, inputLabel)
@@ -183,11 +186,16 @@ func (f *formatting) side(value any, role Role) (OutputMessages, string) {
 				return f.semconv(v), FormatterSemconv
 			case isAnthropicMessages(v):
 				return f.anthropicMessages(v), FormatterAnthropic
+			case isResponsesItems(v):
+				return foldAssistant(f.responsesItems(v)), FormatterOpenAIResponses
 			case first["role"] != nil:
 				return f.openAIChatMessages(v), FormatterOpenAIChat
 			}
 		}
 	case map[string]any:
+		if inner, ok := superJSON(v); ok {
+			return f.side(decode(inner), role)
+		}
 		switch {
 		case isLangChainRun(v):
 			return f.langChainRun(v), FormatterLangChain
@@ -214,6 +222,9 @@ func (f *formatting) side(value any, role Role) (OutputMessages, string) {
 		if messages, ok := v["messages"].([]any); ok {
 			return f.openAIChatMessages(messages), FormatterOpenAIChat
 		}
+		if (v["role"] != nil && hasKeys(v, "content")) || v["tool_calls"] != nil {
+			return OutputMessages{f.openAIChatSingle(v)}, FormatterOpenAIChat
+		}
 	case string:
 		if v == "" {
 			return OutputMessages{}, FormatterText
@@ -309,6 +320,23 @@ func foldAssistant(msgs OutputMessages) OutputMessages {
 	return out
 }
 
+// superJSON unwraps {json, meta?}, the envelope superjson, the transformer tRPC apps serialise with,
+// puts around a value; a Langfuse generation records it around the request when the app passes the
+// serialised form:
+// https://github.com/flightcontrolhq/superjson/blob/aaa65e36e8e31da740265a56b27c965ca1c7b754/README.md
+func superJSON(m map[string]any) (any, bool) {
+	inner, ok := m["json"]
+	if !ok {
+		return nil, false
+	}
+	for key := range m {
+		if key != "json" && key != "meta" {
+			return nil, false
+		}
+	}
+	return inner, true
+}
+
 // decode parses a JSON-encoded attribute value, twice when an SDK encoded the string again.
 // Anything that is not JSON is returned as is.
 func decode(value any) any {

pkg/types/aiobservabilitytypes/message_format_openai_chat.go

@@ -4,8 +4,9 @@ import (
 	"strings"
 )
 
-// Format: OpenAI Chat Completions, a request {messages}, a response {choices}, or a bare message list.
-// Written by: bifrost, openrouter requests, the raw OpenAI calls OpenInference records.
+// Format: OpenAI Chat Completions, a request {messages}, a response {choices}, a bare message list, or a
+// single message.
+// Written by: bifrost, openrouter requests, the raw OpenAI calls OpenInference records, the Langfuse SDK.
 // Handled: string and block content, tool_calls nested or flattened, tool messages, refusal,
 // reasoning_content, finish reasons.
 // Not yet: streaming delta chunks, legacy completions choices[].text, audio output.
@@ -88,6 +89,19 @@ func (f *formatting) openAIChatMessage(m map[string]any) OutputMessage {
 	return msg
 }
 
+// openAIChatSingle converts a message recorded outside a list: the first choice's message the Langfuse
+// OpenAI wrapper writes as the output, or the additional_kwargs of a LangChain AIMessage, {tool_calls}
+// without a role, which the Langfuse LangChain handler writes for a tool-calling turn:
+// https://github.com/langfuse/langfuse-python/blob/18baf80450dd5f9c1f85916eb2d1b3c72d2052e1/langfuse/openai.py#L446
+// https://github.com/langfuse/langfuse-python/blob/18baf80450dd5f9c1f85916eb2d1b3c72d2052e1/langfuse/langchain/CallbackHandler.py#L1558
+func (f *formatting) openAIChatSingle(m map[string]any) OutputMessage {
+	msg := f.openAIChatMessage(m)
+	if msg.Role.IsZero() {
+		msg.Role = RoleAssistant
+	}
+	return msg
+}
+
 // toolCallPart reads {id, function: {name, arguments}}; a gateway may flatten it to {id, name, arguments|args}.
 func toolCallPart(c map[string]any) Part {
 	call := ToolCallRequestPart{Type: PartTypeToolCall, ID: optionalString(c["id"]), Name: stringOf(c["name"])}

pkg/types/aiobservabilitytypes/message_format_openai_chat_test.go

@@ -205,6 +205,30 @@ func TestFormat_OpenAIChat(t *testing.T) {
 			]`,
 			wantOutput: `[]`,
 		},
+		{
+			// Langfuse SDK generation: the history as a list, the first choice's message as the output
+			name:       "SingleMessageOutput_AssistantMessage",
+			input:      `[{"role": "system", "content": "You are a support agent on a phone call."}, {"role": "user", "content": "Hello?"}]`,
+			output:     `{"role": "assistant", "content": "Hi, this is a courtesy call about your order. Do you have a minute?"}`,
+			wantInput:  `[{"role": "system", "parts": [{"type": "text", "content": "You are a support agent on a phone call."}]}, {"role": "user", "parts": [{"type": "text", "content": "Hello?"}]}]`,
+			wantOutput: `[{"role": "assistant", "parts": [{"type": "text", "content": "Hi, this is a courtesy call about your order. Do you have a minute?"}]}]`,
+		},
+		{
+			// a structured output that happens to carry a role key is not a message
+			name:       "ObjectWithRoleKeyButNoContent_KeptAsGeneric",
+			output:     `{"role": "admin", "permissions": ["read", "write"]}`,
+			warnings:   []string{"message format not recognised, kept as generic"},
+			wantInput:  `[]`,
+			wantOutput: `[{"role": "", "parts": [{"type": "generic", "content": "{\"permissions\":[\"read\",\"write\"],\"role\":\"admin\"}"}]}]`,
+		},
+		{
+			// Langfuse LangChain handler: the additional_kwargs of the AIMessage, without a role, as the output
+			// of a tool-calling turn
+			name:       "RolelessToolCallsOutput_AssistantToolCallPart",
+			output:     `{"tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Oslo\"}"}}], "refusal": null}`,
+			wantInput:  `[]`,
+			wantOutput: `[{"role": "assistant", "parts": [{"type": "tool_call", "name": "get_weather", "id": "call_1", "arguments": {"city": "Oslo"}}]}]`,
+		},
 		{
 			name:      "ResponseWithScalarChoice_KeptAsGeneric",
 			output:    `{"choices":["oops",{"finish_reason":"stop","index":1,"message":{"role":"assistant","content":"hi"}}]}`,

pkg/types/aiobservabilitytypes/message_format_openai_responses.go

@@ -2,13 +2,15 @@ package aiobservabilitytypes
 
 import "strings"
 
-// Format: OpenAI Responses, a request {input, instructions} or a response {output, status} of items:
+// Format: OpenAI Responses, a request {input, instructions}, a response {output, status} or a bare list
+// of items:
 // https://github.com/openai/openai-openapi/blob/aeb6a041dc56c011ca9f12127ec898f911100aa0/openapi.yaml#L47597
 // https://github.com/openai/openai-openapi/blob/aeb6a041dc56c011ca9f12127ec898f911100aa0/openapi.yaml#L69691
 // https://github.com/openai/openai-openapi/blob/aeb6a041dc56c011ca9f12127ec898f911100aa0/openapi.yaml#L53834
 // https://github.com/openai/openai-openapi/blob/aeb6a041dc56c011ca9f12127ec898f911100aa0/openapi.yaml#L57596
-// Written by: gateways recording raw Responses bodies, and the raw calls OpenInference records for the
-// Responses API and the OpenAI Agents SDK when its flattened messages are absent.
+// Written by: gateways recording raw Responses bodies, the raw calls OpenInference records for the
+// Responses API and the OpenAI Agents SDK when its flattened messages are absent, and LiteLLM's
+// langfuse_otel callback, the function_call items of a tool-calling turn as a bare list.
 // Handled: message items with input_text, output_text, refusal, input_image and input_file content;
 // function_call and custom_tool_call with their outputs; reasoning summaries; compaction; every other
 // *_call item as a server tool call and *_output item as its response; the status and incomplete_details
@@ -32,6 +34,19 @@ func isResponsesRequest(m map[string]any) bool {
 	return isItem
 }
 
+// isResponsesItems looks for an item only the Responses API has, a tool call or its output, a reasoning
+// or compaction item, all without a role; a list of messages has none.
+func isResponsesItems(list []any) bool {
+	for _, item := range list {
+		m, _ := item.(map[string]any)
+		typ := stringOf(m["type"])
+		if m["role"] == nil && (typ == "reasoning" || typ == "compaction" || typ == "mcp_approval_response" || strings.HasSuffix(typ, "_call") || strings.HasSuffix(typ, "_output")) {
+			return true
+		}
+	}
+	return false
+}
+
 func isResponsesResponse(m map[string]any) bool {
 	if _, ok := m["output"].([]any); !ok {
 		return false

pkg/types/aiobservabilitytypes/message_format_openai_responses_test.go

@@ -73,6 +73,23 @@ func TestFormat_OpenAIResponses(t *testing.T) {
 			wantInput:  `[{"role": "system", "parts": [{"type": "text", "content": "Be brief."}]}, {"role": "user", "parts": [{"type": "text", "content": "Say hi"}]}]`,
 			wantOutput: `[]`,
 		},
+		{
+			// LiteLLM langfuse_otel: the function_call items of a tool-calling turn as a bare list
+			name:       "BareFunctionCallItems_OneAssistantMessage",
+			output:     `[{"id": "chatcmpl-1", "name": "set_stage", "call_id": "call_7", "type": "function_call", "arguments": {"fields": {"stage": "consent"}}}]`,
+			wantInput:  `[]`,
+			wantOutput: `[{"role": "assistant", "parts": [{"type": "tool_call", "name": "set_stage", "id": "call_7", "arguments": {"fields": {"stage": "consent"}}}]}]`,
+		},
+		{
+			name:  "BareItemsWithMessage_ToolRound",
+			input: `[{"type": "message", "role": "user", "content": [{"type": "input_text", "text": "Weather in Oslo?"}]}, {"type": "function_call", "call_id": "call_9", "name": "get_weather", "arguments": "{\"city\":\"Oslo\"}"}, {"type": "function_call_output", "call_id": "call_9", "output": "4C, cloudy"}]`,
+			wantInput: `[
+			  {"role": "user", "parts": [{"type": "text", "content": "Weather in Oslo?"}]},
+			  {"role": "assistant", "parts": [{"type": "tool_call", "name": "get_weather", "id": "call_9", "arguments": {"city": "Oslo"}}]},
+			  {"role": "tool", "parts": [{"type": "tool_call_response", "id": "call_9", "response": "4C, cloudy"}]}
+			]`,
+			wantOutput: `[]`,
+		},
 		{
 			// openinference capture, CreateEmbeddings span: an embeddings request also has input and model
 			name:       "EmbeddingsRequest_NotResponses_KeptAsGeneric",

pkg/types/aiobservabilitytypes/message_format_openinference.go

@@ -4,8 +4,9 @@ import "github.com/SigNoz/signoz/pkg/valuer"
 
 // Format: the flattened llm.input_messages.N.message.* and llm.output_messages.N.message.* keys, the
 // provider-neutral copy of the conversation the OpenInference instrumentations write on LLM spans. A side
-// without them falls back to the raw provider payload the mapper moved onto gen_ai.*.messages; pipecat
-// streams the reply into output.value alone.
+// without them, or whose buckets hold no message, falls back to the raw provider payload the mapper moved
+// onto gen_ai.*.messages; pipecat streams the reply into output.value alone, LiteLLM's OpenInference mode
+// leaves the output bucket empty on a tool-calling turn.
 // https://github.com/Arize-ai/openinference/blob/12f2cc035edfb00b60b656d977f07abfb7a69900/spec/semantic_conventions.md#L77
 // https://github.com/Arize-ai/openinference/blob/12f2cc035edfb00b60b656d977f07abfb7a69900/spec/tool_calling.md#L37
 // https://github.com/Arize-ai/openinference/blob/12f2cc035edfb00b60b656d977f07abfb7a69900/spec/multimodal_attributes.md#L35
@@ -28,15 +29,14 @@ func (f *formatting) openInference() bool {
 		return false
 	}
 	inputLabel, outputLabel := FormatterOpenInference, FormatterOpenInference
-	if len(input) > 0 {
-		f.out.Input = inputMessages(openInferenceMessages(input))
+	if msgs := openInferenceMessages(input); len(msgs) > 0 {
+		f.out.Input = inputMessages(msgs)
 	} else if raw := attribute(f.attributes, GenAIInputMessages); raw != nil {
-		var msgs OutputMessages
 		msgs, inputLabel = f.side(decode(raw), RoleUser)
 		f.out.Input = inputMessages(msgs)
 	}
-	if len(output) > 0 {
-		f.out.Output = openInferenceMessages(output)
+	if msgs := openInferenceMessages(output); len(msgs) > 0 {
+		f.out.Output = msgs
 		if fr := finishReason(stringOf(attribute(f.attributes, openInferenceFinishReason))); fr != nil {
 			for i := range f.out.Output {
 				f.out.Output[i].FinishReason = fr

pkg/types/aiobservabilitytypes/message_format_openinference_test.go

@@ -160,16 +160,30 @@ func TestFormat_OpenInference(t *testing.T) {
 			// which the mapper moves onto gen_ai.output.messages
 			// https://github.com/Arize-ai/openinference/blob/12f2cc035edfb00b60b656d977f07abfb7a69900/python/instrumentation/openinference-instrumentation-pipecat/src/openinference/instrumentation/pipecat/_attributes.py#L430
 			// https://github.com/Arize-ai/openinference/blob/12f2cc035edfb00b60b656d977f07abfb7a69900/python/instrumentation/openinference-instrumentation-pipecat/src/openinference/instrumentation/pipecat/_observer.py#L628
-			name: "OutputKeysAbsent_RawOutputFormatted_Warns",
+			name: "OutputKeysAbsent_RawOutputFormatted",
 			attributes: map[string]any{
 				"llm.input_messages.0.message.role":    "user",
 				"llm.input_messages.0.message.content": "hi",
 				GenAIOutputMessages:                    "Hello there.",
 			},
-			warnings:   []string{"bare assistant text, role assumed", "output formatted as text, input as openinference"},
+			warnings:   []string{"bare assistant text, role assumed"},
 			wantInput:  `[{"role": "user", "parts": [{"type": "text", "content": "hi"}]}]`,
 			wantOutput: `[{"role": "assistant", "parts": [{"type": "text", "content": "Hello there."}]}]`,
 		},
+		{
+			// LiteLLM's OpenInference mode leaves the output bucket empty on a tool-calling turn; the calls are
+			// the Langfuse output the mapper moved onto gen_ai.output.messages
+			name: "EmptyOutputBucket_RawOutputFormatted",
+			attributes: map[string]any{
+				"llm.input_messages.0.message.role":     "user",
+				"llm.input_messages.0.message.content":  "Yes, go ahead.",
+				"llm.output_messages.0.message.role":    "assistant",
+				"llm.output_messages.0.message.content": "",
+				GenAIOutputMessages:                     `[{"id": "chatcmpl-2", "name": "set_stage", "call_id": "call_8", "type": "function_call", "arguments": {"fields": {"stage": "email"}}}]`,
+			},
+			wantInput:  `[{"role": "user", "parts": [{"type": "text", "content": "Yes, go ahead."}]}]`,
+			wantOutput: `[{"role": "assistant", "parts": [{"type": "tool_call", "name": "set_stage", "id": "call_8", "arguments": {"fields": {"stage": "email"}}}]}]`,
+		},
 		{
 			// message.role left unset by some instrumentors: tool_call_id, tool_calls or neither decide it,
 			// as the collector-contrib normalizer does

pkg/types/aiobservabilitytypes/message_format_test.go

@@ -122,6 +122,31 @@ So, the ball costs 5 cents.`,
 			wantInput:  `[]`,
 			wantOutput: `[]`,
 		},
+		{
+			// Langfuse SDK generation: the app passes the request through superjson, the output is the parsed
+			// structured output
+			name:       "SuperJSONEnvelope_Unwrapped_StructuredOutputGeneric",
+			input:      `{"json": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "Extract the main article of the page as JSON."}, {"role": "user", "content": "# Council approves new bike lanes"}], "temperature": 0}}`,
+			output:     `{"hasArticle": true, "article": {"title": "Council approves new bike lanes"}}`,
+			warnings:   []string{"message format not recognised, kept as generic"},
+			wantInput:  `[{"role": "system", "parts": [{"type": "text", "content": "Extract the main article of the page as JSON."}]}, {"role": "user", "parts": [{"type": "text", "content": "# Council approves new bike lanes"}]}]`,
+			wantOutput: `[{"role": "", "parts": [{"type": "generic", "content": "{\"article\":{\"title\":\"Council approves new bike lanes\"},\"hasArticle\":true}"}]}]`,
+		},
+		{
+			name:       "SuperJSONEnvelopeWithMeta_Unwrapped",
+			input:      `{"json": {"messages": [{"role": "user", "content": "What time is it in Oslo?"}]}, "meta": {"values": {"messages.0.at": ["Date"]}}}`,
+			output:     `{"json": {"choices": [{"finish_reason": "stop", "message": {"role": "assistant", "content": "It is 14:05 in Oslo."}}]}, "meta": {"values": {}}}`,
+			wantInput:  `[{"role": "user", "parts": [{"type": "text", "content": "What time is it in Oslo?"}]}]`,
+			wantOutput: `[{"role": "assistant", "parts": [{"type": "text", "content": "It is 14:05 in Oslo."}], "finish_reason": "stop"}]`,
+		},
+		{
+			// a json key beside others is a payload of its own, not the envelope
+			name:       "JSONKeyBesideOthers_NotUnwrapped_Generic",
+			input:      `{"json": {"messages": []}, "schema": "v1"}`,
+			warnings:   []string{"message format not recognised, kept as generic"},
+			wantInput:  `[{"role": "", "parts": [{"type": "generic", "content": "{\"json\":{\"messages\":[]},\"schema\":\"v1\"}"}]}]`,
+			wantOutput: `[]`,
+		},
 		{
 			name:       "NoMessageAttributes_EmptyLists",
 			attributes: map[string]any{GenAIRequestModel: "gpt-4o", GenAIToolName: "get_weather"},

pkg/types/annotationtypes/annotations.go

@@ -1,125 +0,0 @@
-package annotationtypes
-
-import (
-	"encoding/json"
-	"slices"
-
-	"github.com/SigNoz/signoz/pkg/errors"
-	qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-	"github.com/swaggest/jsonschema-go"
-)
-
-var ValidSignals = []telemetrytypes.Signal{telemetrytypes.SignalTraces, telemetrytypes.SignalLogs}
-
-type AnnotationKind struct {
-	valuer.String
-}
-
-var (
-	AnnotationKindDeployment = AnnotationKind{valuer.NewString("deployment")}
-)
-
-func (AnnotationKind) Enum() []any {
-	return []any{
-		AnnotationKindDeployment,
-	}
-}
-
-type Annotations struct {
-	Records              []Annotation `json:"records" required:"true"`
-	Total                int          `json:"total" required:"true"`
-	HistoryAvailableFrom int64        `json:"historyAvailableFrom" required:"true"`
-}
-
-type Annotation struct {
-	Kind AnnotationKind `json:"kind" required:"true"`
-	Spec any            `json:"spec" required:"true"`
-}
-
-type PostableAnnotations struct {
-	Kind    AnnotationKind          `json:"kind" required:"true"`
-	Start   int64                   `json:"start" required:"true"`
-	End     int64                   `json:"end" required:"true"`
-	Signals []telemetrytypes.Signal `json:"signals"`
-	Filter  *qbtypes.Filter         `json:"filter"`
-	Limit   int                     `json:"limit" required:"true"`
-}
-
-var (
-	_ jsonschema.OneOfExposer = Annotation{}
-	_ jsonschema.Preparer     = Annotation{}
-)
-
-// annotationDeployment is the OpenAPI schema for an Annotation with kind=deployment.
-type annotationDeployment struct {
-	Kind AnnotationKind `json:"kind" required:"true"`
-	Spec DeploymentSpec `json:"spec" required:"true"`
-}
-
-func (Annotation) JSONSchemaOneOf() []any {
-	return []any{
-		annotationDeployment{},
-	}
-}
-
-func (Annotation) PrepareJSONSchema(schema *jsonschema.Schema) error {
-	if schema.ExtraProperties == nil {
-		schema.ExtraProperties = map[string]any{}
-	}
-
-	schema.ExtraProperties["x-signoz-discriminator"] = map[string]any{
-		"propertyName": "kind",
-		"mapping": map[string]string{
-			AnnotationKindDeployment.StringValue(): "#/components/schemas/AnnotationtypesAnnotationDeployment",
-		},
-	}
-
-	return nil
-}
-
-func (req *PostableAnnotations) Validate() error {
-	if req == nil {
-		return errors.NewInvalidInputf(errors.CodeInvalidInput, "request is nil")
-	}
-
-	if req.Kind != AnnotationKindDeployment {
-		return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid kind %q: only %q is supported", req.Kind.StringValue(), AnnotationKindDeployment.StringValue())
-	}
-
-	if req.Start <= 0 {
-		return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid start time %d: start must be greater than 0", req.Start)
-	}
-
-	if req.End <= 0 {
-		return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid end time %d: end must be greater than 0", req.End)
-	}
-
-	if req.Start >= req.End {
-		return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid time range: start (%d) must be less than end (%d)", req.Start, req.End)
-	}
-
-	for _, signal := range req.Signals {
-		if !slices.Contains(ValidSignals, signal) {
-			return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid signal %q: must be one of %q, %q", signal.StringValue(), telemetrytypes.SignalTraces.StringValue(), telemetrytypes.SignalLogs.StringValue())
-		}
-	}
-
-	if req.Limit < 1 || req.Limit > 5000 {
-		return errors.NewInvalidInputf(errors.CodeInvalidInput, "limit must be between 1 and 5000")
-	}
-
-	return nil
-}
-
-// UnmarshalJSON validates input immediately after decoding.
-func (req *PostableAnnotations) UnmarshalJSON(data []byte) error {
-	type raw PostableAnnotations
-	var decoded raw
-	if err := json.Unmarshal(data, &decoded); err != nil {
-		return err
-	}
-	*req = PostableAnnotations(decoded)
-	return req.Validate()
-}

pkg/types/annotationtypes/annotations_test.go

@@ -1,159 +0,0 @@
-package annotationtypes
-
-import (
-	"encoding/json"
-	"testing"
-
-	"github.com/SigNoz/signoz/pkg/errors"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/SigNoz/signoz/pkg/valuer"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-)
-
-func TestPostableAnnotations_Validate(t *testing.T) {
-	testCases := []struct {
-		name    string
-		req     *PostableAnnotations
-		wantErr bool
-	}{
-		{
-			name:    "DeploymentKind_BothSignals_Valid",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Signals: []telemetrytypes.Signal{telemetrytypes.SignalTraces, telemetrytypes.SignalLogs}, Limit: 100},
-			wantErr: false,
-		},
-		{
-			name:    "NoSignals_Valid",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Limit: 1},
-			wantErr: false,
-		},
-		{
-			name:    "NilRequest_Rejected",
-			req:     nil,
-			wantErr: true,
-		},
-		{
-			name:    "UnknownKind_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKind{valuer.NewString("incident")}, Start: 1000, End: 2000, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "EmptyKind_Rejected",
-			req:     &PostableAnnotations{Start: 1000, End: 2000, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "StartZero_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 0, End: 2000, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "EndZero_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 0, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "StartAfterEnd_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 3000, End: 2000, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "StartEqualsEnd_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 2000, End: 2000, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "MetricsSignal_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Signals: []telemetrytypes.Signal{telemetrytypes.SignalMetrics}, Limit: 100},
-			wantErr: true,
-		},
-		{
-			name:    "LimitZero_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Limit: 0},
-			wantErr: true,
-		},
-		{
-			name:    "LimitNegative_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Limit: -5},
-			wantErr: true,
-		},
-		{
-			name:    "LimitAboveMax_Rejected",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Limit: 5001},
-			wantErr: true,
-		},
-		{
-			name:    "LimitAtMax_Valid",
-			req:     &PostableAnnotations{Kind: AnnotationKindDeployment, Start: 1000, End: 2000, Limit: 5000},
-			wantErr: false,
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			err := testCase.req.Validate()
-			if !testCase.wantErr {
-				assert.NoError(t, err)
-				return
-			}
-			require.Error(t, err)
-			assert.True(t, errors.Ast(err, errors.TypeInvalidInput))
-		})
-	}
-}
-
-func TestPostableAnnotations_UnmarshalJSON(t *testing.T) {
-	testCases := []struct {
-		name    string
-		body    string
-		want    PostableAnnotations
-		wantErr bool
-	}{
-		{
-			name: "FullBody_Decoded",
-			body: `{"kind":"deployment","start":1000,"end":2000,"signals":["traces"],"filter":{"expression":"service.name = 'cart'"},"limit":50}`,
-			want: PostableAnnotations{
-				Kind:    AnnotationKindDeployment,
-				Start:   1000,
-				End:     2000,
-				Signals: []telemetrytypes.Signal{telemetrytypes.SignalTraces},
-				Limit:   50,
-			},
-			wantErr: false,
-		},
-		{
-			name:    "InvalidKind_ValidatedOnDecode",
-			body:    `{"kind":"release","start":1000,"end":2000,"limit":50}`,
-			wantErr: true,
-		},
-		{
-			name:    "MissingLimit_ValidatedOnDecode",
-			body:    `{"kind":"deployment","start":1000,"end":2000}`,
-			wantErr: true,
-		},
-		{
-			name:    "MalformedJSON_Rejected",
-			body:    `{"kind":"deployment",`,
-			wantErr: true,
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			var got PostableAnnotations
-			err := json.Unmarshal([]byte(testCase.body), &got)
-			if testCase.wantErr {
-				assert.Error(t, err)
-				return
-			}
-			require.NoError(t, err)
-			assert.Equal(t, testCase.want.Kind, got.Kind)
-			assert.Equal(t, testCase.want.Start, got.Start)
-			assert.Equal(t, testCase.want.End, got.End)
-			assert.Equal(t, testCase.want.Signals, got.Signals)
-			assert.Equal(t, testCase.want.Limit, got.Limit)
-			require.NotNil(t, got.Filter)
-			assert.Equal(t, "service.name = 'cart'", got.Filter.Expression)
-		})
-	}
-}

pkg/types/annotationtypes/deployments.go

@@ -1,140 +0,0 @@
-package annotationtypes
-
-import (
-	"crypto/sha256"
-	"encoding/hex"
-	"encoding/json"
-	"sort"
-	"strings"
-
-	"github.com/SigNoz/signoz/pkg/errors"
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-)
-
-var deploymentIdentityKeys = []string{"service.name", "deployment.environment", "deployment.environment.name", "service.version"}
-
-type DeploymentSpec struct {
-	ID                 string                `json:"id" required:"true"`
-	Signal             telemetrytypes.Signal `json:"signal" required:"true"`
-	Service            string                `json:"service" required:"true"`
-	Environment        string                `json:"environment" required:"true"`
-	Version            string                `json:"version" required:"true"`
-	FirstSeen          int64                 `json:"firstSeen" required:"true"`
-	TraceID            string                `json:"traceId" required:"true"`
-	SpanID             string                `json:"spanId" required:"true"`
-	LogID              string                `json:"logId" required:"true"`
-	ResourceAttributes map[string]string     `json:"resourceAttributes" required:"true"`
-}
-
-// DeploymentRow is the storage shape one marker row is scanned into; FirstSeen is epoch ms.
-type DeploymentRow struct {
-	Signal      telemetrytypes.Signal
-	Service     string
-	Environment string
-	Version     string
-	FirstSeen   int64
-	TraceID     string
-	SpanID      string
-	LogID       string
-	Labels      string
-}
-
-func NewDeploymentSpecFromRow(row DeploymentRow) (DeploymentSpec, error) {
-	resourceAttributes := map[string]string{}
-	if row.Labels != "" {
-		if err := json.Unmarshal([]byte(row.Labels), &resourceAttributes); err != nil {
-			return DeploymentSpec{}, errors.WrapInternalf(err, errors.CodeInternal, "failed to parse resource labels for %s/%s/%s", row.Service, row.Environment, row.Version)
-		}
-	}
-	for _, key := range deploymentIdentityKeys {
-		delete(resourceAttributes, key)
-	}
-
-	return DeploymentSpec{
-		ID:                 NewDeploymentID(row.Service, row.Environment, row.Version),
-		Signal:             row.Signal,
-		Service:            row.Service,
-		Environment:        row.Environment,
-		Version:            row.Version,
-		FirstSeen:          row.FirstSeen,
-		TraceID:            row.TraceID,
-		SpanID:             row.SpanID,
-		LogID:              row.LogID,
-		ResourceAttributes: resourceAttributes,
-	}, nil
-}
-
-func NewDeploymentSpecsFromRows(rows []DeploymentRow) ([]DeploymentSpec, error) {
-	specs := make([]DeploymentSpec, 0, len(rows))
-	for _, row := range rows {
-		spec, err := NewDeploymentSpecFromRow(row)
-		if err != nil {
-			return nil, err
-		}
-		specs = append(specs, spec)
-	}
-	return specs, nil
-}
-
-func NewDeploymentID(service, environment, version string) string {
-	sum := sha256.Sum256([]byte(strings.Join([]string{service, environment, version}, "\x00")))
-	return "dep_" + hex.EncodeToString(sum[:4])
-}
-
-// MergeDeployments keeps the earliest FirstSeen per (service, environment, version); on a tie the first input wins.
-func MergeDeployments(specs ...[]DeploymentSpec) []DeploymentSpec {
-	byTuple := map[string]DeploymentSpec{}
-	for _, list := range specs {
-		for _, spec := range list {
-			key := strings.Join([]string{spec.Service, spec.Environment, spec.Version}, "\x00")
-			existing, ok := byTuple[key]
-			if !ok || spec.FirstSeen < existing.FirstSeen {
-				byTuple[key] = spec
-			}
-		}
-	}
-
-	merged := make([]DeploymentSpec, 0, len(byTuple))
-	for _, spec := range byTuple {
-		merged = append(merged, spec)
-	}
-	sort.Slice(merged, func(i, j int) bool {
-		if merged[i].FirstSeen != merged[j].FirstSeen {
-			return merged[i].FirstSeen < merged[j].FirstSeen
-		}
-		if merged[i].Service != merged[j].Service {
-			return merged[i].Service < merged[j].Service
-		}
-		if merged[i].Environment != merged[j].Environment {
-			return merged[i].Environment < merged[j].Environment
-		}
-		return merged[i].Version < merged[j].Version
-	})
-	return merged
-}
-
-// EarliestHistoryEdge returns the smallest non-zero edge, 0 when no signal has history.
-func EarliestHistoryEdge(edges ...int64) int64 {
-	var earliest int64
-	for _, edge := range edges {
-		if edge != 0 && (earliest == 0 || edge < earliest) {
-			earliest = edge
-		}
-	}
-	return earliest
-}
-
-func NewAnnotationsFromDeployments(specs []DeploymentSpec, total int, historyAvailableFrom int64) *Annotations {
-	records := make([]Annotation, 0, len(specs))
-	for _, spec := range specs {
-		records = append(records, Annotation{Kind: AnnotationKindDeployment, Spec: spec})
-	}
-	return &Annotations{Records: records, Total: total, HistoryAvailableFrom: historyAvailableFrom}
-}
-
-func ApplyLimit(specs []DeploymentSpec, limit int) ([]DeploymentSpec, int) {
-	if limit >= len(specs) {
-		return specs, len(specs)
-	}
-	return specs[:limit], len(specs)
-}

pkg/types/annotationtypes/deployments_test.go

@@ -1,204 +0,0 @@
-package annotationtypes
-
-import (
-	"testing"
-
-	"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-)
-
-func TestNewDeploymentID(t *testing.T) {
-	testCases := []struct {
-		name    string
-		service string
-		env     string
-		version string
-		wantEq  bool
-	}{
-		{name: "SameTuple_SameID", service: "checkout", env: "prod", version: "1.2.0", wantEq: true},
-		{name: "DifferentVersion_DifferentID", service: "checkout", env: "prod", version: "1.2.1", wantEq: false},
-		{name: "DifferentEnv_DifferentID", service: "checkout", env: "staging", version: "1.2.0", wantEq: false},
-		{name: "DifferentService_DifferentID", service: "cart", env: "prod", version: "1.2.0", wantEq: false},
-	}
-
-	base := NewDeploymentID("checkout", "prod", "1.2.0")
-	require.Len(t, base, len("dep_")+8)
-	assert.True(t, len(base) > 4 && base[:4] == "dep_")
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			got := NewDeploymentID(testCase.service, testCase.env, testCase.version)
-			assert.Equal(t, testCase.wantEq, got == base)
-		})
-	}
-}
-
-func TestNewDeploymentSpecFromRow(t *testing.T) {
-	testCases := []struct {
-		name      string
-		row       DeploymentRow
-		wantAttrs map[string]string
-		wantErr   bool
-	}{
-		{
-			name: "IdentityKeysStripped_OthersKept",
-			row: DeploymentRow{
-				Signal: telemetrytypes.SignalTraces, Service: "checkout", Environment: "prod", Version: "1.2.0", FirstSeen: 1700000000123,
-				TraceID: "abc", SpanID: "def",
-				Labels: `{"service.name":"checkout","deployment.environment":"prod","deployment.environment.name":"prod","service.version":"1.2.0","k8s.namespace.name":"shop","host.name":"node-1"}`,
-			},
-			wantAttrs: map[string]string{"k8s.namespace.name": "shop", "host.name": "node-1"},
-		},
-		{
-			name:      "EmptyLabels_EmptyMap",
-			row:       DeploymentRow{Signal: telemetrytypes.SignalLogs, Service: "cart", Environment: "prod", Version: "2.0.0", LogID: "log-1"},
-			wantAttrs: map[string]string{},
-		},
-		{
-			name:    "MalformedLabels_Error",
-			row:     DeploymentRow{Service: "cart", Environment: "prod", Version: "2.0.0", Labels: `{"service.name":`},
-			wantErr: true,
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			spec, err := NewDeploymentSpecFromRow(testCase.row)
-			if testCase.wantErr {
-				assert.Error(t, err)
-				return
-			}
-			require.NoError(t, err)
-			assert.Equal(t, NewDeploymentID(testCase.row.Service, testCase.row.Environment, testCase.row.Version), spec.ID)
-			assert.Equal(t, testCase.row.Signal, spec.Signal)
-			assert.Equal(t, testCase.row.FirstSeen, spec.FirstSeen)
-			assert.Equal(t, testCase.row.TraceID, spec.TraceID)
-			assert.Equal(t, testCase.row.SpanID, spec.SpanID)
-			assert.Equal(t, testCase.row.LogID, spec.LogID)
-			assert.Equal(t, testCase.wantAttrs, spec.ResourceAttributes)
-		})
-	}
-}
-
-func TestMergeDeployments(t *testing.T) {
-	traces := []DeploymentSpec{
-		{Service: "checkout", Environment: "prod", Version: "1.2.0", FirstSeen: 200, Signal: telemetrytypes.SignalTraces, TraceID: "t1"},
-		{Service: "cart", Environment: "prod", Version: "3.0.0", FirstSeen: 100, Signal: telemetrytypes.SignalTraces, TraceID: "t2"},
-		{Service: "api", Environment: "prod", Version: "9.0.0", FirstSeen: 300, Signal: telemetrytypes.SignalTraces, TraceID: "t3"},
-	}
-	logs := []DeploymentSpec{
-		{Service: "checkout", Environment: "prod", Version: "1.2.0", FirstSeen: 150, Signal: telemetrytypes.SignalLogs, LogID: "l1"},
-		{Service: "cart", Environment: "prod", Version: "3.0.0", FirstSeen: 100, Signal: telemetrytypes.SignalLogs, LogID: "l2"},
-		{Service: "worker", Environment: "prod", Version: "0.1.0", FirstSeen: 300, Signal: telemetrytypes.SignalLogs, LogID: "l3"},
-	}
-
-	testCases := []struct {
-		name         string
-		inputs       [][]DeploymentSpec
-		wantOrder    []string
-		wantSignalOf map[string]telemetrytypes.Signal
-	}{
-		{
-			name:      "EarliestWins_TieKeepsFirstInput_SortedByFirstSeenThenService",
-			inputs:    [][]DeploymentSpec{traces, logs},
-			wantOrder: []string{"cart", "checkout", "api", "worker"},
-			wantSignalOf: map[string]telemetrytypes.Signal{
-				"cart":     telemetrytypes.SignalTraces,
-				"checkout": telemetrytypes.SignalLogs,
-				"api":      telemetrytypes.SignalTraces,
-				"worker":   telemetrytypes.SignalLogs,
-			},
-		},
-		{
-			name:         "SingleInput_Sorted",
-			inputs:       [][]DeploymentSpec{traces},
-			wantOrder:    []string{"cart", "checkout", "api"},
-			wantSignalOf: map[string]telemetrytypes.Signal{"cart": telemetrytypes.SignalTraces, "checkout": telemetrytypes.SignalTraces, "api": telemetrytypes.SignalTraces},
-		},
-		{
-			name:         "NoInput_Empty",
-			inputs:       nil,
-			wantOrder:    []string{},
-			wantSignalOf: map[string]telemetrytypes.Signal{},
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			merged := MergeDeployments(testCase.inputs...)
-			require.Len(t, merged, len(testCase.wantOrder))
-			for i, spec := range merged {
-				assert.Equal(t, testCase.wantOrder[i], spec.Service)
-				assert.Equal(t, testCase.wantSignalOf[spec.Service], spec.Signal)
-			}
-		})
-	}
-}
-
-func TestApplyLimit(t *testing.T) {
-	specs := []DeploymentSpec{{Service: "a"}, {Service: "b"}, {Service: "c"}}
-
-	testCases := []struct {
-		name      string
-		limit     int
-		wantLen   int
-		wantTotal int
-	}{
-		{name: "LimitBelowCount_Truncated", limit: 2, wantLen: 2, wantTotal: 3},
-		{name: "LimitEqualsCount_All", limit: 3, wantLen: 3, wantTotal: 3},
-		{name: "LimitAboveCount_All", limit: 100, wantLen: 3, wantTotal: 3},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			page, total := ApplyLimit(specs, testCase.limit)
-			assert.Len(t, page, testCase.wantLen)
-			assert.Equal(t, testCase.wantTotal, total)
-		})
-	}
-}
-
-func TestEarliestHistoryEdge(t *testing.T) {
-	testCases := []struct {
-		name  string
-		edges []int64
-		want  int64
-	}{
-		{name: "BothZero_Zero", edges: []int64{0, 0}, want: 0},
-		{name: "OneZero_OtherWins", edges: []int64{0, 500}, want: 500},
-		{name: "BothSet_Smallest", edges: []int64{900, 500}, want: 500},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			assert.Equal(t, testCase.want, EarliestHistoryEdge(testCase.edges...))
-		})
-	}
-}
-
-func TestNewDeploymentSpecsFromRows(t *testing.T) {
-	specs, err := NewDeploymentSpecsFromRows([]DeploymentRow{
-		{Service: "a", Environment: "prod", Version: "1", Labels: `{"service.name":"a","host.name":"h1"}`},
-		{Service: "b", Environment: "prod", Version: "2"},
-	})
-	require.NoError(t, err)
-	require.Len(t, specs, 2)
-	assert.Equal(t, map[string]string{"host.name": "h1"}, specs[0].ResourceAttributes)
-	assert.Equal(t, map[string]string{}, specs[1].ResourceAttributes)
-
-	_, err = NewDeploymentSpecsFromRows([]DeploymentRow{{Labels: `{`}})
-	assert.Error(t, err)
-}
-
-func TestNewAnnotationsFromDeployments(t *testing.T) {
-	got := NewAnnotationsFromDeployments([]DeploymentSpec{{Service: "a"}}, 7, 1234)
-	require.Len(t, got.Records, 1)
-	assert.Equal(t, AnnotationKindDeployment, got.Records[0].Kind)
-	assert.Equal(t, 7, got.Total)
-	assert.Equal(t, int64(1234), got.HistoryAvailableFrom)
-
-	empty := NewAnnotationsFromDeployments(nil, 0, 0)
-	assert.NotNil(t, empty.Records)
-	assert.Len(t, empty.Records, 0)
-}

tests/integration/testdata/annotations/edge_logs.jsonl

@@ -1,2 +0,0 @@
-{ "timestamp": "2026-01-01T01:00:00.000000Z", "resources": { "service.name": "checkout", "deployment.environment": "prod", "service.version": "1.0.0" }, "attributes": {}, "body": "checkout 1.0.0 started", "severity_text": "INFO" }
-{ "timestamp": "2026-01-01T02:39:30.000000Z", "resources": { "service.name": "checkout", "deployment.environment": "prod", "service.version": "1.1.0" }, "attributes": {}, "body": "checkout 1.1.0 started", "severity_text": "INFO" }

tests/integration/testdata/annotations/edge_traces.jsonl

@@ -1,2 +0,0 @@
-{"timestamp": "2026-01-01T00:00:00.000000Z", "duration": "PT1.0S", "trace_id": "b0000000000000000000000000000001", "span_id": "b000000000000001", "parent_span_id": "", "name": "GET /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "checkout", "deployment.environment": "prod", "service.version": "1.0.0", "host.name": "ip-10-0-1-23", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/checkout"}}
-{"timestamp": "2026-01-01T02:40:00.000000Z", "duration": "PT1.2S", "trace_id": "b0000000000000000000000000000002", "span_id": "b000000000000002", "parent_span_id": "", "name": "GET /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "checkout", "deployment.environment": "prod", "service.version": "1.1.0", "host.name": "ip-10-0-1-23", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/checkout"}}

tests/integration/testdata/annotations/logs.jsonl

@@ -1,4 +0,0 @@
-{ "timestamp": "2026-01-01T00:00:00.000000Z", "resources": { "service.name": "checkout", "deployment.environment": "prod", "service.version": "1.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a" }, "attributes": { "code.file": "checkout.py" }, "body": "checkout 1.0.0 started", "severity_text": "INFO" }
-{ "timestamp": "2026-01-01T02:39:30.000000Z", "resources": { "service.name": "checkout", "deployment.environment": "prod", "service.version": "1.1.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a" }, "attributes": { "code.file": "checkout.py" }, "body": "checkout 1.1.0 started", "severity_text": "INFO" }
-{ "timestamp": "2026-01-01T02:44:00.000000Z", "resources": { "service.name": "worker", "deployment.environment": "prod", "service.version": "3.0.0", "k8s.namespace.name": "jobs", "custom.team": "payments", "signoz.workspace.key.id": "key-b" }, "attributes": { "code.file": "worker.py" }, "body": "worker 3.0.0 started", "severity_text": "INFO" }
-{ "timestamp": "2026-01-01T02:47:00.000000Z", "resources": { "service.name": "api", "deployment.environment": "prod", "service.version": "6.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-b" }, "attributes": { "code.file": "api.py" }, "body": "api 6.0.0 started", "severity_text": "INFO" }

tests/integration/testdata/annotations/traces.jsonl

@@ -1,10 +0,0 @@
-{"timestamp": "2026-01-01T00:00:00.000000Z", "duration": "PT1.0S", "trace_id": "a0000000000000000000000000000001", "span_id": "a000000000000001", "parent_span_id": "", "name": "GET /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "checkout", "deployment.environment": "prod", "service.version": "1.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-23", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/checkout"}}
-{"timestamp": "2026-01-01T01:00:00.000000Z", "duration": "PT1.2S", "trace_id": "a0000000000000000000000000000002", "span_id": "a000000000000002", "parent_span_id": "", "name": "GET /legacy", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "legacy", "deployment.environment": "prod", "service.version": "0.5.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-24", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/legacy"}}
-{"timestamp": "2026-01-01T01:40:00.000000Z", "duration": "PT1.4S", "trace_id": "a0000000000000000000000000000003", "span_id": "a000000000000003", "parent_span_id": "", "name": "GET /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "checkout", "deployment.environment": "prod", "service.version": "1.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-23", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/checkout"}}
-{"timestamp": "2026-01-01T02:40:00.000000Z", "duration": "PT1.0S", "trace_id": "a0000000000000000000000000000004", "span_id": "a000000000000004", "parent_span_id": "", "name": "GET /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "checkout", "deployment.environment": "prod", "service.version": "1.1.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-23", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/checkout"}}
-{"timestamp": "2026-01-01T02:42:00.000000Z", "duration": "PT1.2S", "trace_id": "a0000000000000000000000000000005", "span_id": "a000000000000005", "parent_span_id": "", "name": "GET /cart", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "cart", "deployment.environment": "prod", "service.version": "2.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-b", "host.name": "ip-10-0-1-25", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/cart"}}
-{"timestamp": "2026-01-01T02:44:00.000000Z", "duration": "PT1.4S", "trace_id": "a0000000000000000000000000000006", "span_id": "a000000000000006", "parent_span_id": "", "name": "run job", "kind": 1, "status_code": 1, "status_message": "", "resources": {"service.name": "worker", "deployment.environment": "prod", "service.version": "3.0.0", "k8s.namespace.name": "jobs", "custom.team": "payments", "signoz.workspace.key.id": "key-b", "host.name": "ip-10-0-2-10", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "job"}}
-{"timestamp": "2026-01-01T02:46:00.000000Z", "duration": "PT1.0S", "trace_id": "a0000000000000000000000000000007", "span_id": "a000000000000007", "parent_span_id": "", "name": "GET /tie", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "tieA", "deployment.environment": "prod", "service.version": "4.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-26", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/tie"}}
-{"timestamp": "2026-01-01T02:46:00.000000Z", "duration": "PT1.2S", "trace_id": "a0000000000000000000000000000008", "span_id": "a000000000000008", "parent_span_id": "", "name": "GET /tie", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "tieB", "deployment.environment": "prod", "service.version": "4.0.0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-27", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/tie"}}
-{"timestamp": "2026-01-01T02:48:00.000000Z", "duration": "PT1.4S", "trace_id": "a0000000000000000000000000000009", "span_id": "a000000000000009", "parent_span_id": "", "name": "GET /special", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "sp`ec'ial", "deployment.environment": "pr$0d", "service.version": "5.0.0$0", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-28", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/special"}}
-{"timestamp": "2026-01-01T02:50:00.000000Z", "duration": "PT1.0S", "trace_id": "a000000000000000000000000000000a", "span_id": "a00000000000000a", "parent_span_id": "", "name": "GET /noversion", "kind": 2, "status_code": 1, "status_message": "", "resources": {"service.name": "noversion", "deployment.environment": "prod", "service.version": "", "k8s.namespace.name": "shop", "custom.team": "payments", "signoz.workspace.key.id": "key-a", "host.name": "ip-10-0-1-29", "os.type": "linux"}, "attributes": {"net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "GET", "http.response.status_code": "200", "http.request.path": "/noversion"}}

tests/integration/tests/annotations/01_deployments.py

@@ -1,418 +0,0 @@
-import re
-from datetime import UTC, datetime, timedelta
-from http import HTTPStatus
-
-import pytest
-import requests
-
-from fixtures import types
-from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
-from fixtures.fs import get_testdata_file_path
-from fixtures.logs import Logs
-from fixtures.traces import Traces
-
-ENDPOINT = "/api/v1/annotations"
-IDENTITY_KEYS = ("service.name", "deployment.environment", "deployment.environment.name", "service.version")
-# load_from_file shifts the earliest dataset row (the baseline tuple) to now - BASELINE_OFFSET.
-BASELINE_OFFSET = timedelta(hours=3)
-ALL_MARKERS = {("checkout", "1.1.0"), ("cart", "2.0.0"), ("worker", "3.0.0"), ("tieA", "4.0.0"), ("tieB", "4.0.0"), ("api", "6.0.0"), ("sp`ec'ial", "5.0.0$0")}
-
-
-def test_deployments_empty_tables(
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,  # pylint: disable=unused-argument
-    insert_logs,  # pylint: disable=unused-argument
-) -> None:
-    """No telemetry at all: 200 with no records and no history edge."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50},
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.OK, response.text
-    assert response.json()["data"] == {"records": [], "total": 0, "historyAvailableFrom": 0}
-
-
-def test_deployments_accuracy(
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,
-    insert_logs,
-) -> None:
-    """Seed both signals; assert the full record contract, exact first-seen times,
-    deep-link ids, stripped identity keys, ordering, merge winners and the history edge."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    traces = Traces.load_from_file(get_testdata_file_path("annotations/traces.jsonl"), base_time=now - BASELINE_OFFSET)
-    logs = Logs.load_from_file(get_testdata_file_path("annotations/logs.jsonl"), base_time=now - BASELINE_OFFSET)
-    insert_traces(traces)
-    insert_logs(logs)
-    span_of: dict[tuple[str, str], Traces] = {}
-    for span in sorted(traces, key=lambda span: span.timestamp):
-        span_of.setdefault((span.resources_string["service.name"], span.resources_string["service.version"]), span)
-    log_of: dict[tuple[str, str], Logs] = {}
-    for log in sorted(logs, key=lambda log: log.timestamp):
-        log_of.setdefault((log.resources_string["service.name"], log.resources_string["service.version"]), log)
-
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50},
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.OK, response.text
-    data = response.json()["data"]
-
-    # Ordered by firstSeen then service. checkout 1.1.0: logs 30s before traces, logs wins.
-    # worker 3.0.0: same timestamp on both signals, traces wins. tieA/tieB: same timestamp.
-    # checkout 1.0.0 (oldest bucket), legacy 0.5.0 (before start), noversion (empty version) are not markers.
-    expected = [
-        ("checkout", "1.1.0", "logs"),
-        ("cart", "2.0.0", "traces"),
-        ("worker", "3.0.0", "traces"),
-        ("tieA", "4.0.0", "traces"),
-        ("tieB", "4.0.0", "traces"),
-        ("api", "6.0.0", "logs"),
-        ("sp`ec'ial", "5.0.0$0", "traces"),
-    ]
-    assert [(r["spec"]["service"], r["spec"]["version"], r["spec"]["signal"]) for r in data["records"]] == expected
-    assert data["total"] == len(expected)
-    assert data["historyAvailableFrom"] == (int((now - BASELINE_OFFSET).timestamp()) // 1800 * 1800 + 1800) * 1000
-
-    for record in data["records"]:
-        assert record["kind"] == "deployment"
-        spec = record["spec"]
-        for field in ("id", "signal", "service", "environment", "version", "firstSeen", "traceId", "spanId", "logId", "resourceAttributes"):
-            assert field in spec, f"missing {field} in {spec!r}"
-        assert re.fullmatch(r"dep_[0-9a-f]{8}", spec["id"])
-        key = (spec["service"], spec["version"])
-        if spec["signal"] == "traces":
-            span = span_of[key]
-            assert spec["firstSeen"] == int(span.timestamp.timestamp() * 1000)
-            assert (spec["traceId"], spec["spanId"], spec["logId"]) == (span.trace_id, span.span_id, "")
-            resources = span.resources_string
-        else:
-            log = log_of[key]
-            assert spec["firstSeen"] == int(log.timestamp) // 1_000_000
-            assert (spec["traceId"], spec["spanId"], spec["logId"]) == ("", "", log.id)
-            resources = log.resources_string
-        assert spec["environment"] == resources["deployment.environment"]
-        assert spec["resourceAttributes"] == {k: v for k, v in resources.items() if k not in IDENTITY_KEYS}
-
-    assert len({r["spec"]["id"] for r in data["records"]}) == len(expected)
-
-
-@pytest.mark.parametrize(
-    ("start_anchor", "start_offset", "end_anchor", "end_offset", "expected"),
-    [
-        pytest.param("cart", 0, "now", 0, ALL_MARKERS - {("checkout", "1.1.0")}, id="start_at_first_row_inclusive"),
-        pytest.param("cart", 1, "now", 0, ALL_MARKERS - {("checkout", "1.1.0"), ("cart", "2.0.0")}, id="start_after_first_row"),
-        pytest.param("baseline", 7200, "cart", 0, {("checkout", "1.1.0"), ("cart", "2.0.0")}, id="end_at_first_row_inclusive"),
-        pytest.param("baseline", 7200, "cart", -1, {("checkout", "1.1.0")}, id="end_before_first_row"),
-        pytest.param("baseline", -60, "baseline", 60, set(), id="oldest_bucket_only"),
-        pytest.param("baseline", -7200, "baseline", -3600, set(), id="window_before_data"),
-        pytest.param("now", -300, "now", 0, set(), id="window_after_data"),
-    ],
-)
-def test_deployments_exclusions(  # pylint: disable=too-many-arguments,too-many-positional-arguments
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,
-    insert_logs,
-    start_anchor: str,
-    start_offset: int,
-    end_anchor: str,
-    end_offset: int,
-    expected: set[tuple[str, str]],
-) -> None:
-    """Window bounds bind exactly on first_seen (inclusive both ends). Rows in the oldest
-    bucket, before start, or outside the window never produce markers, and the history
-    edge is reported even when nothing matches."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    traces = Traces.load_from_file(get_testdata_file_path("annotations/traces.jsonl"), base_time=now - BASELINE_OFFSET)
-    logs = Logs.load_from_file(get_testdata_file_path("annotations/logs.jsonl"), base_time=now - BASELINE_OFFSET)
-    insert_traces(traces)
-    insert_logs(logs)
-
-    anchors = {"cart": next(span.timestamp for span in traces if span.resources_string["service.name"] == "cart"), "baseline": now - BASELINE_OFFSET, "now": now}
-    start = anchors[start_anchor] + timedelta(seconds=start_offset)
-    end = anchors[end_anchor] + timedelta(seconds=end_offset)
-
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={"kind": "deployment", "start": int(start.timestamp() * 1000), "end": int(end.timestamp() * 1000), "limit": 50},
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.OK, response.text
-    data = response.json()["data"]
-    assert {(r["spec"]["service"], r["spec"]["version"]) for r in data["records"]} == expected
-    assert data["total"] == len(expected)
-    assert data["historyAvailableFrom"] == (int((now - BASELINE_OFFSET).timestamp()) // 1800 * 1800 + 1800) * 1000
-
-
-@pytest.mark.parametrize(
-    ("signals", "expected"),
-    [
-        pytest.param(None, {("checkout", "1.1.0", "logs"), ("cart", "2.0.0", "traces"), ("worker", "3.0.0", "traces"), ("tieA", "4.0.0", "traces"), ("tieB", "4.0.0", "traces"), ("api", "6.0.0", "logs"), ("sp`ec'ial", "5.0.0$0", "traces")}, id="unset_both"),
-        pytest.param([], {("checkout", "1.1.0", "logs"), ("cart", "2.0.0", "traces"), ("worker", "3.0.0", "traces"), ("tieA", "4.0.0", "traces"), ("tieB", "4.0.0", "traces"), ("api", "6.0.0", "logs"), ("sp`ec'ial", "5.0.0$0", "traces")}, id="empty_list_both"),
-        pytest.param(["traces"], {("checkout", "1.1.0", "traces"), ("cart", "2.0.0", "traces"), ("worker", "3.0.0", "traces"), ("tieA", "4.0.0", "traces"), ("tieB", "4.0.0", "traces"), ("sp`ec'ial", "5.0.0$0", "traces")}, id="traces_only"),
-        pytest.param(["logs"], {("checkout", "1.1.0", "logs"), ("worker", "3.0.0", "logs"), ("api", "6.0.0", "logs")}, id="logs_only"),
-        pytest.param(["logs", "traces"], {("checkout", "1.1.0", "logs"), ("cart", "2.0.0", "traces"), ("worker", "3.0.0", "traces"), ("tieA", "4.0.0", "traces"), ("tieB", "4.0.0", "traces"), ("api", "6.0.0", "logs"), ("sp`ec'ial", "5.0.0$0", "traces")}, id="explicit_both"),
-    ],
-)
-def test_deployments_signals(  # pylint: disable=too-many-arguments,too-many-positional-arguments
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,
-    insert_logs,
-    signals: list[str] | None,
-    expected: set[tuple[str, str, str]],
-) -> None:
-    """Signal selection: a single signal only ever reports that signal and its ids; the
-    deployment id of a tuple is the same whichever signal reports it."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    traces = Traces.load_from_file(get_testdata_file_path("annotations/traces.jsonl"), base_time=now - BASELINE_OFFSET)
-    logs = Logs.load_from_file(get_testdata_file_path("annotations/logs.jsonl"), base_time=now - BASELINE_OFFSET)
-    insert_traces(traces)
-    insert_logs(logs)
-
-    body: dict = {"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50}
-    if signals is not None:
-        body["signals"] = signals
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(signoz.self.host_configs["8080"].get(ENDPOINT), headers={"authorization": f"Bearer {token}"}, json=body, timeout=5)
-    assert response.status_code == HTTPStatus.OK, response.text
-    data = response.json()["data"]
-    assert {(r["spec"]["service"], r["spec"]["version"], r["spec"]["signal"]) for r in data["records"]} == expected
-    assert data["total"] == len(expected)
-    for record in data["records"]:
-        spec = record["spec"]
-        if spec["signal"] == "traces":
-            assert spec["traceId"] and spec["spanId"] and spec["logId"] == ""
-        else:
-            assert spec["logId"] and spec["traceId"] == "" and spec["spanId"] == ""
-
-    # Same tuple, other signal: same deployment id.
-    other = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={**body, "signals": ["traces"] if signals == ["logs"] else ["logs"]},
-        timeout=5,
-    )
-    assert other.status_code == HTTPStatus.OK, other.text
-    ids = {(r["spec"]["service"], r["spec"]["version"]): r["spec"]["id"] for r in data["records"]}
-    for record in other.json()["data"]["records"]:
-        key = (record["spec"]["service"], record["spec"]["version"])
-        if key in ids:
-            assert record["spec"]["id"] == ids[key]
-
-
-@pytest.mark.parametrize(
-    ("signals", "edge_offset"),
-    [
-        pytest.param(None, timedelta(hours=3), id="both_earliest_edge"),
-        pytest.param(["traces"], timedelta(hours=3), id="traces_edge"),
-        pytest.param(["logs"], timedelta(hours=2), id="logs_edge"),
-    ],
-)
-def test_deployments_history_edge(  # pylint: disable=too-many-arguments,too-many-positional-arguments
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,
-    insert_logs,
-    signals: list[str] | None,
-    edge_offset: timedelta,
-) -> None:
-    """historyAvailableFrom is the bucket after the oldest resource bucket of the queried
-    signals; with both signals it is the earlier of the two edges."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    insert_traces(Traces.load_from_file(get_testdata_file_path("annotations/edge_traces.jsonl"), base_time=now - timedelta(hours=3)))
-    insert_logs(Logs.load_from_file(get_testdata_file_path("annotations/edge_logs.jsonl"), base_time=now - timedelta(hours=2)))
-
-    body: dict = {"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50}
-    if signals is not None:
-        body["signals"] = signals
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(signoz.self.host_configs["8080"].get(ENDPOINT), headers={"authorization": f"Bearer {token}"}, json=body, timeout=5)
-    assert response.status_code == HTTPStatus.OK, response.text
-    data = response.json()["data"]
-    assert data["historyAvailableFrom"] == (int((now - edge_offset).timestamp()) // 1800 * 1800 + 1800) * 1000
-    assert {(r["spec"]["service"], r["spec"]["version"]) for r in data["records"]} == {("checkout", "1.1.0")}
-
-
-@pytest.mark.parametrize(
-    ("expression", "expected"),
-    [
-        pytest.param("k8s.namespace.name = 'jobs'", {("worker", "3.0.0")}, id="eq"),
-        pytest.param("k8s.namespace.name IN ('jobs', 'shop')", ALL_MARKERS, id="in"),
-        pytest.param("k8s.namespace.name NOT IN ('shop')", {("worker", "3.0.0")}, id="not_in"),
-        pytest.param("service.name CONTAINS 'tie'", {("tieA", "4.0.0"), ("tieB", "4.0.0")}, id="contains"),
-        pytest.param("service.name LIKE 'tie%'", {("tieA", "4.0.0"), ("tieB", "4.0.0")}, id="like"),
-        pytest.param("k8s.namespace.name = 'shop' AND service.name = 'cart'", {("cart", "2.0.0")}, id="and"),
-        pytest.param("service.name = 'cart' OR service.name = 'api'", {("cart", "2.0.0"), ("api", "6.0.0")}, id="or_across_signals"),
-        pytest.param("deployment.environment = 'pr$0d'", {("sp`ec'ial", "5.0.0$0")}, id="dollar_in_value"),
-        pytest.param('service.name = "sp`ec\'ial"', {("sp`ec'ial", "5.0.0$0")}, id="backtick_quote_in_value"),
-        pytest.param("custom.team = 'payments'", ALL_MARKERS, id="custom_resource_attribute"),
-        pytest.param("signoz.workspace.key.id = 'key-b'", {("cart", "2.0.0"), ("worker", "3.0.0"), ("api", "6.0.0")}, id="workspace_key"),
-        pytest.param("service.name = 'nope'", set(), id="no_match"),
-        pytest.param("k8s.namespace.namee = 'jobs'", ALL_MARKERS, id="unresolved_key_noop"),
-        pytest.param("http.request.method = 'GET'", ALL_MARKERS, id="span_attribute_noop"),
-        pytest.param("", ALL_MARKERS, id="empty_expression"),
-    ],
-)
-def test_deployments_filter(  # pylint: disable=too-many-arguments,too-many-positional-arguments
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,
-    insert_logs,
-    expression: str,
-    expected: set[tuple[str, str]],
-) -> None:
-    """Filters apply to resource attributes on both signals; keys that are not resource
-    attributes compile to nothing and leave the result unfiltered."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    insert_traces(Traces.load_from_file(get_testdata_file_path("annotations/traces.jsonl"), base_time=now - BASELINE_OFFSET))
-    insert_logs(Logs.load_from_file(get_testdata_file_path("annotations/logs.jsonl"), base_time=now - BASELINE_OFFSET))
-
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50, "filter": {"expression": expression}},
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.OK, response.text
-    data = response.json()["data"]
-    assert {(r["spec"]["service"], r["spec"]["version"]) for r in data["records"]} == expected
-    assert data["total"] == len(expected)
-
-
-@pytest.mark.parametrize(
-    "expression",
-    [
-        pytest.param("service.name =", id="trailing_op"),
-        pytest.param("(service.name = 'cart'", id="unclosed_paren"),
-    ],
-)
-def test_deployments_filter_invalid(
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    expression: str,
-) -> None:
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50, "filter": {"expression": expression}},
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
-    assert "syntax error" in response.json()["error"]["message"]
-
-
-@pytest.mark.parametrize(
-    ("limit", "expected_len"),
-    [
-        pytest.param(1, 1, id="below_count"),
-        pytest.param(7, 7, id="equals_count"),
-        pytest.param(5000, 7, id="above_count"),
-    ],
-)
-def test_deployments_limit(  # pylint: disable=too-many-arguments,too-many-positional-arguments
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    insert_traces,
-    insert_logs,
-    limit: int,
-    expected_len: int,
-) -> None:
-    """limit truncates the merged, ordered list; total always counts every marker."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    insert_traces(Traces.load_from_file(get_testdata_file_path("annotations/traces.jsonl"), base_time=now - BASELINE_OFFSET))
-    insert_logs(Logs.load_from_file(get_testdata_file_path("annotations/logs.jsonl"), base_time=now - BASELINE_OFFSET))
-
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}"},
-        json={"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": limit},
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.OK, response.text
-    data = response.json()["data"]
-    assert len(data["records"]) == expected_len
-    assert data["total"] == len(ALL_MARKERS)
-    assert (data["records"][0]["spec"]["service"], data["records"][0]["spec"]["version"]) == ("checkout", "1.1.0")
-    assert [r["spec"]["firstSeen"] for r in data["records"]] == sorted(r["spec"]["firstSeen"] for r in data["records"])
-
-
-@pytest.mark.parametrize(
-    ("overrides", "message"),
-    [
-        pytest.param({"kind": None}, "invalid kind", id="kind_missing"),
-        pytest.param({"kind": "release"}, "invalid kind", id="kind_unknown"),
-        pytest.param({"start": 0}, "start must be greater than 0", id="start_zero"),
-        pytest.param({"start": -1}, "start must be greater than 0", id="start_negative"),
-        pytest.param({"end": 0}, "end must be greater than 0", id="end_zero"),
-        pytest.param({"end": "same"}, "must be less than end", id="start_equals_end"),
-        pytest.param({"end": "before"}, "must be less than end", id="start_greater_than_end"),
-        pytest.param({"signals": ["metrics"]}, "invalid signal", id="signal_unknown"),
-        pytest.param({"limit": None}, "limit must be between", id="limit_missing"),
-        pytest.param({"limit": 0}, "limit must be between", id="limit_zero"),
-        pytest.param({"limit": -1}, "limit must be between", id="limit_negative"),
-        pytest.param({"limit": 5001}, "limit must be between", id="limit_too_large"),
-    ],
-)
-def test_deployments_validation_errors(
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-    overrides: dict,
-    message: str,
-) -> None:
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    body: dict = {"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50}
-    for key, value in overrides.items():
-        if value is None:
-            body.pop(key)
-        elif value == "same":
-            body[key] = body["start"]
-        elif value == "before":
-            body[key] = body["start"] - 1
-        else:
-            body[key] = value
-
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(signoz.self.host_configs["8080"].get(ENDPOINT), headers={"authorization": f"Bearer {token}"}, json=body, timeout=5)
-    assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
-    assert message in response.json()["error"]["message"]
-
-
-def test_deployments_malformed_body(
-    signoz: types.SigNoz,
-    create_user_admin: None,  # pylint: disable=unused-argument
-    get_token,
-) -> None:
-    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        headers={"authorization": f"Bearer {token}", "content-type": "application/json"},
-        data='{"kind": "deployment", "start": ',
-        timeout=5,
-    )
-    assert response.status_code == HTTPStatus.BAD_REQUEST, response.text

tests/integration/tests/annotations/02_authz.py

@@ -1,150 +0,0 @@
-from collections.abc import Callable
-from datetime import UTC, datetime, timedelta
-from http import HTTPStatus
-
-import pytest
-import requests
-from wiremock.resources.mappings import Mapping
-
-from fixtures import types
-from fixtures.auth import (
-    USER_ADMIN_EMAIL,
-    USER_ADMIN_PASSWORD,
-    add_license,
-    change_user_role,
-    create_active_user,
-)
-from fixtures.fs import get_testdata_file_path
-from fixtures.logs import Logs
-from fixtures.role import transaction_group
-from fixtures.traces import Traces
-
-ENDPOINT = "/api/v1/annotations"
-user_password = "password123Z$"
-users = {
-    "metrics_only": ("annotations-metrics-only", "annotations-metrics-only@telemetry.test"),
-    "traces_only": ("annotations-traces-only", "annotations-traces-only@telemetry.test"),
-    "logs_only": ("annotations-logs-only", "annotations-logs-only@telemetry.test"),
-    "both_wildcard": ("annotations-both-wildcard", "annotations-both-wildcard@telemetry.test"),
-    "both_scoped": ("annotations-both-scoped", "annotations-both-scoped@telemetry.test"),
-}
-
-
-def test_setup(
-    signoz: types.SigNoz,
-    create_user_admin: types.Operation,  # pylint: disable=unused-argument
-    make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
-    get_token: Callable[[str, str], str],
-    create_role: Callable[..., str],
-) -> None:
-    add_license(signoz, make_http_mocks, get_token)
-    admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
-
-    grants = {
-        "metrics_only": [transaction_group("read", "telemetryresource", "metrics", ["builder_query/*"])],
-        "traces_only": [transaction_group("read", "telemetryresource", "traces", ["builder_query/*"])],
-        "logs_only": [transaction_group("read", "telemetryresource", "logs", ["builder_query/*"])],
-        "both_wildcard": [
-            transaction_group("read", "telemetryresource", "traces", ["builder_query/*"]),
-            transaction_group("read", "telemetryresource", "logs", ["builder_query/*"]),
-        ],
-        "both_scoped": [
-            transaction_group("read", "telemetryresource", "traces", ["builder_query/signoz.workspace.key.id/key-a"]),
-            transaction_group("read", "telemetryresource", "logs", ["builder_query/signoz.workspace.key.id/key-a"]),
-        ],
-    }
-    for name, (role, email) in users.items():
-        create_role(admin_token, role, grants[name])
-        user = create_active_user(signoz, admin_token, email=email, role="signoz-viewer", password=user_password)
-        change_user_role(signoz, admin_token, user, "signoz-viewer", role)
-
-
-@pytest.mark.parametrize(
-    ("user", "expected"),
-    [
-        pytest.param("metrics_only", HTTPStatus.FORBIDDEN, id="metrics_only_denied"),
-        pytest.param("traces_only", HTTPStatus.FORBIDDEN, id="traces_only_denied"),
-        pytest.param("logs_only", HTTPStatus.FORBIDDEN, id="logs_only_denied"),
-        pytest.param("both_wildcard", HTTPStatus.OK, id="both_wildcard_allowed"),
-        pytest.param("both_scoped", HTTPStatus.FORBIDDEN, id="both_scoped_unfiltered_denied"),
-        pytest.param("admin", HTTPStatus.OK, id="admin_allowed"),
-        pytest.param(None, HTTPStatus.UNAUTHORIZED, id="anonymous"),
-    ],
-)
-def test_list_route(
-    signoz: types.SigNoz,
-    get_token: Callable[[str, str], str],
-    user: str | None,
-    expected: HTTPStatus,
-) -> None:
-    """The route declares traces and logs read; a grant on only one signal is denied."""
-    now = datetime.now(tz=UTC)
-    headers = {}
-    if user == "admin":
-        headers["Authorization"] = f"Bearer {get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)}"
-    elif user is not None:
-        headers["Authorization"] = f"Bearer {get_token(users[user][1], user_password)}"
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        json={"kind": "deployment", "start": int((now - timedelta(minutes=10)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 10},
-        headers=headers,
-        timeout=10,
-    )
-    assert response.status_code == expected, response.text
-
-
-@pytest.mark.parametrize("user", ["metrics_only", "traces_only", "logs_only"])
-def test_denied_message_names_wildcard(
-    signoz: types.SigNoz,
-    get_token: Callable[[str, str], str],
-    user: str,
-) -> None:
-    now = datetime.now(tz=UTC)
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        json={"kind": "deployment", "start": int((now - timedelta(minutes=10)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 10},
-        headers={"Authorization": f"Bearer {get_token(users[user][1], user_password)}"},
-        timeout=10,
-    )
-    assert response.status_code == HTTPStatus.FORBIDDEN, response.text
-    assert "builder_query/*" in response.text
-
-
-@pytest.mark.parametrize(
-    ("user", "expression", "expected", "expected_markers"),
-    [
-        pytest.param("both_scoped", "signoz.workspace.key.id = 'key-a'", HTTPStatus.OK, {("checkout", "1.1.0"), ("tieA", "4.0.0"), ("tieB", "4.0.0"), ("sp`ec'ial", "5.0.0$0")}, id="scoped_granted_key"),
-        pytest.param("both_scoped", "signoz.workspace.key.id = 'key-b'", HTTPStatus.FORBIDDEN, set(), id="scoped_other_key"),
-        pytest.param("both_scoped", "signoz.workspace.key.id IN ('key-a', 'key-b')", HTTPStatus.FORBIDDEN, set(), id="scoped_both_keys"),
-        pytest.param("both_scoped", "signoz.workspace.key.id = 'key-a' AND k8s.namespace.name = 'shop'", HTTPStatus.OK, {("checkout", "1.1.0"), ("tieA", "4.0.0"), ("tieB", "4.0.0"), ("sp`ec'ial", "5.0.0$0")}, id="scoped_granted_key_with_extra_filter"),
-        pytest.param("both_wildcard", None, HTTPStatus.OK, {("checkout", "1.1.0"), ("cart", "2.0.0"), ("worker", "3.0.0"), ("tieA", "4.0.0"), ("tieB", "4.0.0"), ("api", "6.0.0"), ("sp`ec'ial", "5.0.0$0")}, id="wildcard_unfiltered"),
-    ],
-)
-def test_scoped_read(  # pylint: disable=too-many-arguments,too-many-positional-arguments
-    signoz: types.SigNoz,
-    get_token: Callable[[str, str], str],
-    insert_traces: Callable[[list[Traces]], None],
-    insert_logs: Callable[[list[Logs]], None],
-    user: str,
-    expression: str | None,
-    expected: HTTPStatus,
-    expected_markers: set[tuple[str, str]],
-) -> None:
-    """A key-scoped grant on both signals passes only when the filter pins that key, and
-    the response then carries only that key's markers from either signal."""
-    now = datetime.now(tz=UTC).replace(microsecond=0)
-    insert_traces(Traces.load_from_file(get_testdata_file_path("annotations/traces.jsonl"), base_time=now - timedelta(hours=3)))
-    insert_logs(Logs.load_from_file(get_testdata_file_path("annotations/logs.jsonl"), base_time=now - timedelta(hours=3)))
-
-    body: dict = {"kind": "deployment", "start": int((now - timedelta(hours=1)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50}
-    if expression:
-        body["filter"] = {"expression": expression}
-    response = requests.post(
-        signoz.self.host_configs["8080"].get(ENDPOINT),
-        json=body,
-        headers={"Authorization": f"Bearer {get_token(users[user][1], user_password)}"},
-        timeout=10,
-    )
-    assert response.status_code == expected, response.text
-    if expected == HTTPStatus.OK:
-        assert {(r["spec"]["service"], r["spec"]["version"]) for r in response.json()["data"]["records"]} == expected_markers