fix: update openapi

.github/workflows/integrationci.yaml

@@ -38,6 +38,7 @@ jobs:
       fail-fast: false
       matrix:
         suite:
+          - aiobservability
           - alerts
           - basepath
           - callbackauthn

deploy/MIGRATION.md

@@ -1,76 +1,48 @@
-# Migrating from the install script and `deploy/` to Foundry
-
-The install script (`install.sh`) and the bundled Compose and Swarm files
-under `deploy/` are deprecated in favor of [Foundry][foundry], the supported
-way to install and manage SigNoz. This guide moves an existing Docker Compose
-or Docker Swarm deployment to Foundry and reattaches your existing volumes, so
-your data is preserved.
+# Migrating from the install script to Foundry
 
 > [!IMPORTANT]
-> This guide is only for **existing** `install.sh` / `deploy/` deployments.
-> Setting up SigNoz for the first time? Skip migration and install Foundry
-> directly: [SigNoz install docs][install-docs].
+> The install script is now deprecated and will no longer receive updates.
 
-## How it works
+This guide walks you through migrating an existing SigNoz deployment running via
+Docker Compose to [Foundry](https://signoz.io/docs/install/docker/).
 
-Foundry splits a deployment into two commands:
+> [!NOTE]
+> Setting up SigNoz for the first time? You don't need this guide — follow the [SigNoz installation docs](https://signoz.io/docs/install/) instead.
 
-- `foundryctl forge` generates the deployment manifests from a `casting.yaml`.
-  It never touches running containers, so it is safe to re-run while you
-  iterate.
-- `foundryctl cast` applies those manifests: it (re)creates the containers and
-  reuses the volumes you point it at.
+## Overview
+To stay up to date on new installation platforms and patterns, please refer to [Foundry](https://github.com/SigNoz/foundry).
 
-You write one `casting.yaml`, point a few patches at your existing data
-volumes, then cast. The steps below are the same for Compose and Swarm; they
-differ only in the casting (step 3) and how you stop the old stack (step 5).
+Two `foundryctl` commands are used throughout this guide:
+- **`forge`** — generates deployment manifests from your `casting.yaml`. It does not touch running containers, so it is safe to re-run while you iterate.
+- **`cast`** — applies the generated manifests: it creates and starts the containers (and pulls new images).
 
 ## Prerequisites
+- [ ] Install Foundry - `curl -fsSL https://signoz.io/foundry.sh | bash`
 
-- An existing SigNoz deployment from `install.sh` or `deploy/` (Compose or
-  Swarm).
-- `foundryctl` (installed in step 1).
+## Migration Steps
+> [!WARNING]
+> **Before proceeding, back up both:**
+> - **Your docker volumes** — these hold your data.
+> - **Your existing `docker-compose.yaml` (and any config it references)** — keep a copy somewhere safe. The compose manifests are no longer distributed by SigNoz, so this backup is your only way to roll back to your previous setup.
 
-## Migrate
+1. Make a note of the volume names used by your existing deployment for the following components:
+- ClickHouse
+- SigNoz
+- ZooKeeper
 
-### 1. Install Foundry
+> If you used the docker compose file we provided, the volumes will be `signoz-clickhouse`, `signoz-sqlite`, and `signoz-zookeeper-1`.
 
-```bash
-curl -fsSL https://signoz.io/foundry.sh | bash
-```
+2. Generate your `casting.yaml`. Based on internal testing, the following casting should generate the manifests that mimic the legacy docker compose setup (compare against your backed-up `docker-compose.yaml`). Once created, run `foundryctl forge -f casting.yaml`.
 
-### 2. Keep your rollback path
+Foundry has a [Docker Compose example](https://github.com/SigNoz/foundry/tree/main/docs/examples/docker/compose). Please use it as a reference.
 
-This migration reattaches your existing volumes in place; it does not move or
-delete your data. The only destructive action is passing `--volumes` / `-v`
-when you stop the old stack (step 5), so avoid that flag.
+> [!WARNING]
+> If your deployment had more than 1 shard or replica, you will need to adjust your manifest volumes accordingly.
 
 > [!IMPORTANT]
-> Keep a copy of your existing `docker-compose.yaml` / stack file (and any
-> config it references). SigNoz no longer distributes these files, so this copy
-> is your only way to roll back.
-
-### 3. Write your `casting.yaml`
-
-Use the casting for your deployment. Both reproduce the legacy single-node
-setup (ClickHouse + ZooKeeper + SQLite) and reattach your existing volumes;
-they differ only in `spec.deployment.flavor` and the volume-reuse patch
-(Compose volumes have a `name` to replace; Swarm volumes are bare, so the whole
-entry is replaced). If your deployment ran more than one shard or replica,
-adjust the volume patches accordingly. The
-[Docker Compose example][compose-example] is a useful reference.
-
-> [!IMPORTANT]
-> The `replica` and `shard` macros are placeholders. Replace them with the
-> values from your existing ClickHouse config (the `macros` section of
-> `config.xml` / `metrika.xml`), or the generated manifests will not match your
-> existing data.
-
-<details>
-<summary><b>Docker Compose</b> casting.yaml</summary>
+> The `replica` and `shard` macros below are placeholders. Replace them with the values from your existing ClickHouse configuration (check the `macros` section of your current ClickHouse config, e.g. `config.xml`/`metrika.xml`), otherwise the generated manifests will not match your existing data.
 
 ```yaml
-# casting.yaml
 apiVersion: v1alpha1
 kind: Installation
 metadata:
@@ -89,8 +61,8 @@ spec:
         data:
           config-0-0.yaml: |
             macros:
-              replica: "example01-01-1" # replace with your replica macro
-              shard: "01" # replace with your shard macro
+              replica: "example01-01-1" # replace with your existing ClickHouse replica macro (see legacy configuration files for reference)
+              shard: "01"               # replace with your existing ClickHouse shard macro (see legacy configuration files for reference)
   patches:
     - target: "deployment/compose.yaml"
       operations:
@@ -108,165 +80,50 @@ spec:
           value: root
 ```
 
-</details>
+> [!NOTE]
+> The `user: root` patch on the ZooKeeper service is required so the container can read/write the data in your reused ZooKeeper volume, which was created with `root`-owned files by the legacy compose setup. Without it, ZooKeeper may fail to start with permission errors.
 
-<details>
-<summary><b>Docker Swarm</b> casting.yaml</summary>
+If you had custom configurations for features like SMTP or additional ingestion processors/receivers, you will need to include those in your casting file via [patches](https://github.com/SigNoz/foundry/blob/main/docs/concepts/patches.md), [custom configuration](https://github.com/SigNoz/foundry/blob/main/docs/concepts/moldings.md#custom-config-files) or [environment variables](https://github.com/SigNoz/foundry/blob/main/docs/reference/casting-file.md#molding-spec) based on your previous configuration.
 
-```yaml
-# casting.yaml
-apiVersion: v1alpha1
-kind: Installation
-metadata:
-  name: signoz
-spec:
-  deployment:
-    flavor: swarm
-    mode: docker
-  metastore:
-    kind: sqlite
-  telemetrykeeper:
-    kind: zookeeper
-  telemetrystore:
-    spec:
-      config:
-        data:
-          config-0-0.yaml: |
-            macros:
-              replica: "example01-01-1" # replace with your replica macro
-              shard: "01" # replace with your shard macro
-  patches:
-    - target: "deployment/compose.yaml"
-      operations:
-        - op: replace
-          path: /volumes/signoz-telemetrykeeper-0-data
-          value:
-            name: signoz-zookeeper-1
-        - op: replace
-          path: /volumes/signoz-telemetrystore-0-0-data
-          value:
-            name: signoz-clickhouse
-        - op: replace
-          path: /volumes/signoz-metastore-sqlite-0-data
-          value:
-            name: signoz-sqlite
-        - op: add
-          path: /services/signoz-telemetrykeeper-zookeeper-0/user
-          value: root
-```
+3. Review your manifests, we suggest executing the following checks on your manifests before proceeding:
+- [ ] Validate the container images match what your deployment had, Foundry uses `latest` on generation by default.
+- [ ] If your signoz version was older than latest, please check the [upgrade path](https://signoz.io/docs/operate/upgrade/) first.
+- [ ] Check the produced manifests in `pours/deployment` match your older configurations. Extra consideration and review needs to be done on `compose.yaml` as this will be the main entry point for your new deployment.
+- [ ] The configuration files for clickhouse are now in YAML so validate your custom settings are present.
 
-</details>
+4. Execute a `docker compose down`. **Do not** include parameters such as `--volumes` (or `-v`), as it will wipe the volumes we need to maintain and reuse to avoid data loss. 
 
 > [!NOTE]
-> The `user: root` patch on the ZooKeeper service lets the container read and
-> write the data in your reused ZooKeeper volume, whose files the legacy setup
-> created as `root`. Without it, ZooKeeper may fail to start with permission
-> errors.
+> This will generate downtime so please plan accordingly.
 
-If you had custom configuration (SMTP, extra ingestion receivers/processors,
-or custom ClickHouse settings), carry it over via [patches][patches],
-[custom config files][custom-config], or [environment variables][env-vars].
+5. Validate the SigNoz containers are down with `docker ps -a`. Multiple containers cannot bind the same volume.
 
-### 4. Generate and review the manifests
-
-```bash
-foundryctl forge -f casting.yaml
-```
-
-Review `pours/deployment/` before deploying:
-
-- [ ] Container images match your current deployment. Foundry generates with
-  `latest` by default; if your SigNoz version was older than latest, check the
-  [upgrade path][upgrade-path] first.
-- [ ] The generated manifests match your previous configuration, especially
-  `compose.yaml` (the new entry point for your deployment).
-- [ ] The ClickHouse config is now YAML rather than XML; confirm your custom
-  settings carried over (see [ClickHouse configuration files][ch-config] for
-  the XML-to-YAML mapping).
-
-### 5. Stop the old deployment
-
-Use the command for your deployment. Do **not** pass `--volumes` / `-v`; that
-would delete the data you are migrating.
-
-```bash
-docker compose down        # Compose
-docker stack rm signoz     # Swarm
-```
+6. Run `foundryctl cast -f casting.yaml`. This will recreate the containers based on the spec. This process will download new container images.
 
 > [!NOTE]
-> This causes downtime, so plan accordingly.
+> When `cast` is run, the migration container will execute its migrations.
 
-Confirm nothing is still bound to the volumes before continuing:
+## Verifying the Migration
+- SigNoz containers will be up and running.
+- Log in to the SigNoz UI and verify that data is present.
+    - Signoz will run on localhost:8080
+- Validate that your data ingestion is receiving data.
+    - Ingesters will receive data on localhost:4317(grpc) and localhost:4318(http)
+- Review the logs from both ClickHouse and ZooKeeper; no errors should be present.
 
-```bash
-docker ps -a
-```
+## Rolling Back
+Because step 4 brought the legacy stack down *without* `-v`, your original volumes
+are untouched and still hold your data. To roll back:
 
-### 6. Deploy with Foundry
-
-```bash
-foundryctl cast -f casting.yaml
-```
-
-This recreates the containers against your existing volumes and pulls the
-images. The migration container runs the schema migrations as part of `cast`.
-
-**Prefer not to use `cast`?** The manifests in `pours/deployment/` are standard
-Docker artifacts you can apply yourself. Run the command from that directory so
-the relative config paths resolve:
-
-```bash
-cd pours/deployment
-docker compose up -d                        # Compose
-docker stack deploy -c compose.yaml signoz  # Swarm
-```
-
-## Verify
-
-- All SigNoz containers are running.
-- The UI is reachable on `http://localhost:8080`, and OTLP on `4317` (gRPC)
-  and `4318` (HTTP), so already-instrumented apps and saved bookmarks keep
-  working.
-- Your existing data is present in the UI, and new data is being ingested.
-- ClickHouse and ZooKeeper logs show no errors.
-
-## Roll back
-
-Step 5 left your volumes untouched, so your data is intact. To return to the
-previous setup:
-
-1. Bring down the Foundry deployment (`docker compose down` or
-   `docker stack rm signoz`, again without `-v`).
-2. Confirm the containers are gone with `docker ps -a`.
-3. Re-apply your backed-up stack: `docker compose up -d` (Compose) or
-   `docker stack deploy -c docker-compose.yaml signoz` (Swarm). It reattaches
-   the same volumes and restores your prior state.
+- Stop and remove the containers created by Foundry (`docker compose down`, again without `-v`).
+- Confirm the containers are gone with `docker ps -a` so nothing else is bound to the volumes.
+- Reapply your original docker compose file (`docker compose up -d`). It will reattach to the
+  existing volumes and restore your prior state.
 
 ## Troubleshooting
-
-If the migration runs into trouble, see
-[Troubleshooting Foundry][troubleshooting] for how to capture what we need to
-help (the `--debug` output, the exit code, and your `casting.yaml`), then reach
-out on [Slack][slack].
+- Please reach out to our community on [Slack](https://signoz.io/slack).
 
 ## References
-
-- [Foundry][foundry]
-- [Casting file reference][casting-ref]
-- [Custom config files][custom-config]
-- [Patches][patches]
-- [SigNoz documentation][signoz-docs]
-
-[foundry]: https://github.com/SigNoz/foundry
-[install-docs]: https://signoz.io/docs/install/
-[compose-example]: https://github.com/SigNoz/foundry/tree/main/docs/examples/docker/compose
-[patches]: https://github.com/SigNoz/foundry/blob/main/docs/concepts/patches.md
-[custom-config]: https://github.com/SigNoz/foundry/blob/main/docs/concepts/moldings.md#custom-config-files
-[env-vars]: https://github.com/SigNoz/foundry/blob/main/docs/reference/casting-file.md#molding-spec
-[casting-ref]: https://github.com/SigNoz/foundry/blob/main/docs/reference/casting-file.md
-[ch-config]: https://clickhouse.com/docs/operations/configuration-files
-[upgrade-path]: https://signoz.io/docs/operate/upgrade/
-[troubleshooting]: https://signoz.io/docs/setup/foundry/troubleshooting/faq/
-[slack]: https://signoz.io/slack
-[signoz-docs]: https://signoz.io/docs
+- [SigNoz Docker installation docs](https://signoz.io/docs/install/docker/)
+- [SigNoz documentation](https://signoz.io/docs)
+- [Foundry](https://github.com/SigNoz/foundry)

docs/api/openapi.yml

@@ -7108,6 +7108,21 @@ components:
       required:
       - items
       type: object
+    SpantypesGettableSpanMapperTest:
+      properties:
+        collectorLogs:
+          items:
+            type: string
+          nullable: true
+          type: array
+        spans:
+          items:
+            $ref: '#/components/schemas/SpantypesSpanMapperTestSpan'
+          nullable: true
+          type: array
+      required:
+      - spans
+      type: object
     SpantypesGettableTraceAggregations:
       properties:
         aggregations:
@@ -7195,6 +7210,39 @@ components:
       - name
       - condition
       type: object
+    SpantypesPostableSpanMapperTest:
+      properties:
+        groups:
+          items:
+            $ref: '#/components/schemas/SpantypesPostableSpanMapperTestGroup'
+          nullable: true
+          type: array
+        spans:
+          items:
+            $ref: '#/components/schemas/SpantypesSpanMapperTestSpan'
+          nullable: true
+          type: array
+      required:
+      - spans
+      - groups
+      type: object
+    SpantypesPostableSpanMapperTestGroup:
+      properties:
+        condition:
+          $ref: '#/components/schemas/SpantypesSpanMapperGroupCondition'
+        enabled:
+          type: boolean
+        mappers:
+          items:
+            $ref: '#/components/schemas/SpantypesPostableSpanMapper'
+          nullable: true
+          type: array
+        name:
+          type: string
+      required:
+      - name
+      - condition
+      type: object
     SpantypesPostableTraceAggregations:
       properties:
         aggregations:
@@ -7356,6 +7404,17 @@ components:
       - operation
       - priority
       type: object
+    SpantypesSpanMapperTestSpan:
+      properties:
+        attributes:
+          additionalProperties: {}
+          nullable: true
+          type: object
+        resource:
+          additionalProperties: {}
+          nullable: true
+          type: object
+      type: object
     SpantypesUpdatableSpanMapper:
       properties:
         config:
@@ -12984,6 +13043,69 @@ paths:
       summary: Update a span mapper
       tags:
       - spanmapper
+  /api/v1/span_mapper_groups/test:
+    post:
+      deprecated: false
+      description: Tests how span mappers would transform sample spans
+      operationId: TestSpanMappers
+      requestBody:
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/SpantypesPostableSpanMapperTest'
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                properties:
+                  data:
+                    $ref: '#/components/schemas/SpantypesGettableSpanMapperTest'
+                  status:
+                    type: string
+                required:
+                - status
+                - data
+                type: object
+          description: OK
+        "400":
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RenderErrorResponse'
+          description: Bad Request
+        "401":
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RenderErrorResponse'
+          description: Unauthorized
+        "403":
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RenderErrorResponse'
+          description: Forbidden
+        "404":
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RenderErrorResponse'
+          description: Not Found
+        "500":
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RenderErrorResponse'
+          description: Internal Server Error
+      security:
+      - api_key:
+        - VIEWER
+      - tokenizer:
+        - VIEWER
+      summary: Test span mappers against sample spans
+      tags:
+      - spanmapper
   /api/v1/stats:
     get:
       deprecated: false

frontend/src/api/generated/services/sigNoz.schemas.ts

@@ -8247,6 +8247,48 @@ export interface SpantypesGettableSpanMapperGroupsDTO {
 	items: SpantypesSpanMapperGroupDTO[];
 }
 
+export type SpantypesSpanMapperTestSpanDTOAttributesAnyOf = {
+	[key: string]: unknown;
+};
+
+/**
+ * @nullable
+ */
+export type SpantypesSpanMapperTestSpanDTOAttributes =
+	SpantypesSpanMapperTestSpanDTOAttributesAnyOf | null;
+
+export type SpantypesSpanMapperTestSpanDTOResourceAnyOf = {
+	[key: string]: unknown;
+};
+
+/**
+ * @nullable
+ */
+export type SpantypesSpanMapperTestSpanDTOResource =
+	SpantypesSpanMapperTestSpanDTOResourceAnyOf | null;
+
+export interface SpantypesSpanMapperTestSpanDTO {
+	/**
+	 * @type object,null
+	 */
+	attributes?: SpantypesSpanMapperTestSpanDTOAttributes;
+	/**
+	 * @type object,null
+	 */
+	resource?: SpantypesSpanMapperTestSpanDTOResource;
+}
+
+export interface SpantypesGettableSpanMapperTestDTO {
+	/**
+	 * @type array,null
+	 */
+	collectorLogs?: string[] | null;
+	/**
+	 * @type array,null
+	 */
+	spans: SpantypesSpanMapperTestSpanDTO[] | null;
+}
+
 export enum SpantypesSpanAggregationTypeDTO {
 	span_count = 'span_count',
 	execution_time_percentage = 'execution_time_percentage',
@@ -8542,6 +8584,33 @@ export interface SpantypesPostableSpanMapperGroupDTO {
 	name: string;
 }
 
+export interface SpantypesPostableSpanMapperTestGroupDTO {
+	condition: SpantypesSpanMapperGroupConditionDTO | null;
+	/**
+	 * @type boolean
+	 */
+	enabled?: boolean;
+	/**
+	 * @type array,null
+	 */
+	mappers?: SpantypesPostableSpanMapperDTO[] | null;
+	/**
+	 * @type string
+	 */
+	name: string;
+}
+
+export interface SpantypesPostableSpanMapperTestDTO {
+	/**
+	 * @type array,null
+	 */
+	groups: SpantypesPostableSpanMapperTestGroupDTO[] | null;
+	/**
+	 * @type array,null
+	 */
+	spans: SpantypesSpanMapperTestSpanDTO[] | null;
+}
+
 export interface SpantypesSpanAggregationDTO {
 	aggregation: SpantypesSpanAggregationTypeDTO;
 	field: TelemetrytypesTelemetryFieldKeyDTO;
@@ -9923,6 +9992,14 @@ export type UpdateSpanMapperPathParameters = {
 	groupId: string;
 	mapperId: string;
 };
+export type TestSpanMappers200 = {
+	data: SpantypesGettableSpanMapperTestDTO;
+	/**
+	 * @type string
+	 */
+	status: string;
+};
+
 export type GetStats200Data = { [key: string]: unknown };
 
 export type GetStats200 = {

frontend/src/api/generated/services/spanmapper/index.ts

@@ -30,8 +30,10 @@ import type {
 	RenderErrorResponseDTO,
 	SpantypesPostableSpanMapperDTO,
 	SpantypesPostableSpanMapperGroupDTO,
+	SpantypesPostableSpanMapperTestDTO,
 	SpantypesUpdatableSpanMapperDTO,
 	SpantypesUpdatableSpanMapperGroupDTO,
+	TestSpanMappers200,
 	UpdateSpanMapperGroupPathParameters,
 	UpdateSpanMapperPathParameters,
 } from '../sigNoz.schemas';
@@ -780,3 +782,86 @@ export const useUpdateSpanMapper = <
 > => {
 	return useMutation(getUpdateSpanMapperMutationOptions(options));
 };
+/**
+ * Tests how span mappers would transform sample spans
+ * @summary Test span mappers against sample spans
+ */
+export const testSpanMappers = (
+	spantypesPostableSpanMapperTestDTO?: BodyType<SpantypesPostableSpanMapperTestDTO>,
+	signal?: AbortSignal,
+) => {
+	return GeneratedAPIInstance<TestSpanMappers200>({
+		url: `/api/v1/span_mapper_groups/test`,
+		method: 'POST',
+		headers: { 'Content-Type': 'application/json' },
+		data: spantypesPostableSpanMapperTestDTO,
+		signal,
+	});
+};
+
+export const getTestSpanMappersMutationOptions = <
+	TError = ErrorType<RenderErrorResponseDTO>,
+	TContext = unknown,
+>(options?: {
+	mutation?: UseMutationOptions<
+		Awaited<ReturnType<typeof testSpanMappers>>,
+		TError,
+		{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
+		TContext
+	>;
+}): UseMutationOptions<
+	Awaited<ReturnType<typeof testSpanMappers>>,
+	TError,
+	{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
+	TContext
+> => {
+	const mutationKey = ['testSpanMappers'];
+	const { mutation: mutationOptions } = options
+		? options.mutation &&
+			'mutationKey' in options.mutation &&
+			options.mutation.mutationKey
+			? options
+			: { ...options, mutation: { ...options.mutation, mutationKey } }
+		: { mutation: { mutationKey } };
+
+	const mutationFn: MutationFunction<
+		Awaited<ReturnType<typeof testSpanMappers>>,
+		{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> }
+	> = (props) => {
+		const { data } = props ?? {};
+
+		return testSpanMappers(data);
+	};
+
+	return { mutationFn, ...mutationOptions };
+};
+
+export type TestSpanMappersMutationResult = NonNullable<
+	Awaited<ReturnType<typeof testSpanMappers>>
+>;
+export type TestSpanMappersMutationBody =
+	| BodyType<SpantypesPostableSpanMapperTestDTO>
+	| undefined;
+export type TestSpanMappersMutationError = ErrorType<RenderErrorResponseDTO>;
+
+/**
+ * @summary Test span mappers against sample spans
+ */
+export const useTestSpanMappers = <
+	TError = ErrorType<RenderErrorResponseDTO>,
+	TContext = unknown,
+>(options?: {
+	mutation?: UseMutationOptions<
+		Awaited<ReturnType<typeof testSpanMappers>>,
+		TError,
+		{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
+		TContext
+	>;
+}): UseMutationResult<
+	Awaited<ReturnType<typeof testSpanMappers>>,
+	TError,
+	{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
+	TContext
+> => {
+	return useMutation(getTestSpanMappersMutationOptions(options));
+};

go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/DATA-DOG/go-sqlmock v1.5.2
 	github.com/SigNoz/clickhouse-go-mock v0.14.0
 	github.com/SigNoz/govaluate v0.0.0-20240203125216-988004ccc7fd
-	github.com/SigNoz/signoz-otel-collector v0.144.3
+	github.com/SigNoz/signoz-otel-collector v0.144.6-rc.1.0.20260625043036-fb3dbb450bbd
 	github.com/antlr4-go/antlr/v4 v4.13.1
 	github.com/antonmedv/expr v1.15.3
 	github.com/bytedance/sonic v1.14.1
@@ -150,10 +150,6 @@ require (
 	github.com/ugorji/go/codec v1.3.0 // indirect
 	github.com/uptrace/opentelemetry-go-extra/otelsql v0.3.2 // indirect
 	github.com/x448/float16 v0.8.4 // indirect
-	go.opentelemetry.io/collector/client v1.54.0 // indirect
-	go.opentelemetry.io/collector/config/configoptional v1.50.0 // indirect
-	go.opentelemetry.io/collector/config/configretry v1.50.0 // indirect
-	go.opentelemetry.io/collector/exporter/exporterhelper v0.144.0 // indirect
 	go.opentelemetry.io/collector/internal/componentalias v0.148.0 // indirect
 	go.opentelemetry.io/collector/pdata/xpdata v0.148.0 // indirect
 	go.uber.org/goleak v1.3.0 // indirect

go.sum

@@ -110,11 +110,13 @@ github.com/SigNoz/expr v1.17.7-beta h1:FyZkleM5dTQ0O6muQfwGpoH5A2ohmN/XTasRCO72g
 github.com/SigNoz/expr v1.17.7-beta/go.mod h1:8/vRC7+7HBzESEqt5kKpYXxrxkr31SaO8r40VO/1IT4=
 github.com/SigNoz/govaluate v0.0.0-20240203125216-988004ccc7fd h1:Bk43AsDYe0fhkbj57eGXx8H3ZJ4zhmQXBnrW523ktj8=
 github.com/SigNoz/govaluate v0.0.0-20240203125216-988004ccc7fd/go.mod h1:nxRcH/OEdM8QxzH37xkGzomr1O0JpYBRS6pwjsWW6Pc=
-github.com/SigNoz/signoz-otel-collector v0.144.3 h1:/7PPIqIpPsaWtrgnfHam2XVYP41ZlgEKLHzQO8oVxcA=
-github.com/SigNoz/signoz-otel-collector v0.144.3/go.mod h1:9pLVpcIQvUT88ZiNnZN/MI+XLruvwC+Xm2UzPmGjNfA=
+github.com/SigNoz/signoz-otel-collector v0.144.6-rc.1.0.20260625043036-fb3dbb450bbd h1:MGQATRMNEsBrdHzHb7uj2JIzLXXgQNcNxZHfmAAIIs8=
+github.com/SigNoz/signoz-otel-collector v0.144.6-rc.1.0.20260625043036-fb3dbb450bbd/go.mod h1:Jw1Y3s/ZOSz6RmLjZtj2A7wK6bse0T4pxs+oggBQnSs=
 github.com/Yiling-J/theine-go v0.6.2 h1:1GeoXeQ0O0AUkiwj2S9Jc0Mzx+hpqzmqsJ4kIC4M9AY=
 github.com/Yiling-J/theine-go v0.6.2/go.mod h1:08QpMa5JZ2pKN+UJCRrCasWYO1IKCdl54Xa836rpmDU=
 github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c=
+github.com/alecthomas/participle/v2 v2.1.4 h1:W/H79S8Sat/krZ3el6sQMvMaahJ+XcM9WSI2naI7w2U=
+github.com/alecthomas/participle/v2 v2.1.4/go.mod h1:8tqVbpTX20Ru4NfYQgZf4mP18eXPTBViyMWiArNEgGI=
 github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
 github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
 github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
@@ -124,6 +126,10 @@ github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vS
 github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs=
 github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
 github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
+github.com/antchfx/xmlquery v1.5.0 h1:uAi+mO40ZWfyU6mlUBxRVvL6uBNZ6LMU4M3+mQIBV4c=
+github.com/antchfx/xmlquery v1.5.0/go.mod h1:lJfWRXzYMK1ss32zm1GQV3gMIW/HFey3xDZmkP1SuNc=
+github.com/antchfx/xpath v1.3.6 h1:s0y+ElRRtTQdfHP609qFu0+c6bglDv20pqOViQjjdPI=
+github.com/antchfx/xpath v1.3.6/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
 github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
 github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
 github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
@@ -286,6 +292,8 @@ github.com/ebitengine/purego v0.9.1 h1:a/k2f2HQU3Pi399RPW1MOaZyhKJL9w/xFpKAg4q1s
 github.com/ebitengine/purego v0.9.1/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
 github.com/edsrzf/mmap-go v1.2.0 h1:hXLYlkbaPzt1SaQk+anYwKSRNhufIDCchSPkUD6dD84=
 github.com/edsrzf/mmap-go v1.2.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q=
+github.com/elastic/go-grok v0.3.1 h1:WEhUxe2KrwycMnlvMimJXvzRa7DoByJB4PVUIE1ZD/U=
+github.com/elastic/go-grok v0.3.1/go.mod h1:n38ls8ZgOboZRgKcjMY8eFeZFMmcL9n2lP0iHhIDk64=
 github.com/elastic/lunes v0.2.0 h1:WI3bsdOTuaYXVe2DS1KbqA7u7FOHN4o8qJw80ZyZoQs=
 github.com/elastic/lunes v0.2.0/go.mod h1:u3W/BdONWTrh0JjNZ21C907dDc+cUZttZrGa625nf2k=
 github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 h1:oP4q0fw+fOSWn3DfFi4EXdT+B+gTtzx8GC9xsc26Znk=
@@ -461,6 +469,7 @@ github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfU
 github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
 github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
 github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
@@ -674,6 +683,8 @@ github.com/huandu/xstrings v1.5.0 h1:2ag3IFq9ZDANvthTwTiqSSZLjDc+BedvHPAp5tJy2TI
 github.com/huandu/xstrings v1.5.0/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
 github.com/iancoleman/orderedmap v0.3.0 h1:5cbR2grmZR/DiVt+VJopEhtVs9YGInGIxAoMJn+Ichc=
 github.com/iancoleman/orderedmap v0.3.0/go.mod h1:XuLcCUkdL5owUCQeF2Ue9uuw1EptkJDkXXS7VoV7XGE=
+github.com/iancoleman/strcase v0.3.0 h1:nTXanmYxhfFAMjZL34Ov6gkzEsSJZ5DbhxWjvSASxEI=
+github.com/iancoleman/strcase v0.3.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47ZCWhYzw7ho=
 github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
@@ -866,12 +877,18 @@ github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal
 github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.144.0/go.mod h1:O2rZKRXk1WeYhzfJBVXES/g7+PlIds/TzPZW/4NfTNA=
 github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.148.0 h1:CiTjQE/Hh5xK2t56ogrDK4nl0+tJPNmASCs4zEYZ/xU=
 github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.148.0/go.mod h1:WUFkzTiOpt7EYyL67gv1GOf3RD8qKWGtin3lY9LYzW4=
+github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter v0.144.0 h1:Ywu5mU4K5TMJigiXdyZloCRs/cq3/2OnoK3WjxNHWJo=
+github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter v0.144.0/go.mod h1:iebqlu6UvpiV1hO37r1sXA9fXaCaA8sQXilG0///xss=
+github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl v0.144.0 h1:TMRTvQSAeeLtkKwSrqcbectxDRPiqB6yYM3IvjC75es=
+github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl v0.144.0/go.mod h1:1HU0qJ4hFrphDebuBs3I4DPQ6zyBFGinQ5/bXEUM7pw=
 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.148.0 h1:i12duJOl5VCb9mbb8FfZCaP2CjeXbNsbg82JjSe7sy8=
 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.148.0/go.mod h1:jyw+QvkmCrF/oYy31O2ndb5KZZK4l+iR89msnV3LN/k=
 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.148.0 h1:1TLg6YrS3Au6F7xw3ws2Njbwj13IMqPplvGFi+18fWs=
 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.148.0/go.mod h1:P8hZEDIQk4REgUWyLhSVRHwTxK6KkifKfg36BmmQ/DI=
 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza v0.144.0 h1:EIAygME70IOdEwaSr6bA3Wcdp7hXEqRsGsVfrI5v8OA=
 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza v0.144.0/go.mod h1:3Y6ctEEwRg19B0jqsrQH6Hiquqte+zC0ZxpXLLSa5sA=
+github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.144.0 h1:gRk73SsIJv3q/HI0kxMRN5TiIiJj+MRxrz0GEYx3jZw=
+github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.144.0/go.mod h1:7b8ZcPdN6JSm6+LUq3rienoqHHiX60kyoOHDPCMM+L8=
 github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.148.0 h1:xgD/kNGp/wWY+bwY599Pc01OamYN17phRiTP934bM5Y=
 github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.148.0/go.mod h1:ZK7wvaefla9lB3bAW0rNKt7IzRPcTRQoOFqr4sZy/XM=
 github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor v0.144.0 h1:HbmpzTixpQG/xGhQuQoiJTXQPrixe+yivAsF6tl2o4g=
@@ -1119,6 +1136,10 @@ github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc/go.mod h1:bciPuU6GH
 github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
 github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
 github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
+github.com/twmb/murmur3 v1.1.8 h1:8Yt9taO/WN3l08xErzjeschgZU2QSrwm1kclYq+0aRg=
+github.com/twmb/murmur3 v1.1.8/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
+github.com/ua-parser/uap-go v0.0.0-20240611065828-3a4781585db6 h1:SIKIoA4e/5Y9ZOl0DCe3eVMLPOQzJxgZpfdHHeauNTM=
+github.com/ua-parser/uap-go v0.0.0-20240611065828-3a4781585db6/go.mod h1:BUbeWZiieNxAuuADTBNb3/aeje6on3DhU3rpWsQSB1E=
 github.com/ugorji/go/codec v1.3.0 h1:Qd2W2sQawAfG8XSvzwhBeoGq71zXOC/Q1E9y/wUcsUA=
 github.com/ugorji/go/codec v1.3.0/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
 github.com/uptrace/bun v1.2.9 h1:OOt2DlIcRUMSZPr6iXDFg/LaQd59kOxbAjpIVHddKRs=

pkg/apiserver/signozapiserver/spanmapper.go

@@ -51,6 +51,26 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
 		return err
 	}
 
+	if err := router.Handle("/api/v1/span_mapper_groups/test", handler.New(
+		provider.authzMiddleware.ViewAccess(provider.spanMapperHandler.TestMappers),
+		handler.OpenAPIDef{
+			ID:                  "TestSpanMappers",
+			Tags:                []string{"spanmapper"},
+			Summary:             "Test span mappers against sample spans",
+			Description:         "Tests how span mappers would transform sample spans",
+			Request:             new(spantypes.PostableSpanMapperTest),
+			RequestContentType:  "application/json",
+			Response:            new(spantypes.GettableSpanMapperTest),
+			ResponseContentType: "application/json",
+			SuccessStatusCode:   http.StatusOK,
+			ErrorStatusCodes:    []int{http.StatusBadRequest, http.StatusNotFound},
+			Deprecated:          false,
+			SecuritySchemes:     newSecuritySchemes(types.RoleViewer),
+		},
+	)).Methods(http.MethodPost).GetError(); err != nil {
+		return err
+	}
+
 	if err := router.Handle("/api/v1/span_mapper_groups/{groupId}", handler.New(
 		provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.UpdateGroup),
 		handler.OpenAPIDef{

pkg/modules/spanmapper/implspanmapper/handler.go

@@ -273,6 +273,35 @@ func (h *handler) DeleteMapper(rw http.ResponseWriter, r *http.Request) {
 	render.Success(rw, http.StatusNoContent, nil)
 }
 
+// TestMappers handles POST /api/v1/span_mapper_groups/test.
+func (h *handler) TestMappers(rw http.ResponseWriter, r *http.Request) {
+	ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
+	defer cancel()
+
+	claims, err := authtypes.ClaimsFromContext(ctx)
+	if err != nil {
+		render.Error(rw, err)
+		return
+	}
+
+	orgID := valuer.MustNewUUID(claims.OrgID)
+
+	req := new(spantypes.PostableSpanMapperTest)
+	if err := binding.JSON.BindBody(r.Body, req); err != nil {
+		render.Error(rw, err)
+		return
+	}
+
+	groups := spantypes.NewSpanMapperGroupsWithMappersFromPostable(orgID, req.Groups)
+	out, collectorLogs, err := h.module.TestMappers(ctx, orgID, req.Spans, groups)
+	if err != nil {
+		render.Error(rw, err)
+		return
+	}
+
+	render.Success(rw, http.StatusOK, &spantypes.GettableSpanMapperTest{Spans: out, CollectorLogs: collectorLogs})
+}
+
 // groupIDFromPath extracts and validates the {id} or {groupId} path variable.
 func groupIDFromPath(r *http.Request) (valuer.UUID, error) {
 	vars := mux.Vars(r)

pkg/modules/spanmapper/implspanmapper/module.go

@@ -4,6 +4,7 @@ import (
 	"context"
 	"encoding/json"
 
+	"github.com/SigNoz/signoz/pkg/errors"
 	"github.com/SigNoz/signoz/pkg/modules/spanmapper"
 	"github.com/SigNoz/signoz/pkg/query-service/agentConf"
 	"github.com/SigNoz/signoz/pkg/types/opamptypes"
@@ -102,6 +103,56 @@ func (module *module) DeleteMapper(ctx context.Context, orgID, groupID, id value
 	return nil
 }
 
+func (module *module) TestMappers(ctx context.Context, orgID valuer.UUID, spans []spantypes.SpanMapperTestSpan, groups []*spantypes.SpanMapperGroupWithMappers) ([]spantypes.SpanMapperTestSpan, []string, error) {
+	if len(spans) == 0 {
+		return nil, nil, errors.New(errors.TypeInvalidInput, spantypes.ErrCodeMappingInvalidInput, "'spans' must contain at least one span")
+	}
+
+	resolved, err := module.backfillMappers(ctx, orgID, groups)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	out, collectorLogs, err := spantypes.SimulateSpanMappersProcessing(ctx, resolved, spans)
+	if err != nil {
+		return nil, nil, err
+	}
+	return out, collectorLogs, nil
+}
+
+// backfillMappers loads saved mappers for any group whose Mappers is nil.
+func (module *module) backfillMappers(ctx context.Context, orgID valuer.UUID, groups []*spantypes.SpanMapperGroupWithMappers) ([]*spantypes.SpanMapperGroupWithMappers, error) {
+	// Load the enabled saved groups for this org, so we can look up by name.
+	// Disabled groups aren't part of the live collector config, so there's
+	// nothing to backfill from them.
+	enabled := true
+	savedGroups, err := module.store.ListGroups(ctx, orgID, &spantypes.ListSpanMapperGroupsQuery{Enabled: &enabled})
+	if err != nil {
+		return nil, err
+	}
+	savedByName := make(map[string]*spantypes.SpanMapperGroup, len(savedGroups))
+	for _, g := range savedGroups {
+		savedByName[g.Name] = g
+	}
+
+	// For each group in the request, if Mappers is nil, load the saved mappers for that group name.
+	for _, g := range groups {
+		if g.Mappers != nil {
+			continue
+		}
+		saved, ok := savedByName[g.Group.Name]
+		if !ok {
+			return nil, errors.Newf(errors.TypeInvalidInput, spantypes.ErrCodeMappingGroupNotFound, "no saved group named %q to load mappers from; send 'mappers' for new or edited groups", g.Group.Name)
+		}
+		loaded, err := module.store.ListMappers(ctx, orgID, saved.ID)
+		if err != nil {
+			return nil, err
+		}
+		g.Mappers = loaded
+	}
+	return groups, nil
+}
+
 func (module *module) AgentFeatureType() agentConf.AgentFeatureType {
 	return spantypes.SpanAttrMappingFeatureType
 }

pkg/modules/spanmapper/spanmapper.go

@@ -27,6 +27,7 @@ type Module interface {
 	CreateMapper(ctx context.Context, orgID, groupID valuer.UUID, mapper *spantypes.SpanMapper) error
 	UpdateMapper(ctx context.Context, orgID, groupID, id valuer.UUID, fieldContext spantypes.FieldContext, config *spantypes.SpanMapperConfig, enabled *bool, updatedBy string) error
 	DeleteMapper(ctx context.Context, orgID, groupID, id valuer.UUID) error
+	TestMappers(ctx context.Context, orgID valuer.UUID, spans []spantypes.SpanMapperTestSpan, groups []*spantypes.SpanMapperGroupWithMappers) ([]spantypes.SpanMapperTestSpan, []string, error)
 }
 
 // Handler defines the HTTP handler interface for mapping group and mapper endpoints.
@@ -42,4 +43,5 @@ type Handler interface {
 	CreateMapper(rw http.ResponseWriter, r *http.Request)
 	UpdateMapper(rw http.ResponseWriter, r *http.Request)
 	DeleteMapper(rw http.ResponseWriter, r *http.Request)
+	TestMappers(rw http.ResponseWriter, r *http.Request)
 }

pkg/types/spantypes/spanmappersimulator.go

@@ -0,0 +1,186 @@
+package spantypes
+
+import (
+	"context"
+	"sort"
+	"strings"
+	"time"
+
+	"github.com/SigNoz/signoz-otel-collector/pkg/collectorsimulator"
+	"github.com/SigNoz/signoz-otel-collector/processor/signozspanmapperprocessor"
+	"github.com/SigNoz/signoz/pkg/errors"
+	"go.opentelemetry.io/collector/otelcol"
+	"go.opentelemetry.io/collector/pdata/ptrace"
+	"gopkg.in/yaml.v3"
+)
+
+var (
+	ErrCodeProcessorFactoryMapFailed  = errors.MustNewCode("processor_factory_map_failed")
+	ErrCodeSpanMapperSimulationFailed = errors.MustNewCode("span_mapper_simulation_failed")
+)
+
+const spanInputOrderAttr = "__signoz_input_idx__"
+
+func SimulateSpanMappersProcessing(ctx context.Context, groups []*SpanMapperGroupWithMappers, spans []SpanMapperTestSpan) ([]SpanMapperTestSpan, []string, error) {
+	enabled := filterEnabledGroupsWithMappers(groups)
+	if len(enabled) < 1 {
+		return spans, nil, nil
+	}
+
+	// this is done to preserve the order in which the request was sent
+	for i := range spans {
+		if spans[i].Attributes == nil {
+			spans[i].Attributes = map[string]any{}
+		}
+		spans[i].Attributes[spanInputOrderAttr] = int64(i)
+	}
+	simulatorInput := SpansToPTraces(spans)
+
+	processorFactories, err := otelcol.MakeFactoryMap(signozspanmapperprocessor.NewFactory())
+	if err != nil {
+		return nil, nil, errors.WrapInternalf(err, ErrCodeProcessorFactoryMapFailed, "could not construct processor factory map")
+	}
+
+	configGenerator := func(baseConf []byte) ([]byte, error) {
+		withProcessor, err := GenerateCollectorConfigWithSpanMapperProcessor(baseConf, enabled)
+		if err != nil {
+			return nil, err
+		}
+		return wireSpanMapperIntoTracesPipeline(withProcessor)
+	}
+
+	// signozspanmapperprocessor does no batching; spans flow through immediately.
+	timeout := 200 * time.Millisecond
+
+	outputTraces, collectorErrs, simErr := collectorsimulator.SimulateTracesProcessing(
+		ctx,
+		processorFactories,
+		configGenerator,
+		simulatorInput,
+		timeout,
+	)
+	if simErr != nil {
+		if errors.Is(simErr, collectorsimulator.ErrInvalidConfig) {
+			return nil, nil, errors.WrapInvalidInputf(simErr, errors.CodeInvalidInput, "invalid config")
+		}
+		return nil, nil, errors.WrapInternalf(simErr, ErrCodeSpanMapperSimulationFailed, "could not simulate span mapper processing")
+	}
+
+	outputSpans := PTracesToSpans(outputTraces)
+
+	sort.Slice(outputSpans, func(i, j int) bool {
+		iIdx, _ := outputSpans[i].Attributes[spanInputOrderAttr].(int64)
+		jIdx, _ := outputSpans[j].Attributes[spanInputOrderAttr].(int64)
+		return iIdx < jIdx
+	})
+	for _, s := range outputSpans {
+		delete(s.Attributes, spanInputOrderAttr)
+	}
+
+	collectorWarnAndErrorLogs := []string{}
+	for _, log := range collectorErrs {
+		if log == "" || strings.Contains(log, "featuregate.go") {
+			continue
+		}
+		collectorWarnAndErrorLogs = append(collectorWarnAndErrorLogs, log)
+	}
+
+	return outputSpans, collectorWarnAndErrorLogs, nil
+}
+
+func SpansToPTraces(spans []SpanMapperTestSpan) []ptrace.Traces {
+	result := make([]ptrace.Traces, 0, len(spans))
+	for _, s := range spans {
+		td := ptrace.NewTraces()
+		rs := td.ResourceSpans().AppendEmpty()
+		if s.Resource != nil {
+			_ = rs.Resource().Attributes().FromRaw(s.Resource)
+		}
+		sl := rs.ScopeSpans().AppendEmpty()
+		span := sl.Spans().AppendEmpty()
+		if s.Attributes != nil {
+			_ = span.Attributes().FromRaw(s.Attributes)
+		}
+		result = append(result, td)
+	}
+	return result
+}
+
+func PTracesToSpans(traces []ptrace.Traces) []SpanMapperTestSpan {
+	result := []SpanMapperTestSpan{}
+	for _, td := range traces {
+		rss := td.ResourceSpans()
+		for i := 0; i < rss.Len(); i++ {
+			rs := rss.At(i)
+			resourceAttrs := rs.Resource().Attributes().AsRaw()
+			ilss := rs.ScopeSpans()
+			for j := 0; j < ilss.Len(); j++ {
+				spans := ilss.At(j).Spans()
+				for k := 0; k < spans.Len(); k++ {
+					result = append(result, SpanMapperTestSpan{
+						Attributes: spans.At(k).Attributes().AsRaw(),
+						Resource:   resourceAttrs,
+					})
+				}
+			}
+		}
+	}
+	return result
+}
+
+// wireSpanMapperIntoTracesPipeline appends "signozspanmapper" to
+// service.pipelines.traces.processors so the processor defined by
+// GenerateCollectorConfigWithSpanMapperProcessor actually runs against the
+// traces flowing through the simulator. Idempotent: skips appending if the
+// processor name is already present.
+func wireSpanMapperIntoTracesPipeline(confYaml []byte) ([]byte, error) {
+	var conf map[string]any
+	if err := yaml.Unmarshal(confYaml, &conf); err != nil {
+		return nil, errors.Wrapf(err, errors.TypeInvalidInput, ErrCodeInvalidCollectorConfig, "failed to unmarshal collector config for pipeline wiring")
+	}
+	service, _ := conf["service"].(map[string]any)
+	if service == nil {
+		return confYaml, nil
+	}
+	pipelines, _ := service["pipelines"].(map[string]any)
+	if pipelines == nil {
+		return confYaml, nil
+	}
+	traces, _ := pipelines["traces"].(map[string]any)
+	if traces == nil {
+		return confYaml, nil
+	}
+
+	procs, _ := traces["processors"].([]any)
+	for _, p := range procs {
+		if name, ok := p.(string); ok && name == ProcessorName {
+			return confYaml, nil
+		}
+	}
+	traces["processors"] = append(procs, ProcessorName)
+
+	out, err := yaml.Marshal(conf)
+	if err != nil {
+		return nil, errors.Wrapf(err, errors.TypeInternal, ErrCodeBuildMappingProcessorConfig, "failed to marshal collector config after pipeline wiring")
+	}
+	return out, nil
+}
+
+func filterEnabledGroupsWithMappers(groups []*SpanMapperGroupWithMappers) []*SpanMapperGroupWithMappers {
+	out := make([]*SpanMapperGroupWithMappers, 0, len(groups))
+	for _, gm := range groups {
+		if gm == nil || gm.Group == nil || !gm.Group.Enabled {
+			continue
+		}
+		enabled := make([]*SpanMapper, 0, len(gm.Mappers))
+		for _, m := range gm.Mappers {
+			if m != nil && m.Enabled {
+				enabled = append(enabled, m)
+			}
+		}
+		if len(enabled) > 0 {
+			out = append(out, &SpanMapperGroupWithMappers{Group: gm.Group, Mappers: enabled})
+		}
+	}
+	return out
+}

pkg/types/spantypes/spanmappersimulator_test.go

@@ -0,0 +1,43 @@
+package spantypes
+
+import (
+	"context"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+// TestSimulateSpanMappersProcessing_EndToEnd is an integration test: it spins
+// up an actual in-memory otel-collector pipeline with signozspanmapperprocessor
+// and verifies the produced span has the expected target attribute.
+func TestSimulateSpanMappersProcessing_EndToEnd(t *testing.T) {
+	groups := []*SpanMapperGroupWithMappers{{
+		Group: &SpanMapperGroup{
+			Name:      "llm",
+			Condition: SpanMapperGroupCondition{Attributes: []string{"model"}},
+			Enabled:   true,
+		},
+		Mappers: []*SpanMapper{{
+			Name:         "gen_ai.request.model",
+			FieldContext: FieldContextSpanAttribute,
+			Config: SpanMapperConfig{Sources: []SpanMapperSource{
+				{Key: "llm.model", Context: FieldContextSpanAttribute, Operation: SpanMapperOperationCopy, Priority: 1},
+			}},
+			Enabled: true,
+		}},
+	}}
+
+	spans := []SpanMapperTestSpan{{Attributes: map[string]any{"llm.model": "gpt-4"}}}
+
+	out, _, err := SimulateSpanMappersProcessing(context.Background(), groups, spans)
+
+	require.NoError(t, err)
+	require.Len(t, out, 1)
+	assert.Equal(t, "gpt-4", out[0].Attributes["gen_ai.request.model"], "target attribute must be populated by the spanmapper processor")
+	// Source attribute is preserved (copy, not move).
+	assert.Equal(t, "gpt-4", out[0].Attributes["llm.model"])
+	// Order-tracking attribute is stripped from the output.
+	_, hasOrderAttr := out[0].Attributes[spanInputOrderAttr]
+	assert.False(t, hasOrderAttr, "internal ordering attribute must be removed from the response")
+}

pkg/types/spantypes/spanmappertest.go

@@ -0,0 +1,54 @@
+package spantypes
+
+import (
+	"github.com/SigNoz/signoz/pkg/valuer"
+)
+
+type SpanMapperTestSpan struct {
+	Attributes map[string]any `json:"attributes"`
+	Resource   map[string]any `json:"resource"`
+}
+
+// Mappers is optional because the module can backfill from the store by Group.Name.
+type PostableSpanMapperTestGroup struct {
+	PostableSpanMapperGroup
+	Mappers []PostableSpanMapper `json:"mappers"`
+}
+
+type PostableSpanMapperTest struct {
+	Spans  []SpanMapperTestSpan          `json:"spans"  required:"true"`
+	Groups []PostableSpanMapperTestGroup `json:"groups" required:"true"`
+}
+
+type GettableSpanMapperTest struct {
+	Spans         []SpanMapperTestSpan `json:"spans" required:"true"`
+	CollectorLogs []string             `json:"collectorLogs" nullable:"true"`
+}
+
+func NewSpanMapperGroupsWithMappersFromPostable(orgID valuer.UUID, in []PostableSpanMapperTestGroup) []*SpanMapperGroupWithMappers {
+	out := make([]*SpanMapperGroupWithMappers, 0, len(in))
+	for _, pg := range in {
+		var mappers []*SpanMapper
+		if pg.Mappers != nil {
+			mappers = make([]*SpanMapper, 0, len(pg.Mappers))
+			for _, pm := range pg.Mappers {
+				mappers = append(mappers, &SpanMapper{
+					Name:         pm.Name,
+					FieldContext: pm.FieldContext,
+					Config:       pm.Config,
+					Enabled:      pm.Enabled,
+				})
+			}
+		}
+		out = append(out, &SpanMapperGroupWithMappers{
+			Group: &SpanMapperGroup{
+				OrgID:     orgID,
+				Name:      pg.Name,
+				Condition: pg.Condition,
+				Enabled:   pg.Enabled,
+			},
+			Mappers: mappers,
+		})
+	}
+	return out
+}

pkg/types/telemetrytypes/field.go

@@ -4,7 +4,6 @@ import (
 	"fmt"
 	"strings"
 
-	"github.com/SigNoz/signoz-otel-collector/exporter/jsontypeexporter"
 	"github.com/SigNoz/signoz/pkg/valuer"
 )
 
@@ -22,7 +21,7 @@ const (
 	// BodyJSONStringSearchPrefix is the prefix used for body JSON search queries.
 	// e.g., "body.status" where "body." is the prefix.
 	BodyJSONStringSearchPrefix = "body."
-	ArraySep                   = jsontypeexporter.ArraySeparator
+	ArraySep                   = "[]."
 	ArraySepSuffix             = "[]"
 	// TODO(Piyush): Remove once we've migrated to the new array syntax.
 	ArrayAnyIndex       = "[*]."

pkg/types/telemetrytypes/json_access_plan.go

@@ -6,7 +6,6 @@ import (
 	"slices"
 	"strings"
 
-	"github.com/SigNoz/signoz-otel-collector/exporter/jsontypeexporter"
 	"github.com/SigNoz/signoz/pkg/errors"
 	"github.com/SigNoz/signoz/pkg/valuer"
 )
@@ -76,7 +75,7 @@ func (n *JSONAccessNode) Alias() string {
 	parentAlias := strings.TrimLeft(n.Parent.Alias(), "`")
 	parentAlias = strings.TrimRight(parentAlias, "`")
 
-	sep := jsontypeexporter.ArraySeparator
+	sep := "[]."
 	if n.Parent.isRoot {
 		sep = "."
 	}

tests/integration/tests/aiobservability/01_span_mapper.py

@@ -0,0 +1,174 @@
+from collections.abc import Callable
+from http import HTTPStatus
+
+import requests
+
+from fixtures import types
+from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
+
+GROUPS_PATH = "/api/v1/span_mapper_groups"
+
+
+def _auth_headers(token: str) -> dict[str, str]:
+    return {
+        "authorization": f"Bearer {token}",
+        "content-type": "application/json",
+    }
+
+
+def test_create_groups_and_simulate_with_backfill(
+    signoz: types.SigNoz,
+    create_user_admin: types.Operation,  # pylint: disable=unused-argument
+    get_token: Callable[[str, str], str],
+) -> None:
+    """
+    Setup:
+    Create a mapping group and a mapper through the CRUD API, then hit the
+    simulate endpoint with two groups: the saved one referenced by name only
+    (server backfills its mappers from the store) and a second, unsaved group
+    sent with inline mappers.
+
+    Tests:
+    1. Create a group (POST /span_mapper_groups) and read back its id
+    2. Create a mapper in that group (POST .../{groupId}/span_mappers)
+    3. List groups and mappers and verify what we created is persisted
+    4. Simulate spans against both groups: a matching span gets the saved
+       group's mappers (backfilled, copy) and the inline group's mappers
+       (move) applied, while a span matching no condition passes through
+    5. Delete the group and verify it (and its mappers) are gone
+    """
+    token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
+
+    create_group = requests.post(
+        signoz.self.host_configs["8080"].get(GROUPS_PATH),
+        timeout=10,
+        headers=_auth_headers(token),
+        json={
+            "name": "llm-backfill",
+            "condition": {"attributes": ["model"], "resource": []},
+            "enabled": True,
+        },
+    )
+
+    assert create_group.status_code == HTTPStatus.CREATED
+    group = create_group.json()["data"]
+    group_id = group["id"]
+    assert group["name"] == "llm-backfill"
+    assert group["enabled"] is True
+
+    create_mapper = requests.post(
+        signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{group_id}/span_mappers"),
+        timeout=10,
+        headers=_auth_headers(token),
+        json={
+            "name": "gen_ai.request.model",
+            "fieldContext": "attribute",
+            "config": {
+                "sources": [
+                    {
+                        "key": "llm.model",
+                        "context": "attribute",
+                        "operation": "copy",
+                        "priority": 1,
+                    }
+                ]
+            },
+            "enabled": True,
+        },
+    )
+
+    assert create_mapper.status_code == HTTPStatus.CREATED
+    mapper = create_mapper.json()["data"]
+    assert mapper["name"] == "gen_ai.request.model"
+    assert mapper["group_id"] == group_id
+
+    list_groups = requests.get(
+        signoz.self.host_configs["8080"].get(GROUPS_PATH),
+        timeout=10,
+        headers=_auth_headers(token),
+    )
+    assert list_groups.status_code == HTTPStatus.OK
+    assert group_id in [g["id"] for g in list_groups.json()["data"]["items"]]
+
+    list_mappers = requests.get(
+        signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{group_id}/span_mappers"),
+        timeout=10,
+        headers=_auth_headers(token),
+    )
+    assert list_mappers.status_code == HTTPStatus.OK
+    assert "gen_ai.request.model" in [m["name"] for m in list_mappers.json()["data"]["items"]]
+
+    simulate = requests.post(
+        signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/test"),
+        timeout=10,
+        headers=_auth_headers(token),
+        json={
+            "spans": [
+                {
+                    "attributes": {"llm.model": "gpt-4", "db.system": "postgres"},
+                    "resource": {},
+                },
+                # Matches no group condition, so it must pass through untouched.
+                {
+                    "attributes": {"http.method": "GET"},
+                    "resource": {},
+                },
+            ],
+            "groups": [
+                # No "mappers" key: the server backfills them from the saved group.
+                {
+                    "name": "llm-backfill",
+                    "condition": {"attributes": ["model"], "resource": []},
+                    "enabled": True,
+                },
+                # Unsaved group; mappers provided inline.
+                {
+                    "name": "db-inline",
+                    "condition": {"attributes": ["db"], "resource": []},
+                    "enabled": True,
+                    "mappers": [
+                        {
+                            "name": "db.name",
+                            "fieldContext": "attribute",
+                            "config": {
+                                "sources": [
+                                    {
+                                        "key": "db.system",
+                                        "context": "attribute",
+                                        "operation": "move",
+                                        "priority": 1,
+                                    }
+                                ]
+                            },
+                            "enabled": True,
+                        }
+                    ],
+                },
+            ],
+        },
+    )
+
+    assert simulate.status_code == HTTPStatus.OK
+    out_spans = simulate.json()["data"]["spans"]
+    assert len(out_spans) == 2
+    attrs = out_spans[0]["attributes"]
+    assert attrs["gen_ai.request.model"] == "gpt-4"  # backfilled mapper applied
+    assert attrs["llm.model"] == "gpt-4"  # copy preserves source
+    assert attrs["db.name"] == "postgres"  # inline mapper applied
+    assert "db.system" not in attrs  # move removes source
+    assert out_spans[1]["attributes"] == {"http.method": "GET"}  # unchanged
+
+    delete_group = requests.delete(
+        signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{group_id}"),
+        timeout=10,
+        headers=_auth_headers(token),
+    )
+    assert delete_group.status_code == HTTPStatus.NO_CONTENT
+
+    list_after_delete = requests.get(
+        signoz.self.host_configs["8080"].get(GROUPS_PATH),
+        timeout=10,
+        headers=_auth_headers(token),
+    )
+    assert list_after_delete.status_code == HTTPStatus.OK
+    assert group_id not in [g["id"] for g in list_after_delete.json()["data"]["items"]]

tests/integration/tests/querier/03_metrics.py

@@ -15,7 +15,6 @@ from fixtures.querier import (
     build_builder_query,
     find_named_result,
     get_all_warnings,
-    get_error_message,
     index_series_by_label,
     make_query_request,
 )