Compare commits

..

14 Commits

Author SHA1 Message Date
Nikhil Soni
ef6ac791e8 chore: generate api spec
Regenerated via: go run cmd/enterprise/*.go generate openapi && cd frontend && pnpm generate:api
Picks up UpdateSavedView's response becoming void (no body).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:46:57 +05:30
Nikhil Soni
b3ce3a5d1f refactor(savedview): drop redundant GetView after UpdateView
No consumer needs the full view back from Update: neither active
frontend caller of useUpdateView (ExplorerOptions.tsx, pages/SaveView/
index.tsx) reads the mutation's response data -- both just check
success and separately call refetchAllView(). The signoz-mcp-server
does its own GetView before calling Update for its own validation, so
it doesn't depend on Update's response either.

The GetView call was, however, the only thing making PUT on a
nonexistent view id return an error -- bun's raw UPDATE with a
non-matching WHERE clause doesn't error on its own, so removing the
extra fetch without replacement would have silently turned "update a
bogus id" into a fake success. Replaced it with a RowsAffected() check
on the UPDATE result instead (same pattern already used in
llmpricingrule/spanmapper stores) -- no extra DB round trip, and it's
strictly more correct: a bogus id now returns a proper "not found"
error (matching the ErrorStatusCodes: 404 already declared on
UpdateSavedView's OpenAPIDef, which wasn't actually enforced before).

Response for Update is now nil (matching Delete's convention), since
nothing consumes the body.

Verified with go build, golangci-lint (0 issues), and a live smoke
test: a normal update still persists (confirmed via a follow-up GET),
and updating a nonexistent id now returns saved_view_not_found instead
of a silent success. Also re-verified /api/v1/explorer/views (same
shared handler) is unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:44:00 +05:30
Nikhil Soni
8cddda3ed6 chore: generate api spec
Regenerated via: go run cmd/enterprise/*.go generate openapi && cd frontend && pnpm generate:api
docs/api/openapi.yml had no diff (already up to date from prior commits
this session); the frontend generated schemas pick up the savedview
type changes (SourcePage/PanelType/QueryType enums, dropped category/
tags).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:29:54 +05:30
Nikhil Soni
33fdde9939 refactor(savedview): keep Module.GetViewsForFilters on individual args
Passing the whole *ListSavedViewsParams through to the module ties the
module's signature to the handler's request-binding shape. Revert to
individual args (sourcePage, name) -- the handler still binds/validates
via ListSavedViewsParams, it just unpacks before calling into the
module.

Verified with go build, golangci-lint (0 issues), no openapi.yml diff
(pure module-boundary change), and a live List smoke test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:19:23 +05:30
Nikhil Soni
affb94031d refactor(savedview): drop unused category filter, bind List params via struct
category was never passed by the frontend on any List call -- confirmed
via grep, no call site sets it. Drop it from ListSavedViewsParams and
the category-branching query in GetViewsForFilters.

Also switch List's handler to the binding.Query.BindQuery(...) +
params.Validate() pattern already used by dashboard v2's list handler
(pkg/modules/dashboard/impldashboard/v2_handler.go), instead of manually
pulling each field off r.URL.Query(). GetViewsForFilters now takes the
whole *ListSavedViewsParams instead of separate sourcePage/name/category
strings, matching Module.ListV2's params-struct signature.

ListSavedViewsParams.Validate() skips the SourcePage check when it's
zero (unset), consistent with how ListSort/ListOrder handle optional
enum query params in ListFilter.Validate() -- but validates it strictly
otherwise, so an invalid sourcePage on List now returns a clear error
instead of silently matching zero rows.

Verified with go build, golangci-lint (0 issues), an openapi.yml
regeneration (category query param removed, clean diff), and a live
List smoke test: valid sourcePage filters correctly, no sourcePage
returns empty (unchanged from before), invalid sourcePage is now
rejected with a validation error.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:15:59 +05:30
Nikhil Soni
54787a63d8 feat(savedview): drop category/tags from Postable/GettableSavedView
Both are unused: category is set nowhere in the frontend and read
nowhere either (never in SaveViewProps/UpdateViewProps, only appears
in the response-only ViewProps type); tags has no write path at all
(no UI to add them) -- the one place it's read (a homepage widget
badge list) already guards against the empty-string artifact this
produces, confirming nothing ever populates it.

Keeping StorableSavedView.Category/.Tags as-is (still NOT NULL text
columns) to avoid a migration in this PR -- marked `// TODO:
deprecated, remove it` for a follow-up that drops the columns.
NewStorableSavedView/NewGettableSavedViewFromStorable no longer
read/write them, so new rows get empty-string defaults same as before
tags was ever set, and existing rows' values are simply never
surfaced.

Verified with go build, golangci-lint (0 issues), an openapi.yml
regeneration (clean field removal from both request/response schemas),
and a live create/get/update/list/delete smoke test with no category/
tags in the payload.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:06:28 +05:30
Nikhil Soni
06ab96984f refactor(savedview): separate createdBy/updatedBy in NewStorableSavedView
NewStorableSavedView took a single createdBy param and used it for both
CreatedBy and UpdatedBy, which is only correct for a fresh create. Split
it into separate createdBy/updatedBy args so a caller can vary them
independently (e.g. an update path that only needs to bump updatedBy).
Both CreateView and UpdateView currently pass claims.Email for both --
UpdateView's result is used to patch update_at/update_by plus the
content columns via Set(), so the throwaway id/createdAt/createdBy it
computes stay unused there, same as before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 16:02:01 +05:30
Nikhil Soni
104bcc55b9 refactor(savedview): move type construction/conversion into savedviewtypes
implsavedview/module.go was hand-building StorableSavedView/GettableSavedView
struct literals and doing json.Marshal/Unmarshal + strings.Join/Split inline
in every method. Mirrors the tagtypes convention (NewTag,
NewGettableTagFromTag/NewGettableTagsFromTags) instead:

- NewStorableSavedView(orgID, createdBy, PostableSavedView) (*StorableSavedView, error)
  builds the DB row from a request, generating id/timestamps.
- NewGettableSavedViewFromStorable(*StorableSavedView) (*GettableSavedView, error)
  and NewGettableSavedViewsFromStorable (batch) build the API response,
  unmarshalling the JSON-encoded query blob.

UpdateView reuses NewStorableSavedView too -- it only pulls Name/Category/
SourcePage/Tags/Data/ExtraData/UpdatedAt/UpdatedBy out of the result for its
column-level Set(), so the throwaway id/createdAt/createdBy it also computes
are harmless.

No behavior change: verified with go build, golangci-lint (0 issues), a
no-op openapi.yml regeneration (pure internal refactor, no type-shape
change), and a live create/get/update/list/delete smoke test confirming
createdAt/createdBy survive an update while updatedAt changes, and tags/
category/compositeQuery all round-trip correctly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 15:42:50 +05:30
Nikhil Soni
ffbdf26ed7 feat(savedview): add SourcePage enum with validation
sourcePage was a bare string. The frontend only ever sends one of 4
values -- traces/logs/metrics (its own DataSource enum) plus a
special-cased "meter" string literal for the meter explorer -- matching
the legacy v3.DataSource enum exactly. Category has no such fixed set
(unused by the frontend entirely, always empty) so it stays a string.

SourcePage is local to savedviewtypes (valuer.String-backed, same
pattern as PanelType/QueryType), validated on PostableSavedView.Validate(),
and used directly as the StorableSavedView bun column type -- valuer.String
already implements driver.Valuer/sql.Scanner so no extra plumbing is
needed for it to round-trip through the DB.

Verified with go build, golangci-lint (0 issues), an openapi.yml
regeneration (sourcePage now has a proper enum schema instead of a bare
string, in both the request/response bodies and the List query param),
and a live smoke test: valid sourcePage round-trips through create/get/
list, invalid values are rejected on create.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 15:39:14 +05:30
Nikhil Soni
97ee59636b refactor(savedview): drop legacy v3 dependency for panel/query type
savedviewtypes.CompositeQuery used pkg/query-service/model/v3.PanelType
and v3.QueryType, pulling in the legacy package purely for two small
enums. Alerting already solved this exact problem for its own
qbtypes.QueryEnvelope-based composite query (ruletypes.PanelType/
QueryType in pkg/types/ruletypes/alerting.go) -- mirror that pattern
here instead:

- savedviewtypes.PanelType: local enum with 5 values (value/graph/
  table/list/trace). ruletypes.PanelType only has 3 (value/table/
  graph), which isn't enough for saved views -- log/trace explorer
  views need list/trace too.
- savedviewtypes.QueryType: local enum (builder/clickhouse_sql/
  promql). This is a UI-tab-selector concept (which query-builder mode
  the view was last edited in), distinct from qbtypes.QueryEnvelope.Type
  (the per-query envelope discriminator, e.g. builder_query/
  builder_formula/clickhouse_sql/promql, which can differ across
  entries in the same Queries array). Keeping it, just re-typed
  locally instead of importing v3 for it.

Bonus: unlike v3.PanelType (a bare Go string with no schema enum),
these implement jsonschema.Enum, so the generated OpenAPI spec now
lists the acceptable values instead of a bare `type: string`.

Verified with go build, golangci-lint (0 issues), an openapi.yml
regeneration (diff is just the two new enum schemas plus $ref swaps),
and a live smoke test confirming the wire format is unchanged and
invalid panelType/queryType values are rejected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-24 12:53:44 +05:30
Nikhil Soni
6e87eff704 chore: generate api spec 2026-07-23 17:05:37 +05:30
Nikhil Soni
21ac0f44e0 feat(savedview): make SavedView.CompositeQuery v5-only
Splits the domain type into request/response shapes so create/update
don't require id/createdAt/createdBy/updatedAt/updatedBy from the
client:

- PostableSavedView: request body for create/update (no id or
  server-populated audit fields).
- UpdatableSavedView: alias of PostableSavedView (a saved view is
  always replaced in full).
- GettableSavedView: response shape for get/list/create's-echo,
  carrying id and audit fields.

Module and Handler interfaces updated accordingly (CreateView/
UpdateView take PostableSavedView/UpdatableSavedView, GetView/
GetViewsForFilters return *GettableSavedView). The Update handler now
re-fetches and returns the persisted view instead of echoing the
request body, since callers (including the existing frontend type,
UpdateViewPayloadProps.data: ViewProps) expect id/timestamps back.

Also, per the v5-only typing work this continues:
- CompositeQuery is the new name for the saved-view query type
  (matches the established qbtypes.CompositeQuery naming), replacing
  the legacy v3.CompositeQuery for this domain.
- pkg/query-service/model/v3/v3.go now only differs from origin/main
  by the SavedView struct removal (and its now-unused valuer import)
  -- no stray struct tags left over from earlier iterations.
- Validate() uses errors.NewInvalidInputf with a package error code
  instead of fmt.Errorf, matching the forbidigo-clean pattern used
  elsewhere (e.g. dashboardtypes).
- pkg/types/savedviewtypes consolidated down to query.go and
  savedview.go; the separate list.go/domain.go files are gone.

Verified with go build, golangci-lint (0 issues), an openapi.yml
regeneration (clean diff), and a live create/get/update/list/delete
smoke test against /api/v2/saved_views.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb5R7Eo19HUukBMVCgKPjS
2026-07-22 16:39:02 +05:30
Nikhil Soni
735b9e7d68 refactor(savedview): move SavedView domain type into savedviewtypes
Relocates the saved-view domain type from pkg/query-service/model/v3
(v3.SavedView) to pkg/types/savedviewtypes.SavedView, the conventional
home for domain types, following the same Handler/Module signatures.
The existing bun-persisted row type in that package is renamed from
SavedView to StorableSavedView to avoid a name collision (matching the
Storable*/domain-type naming convention used elsewhere, e.g.
dashboardtypes.StorableDashboard).

v3.SavedView was only referenced by the savedview module/handler
(verified via grep), so this is a mechanical move -- CompositeQuery
itself stays in model/v3 since ~25 other files depend on it.

No new handler methods or request/response types: the v2 routes added
in the previous commit keep using the same Create/Get/Update/Delete/
List methods as /api/v1/explorer/views. CompositeQuery already has
omitempty legacy (builderQueries/chQueries/promQueries) and v5
(queries) fields side by side, and Validate() already accepts a
v5-only payload, so one type/handler pair genuinely serves both API
versions -- no conversion layer needed.

Refs SigNoz/engineering-pod#4651
2026-07-21 15:31:13 +05:30
Nikhil Soni
29315d8c89 refactor: register savedview handler in signozapiserver
feat(savedview): register v2 saved view routes via handler.New()

Registers List/Create/Get/Update/Delete for saved views at
/api/v2/saved_views(/{viewId}) in signozapiserver using handler.New()
with OpenAPIDef, mirroring the alertmanager migration (#10941). This
unblocks audit-log instrumentation and Terraform resource generation,
which the legacy router.HandleFunc registrations in http_handler.go
can't support.

/api/v1/explorer/views keeps working unchanged. Wires the previously
dead addSavedViewRoutes into provider.AddToRouter, adds the missing
authz middleware (ViewAccess/EditAccess, matching the v1 access split),
adds required/nullable OpenAPI tags to v3.SavedView/CompositeQuery, and
regenerates docs/api/openapi.yml.

Refs SigNoz/engineering-pod#4651
2026-07-21 15:19:57 +05:30
49 changed files with 1464 additions and 3952 deletions

View File

@@ -53,7 +53,6 @@ jobs:
- queriermetrics
- querierscalar
- queriercommon
- querierai
- rawexportdata
- querierauthz
- role

View File

@@ -9,11 +9,6 @@ global:
# the path component (e.g. /signoz in https://example.com/signoz) is used
# as the base path for all HTTP routes (both API and web frontend).
external_url: <unset>
# origins (scheme://host[:port], no path) allowed as login redirect targets. include
# the origin the signoz ui is served on. when not configured, redirect targets are
# not validated.
# allowed_origins:
# - https://signoz.example.com
# the url where the SigNoz backend receives telemetry data (traces, metrics, logs) from instrumented applications.
ingestion_url: <unset>
# the url of the SigNoz MCP server. when unset, the MCP settings page is hidden in the frontend.

View File

@@ -6992,7 +6992,6 @@ components:
Querybuildertypesv5QueryEnvelope:
discriminator:
mapping:
builder_ai_query: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI'
builder_formula: '#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula'
builder_query: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder'
builder_trace_operator: '#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator'
@@ -7001,7 +7000,6 @@ components:
propertyName: type
oneOf:
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopePromQL'
@@ -7016,15 +7014,6 @@ components:
required:
- type
type: object
Querybuildertypesv5QueryEnvelopeBuilderAI:
properties:
spec:
$ref: '#/components/schemas/Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregation'
type:
$ref: '#/components/schemas/Querybuildertypesv5QueryType'
required:
- type
type: object
Querybuildertypesv5QueryEnvelopeClickHouseSQL:
properties:
spec:
@@ -7138,7 +7127,6 @@ components:
Querybuildertypesv5QueryType:
enum:
- builder_query
- builder_ai_query
- builder_formula
- builder_trace_operator
- clickhouse_sql
@@ -7843,6 +7831,92 @@ components:
enum:
- basic
type: string
SavedviewtypesCompositeQuery:
properties:
panelType:
$ref: '#/components/schemas/SavedviewtypesPanelType'
queries:
items:
$ref: '#/components/schemas/Querybuildertypesv5QueryEnvelope'
nullable: true
type: array
queryType:
$ref: '#/components/schemas/SavedviewtypesQueryType'
required:
- panelType
- queryType
- queries
type: object
SavedviewtypesGettableSavedView:
properties:
compositeQuery:
$ref: '#/components/schemas/SavedviewtypesCompositeQuery'
createdAt:
format: date-time
type: string
createdBy:
type: string
extraData:
type: string
id:
type: string
name:
type: string
sourcePage:
$ref: '#/components/schemas/SavedviewtypesSourcePage'
updatedAt:
format: date-time
type: string
updatedBy:
type: string
required:
- id
- name
- createdAt
- createdBy
- updatedAt
- updatedBy
- sourcePage
- compositeQuery
- extraData
type: object
SavedviewtypesPanelType:
enum:
- value
- graph
- table
- list
- trace
type: string
SavedviewtypesPostableSavedView:
properties:
compositeQuery:
$ref: '#/components/schemas/SavedviewtypesCompositeQuery'
extraData:
type: string
name:
type: string
sourcePage:
$ref: '#/components/schemas/SavedviewtypesSourcePage'
required:
- name
- sourcePage
- compositeQuery
- extraData
type: object
SavedviewtypesQueryType:
enum:
- builder
- clickhouse_sql
- promql
type: string
SavedviewtypesSourcePage:
enum:
- traces
- logs
- metrics
- meter
type: string
ServiceaccounttypesDeprecatedPostableServiceAccountRole:
properties:
id:
@@ -22543,6 +22617,300 @@ paths:
summary: Test alert rule
tags:
- rules
/api/v2/saved_views:
get:
deprecated: false
description: Returns saved views for the calling user's org, optionally filtered
by source page, name, and category.
operationId: ListSavedViews
parameters:
- in: query
name: sourcePage
schema:
$ref: '#/components/schemas/SavedviewtypesSourcePage'
- in: query
name: name
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/SavedviewtypesGettableSavedView'
nullable: true
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- VIEWER
- tokenizer:
- VIEWER
summary: List saved views
tags:
- saved_view
post:
deprecated: false
description: Persists a saved view for the explore page. Returns the id of the
created view.
operationId: CreateSavedView
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/SavedviewtypesPostableSavedView'
responses:
"200":
content:
application/json:
schema:
properties:
data:
nullable: true
type: string
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Create saved view
tags:
- saved_view
/api/v2/saved_views/{viewId}:
delete:
deprecated: false
description: Deletes a saved view by id.
operationId: DeleteSavedView
parameters:
- in: path
name: viewId
required: true
schema:
type: string
responses:
"200":
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Delete saved view
tags:
- saved_view
get:
deprecated: false
description: Returns a saved view by id.
operationId: GetSavedView
parameters:
- in: path
name: viewId
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/SavedviewtypesGettableSavedView'
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- VIEWER
- tokenizer:
- VIEWER
summary: Get saved view
tags:
- saved_view
put:
deprecated: false
description: Replaces a saved view's name, tags, and query.
operationId: UpdateSavedView
parameters:
- in: path
name: viewId
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/SavedviewtypesPostableSavedView'
responses:
"200":
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Update saved view
tags:
- saved_view
/api/v2/sessions:
delete:
deprecated: false

View File

@@ -0,0 +1,491 @@
/**
* ! Do not edit manually
* * The file has been auto-generated using Orval for SigNoz
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useMutation, useQuery } from 'react-query';
import type {
InvalidateOptions,
MutationFunction,
QueryClient,
QueryFunction,
QueryKey,
UseMutationOptions,
UseMutationResult,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type {
CreateSavedView200,
DeleteSavedViewPathParameters,
GetSavedView200,
GetSavedViewPathParameters,
ListSavedViews200,
ListSavedViewsParams,
RenderErrorResponseDTO,
SavedviewtypesPostableSavedViewDTO,
UpdateSavedViewPathParameters,
} from '../sigNoz.schemas';
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
/**
* Returns saved views for the calling user's org, optionally filtered by source page, name, and category.
* @summary List saved views
*/
export const listSavedViews = (
params?: ListSavedViewsParams,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<ListSavedViews200>({
url: `/api/v2/saved_views`,
method: 'GET',
params,
signal,
});
};
export const getListSavedViewsQueryKey = (params?: ListSavedViewsParams) => {
return [`/api/v2/saved_views`, ...(params ? [params] : [])] as const;
};
export const getListSavedViewsQueryOptions = <
TData = Awaited<ReturnType<typeof listSavedViews>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
params?: ListSavedViewsParams,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listSavedViews>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey = queryOptions?.queryKey ?? getListSavedViewsQueryKey(params);
const queryFn: QueryFunction<Awaited<ReturnType<typeof listSavedViews>>> = ({
signal,
}) => listSavedViews(params, signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof listSavedViews>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListSavedViewsQueryResult = NonNullable<
Awaited<ReturnType<typeof listSavedViews>>
>;
export type ListSavedViewsQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary List saved views
*/
export function useListSavedViews<
TData = Awaited<ReturnType<typeof listSavedViews>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
params?: ListSavedViewsParams,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listSavedViews>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListSavedViewsQueryOptions(params, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary List saved views
*/
export const invalidateListSavedViews = async (
queryClient: QueryClient,
params?: ListSavedViewsParams,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListSavedViewsQueryKey(params) },
options,
);
return queryClient;
};
/**
* Persists a saved view for the explore page. Returns the id of the created view.
* @summary Create saved view
*/
export const createSavedView = (
savedviewtypesPostableSavedViewDTO?: BodyType<SavedviewtypesPostableSavedViewDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<CreateSavedView200>({
url: `/api/v2/saved_views`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: savedviewtypesPostableSavedViewDTO,
signal,
});
};
export const getCreateSavedViewMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof createSavedView>>,
TError,
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof createSavedView>>,
TError,
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
TContext
> => {
const mutationKey = ['createSavedView'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof createSavedView>>,
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> }
> = (props) => {
const { data } = props ?? {};
return createSavedView(data);
};
return { mutationFn, ...mutationOptions };
};
export type CreateSavedViewMutationResult = NonNullable<
Awaited<ReturnType<typeof createSavedView>>
>;
export type CreateSavedViewMutationBody =
| BodyType<SavedviewtypesPostableSavedViewDTO>
| undefined;
export type CreateSavedViewMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Create saved view
*/
export const useCreateSavedView = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof createSavedView>>,
TError,
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof createSavedView>>,
TError,
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
TContext
> => {
return useMutation(getCreateSavedViewMutationOptions(options));
};
/**
* Deletes a saved view by id.
* @summary Delete saved view
*/
export const deleteSavedView = (
{ viewId }: DeleteSavedViewPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/saved_views/${viewId}`,
method: 'DELETE',
signal,
});
};
export const getDeleteSavedViewMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof deleteSavedView>>,
TError,
{ pathParams: DeleteSavedViewPathParameters },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof deleteSavedView>>,
TError,
{ pathParams: DeleteSavedViewPathParameters },
TContext
> => {
const mutationKey = ['deleteSavedView'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof deleteSavedView>>,
{ pathParams: DeleteSavedViewPathParameters }
> = (props) => {
const { pathParams } = props ?? {};
return deleteSavedView(pathParams);
};
return { mutationFn, ...mutationOptions };
};
export type DeleteSavedViewMutationResult = NonNullable<
Awaited<ReturnType<typeof deleteSavedView>>
>;
export type DeleteSavedViewMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Delete saved view
*/
export const useDeleteSavedView = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof deleteSavedView>>,
TError,
{ pathParams: DeleteSavedViewPathParameters },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof deleteSavedView>>,
TError,
{ pathParams: DeleteSavedViewPathParameters },
TContext
> => {
return useMutation(getDeleteSavedViewMutationOptions(options));
};
/**
* Returns a saved view by id.
* @summary Get saved view
*/
export const getSavedView = (
{ viewId }: GetSavedViewPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetSavedView200>({
url: `/api/v2/saved_views/${viewId}`,
method: 'GET',
signal,
});
};
export const getGetSavedViewQueryKey = ({
viewId,
}: GetSavedViewPathParameters) => {
return [`/api/v2/saved_views/${viewId}`] as const;
};
export const getGetSavedViewQueryOptions = <
TData = Awaited<ReturnType<typeof getSavedView>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ viewId }: GetSavedViewPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getSavedView>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey = queryOptions?.queryKey ?? getGetSavedViewQueryKey({ viewId });
const queryFn: QueryFunction<Awaited<ReturnType<typeof getSavedView>>> = ({
signal,
}) => getSavedView({ viewId }, signal);
return {
queryKey,
queryFn,
enabled: !!viewId,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof getSavedView>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetSavedViewQueryResult = NonNullable<
Awaited<ReturnType<typeof getSavedView>>
>;
export type GetSavedViewQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Get saved view
*/
export function useGetSavedView<
TData = Awaited<ReturnType<typeof getSavedView>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ viewId }: GetSavedViewPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getSavedView>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetSavedViewQueryOptions({ viewId }, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary Get saved view
*/
export const invalidateGetSavedView = async (
queryClient: QueryClient,
{ viewId }: GetSavedViewPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetSavedViewQueryKey({ viewId }) },
options,
);
return queryClient;
};
/**
* Replaces a saved view's name, tags, and query.
* @summary Update saved view
*/
export const updateSavedView = (
{ viewId }: UpdateSavedViewPathParameters,
savedviewtypesPostableSavedViewDTO?: BodyType<SavedviewtypesPostableSavedViewDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/saved_views/${viewId}`,
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
data: savedviewtypesPostableSavedViewDTO,
signal,
});
};
export const getUpdateSavedViewMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof updateSavedView>>,
TError,
{
pathParams: UpdateSavedViewPathParameters;
data?: BodyType<SavedviewtypesPostableSavedViewDTO>;
},
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof updateSavedView>>,
TError,
{
pathParams: UpdateSavedViewPathParameters;
data?: BodyType<SavedviewtypesPostableSavedViewDTO>;
},
TContext
> => {
const mutationKey = ['updateSavedView'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof updateSavedView>>,
{
pathParams: UpdateSavedViewPathParameters;
data?: BodyType<SavedviewtypesPostableSavedViewDTO>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
return updateSavedView(pathParams, data);
};
return { mutationFn, ...mutationOptions };
};
export type UpdateSavedViewMutationResult = NonNullable<
Awaited<ReturnType<typeof updateSavedView>>
>;
export type UpdateSavedViewMutationBody =
| BodyType<SavedviewtypesPostableSavedViewDTO>
| undefined;
export type UpdateSavedViewMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Update saved view
*/
export const useUpdateSavedView = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof updateSavedView>>,
TError,
{
pathParams: UpdateSavedViewPathParameters;
data?: BodyType<SavedviewtypesPostableSavedViewDTO>;
},
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof updateSavedView>>,
TError,
{
pathParams: UpdateSavedViewPathParameters;
data?: BodyType<SavedviewtypesPostableSavedViewDTO>;
},
TContext
> => {
return useMutation(getUpdateSavedViewMutationOptions(options));
};

View File

@@ -4346,18 +4346,6 @@ export interface Querybuildertypesv5QueryEnvelopeBuilderDTO {
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType;
}
export enum Querybuildertypesv5QueryEnvelopeBuilderAIDTOType {
builder_ai_query = 'builder_ai_query',
}
export interface Querybuildertypesv5QueryEnvelopeBuilderAIDTO {
spec?: Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregationDTO;
/**
* @type string
* @enum builder_ai_query
*/
type: Querybuildertypesv5QueryEnvelopeBuilderAIDTOType;
}
export interface Querybuildertypesv5QueryBuilderFormulaDTO {
/**
* @type boolean
@@ -4541,7 +4529,6 @@ export interface Querybuildertypesv5QueryEnvelopeClickHouseSQLDTO {
export type Querybuildertypesv5QueryEnvelopeDTO =
| Querybuildertypesv5QueryEnvelopeBuilderDTO
| Querybuildertypesv5QueryEnvelopeBuilderAIDTO
| Querybuildertypesv5QueryEnvelopeFormulaDTO
| Querybuildertypesv5QueryEnvelopeTraceOperatorDTO
| Querybuildertypesv5QueryEnvelopePromQLDTO
@@ -8386,7 +8373,6 @@ export interface Querybuildertypesv5QueryRangeResponseDTO {
export enum Querybuildertypesv5QueryTypeDTO {
builder_query = 'builder_query',
builder_ai_query = 'builder_ai_query',
builder_formula = 'builder_formula',
builder_trace_operator = 'builder_trace_operator',
clickhouse_sql = 'clickhouse_sql',
@@ -8934,6 +8920,81 @@ export interface RuletypesRuleDTO {
export enum RuletypesThresholdKindDTO {
basic = 'basic',
}
export enum SavedviewtypesPanelTypeDTO {
value = 'value',
graph = 'graph',
table = 'table',
list = 'list',
trace = 'trace',
}
export enum SavedviewtypesQueryTypeDTO {
builder = 'builder',
clickhouse_sql = 'clickhouse_sql',
promql = 'promql',
}
export interface SavedviewtypesCompositeQueryDTO {
panelType: SavedviewtypesPanelTypeDTO;
/**
* @type array,null
*/
queries: Querybuildertypesv5QueryEnvelopeDTO[] | null;
queryType: SavedviewtypesQueryTypeDTO;
}
export enum SavedviewtypesSourcePageDTO {
traces = 'traces',
logs = 'logs',
metrics = 'metrics',
meter = 'meter',
}
export interface SavedviewtypesGettableSavedViewDTO {
compositeQuery: SavedviewtypesCompositeQueryDTO;
/**
* @type string
* @format date-time
*/
createdAt: string;
/**
* @type string
*/
createdBy: string;
/**
* @type string
*/
extraData: string;
/**
* @type string
*/
id: string;
/**
* @type string
*/
name: string;
sourcePage: SavedviewtypesSourcePageDTO;
/**
* @type string
* @format date-time
*/
updatedAt: string;
/**
* @type string
*/
updatedBy: string;
}
export interface SavedviewtypesPostableSavedViewDTO {
compositeQuery: SavedviewtypesCompositeQueryDTO;
/**
* @type string
*/
extraData: string;
/**
* @type string
*/
name: string;
sourcePage: SavedviewtypesSourcePageDTO;
}
export interface ServiceaccounttypesDeprecatedPostableServiceAccountRoleDTO {
/**
* @type string
@@ -12060,6 +12121,57 @@ export type TestRule200 = {
status: string;
};
export type ListSavedViewsParams = {
/**
* @description undefined
*/
sourcePage?: SavedviewtypesSourcePageDTO;
/**
* @type string
* @description undefined
*/
name?: string;
};
export type ListSavedViews200 = {
/**
* @type array,null
*/
data: SavedviewtypesGettableSavedViewDTO[] | null;
/**
* @type string
*/
status: string;
};
export type CreateSavedView200 = {
/**
* @type string,null
*/
data: string | null;
/**
* @type string
*/
status: string;
};
export type DeleteSavedViewPathParameters = {
viewId: string;
};
export type GetSavedViewPathParameters = {
viewId: string;
};
export type GetSavedView200 = {
data: SavedviewtypesGettableSavedViewDTO;
/**
* @type string
*/
status: string;
};
export type UpdateSavedViewPathParameters = {
viewId: string;
};
export type GetSessionContext200 = {
data: AuthtypesSessionContextDTO;
/**

View File

@@ -25,6 +25,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/modules/rawdataexport"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/savedview"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/modules/session"
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
@@ -75,6 +76,7 @@ type provider struct {
rulerHandler ruler.Handler
llmPricingRuleHandler llmpricingrule.Handler
statsHandler statsreporter.Handler
savedViewHandler savedview.Handler
}
func NewFactory(
@@ -110,6 +112,7 @@ func NewFactory(
traceDetailHandler tracedetail.Handler,
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
) factory.ProviderFactory[apiserver.APIServer, apiserver.Config] {
return factory.NewProviderFactory(factory.MustNewName("signoz"), func(ctx context.Context, providerSettings factory.ProviderSettings, config apiserver.Config) (apiserver.APIServer, error) {
return newProvider(
@@ -148,6 +151,7 @@ func NewFactory(
traceDetailHandler,
rulerHandler,
statsHandler,
savedViewHandler,
)
})
}
@@ -188,6 +192,7 @@ func newProvider(
traceDetailHandler tracedetail.Handler,
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
) (apiserver.APIServer, error) {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/apiserver/signozapiserver")
router := mux.NewRouter().UseEncodedPath()
@@ -227,6 +232,7 @@ func newProvider(
rulerHandler: rulerHandler,
llmPricingRuleHandler: llmPricingRuleHandler,
statsHandler: statsHandler,
savedViewHandler: savedViewHandler,
}
provider.authzMiddleware = middleware.NewAuthZ(settings.Logger(), orgGetter, authzService)
@@ -359,6 +365,10 @@ func (provider *provider) AddToRouter(router *mux.Router) error {
return err
}
if err := provider.addSavedViewRoutes(router); err != nil {
return err
}
return nil
}

View File

@@ -0,0 +1,101 @@
package signozapiserver
import (
"net/http"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
func (provider *provider) addSavedViewRoutes(router *mux.Router) error {
if err := router.Handle("/api/v2/saved_views", handler.New(provider.authzMiddleware.ViewAccess(provider.savedViewHandler.List), handler.OpenAPIDef{
ID: "ListSavedViews",
Tags: []string{"saved_view"},
Summary: "List saved views",
Description: "Returns saved views for the calling user's org, optionally filtered by source page, name, and category.",
Request: nil,
RequestQuery: new(savedviewtypes.ListSavedViewsParams),
RequestContentType: "",
Response: new([]*savedviewtypes.GettableSavedView),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/saved_views", handler.New(provider.authzMiddleware.EditAccess(provider.savedViewHandler.Create), handler.OpenAPIDef{
ID: "CreateSavedView",
Tags: []string{"saved_view"},
Summary: "Create saved view",
Description: "Persists a saved view for the explore page. Returns the id of the created view.",
Request: new(savedviewtypes.PostableSavedView),
RequestContentType: "application/json",
Response: new(valuer.UUID),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/saved_views/{viewId}", handler.New(provider.authzMiddleware.ViewAccess(provider.savedViewHandler.Get), handler.OpenAPIDef{
ID: "GetSavedView",
Tags: []string{"saved_view"},
Summary: "Get saved view",
Description: "Returns a saved view by id.",
Request: nil,
RequestContentType: "",
Response: new(savedviewtypes.GettableSavedView),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/saved_views/{viewId}", handler.New(provider.authzMiddleware.EditAccess(provider.savedViewHandler.Update), handler.OpenAPIDef{
ID: "UpdateSavedView",
Tags: []string{"saved_view"},
Summary: "Update saved view",
Description: "Replaces a saved view's name, tags, and query.",
Request: new(savedviewtypes.UpdatableSavedView),
RequestContentType: "application/json",
Response: nil,
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPut).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/saved_views/{viewId}", handler.New(provider.authzMiddleware.EditAccess(provider.savedViewHandler.Delete), handler.OpenAPIDef{
ID: "DeleteSavedView",
Tags: []string{"saved_view"},
Summary: "Delete saved view",
Description: "Deletes a saved view by id.",
Request: nil,
RequestContentType: "",
Response: nil,
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodDelete).GetError(); err != nil {
return err
}
return nil
}

View File

@@ -12,15 +12,13 @@ import (
var (
ErrCodeInvalidGlobalConfig = errors.MustNewCode("invalid_global_config")
ErrCodeOriginNotAllowed = errors.MustNewCode("origin_not_allowed")
)
type Config struct {
ExternalURL *url.URL `mapstructure:"external_url"`
AllowedOrigins []*url.URL `mapstructure:"allowed_origins"`
IngestionURL *url.URL `mapstructure:"ingestion_url"`
MCPURL *url.URL `mapstructure:"mcp_url"`
AIAssistantURL *url.URL `mapstructure:"ai_assistant_url"`
ExternalURL *url.URL `mapstructure:"external_url"`
IngestionURL *url.URL `mapstructure:"ingestion_url"`
MCPURL *url.URL `mapstructure:"mcp_url"`
AIAssistantURL *url.URL `mapstructure:"ai_assistant_url"`
}
func NewConfigFactory() factory.ConfigFactory {
@@ -51,33 +49,9 @@ func (c Config) Validate() error {
}
}
for _, origin := range c.AllowedOrigins {
if origin == nil || origin.Scheme == "" || origin.Host == "" {
return errors.NewInvalidInputf(ErrCodeInvalidGlobalConfig, "global::allowed_origins entries must be of the form scheme://host[:port], got %q", origin)
}
if origin.Path != "" && origin.Path != "/" {
return errors.NewInvalidInputf(ErrCodeInvalidGlobalConfig, "global::allowed_origins entries must not contain a path, got %q", origin)
}
}
return nil
}
func (c Config) IsOriginAllowed(u *url.URL) bool {
if len(c.AllowedOrigins) == 0 {
return true
}
for _, origin := range c.AllowedOrigins {
if strings.EqualFold(origin.Scheme, u.Scheme) && strings.EqualFold(origin.Host, u.Host) {
return true
}
}
return false
}
func (c Config) ExternalPath() string {
if c.ExternalURL == nil || c.ExternalURL.Path == "" || c.ExternalURL.Path == "/" {
return ""

View File

@@ -123,26 +123,6 @@ func TestValidate(t *testing.T) {
config: Config{ExternalURL: &url.URL{Path: "signoz"}},
fail: true,
},
{
name: "ValidAllowedOrigin",
config: Config{AllowedOrigins: []*url.URL{{Scheme: "https", Host: "signoz.example.com"}}},
fail: false,
},
{
name: "AllowedOriginWithoutScheme",
config: Config{AllowedOrigins: []*url.URL{{Host: "signoz.example.com"}}},
fail: true,
},
{
name: "AllowedOriginWithoutHost",
config: Config{AllowedOrigins: []*url.URL{{Scheme: "https"}}},
fail: true,
},
{
name: "AllowedOriginWithPath",
config: Config{AllowedOrigins: []*url.URL{{Scheme: "https", Host: "signoz.example.com", Path: "/login"}}},
fail: true,
},
}
for _, tc := range testCases {
@@ -157,96 +137,3 @@ func TestValidate(t *testing.T) {
})
}
}
func TestIsOriginAllowedWhenUnconfigured(t *testing.T) {
testCases := []struct {
name string
config Config
}{
{
name: "Empty",
config: Config{},
},
{
name: "ExternalURLDoesNotActivateValidation",
config: Config{ExternalURL: &url.URL{Scheme: "https", Host: "signoz.example.com"}},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
u, err := url.Parse("https://anything.example.com/login")
assert.NoError(t, err)
assert.True(t, tc.config.IsOriginAllowed(u))
})
}
}
func TestIsOriginAllowed(t *testing.T) {
config := Config{
AllowedOrigins: []*url.URL{
{Scheme: "https", Host: "signoz.example.com"},
{Scheme: "http", Host: "localhost:3301"},
},
}
testCases := []struct {
name string
input string
expected bool
}{
{
name: "ConfiguredOrigin",
input: "https://signoz.example.com/login",
expected: true,
},
{
name: "ConfiguredOriginWithQuery",
input: "http://localhost:3301/login?next=/dashboards",
expected: true,
},
{
name: "CaseInsensitiveHost",
input: "https://SigNoz.Example.Com/login",
expected: true,
},
{
name: "UnknownHost",
input: "https://attacker.example.com/login",
expected: false,
},
{
name: "SchemeMismatch",
input: "http://signoz.example.com/login",
expected: false,
},
{
name: "PortMismatch",
input: "https://signoz.example.com:8443/login",
expected: false,
},
{
name: "SuffixConfusion",
input: "https://evilsignoz.example.com/login",
expected: false,
},
{
name: "UserInfoConfusion",
input: "https://signoz.example.com@attacker.example.com/login",
expected: false,
},
{
name: "SchemeRelative",
input: "//attacker.example.com/login",
expected: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
u, err := url.Parse(tc.input)
assert.NoError(t, err)
assert.Equal(t, tc.expected, config.IsOriginAllowed(u))
})
}
}

View File

@@ -213,18 +213,18 @@ func (module *module) discoverModels(ctx context.Context, orgID valuer.UUID) ([]
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: fmt.Sprintf("%s EXISTS", telemetrytypes.GenAIRequestModel)},
Filter: &qbtypes.Filter{Expression: fmt.Sprintf("%s EXISTS", llmpricingruletypes.GenAIRequestModel)},
Aggregations: []qbtypes.TraceAggregation{
{Expression: "count()", Alias: "spanCount"},
},
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: telemetrytypes.GenAIRequestModel,
Name: llmpricingruletypes.GenAIRequestModel,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: telemetrytypes.GenAIProviderName,
Name: llmpricingruletypes.GenAIProviderName,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
@@ -254,9 +254,9 @@ func (module *module) discoverModels(ctx context.Context, orgID valuer.UUID) ([]
switch c.Type {
case qbtypes.ColumnTypeGroup:
switch c.Name {
case telemetrytypes.GenAIRequestModel:
case llmpricingruletypes.GenAIRequestModel:
modelIdx = i
case telemetrytypes.GenAIProviderName:
case llmpricingruletypes.GenAIProviderName:
providerIdx = i
}
case qbtypes.ColumnTypeAggregation:

View File

@@ -7,10 +7,11 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/modules/savedview"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
@@ -33,7 +34,7 @@ func (handler *handler) Create(w http.ResponseWriter, r *http.Request) {
return
}
var view v3.SavedView
var view savedviewtypes.PostableSavedView
if err := json.NewDecoder(r.Body).Decode(&view); err != nil {
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to decode request body"))
return
@@ -95,7 +96,7 @@ func (handler *handler) Update(w http.ResponseWriter, r *http.Request) {
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse view id"))
return
}
var view v3.SavedView
var view savedviewtypes.UpdatableSavedView
if err := json.NewDecoder(r.Body).Decode(&view); err != nil {
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to decode request body"))
return
@@ -112,7 +113,7 @@ func (handler *handler) Update(w http.ResponseWriter, r *http.Request) {
return
}
render.Success(w, http.StatusOK, view)
render.Success(w, http.StatusOK, nil)
}
func (handler *handler) Delete(w http.ResponseWriter, r *http.Request) {
@@ -151,11 +152,17 @@ func (handler *handler) List(w http.ResponseWriter, r *http.Request) {
return
}
sourcePage := r.URL.Query().Get("sourcePage")
name := r.URL.Query().Get("name")
category := r.URL.Query().Get("category")
params := new(savedviewtypes.ListSavedViewsParams)
if err := binding.Query.BindQuery(r.URL.Query(), params); err != nil {
render.Error(w, err)
return
}
if err := params.Validate(); err != nil {
render.Error(w, err)
return
}
queries, err := handler.module.GetViewsForFilters(r.Context(), claims.OrgID, sourcePage, name, category)
queries, err := handler.module.GetViewsForFilters(r.Context(), claims.OrgID, params.SourcePage, params.Name)
if err != nil {
render.Error(w, err)
return

View File

@@ -2,15 +2,10 @@ package implsavedview
import (
"context"
"encoding/json"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/savedview"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -24,144 +19,82 @@ func NewModule(sqlstore sqlstore.SQLStore) savedview.Module {
return &module{sqlstore: sqlstore}
}
func (module *module) GetViewsForFilters(ctx context.Context, orgID string, sourcePage string, name string, category string) ([]*v3.SavedView, error) {
var views []savedviewtypes.SavedView
var err error
if len(category) == 0 {
err = module.sqlstore.BunDB().NewSelect().Model(&views).Where("org_id = ? AND source_page = ? AND name LIKE ?", orgID, sourcePage, "%"+name+"%").Scan(ctx)
} else {
err = module.sqlstore.BunDB().NewSelect().Model(&views).Where("org_id = ? AND source_page = ? AND category LIKE ? AND name LIKE ?", orgID, sourcePage, "%"+category+"%", "%"+name+"%").Scan(ctx)
}
func (module *module) GetViewsForFilters(ctx context.Context, orgID string, sourcePage savedviewtypes.SourcePage, name string) ([]*savedviewtypes.GettableSavedView, error) {
var views []*savedviewtypes.StorableSavedView
err := module.sqlstore.BunDB().NewSelect().Model(&views).
Where("org_id = ? AND source_page = ? AND name LIKE ?", orgID, sourcePage, "%"+name+"%").
Scan(ctx)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved views")
}
var savedViews []*v3.SavedView
for _, view := range views {
var compositeQuery v3.CompositeQuery
err = json.Unmarshal([]byte(view.Data), &compositeQuery)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in unmarshalling explorer query data: %s", err.Error())
}
savedViews = append(savedViews, &v3.SavedView{
ID: view.ID,
Name: view.Name,
CreatedAt: view.CreatedAt,
CreatedBy: view.CreatedBy,
UpdatedAt: view.UpdatedAt,
UpdatedBy: view.UpdatedBy,
Tags: strings.Split(view.Tags, ","),
SourcePage: view.SourcePage,
CompositeQuery: &compositeQuery,
ExtraData: view.ExtraData,
})
}
return savedViews, nil
return savedviewtypes.NewGettableSavedViewsFromStorable(views)
}
func (module *module) CreateView(ctx context.Context, orgID string, view v3.SavedView) (valuer.UUID, error) {
data, err := json.Marshal(view.CompositeQuery)
func (module *module) CreateView(ctx context.Context, orgID string, view savedviewtypes.PostableSavedView) (valuer.UUID, error) {
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
return valuer.UUID{}, errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling explorer query data")
}
uuid := valuer.GenerateUUID()
createdAt := time.Now()
updatedAt := time.Now()
claims, errv2 := authtypes.ClaimsFromContext(ctx)
if errv2 != nil {
return valuer.UUID{}, errors.NewInternalf(errors.CodeInternal, "error in getting email from context")
}
createBy := claims.Email
updatedBy := claims.Email
dbView := savedviewtypes.SavedView{
TimeAuditable: types.TimeAuditable{
CreatedAt: createdAt,
UpdatedAt: updatedAt,
},
UserAuditable: types.UserAuditable{
CreatedBy: createBy,
UpdatedBy: updatedBy,
},
OrgID: orgID,
Identifiable: types.Identifiable{
ID: uuid,
},
Name: view.Name,
Category: view.Category,
SourcePage: view.SourcePage,
Tags: strings.Join(view.Tags, ","),
Data: string(data),
ExtraData: view.ExtraData,
dbView, err := savedviewtypes.NewStorableSavedView(orgID, claims.Email, claims.Email, view)
if err != nil {
return valuer.UUID{}, err
}
_, err = module.sqlstore.BunDB().NewInsert().Model(&dbView).Exec(ctx)
_, err = module.sqlstore.BunDB().NewInsert().Model(dbView).Exec(ctx)
if err != nil {
return valuer.UUID{}, errors.WrapInternalf(err, errors.CodeInternal, "error in creating saved view")
}
return uuid, nil
return dbView.ID, nil
}
func (module *module) GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*v3.SavedView, error) {
var view savedviewtypes.SavedView
func (module *module) GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*savedviewtypes.GettableSavedView, error) {
var view savedviewtypes.StorableSavedView
err := module.sqlstore.BunDB().NewSelect().Model(&view).Where("org_id = ? AND id = ?", orgID, uuid.StringValue()).Scan(ctx)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved view")
}
var compositeQuery v3.CompositeQuery
err = json.Unmarshal([]byte(view.Data), &compositeQuery)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in unmarshalling explorer query data")
}
return &v3.SavedView{
ID: view.ID,
Name: view.Name,
Category: view.Category,
CreatedAt: view.CreatedAt,
CreatedBy: view.CreatedBy,
UpdatedAt: view.UpdatedAt,
UpdatedBy: view.UpdatedBy,
SourcePage: view.SourcePage,
Tags: strings.Split(view.Tags, ","),
CompositeQuery: &compositeQuery,
ExtraData: view.ExtraData,
}, nil
return savedviewtypes.NewGettableSavedViewFromStorable(&view)
}
func (module *module) UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view v3.SavedView) error {
data, err := json.Marshal(view.CompositeQuery)
func (module *module) UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view savedviewtypes.UpdatableSavedView) error {
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
return errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling explorer query data")
}
claims, errv2 := authtypes.ClaimsFromContext(ctx)
if errv2 != nil {
return errors.NewInternalf(errors.CodeInternal, "error in getting email from context")
}
updatedAt := time.Now()
updatedBy := claims.Email
dbView, err := savedviewtypes.NewStorableSavedView(orgID, claims.Email, claims.Email, view)
if err != nil {
return err
}
_, err = module.sqlstore.BunDB().NewUpdate().
Model(&savedviewtypes.SavedView{}).
res, err := module.sqlstore.BunDB().NewUpdate().
Model(&savedviewtypes.StorableSavedView{}).
Set("updated_at = ?, updated_by = ?, name = ?, category = ?, source_page = ?, tags = ?, data = ?, extra_data = ?",
updatedAt, updatedBy, view.Name, view.Category, view.SourcePage, strings.Join(view.Tags, ","), data, view.ExtraData).
dbView.UpdatedAt, dbView.UpdatedBy, dbView.Name, dbView.Category, dbView.SourcePage, dbView.Tags, dbView.Data, dbView.ExtraData).
Where("id = ?", uuid.StringValue()).
Where("org_id = ?", orgID).
Exec(ctx)
if err != nil {
return errors.WrapInternalf(err, errors.CodeInternal, "error in updating saved view")
}
rowsAffected, err := res.RowsAffected()
if err != nil {
return errors.WrapInternalf(err, errors.CodeInternal, "error in updating saved view")
}
if rowsAffected == 0 {
return errors.NewNotFoundf(savedviewtypes.ErrCodeSavedViewNotFound, "saved view %s not found", uuid.StringValue())
}
return nil
}
func (module *module) DeleteView(ctx context.Context, orgID string, uuid valuer.UUID) error {
_, err := module.sqlstore.BunDB().NewDelete().
Model(&savedviewtypes.SavedView{}).
Model(&savedviewtypes.StorableSavedView{}).
Where("id = ?", uuid.StringValue()).
Where("org_id = ?", orgID).
Exec(ctx)
@@ -172,7 +105,7 @@ func (module *module) DeleteView(ctx context.Context, orgID string, uuid valuer.
}
func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
savedViews := []*savedviewtypes.SavedView{}
savedViews := []*savedviewtypes.StorableSavedView{}
err := module.
sqlstore.

View File

@@ -4,19 +4,19 @@ import (
"context"
"net/http"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type Module interface {
GetViewsForFilters(ctx context.Context, orgID string, sourcePage string, name string, category string) ([]*v3.SavedView, error)
GetViewsForFilters(ctx context.Context, orgID string, sourcePage savedviewtypes.SourcePage, name string) ([]*savedviewtypes.GettableSavedView, error)
CreateView(ctx context.Context, orgID string, view v3.SavedView) (valuer.UUID, error)
CreateView(ctx context.Context, orgID string, view savedviewtypes.PostableSavedView) (valuer.UUID, error)
GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*v3.SavedView, error)
GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*savedviewtypes.GettableSavedView, error)
UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view v3.SavedView) error
UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view savedviewtypes.UpdatableSavedView) error
DeleteView(ctx context.Context, orgID string, uuid valuer.UUID) error

View File

@@ -12,7 +12,6 @@ import (
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/modules/authdomain"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/session"
@@ -24,28 +23,26 @@ import (
)
type module struct {
settings factory.ScopedProviderSettings
authNs map[authtypes.AuthNProvider]authn.AuthN
userSetter user.Setter
userGetter user.Getter
authDomain authdomain.Module
tokenizer tokenizer.Tokenizer
orgGetter organization.Getter
authz authz.AuthZ
globalConfig global.Config
settings factory.ScopedProviderSettings
authNs map[authtypes.AuthNProvider]authn.AuthN
userSetter user.Setter
userGetter user.Getter
authDomain authdomain.Module
tokenizer tokenizer.Tokenizer
orgGetter organization.Getter
authz authz.AuthZ
}
func NewModule(providerSettings factory.ProviderSettings, authNs map[authtypes.AuthNProvider]authn.AuthN, userSetter user.Setter, userGetter user.Getter, authDomain authdomain.Module, tokenizer tokenizer.Tokenizer, orgGetter organization.Getter, authz authz.AuthZ, globalConfig global.Config) session.Module {
func NewModule(providerSettings factory.ProviderSettings, authNs map[authtypes.AuthNProvider]authn.AuthN, userSetter user.Setter, userGetter user.Getter, authDomain authdomain.Module, tokenizer tokenizer.Tokenizer, orgGetter organization.Getter, authz authz.AuthZ) session.Module {
return &module{
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/session/implsession"),
authNs: authNs,
userSetter: userSetter,
userGetter: userGetter,
authDomain: authDomain,
tokenizer: tokenizer,
orgGetter: orgGetter,
authz: authz,
globalConfig: globalConfig,
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/session/implsession"),
authNs: authNs,
userSetter: userSetter,
userGetter: userGetter,
authDomain: authDomain,
tokenizer: tokenizer,
orgGetter: orgGetter,
authz: authz,
}
}
@@ -143,10 +140,6 @@ func (module *module) CreateCallbackAuthNSession(ctx context.Context, authNProvi
return "", err
}
if callbackIdentity.State.URL.Host != "" && !module.globalConfig.IsOriginAllowed(callbackIdentity.State.URL) {
return "", errors.Newf(errors.TypeForbidden, global.ErrCodeOriginNotAllowed, "state redirect %q is not an allowed origin", callbackIdentity.State.URL.String())
}
authDomain, err := module.authDomain.GetByOrgIDAndID(ctx, callbackIdentity.OrgID, callbackIdentity.State.DomainID)
if err != nil {
return "", err
@@ -224,10 +217,6 @@ func (module *module) getOrgSessionContext(ctx context.Context, org *types.Organ
return nil, err
}
if !module.globalConfig.IsOriginAllowed(siteURL) {
return nil, errors.Newf(errors.TypeInvalidInput, global.ErrCodeOriginNotAllowed, "ref %q is not an allowed origin", siteURL.String())
}
loginURL, err := provider.LoginURL(ctx, siteURL, authDomain)
if err != nil {
return nil, err

View File

@@ -242,7 +242,7 @@ func (handler *handler) ReplaceVariables(rw http.ResponseWriter, req *http.Reque
errs := []error{}
for idx, item := range queryRangeRequest.CompositeQuery.Queries {
if item.Type == qbtypes.QueryTypeBuilder || item.Type == qbtypes.QueryTypeBuilderAI {
if item.Type == qbtypes.QueryTypeBuilder {
switch spec := item.Spec.(type) {
case qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]:
if spec.Filter != nil && spec.Filter.Expression != "" {

View File

@@ -249,7 +249,6 @@ func (q *querier) buildPreviewProviders(
func rendersStandaloneStatement(t qbtypes.QueryType) bool {
switch t {
case qbtypes.QueryTypeBuilder,
qbtypes.QueryTypeBuilderAI,
qbtypes.QueryTypePromQL,
qbtypes.QueryTypeClickHouseSQL,
qbtypes.QueryTypeTraceOperator:

View File

@@ -42,7 +42,6 @@ type querier struct {
metadataStore telemetrytypes.MetadataStore
promEngine prometheus.Prometheus
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation]
@@ -62,7 +61,6 @@ func New(
metadataStore telemetrytypes.MetadataStore,
promEngine prometheus.Prometheus,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
@@ -84,7 +82,6 @@ func New(
metadataStore: metadataStore,
promEngine: promEngine,
traceStmtBuilder: traceStmtBuilder,
aiTraceStmtBuilder: aiTraceStmtBuilder,
logStmtBuilder: logStmtBuilder,
auditStmtBuilder: auditStmtBuilder,
metricStmtBuilder: metricStmtBuilder,
@@ -235,16 +232,6 @@ func (q *querier) buildQueries(
}
queries[traceOpQuery.Name] = toq
steps[traceOpQuery.Name] = traceOpQuery.StepInterval
case qbtypes.QueryTypeBuilderAI:
spec, ok := query.Spec.(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation])
if !ok {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid AI builder query spec %T", query.Spec)
}
spec.ShiftBy = extractShiftFromBuilderQuery(spec)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, req.RequestType)
bq := newBuilderQuery(q.logger, q.telemetryStore, orgID, q.aiTraceStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
queries[spec.Name] = bq
steps[spec.Name] = spec.StepInterval
case qbtypes.QueryTypeBuilder:
switch spec := query.Spec.(type) {
case qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]:
@@ -311,11 +298,6 @@ func (q *querier) populateQBEvent(event *qbtypes.QBEvent, queries []qbtypes.Quer
case qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]:
event.MetricsUsed = true
}
case qbtypes.QueryTypeBuilderAI:
filter := query.GetFilter()
event.FilterApplied = event.FilterApplied || (filter != nil && filter.Expression != "")
event.GroupByApplied = event.GroupByApplied || len(query.GetGroupBy()) > 0
event.TracesUsed = true
case qbtypes.QueryTypePromQL:
event.MetricsUsed = true
case qbtypes.QueryTypeTraceOperator:
@@ -878,9 +860,7 @@ func (q *querier) createRangedQuery(_ valuer.UUID, originalQuery qbtypes.Query,
specCopy := qt.spec.Copy()
specCopy.ShiftBy = extractShiftFromBuilderQuery(specCopy)
adjustedTimeRange := adjustTimeRangeForShift(specCopy, timeRange, qt.kind)
// Reuse the statement builder the original query was created with, so an AI
// query keeps its AI builder without re-deriving it from the spec.
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, qt.stmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, q.traceStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
case *builderQuery[qbtypes.LogAggregation]:
specCopy := qt.spec.Copy()
@@ -1237,8 +1217,6 @@ func (q *querier) adjustStepInterval(queries []qbtypes.QueryEnvelope, start, end
if qe.GetStepInterval().Seconds() == 0 {
qe.SetStepInterval(secondsStep(metricRecommended))
}
case qbtypes.QueryTypeBuilderAI:
clampStep(qe, traceLogRecommended, traceLogMin, &warnings)
case qbtypes.QueryTypeTraceOperator:
clampStep(qe, traceLogRecommended, traceLogMin, &warnings)
}

View File

@@ -49,7 +49,6 @@ func TestQueryRange_MetricTypeMissing(t *testing.T) {
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
@@ -122,7 +121,6 @@ func TestQueryRange_MetricTypeFromStore(t *testing.T) {
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
&mockMetricStmtBuilder{}, // metricStmtBuilder

View File

@@ -9,13 +9,11 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryai"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryscopedtraces"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
)
@@ -94,18 +92,6 @@ func newProvider(
cfg.SkipResourceFingerprint.Threshold,
)
// AI trace statement builder (builder_ai_query). The gen_ai gate/column keys are
// surfaced by the metadata store itself (enrichWithGenAIKeys), so queries work
// before any gen_ai metadata is ingested — no per-builder decoration needed.
// The standard trace builder doubles as the delegate for the span-list path.
aiTraceStmtBuilder := telemetryscopedtraces.NewScopedTraceStatementBuilder(
settings,
telemetryMetadataStore,
telemetryai.Scope(),
traceStmtBuilder,
flagger,
)
// Create trace operator statement builder
traceOperatorStmtBuilder := telemetrytraces.NewTraceOperatorStatementBuilder(
settings,
@@ -195,7 +181,6 @@ func newProvider(
telemetryMetadataStore,
prometheus,
traceStmtBuilder,
aiTraceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,

View File

@@ -14,7 +14,6 @@ import (
"github.com/pkg/errors"
signozerrors "github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/valuer"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
)
@@ -1423,34 +1422,6 @@ func (p *Point) UnmarshalJSON(data []byte) error {
return err
}
// SavedView is a saved query for the explore page
// It is a composite query with a source page name and user defined tags
// The source page name is used to identify the page that initiated the query
// The source page could be "traces", "logs", "metrics".
type SavedView struct {
ID valuer.UUID `json:"id,omitempty"`
Name string `json:"name"`
Category string `json:"category"`
CreatedAt time.Time `json:"createdAt"`
CreatedBy string `json:"createdBy"`
UpdatedAt time.Time `json:"updatedAt"`
UpdatedBy string `json:"updatedBy"`
SourcePage string `json:"sourcePage"`
Tags []string `json:"tags"`
CompositeQuery *CompositeQuery `json:"compositeQuery"`
// ExtraData is JSON encoded data used by frontend to store additional data
ExtraData string `json:"extraData"`
}
func (eq *SavedView) Validate() error {
if eq.CompositeQuery == nil {
return fmt.Errorf("composite query is required")
}
return eq.CompositeQuery.Validate()
}
type LatencyMetricMetadataResponse struct {
Delta bool `json:"delta"`
Le []float64 `json:"le"`

View File

@@ -48,7 +48,6 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
metricStmtBuilder,
@@ -102,7 +101,6 @@ func prepareQuerierForLogs(t *testing.T, telemetryStore telemetrystore.Telemetry
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
logStmtBuilder, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
@@ -152,7 +150,6 @@ func prepareQuerierForTraces(t *testing.T, telemetryStore telemetrystore.Telemet
metadataStore,
nil, // prometheus
traceStmtBuilder, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder

View File

@@ -1,154 +0,0 @@
package querybuilder
import (
"strings"
"github.com/SigNoz/signoz/pkg/errors"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/antlr4-go/antlr/v4"
)
// SplitFilterForAggregates partitions a single filter expression into a span-level
// part (a WHERE over spans) and a trace-level part (a HAVING over per-trace
// aggregates), splitting on the top-level AND.
//
// A key is trace-level when it carries the trace field context (`trace.completion_tokens`)
// or, with no context, its bare name is in aggregateNames. Any other explicit context
// (`span.`, `resource.`, …) is span-level. Trace-level and span-level keys may be
// AND-combined (they run at different query stages) but not OR-combined; an OR that
// mixes the two is an error.
//
// Syntax errors are ignored here — each part is re-parsed downstream (PrepareWhereClause
// for the span part, the HAVING rewriter for the trace part), which surface them.
func SplitFilterForAggregates(query string, aggregateNames map[string]struct{}) (spanExpr string, havingExpr string, err error) {
if strings.TrimSpace(query) == "" {
return "", "", nil
}
s := filterSplitter{query: []rune(query), aggregateNames: aggregateNames}
s.visit(parseFilterQuery(query))
if s.mixed {
return "", "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level and span-level filters cannot be combined within an OR/NOT group; separate them with a top-level AND")
}
return strings.Join(s.span, " AND "), strings.Join(s.having, " AND "), nil
}
func parseFilterQuery(query string) antlr.Tree {
lexer := grammar.NewFilterQueryLexer(antlr.NewInputStream(query))
lexer.RemoveErrorListeners()
parser := grammar.NewFilterQueryParser(antlr.NewCommonTokenStream(lexer, 0))
parser.RemoveErrorListeners()
return parser.Query()
}
// filterSplitter walks the parse tree once, flattening the top-level AND chain and
// routing each atom (a comparison, a NOT expression, or a whole multi-branch OR group)
// to the span or having bucket by the class of the keys it references.
type filterSplitter struct {
query []rune
aggregateNames map[string]struct{}
span []string
having []string
mixed bool
}
func (s *filterSplitter) visit(node antlr.Tree) {
switch n := node.(type) {
case *grammar.QueryContext:
if n.Expression() != nil {
s.visit(n.Expression())
}
case *grammar.ExpressionContext:
if n.OrExpression() != nil {
s.visit(n.OrExpression())
}
case *grammar.OrExpressionContext:
// a single branch is just an AND chain; multiple branches are a real OR, kept
// whole so a class-mixing OR can be rejected.
if ands := n.AllAndExpression(); len(ands) == 1 {
s.visit(ands[0])
} else {
s.route(n)
}
case *grammar.AndExpressionContext:
for _, u := range n.AllUnaryExpression() {
s.visit(u)
}
case *grammar.UnaryExpressionContext:
if n.NOT() != nil {
s.route(n)
} else if n.Primary() != nil {
s.visit(n.Primary())
}
case *grammar.PrimaryContext:
if n.OrExpression() != nil { // parenthesized sub-expression
s.visit(n.OrExpression())
} else {
s.route(n)
}
}
}
// route classifies an atom and appends its original source text to the right bucket.
func (s *filterSplitter) route(atom antlr.ParserRuleContext) {
isTrace, isSpan := classifyKeys(atom, s.aggregateNames)
if isTrace && isSpan {
s.mixed = true
return
}
text := atomSourceText(s.query, atom)
// A multi-branch OR group's source slice excludes its enclosing parens (they belong
// to the parent Primary). Re-wrap it so rejoining a bucket with " AND " cannot invert
// OR/AND precedence, e.g. `a AND (b OR c)` must not flatten to `a AND b OR c`.
if or, ok := atom.(*grammar.OrExpressionContext); ok && len(or.AllAndExpression()) > 1 {
text = "(" + text + ")"
}
if isTrace {
s.having = append(s.having, text)
} else {
s.span = append(s.span, text)
}
}
// classifyKeys reports whether a subtree references trace-level and/or span-level keys.
// A key is trace-level when it carries the trace field context or, with no context,
// its name is a known aggregate; an unknown name under the trace context stays
// trace-level so the aggregate validation rejects it with a targeted error. Any other
// explicit context (`span.`, `resource.`, …) is span-level.
func classifyKeys(node antlr.Tree, aggregateNames map[string]struct{}) (isTrace, isSpan bool) {
kc, ok := node.(*grammar.KeyContext)
if ok {
key := telemetrytypes.GetFieldKeyFromKeyText(kc.GetText())
switch key.FieldContext {
case telemetrytypes.FieldContextTrace:
isTrace = true
case telemetrytypes.FieldContextUnspecified:
_, isTrace = aggregateNames[key.Name]
isSpan = !isTrace
default:
isSpan = true
}
return
}
for i := 0; i < node.GetChildCount(); i++ {
t, s := classifyKeys(node.GetChild(i), aggregateNames)
isTrace = isTrace || t
isSpan = isSpan || s
}
return
}
// atomSourceText returns the original source substring for an atom, preserving
// whitespace. The token stream drops skipped whitespace, which would glue word
// operators (OR/AND/NOT) to their operands, so slice the input by token offsets.
// ANTLR offsets are rune indices (InputStream holds []rune), hence the rune slice.
func atomSourceText(query []rune, atom antlr.ParserRuleContext) string {
start, stop := atom.GetStart(), atom.GetStop()
if start == nil || stop == nil || start.GetStart() < 0 || stop.GetStop() >= len(query) || stop.GetStop() < start.GetStart() {
return atom.GetText()
}
return string(query[start.GetStart() : stop.GetStop()+1])
}

View File

@@ -1,167 +0,0 @@
package querybuilder
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestSplitFilterForAggregates(t *testing.T) {
agg := map[string]struct{}{"completion_tokens": {}, "span_count": {}, "prompt_tokens": {}}
type tc struct {
name string
query string
span string // expected span-level (WHERE) part; "" => empty
having string // expected trace-level (HAVING) part; "" => empty
wantErr bool
}
cases := []tc{
// --- empty input ---------------------------------------------------------
{
name: "empty",
},
{
name: "whitespace only",
query: " ",
},
// --- single class --------------------------------------------------------
{
name: "span only",
query: "service.name = 'x'",
span: "service.name = 'x'",
},
{
name: "agg only bare",
query: "completion_tokens > 1000",
having: "completion_tokens > 1000",
},
{
// the user-facing `trace.` prefix marks a trace-level aggregate.
name: "agg only trace prefix",
query: "trace.completion_tokens > 1000",
having: "trace.completion_tokens > 1000",
},
{
// an unknown name under the trace context still routes trace-level, so the
// aggregate validation rejects it with a targeted error instead of the span
// path failing on an unknown field.
name: "unknown aggregate under trace context stays trace-level",
query: "trace.not_an_aggregate > 1000",
having: "trace.not_an_aggregate > 1000",
},
{
// ANTLR token offsets are rune indices; slicing must not shift after a
// multi-byte char (this used to truncate 1000 → 100).
name: "unicode value before the split",
query: "service.name = 'héllo' AND completion_tokens > 1000",
span: "service.name = 'héllo'",
having: "completion_tokens > 1000",
},
// --- top-level AND splits across the two buckets -------------------------
{
name: "span AND agg",
query: "service.name = 'x' AND completion_tokens > 1000",
span: "service.name = 'x'",
having: "completion_tokens > 1000",
},
{
// order within a bucket is preserved; the two span atoms join with AND.
name: "span AND span AND agg",
query: "service.name = 'x' AND kind_string = 'Internal' AND completion_tokens > 1000",
span: "service.name = 'x' AND kind_string = 'Internal'",
having: "completion_tokens > 1000",
},
{
// a parenthesized top-level AND still splits across the two buckets.
name: "parenthesized span AND agg",
query: "(service.name = 'x' AND completion_tokens > 1000)",
span: "service.name = 'x'",
having: "completion_tokens > 1000",
},
// --- OR groups are re-wrapped in parens so a later AND-join can't invert
// precedence (`a AND (b OR c)` must not flatten to `a AND b OR c`) ------
{
name: "agg OR agg",
query: "completion_tokens > 1000 OR span_count > 3",
having: "(completion_tokens > 1000 OR span_count > 3)",
},
{
name: "span OR span",
query: "service.name = 'x' OR kind_string = 'Internal'",
span: "(service.name = 'x' OR kind_string = 'Internal')",
},
{
name: "span AND (span OR span)",
query: "service.name = 'x' AND (kind_string = 'Internal' OR kind_string = 'Client')",
span: "service.name = 'x' AND (kind_string = 'Internal' OR kind_string = 'Client')",
},
{
name: "agg AND (agg OR agg)",
query: "prompt_tokens > 5 AND (completion_tokens > 1000 OR span_count > 3)",
having: "prompt_tokens > 5 AND (completion_tokens > 1000 OR span_count > 3)",
},
{
// the OR group routes to span, the trailing aggregate to having.
name: "span AND (span OR span) AND agg",
query: "a.b = 'x' AND (c.d = 'y' OR e.f = 'z') AND completion_tokens > 1000",
span: "a.b = 'x' AND (c.d = 'y' OR e.f = 'z')",
having: "completion_tokens > 1000",
},
// --- a nested AND group flattens across the buckets (no spurious parens) --
{
name: "(span AND agg) AND agg",
query: "(service.name = 'x' AND completion_tokens > 1000) AND prompt_tokens > 5",
span: "service.name = 'x'",
having: "completion_tokens > 1000 AND prompt_tokens > 5",
},
// --- NOT wrapping a single-class group is routed whole to that class ------
{
name: "not agg",
query: "NOT (completion_tokens > 1000)",
having: "NOT (completion_tokens > 1000)",
},
{
name: "not span",
query: "NOT (service.name = 'x')",
span: "NOT (service.name = 'x')",
},
// --- class-mixing is rejected in an OR group, a NOT group, or a nested OR -
{
name: "agg OR span rejected",
query: "completion_tokens > 1000 OR service.name = 'x'",
wantErr: true,
},
{
name: "not mixed rejected",
query: "NOT (completion_tokens > 1000 AND service.name = 'x')",
wantErr: true,
},
{
name: "span AND (agg OR span) rejected",
query: "service.name = 'x' AND (completion_tokens > 1000 OR kind_string = 'Client')",
wantErr: true,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
span, having, err := SplitFilterForAggregates(c.query, agg)
if c.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
require.Equal(t, c.span, span, "span part")
require.Equal(t, c.having, having, "having part")
})
}
}

View File

@@ -18,19 +18,6 @@ func NewHavingExpressionRewriter() *HavingExpressionRewriter {
}
}
// Rewrite rewrites and validates a HAVING expression against a caller-supplied
// column map (user-facing name -> SQL identifier/expression). Values are inlined, so
// the result is a bare SQL boolean expression with no bound args. Used by callers
// that project their own aggregate columns (e.g. the AI trace list) rather than the
// query's Aggregations.
func (r *HavingExpressionRewriter) Rewrite(expression string, columnMap map[string]string) (string, error) {
if len(strings.TrimSpace(expression)) == 0 {
return "", nil
}
r.columnMap = columnMap
return r.rewriteAndValidate(expression)
}
// RewriteForTraces rewrites and validates the HAVING expression for a traces query.
func (r *HavingExpressionRewriter) RewriteForTraces(expression string, aggregations []qbtypes.TraceAggregation) (string, error) {
if len(strings.TrimSpace(expression)) == 0 {

View File

@@ -82,10 +82,6 @@ func resourcesForQuery(query gjson.Result, variables map[string]qbtypes.Variable
switch queryType {
case qbtypes.QueryTypeBuilder.StringValue(), qbtypes.QueryTypeSubQuery.StringValue():
return resourcesForBuilderQuery(queryType, query.Get("spec"), variables)
case qbtypes.QueryTypeBuilderAI.StringValue():
// An AI builder query is always a traces query; the signal is implied by the
// type (and may be absent from the payload), so pin the resource directly.
return builderQueryResourceRefs(queryType, coretypes.ResourceTelemetryResourceTraces, query.Get("spec"), variables)
case qbtypes.QueryTypePromQL.StringValue():
return []coretypes.ResourceWithID{{Resource: coretypes.ResourceTelemetryResourceMetrics, ID: typeWildcard}}, nil
case qbtypes.QueryTypeClickHouseSQL.StringValue():
@@ -107,10 +103,7 @@ func resourcesForBuilderQuery(queryType string, spec gjson.Result, variables map
if err != nil {
return nil, err
}
return builderQueryResourceRefs(queryType, resource, spec, variables)
}
func builderQueryResourceRefs(queryType string, resource coretypes.Resource, spec gjson.Result, variables map[string]qbtypes.VariableItem) ([]coretypes.ResourceWithID, error) {
ids, err := builderQuerySelectors(queryType, spec.Get("filter.expression").String(), variables)
if err != nil {
return nil, err

View File

@@ -92,20 +92,6 @@ func TestQueryRangeResources(t *testing.T) {
{Resource: coretypes.ResourceTelemetryResourceAuditLogs, ID: "builder_query/signoz.workspace.key.id/a"},
},
},
{
name: "ai builder query maps to traces resource without a signal",
body: `{"compositeQuery":{"queries":[{"type":"builder_ai_query","spec":{"filter":{"expression":"signoz.workspace.key.id = 'checkout'"}}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_ai_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "ai builder query without filter is wildcard",
body: `{"compositeQuery":{"queries":[{"type":"builder_ai_query","spec":{}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_ai_query/*"},
},
},
{
name: "promql is wildcard only",
body: `{"compositeQuery":{"queries":[{"type":"promql","spec":{"query":"up"}}]}}`,

View File

@@ -149,7 +149,7 @@ func NewModules(
TraceFunnel: impltracefunnel.NewModule(impltracefunnel.NewStore(sqlstore)),
RawDataExport: implrawdataexport.NewModule(querier),
AuthDomain: authDomainModule,
Session: implsession.NewModule(providerSettings, authNs, userSetter, userGetter, authDomainModule, tokenizer, orgGetter, authz, config.Global),
Session: implsession.NewModule(providerSettings, authNs, userSetter, userGetter, authDomainModule, tokenizer, orgGetter, authz),
SpanPercentile: implspanpercentile.NewModule(querier, providerSettings),
Services: implservices.NewModule(querier, telemetryStore),
MetricsExplorer: implmetricsexplorer.NewModule(telemetryStore, telemetryMetadataStore, cache, ruleStore, dashboard, fl, providerSettings, config.MetricsExplorer),

View File

@@ -30,6 +30,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/modules/rawdataexport"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/savedview"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/modules/session"
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
@@ -88,6 +89,7 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
struct{ tracedetail.Handler }{},
struct{ ruler.Handler }{},
struct{ statsreporter.Handler }{},
struct{ savedview.Handler }{},
).New(ctx, instrumentation.ToProviderSettings(), apiserver.Config{})
if err != nil {
return nil, err

View File

@@ -320,6 +320,7 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
handlers.TraceDetail,
handlers.RulerHandler,
handlers.StatsHandler,
handlers.SavedView,
),
)
}

View File

@@ -1,66 +0,0 @@
package telemetryai
import (
"strings"
scopedtraces "github.com/SigNoz/signoz/pkg/telemetryscopedtraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// Scope describes gen_ai for the scoped trace builder: an AI trace has >=1 gen_ai
// LLM, tool, or agent span, and its list adds AI/LLM per-trace metrics on top of the
// common columns. This package holds only gen_ai domain knowledge; the query
// topology lives in telemetryscopedtraces.
func Scope() scopedtraces.TraceScope {
gateKeyNames := []string{telemetrytypes.GenAIRequestModel, telemetrytypes.GenAIToolName, telemetrytypes.GenAIAgentName}
gateExprs := make([]string, 0, len(gateKeyNames))
gateKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(gateKeyNames))
for _, name := range gateKeyNames {
gateExprs = append(gateExprs, name+" EXISTS")
gateKeys = append(gateKeys, &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
})
}
defs := telemetrytypes.GenAIFieldDefinitions
reqModel := defs[telemetrytypes.GenAIRequestModel]
toolName := defs[telemetrytypes.GenAIToolName]
inTok := defs[telemetrytypes.GenAIUsageInputTokens]
outTok := defs[telemetrytypes.GenAIUsageOutputTokens]
cost := defs[telemetrytypes.SignozGenAITotalCost]
inMsg := defs[telemetrytypes.GenAIInputMessages]
outMsg := defs[telemetrytypes.GenAIOutputMessages]
str := telemetrytypes.FieldDataTypeString
columns := append(scopedtraces.CommonTraceColumns(),
// LLM calls only (request model present), not the full gate.
scopedtraces.TraceColumn{Alias: "llm_call_count", Orderable: true, Expr: scopedtraces.CountExists(&reqModel)},
scopedtraces.TraceColumn{Alias: "tool_call_count", Orderable: true, Expr: scopedtraces.CountExists(&toolName)},
scopedtraces.TraceColumn{Alias: "distinct_tool_count", Orderable: true, Expr: scopedtraces.UniqCount(&toolName, str)},
// tokens live only on LLM spans, so a plain sum needs no gate scoping.
scopedtraces.TraceColumn{Alias: "input_tokens", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &inTok)},
scopedtraces.TraceColumn{Alias: "output_tokens", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &outTok)},
scopedtraces.TraceColumn{Alias: "total_tokens", Orderable: true, Expr: scopedtraces.SumOfKeys(telemetrytypes.FieldDataTypeFloat64, &inTok, &outTok)},
// per-span cost attached by the SigNoz LLM pricing processor.
scopedtraces.TraceColumn{Alias: "estimated_total_cost", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &cost)},
// slowest single LLM call in the trace.
scopedtraces.TraceColumn{Alias: "max_llm_duration_nano", Orderable: true, Expr: scopedtraces.ScopedToKeyColumn(scopedtraces.AggMax, scopedtraces.IntrinsicSpanKey("duration_nano"), &reqModel)},
// errors across the whole trace (any span), so display-only.
scopedtraces.TraceColumn{Alias: "error_count", Expr: scopedtraces.CondCount(scopedtraces.IntrinsicSpanKey("has_error"), qbtypes.FilterOperatorEqual, true)},
// timestamp of the last gen_ai span (LLM/tool/agent), hence gate-scoped.
scopedtraces.TraceColumn{Alias: "last_activity_time", Orderable: true, Expr: scopedtraces.ScopedReduce(scopedtraces.AggMax, scopedtraces.IntrinsicSpanKey("timestamp"))},
// previews: first call's input (the prompt), last call's output (the answer).
scopedtraces.TraceColumn{Alias: "input", SpanLevel: true, Expr: scopedtraces.PickBy(&inMsg, str, scopedtraces.IntrinsicSpanKey("timestamp"), scopedtraces.PickEarliest)},
scopedtraces.TraceColumn{Alias: "output", SpanLevel: true, Expr: scopedtraces.PickBy(&outMsg, str, scopedtraces.IntrinsicSpanKey("timestamp"), scopedtraces.PickLatest)},
)
return scopedtraces.TraceScope{
FilterExpression: strings.Join(gateExprs, " OR "),
FieldKeys: gateKeys,
Columns: columns,
DefaultOrderAlias: "last_activity_time",
}
}

View File

@@ -1,988 +0,0 @@
package telemetryai
import (
"context"
"fmt"
"strings"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
scopedtraces "github.com/SigNoz/signoz/pkg/telemetryscopedtraces"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
// otelKeysMap seeds the OpenTelemetry gen_ai semantic-convention keys the AI
// queries reference, so the metadata-backed field resolution succeeds in tests.
func otelKeysMap() map[string][]*telemetrytypes.TelemetryFieldKey {
strKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
numKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeFloat64,
}
}
m := make(map[string][]*telemetrytypes.TelemetryFieldKey)
// gen_ai semconv keys sourced from the single source of truth, mirroring what the
// production metadata store surfaces via enrichWithGenAIKeys.
for name, def := range telemetrytypes.GenAIFieldDefinitions {
keyCopy := def
m[name] = []*telemetrytypes.TelemetryFieldKey{&keyCopy}
}
// Extra keys these tests reference that aren't gen_ai semconv definitions.
m["gen_ai.user.id"] = []*telemetrytypes.TelemetryFieldKey{strKey("gen_ai.user.id")}
m["_signoz.gen_ai.total_cost"] = []*telemetrytypes.TelemetryFieldKey{numKey("_signoz.gen_ai.total_cost")}
m["gen_ai.usage.cached_input_tokens"] = []*telemetrytypes.TelemetryFieldKey{numKey("gen_ai.usage.cached_input_tokens")}
m["has_error"] = []*telemetrytypes.TelemetryFieldKey{{
Name: "has_error",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeBool,
}}
// service.name carries the resource-column evolutions like production metadata, so
// the rendered value expression prefers the JSON resource column over the legacy
// map (matching the standard traces builder tests).
m["service.name"] = []*telemetrytypes.TelemetryFieldKey{{
Name: "service.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
Evolutions: resourceEvolutions(),
}}
return m
}
// resourceEvolutions is the canonical resource-column timeline: the legacy
// resources_string map at epoch 0 and the JSON resource column released inside the
// test window (mirrors telemetrytraces' mockEvolutionData).
func resourceEvolutions() []*telemetrytypes.EvolutionEntry {
return []*telemetrytypes.EvolutionEntry{
{
Signal: telemetrytypes.SignalTraces,
ColumnName: "resources_string",
ColumnType: "Map(LowCardinality(String), String)",
FieldContext: telemetrytypes.FieldContextResource,
FieldName: "__all__",
ReleaseTime: time.Unix(0, 0),
},
{
Signal: telemetrytypes.SignalTraces,
ColumnName: "resource",
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextResource,
FieldName: "__all__",
ReleaseTime: time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC),
},
}
}
// standard test window (ms), matching the traces builder tests.
const (
testStartMs = uint64(1747947419000)
testEndMs = uint64(1747983448000)
)
func newTestBuilder(t *testing.T) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
return newTestBuilderWithKeys(t, otelKeysMap())
}
// newTestBuilderWithKeys mirrors the production wiring in signozquerier's provider.
// The gen_ai keys are seeded via keysMap here; in production the metadata store
// surfaces them itself (enrichWithGenAIKeys).
func newTestBuilderWithKeys(t *testing.T, keysMap map[string][]*telemetrytypes.TelemetryFieldKey) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
t.Helper()
settings := instrumentationtest.New().ToProviderSettings()
fm := telemetrytraces.NewFieldMapper()
cb := telemetrytraces.NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = keysMap
fl := flaggertest.New(t)
// In production the metadata store enriches gen_ai keys (enrichWithGenAIKeys);
// here the mock is seeded directly via keysMap.
metadataStore := telemetrytypes.MetadataStore(mockMetadataStore)
rewriter := querybuilder.NewAggExprRewriter(settings, nil, fm, cb, fl)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
settings,
metadataStore,
fm,
cb,
rewriter,
nil,
fl,
false,
100000,
)
return scopedtraces.NewScopedTraceStatementBuilder(
settings,
metadataStore,
Scope(),
traceStmtBuilder,
fl,
)
}
// ---------------------------------------------------------------------------
// Full-query golden tests
//
// Each pins the WHOLE generated statement, with bound args inlined into the `?`
// placeholders, as ONE self-contained literal — so a failure diff shows the entire
// query and the expected SQL can be copied straight into a ClickHouse client. The
// `want` strings are formatted for readability; the comparison is whitespace- and
// backtick-insensitive (see normalizeSQL), so only the SQL tokens themselves matter.
//
// The four trace-list goldens cover the corners of how `matched` is assembled —
// {no span filter, span filter} × {no aggregate filter, aggregate filter} — plus a
// mixed filter + multi-key order, plus the delegated span list. Note `matched` selects
// only the aggregates ORDER BY / HAVING reference; the rest appear only in enrichment.
//
// Run `go test ./pkg/telemetryai/ -run TestBuild_FullSQL -v` to also print each query.
// ---------------------------------------------------------------------------
// renderSQL substitutes bound args into the `?` placeholders so the whole statement
// reads as one literal SQL string.
func renderSQL(t *testing.T, stmt *qbtypes.Statement) string {
t.Helper()
var b strings.Builder
argi := 0
for i := 0; i < len(stmt.Query); i++ {
if stmt.Query[i] == '?' {
require.Less(t, argi, len(stmt.Args), "more ? than args in query")
b.WriteString(formatArg(stmt.Args[argi]))
argi++
continue
}
b.WriteByte(stmt.Query[i])
}
require.Equal(t, len(stmt.Args), argi, "arg count does not match number of placeholders")
return b.String()
}
func formatArg(a any) string {
if s, ok := a.(string); ok {
return "'" + s + "'"
}
return fmt.Sprintf("%v", a)
}
// normalizeSQL makes the comparison insensitive to formatting: it drops identifier
// backticks, collapses whitespace runs to a single space, and removes spaces directly
// inside parentheses. This lets the golden strings be freely indented/wrapped (and
// written as Go raw literals, which cannot contain backticks) — only the SQL tokens
// and their order matter.
func normalizeSQL(s string) string {
s = strings.Join(strings.Fields(strings.ReplaceAll(s, "`", "")), " ")
s = strings.ReplaceAll(s, "( ", "(")
s = strings.ReplaceAll(s, " )", ")")
return s
}
func requireSQLEqual(t *testing.T, want string, stmt *qbtypes.Statement) {
t.Helper()
got := renderSQL(t, stmt)
t.Logf("\n%s", got)
require.Equal(t, normalizeSQL(want), normalizeSQL(got))
}
// No filter: matched selects only the default order key (last_activity_time), WHERE is
// just window + gate mask, no HAVING.
func TestBuild_FullSQL_TraceList_NoFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces, Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Promotion: a materialized gen_ai attribute must resolve to its materialized column
// everywhere it appears — gate mask, countIf/scoped existence, and value columns —
// while un-promoted attributes stay in the attributes map, so one query mixes both
// forms. Here gen_ai.request.model and gen_ai.usage.input_tokens are materialized:
// the gate/llm_call_count/max_llm_latency use `..._exists`, input_tokens/total_tokens
// use the materialized value column, and tool/output_tokens/cost/messages stay in the map.
func TestBuild_FullSQL_TraceList_MaterializedColumns(t *testing.T) {
keys := otelKeysMap()
for _, name := range []string{"gen_ai.request.model", "gen_ai.usage.input_tokens"} {
for _, k := range keys[name] {
k.Materialized = true
}
}
b := newTestBuilderWithKeys(t, keys)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces, Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(attribute_string_gen_ai$$request$$model_exists) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, attribute_string_gen_ai$$request$$model_exists) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span-level AND trace-level filter, order by the aggregate, pagination. matched selects
// only output_tokens (the sole aggregate referenced by both ORDER BY and HAVING) — not
// input_tokens/llm_call_count/last_activity_time. The span predicate widens the WHERE
// prune and becomes a countIf(...) > 0 existence check alongside the gate countIf.
func TestBuild_FullSQL_TraceList_SpanAndTraceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND output_tokens > 1000"},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "output_tokens"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 10, Offset: 30,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
OR (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model')))
GROUP BY trace_id
HAVING countIf((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) > 0
AND countIf((attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))) > 0
AND output_tokens > 1000
ORDER BY output_tokens DESC, trace_id DESC
LIMIT 10 OFFSET 30
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY output_tokens DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Aggregate-only filter (no span filter). WHERE prune is NOT widened, there is no
// gate/span countIf, just the aggregate HAVING. `trace.output_tokens` rewrites to the
// output_tokens alias. matched selects output_tokens (HAVING) + last_activity_time (default order).
func TestBuild_FullSQL_TraceList_AggregateFilterOnly(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"},
Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
HAVING output_tokens > 1000
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span-only filter (no aggregate filter). WHERE is widened; HAVING has the gate + span
// countIf pair but no trailing aggregate. `has_error = true` resolves to a
// materialized-column predicate (not a map access). matched selects only the default order key.
func TestBuild_FullSQL_TraceList_SpanFilterOnly(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "has_error = true"},
Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
OR has_error = true)
GROUP BY trace_id
HAVING countIf((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) > 0
AND countIf(has_error = true) > 0
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Resource filter: a resource attribute in the filter is pulled into a __resource_filter
// CTE (fingerprints matching the resource condition), and the `matched` scan is narrowed
// by `resource_fingerprint GLOBAL IN (…)`. The resource key is dropped from the span
// predicate (skipResourceFilter), so here there is no span-level existence check — the
// prune stays the gate mask and the whole match is scoped to the resource fingerprints.
func TestBuild_FullSQL_TraceList_ResourceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "resource.service.name = 'checkout'"},
Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH __resource_filter AS (
SELECT fingerprint
FROM signoz_traces.distributed_traces_v3_resource
WHERE (simpleJSONExtractString(labels, 'service.name') = 'checkout' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"checkout%')
AND seen_at_ts_bucket_start >= 1747945619
AND seen_at_ts_bucket_start <= 1747983448
GROUP BY fingerprint
),
matched AS (
SELECT trace_id,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Mixed filter (two span predicates AND'd into one existence check + an aggregate) with
// a two-key order on different aggregates than the filter. matched selects input_tokens
// + last_activity_time (ORDER BY) and output_tokens (HAVING) — three of four; llm_call_count is not.
func TestBuild_FullSQL_TraceList_MixedFiltersMultiOrder(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o' AND has_error = true AND output_tokens > 500"},
Order: []qbtypes.OrderBy{
{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "input_tokens"}}, Direction: qbtypes.OrderDirectionDesc},
{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "last_activity_time"}}, Direction: qbtypes.OrderDirectionAsc},
},
Limit: 15,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
OR ((attributes_string['gen_ai.request.model'] = 'gpt-4o' AND mapContains(attributes_string, 'gen_ai.request.model')) AND has_error = true))
GROUP BY trace_id
HAVING countIf((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) > 0
AND countIf(((attributes_string['gen_ai.request.model'] = 'gpt-4o' AND mapContains(attributes_string, 'gen_ai.request.model')) AND has_error = true)) > 0
AND output_tokens > 500
ORDER BY input_tokens DESC, last_activity_time ASC, trace_id DESC
LIMIT 15
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY input_tokens DESC, last_activity_time ASC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span list (requestType raw): delegated to the traces builder with the gate ANDed
// into the user filter, so only gen_ai spans matching the filter come back. Standard
// span columns, single SELECT (no CTE pipeline).
func TestBuild_FullSQL_SpanList_Raw(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini'"},
Limit: 10,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
SELECT timestamp AS __SELECT_KEY_0_timestamp, trace_id AS __SELECT_KEY_1_trace_id, span_id AS __SELECT_KEY_2_span_id,
trace_state AS __SELECT_KEY_3_trace_state, parent_span_id AS __SELECT_KEY_4_parent_span_id, flags AS __SELECT_KEY_5_flags,
name AS __SELECT_KEY_6_name, kind AS __SELECT_KEY_7_kind, kind_string AS __SELECT_KEY_8_kind_string, duration_nano AS __SELECT_KEY_9_duration_nano,
status_code AS __SELECT_KEY_10_status_code, status_message AS __SELECT_KEY_11_status_message,
status_code_string AS __SELECT_KEY_12_status_code_string, events AS __SELECT_KEY_13_events, links AS __SELECT_KEY_14_links,
response_status_code AS __SELECT_KEY_15_response_status_code, external_http_url AS __SELECT_KEY_16_external_http_url,
http_url AS __SELECT_KEY_17_http_url, external_http_method AS __SELECT_KEY_18_external_http_method,
http_method AS __SELECT_KEY_19_http_method, http_host AS __SELECT_KEY_20_http_host, db_name AS __SELECT_KEY_21_db_name,
db_operation AS __SELECT_KEY_22_db_operation, has_error AS __SELECT_KEY_23_has_error, is_remote AS __SELECT_KEY_24_is_remote,
attributes_string, attributes_number, attributes_bool, resources_string
FROM signoz_traces.distributed_signoz_index_v3
WHERE (((mapContains(attributes_string, 'gen_ai.request.model')
OR mapContains(attributes_string, 'gen_ai.tool.name')
OR mapContains(attributes_string, 'gen_ai.agent.name')))
AND ((attributes_string['gen_ai.request.model'] = 'gpt-4o-mini'
AND mapContains(attributes_string, 'gen_ai.request.model'))))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
LIMIT 10
`, stmt)
}
// ---------------------------------------------------------------------------
// Behavior / branch tests not covered by the goldens above
// ---------------------------------------------------------------------------
// A filter mixing a resource attribute with a span-level and an aggregate condition:
// the resource key routes into __resource_filter (fingerprint prune), the span key stays
// as a countIf existence check, and the aggregate becomes a HAVING — all AND-combined.
// service.name (resource context) comes from otelKeysMap.
func TestBuild_TraceList_ResourcePlusSpanPlusAggregateFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "resource.service.name = 'checkout' AND has_error = true AND output_tokens > 1000"},
Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
// resource condition -> fingerprint CTE + prune, not filtered on the span index
// (the service.name output column still reads the resource map, hence the = form).
require.Contains(t, got, "__resource_filter AS (")
require.Contains(t, got, "resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
require.NotContains(t, got, "resources_string['service.name'] = 'checkout'")
// span condition -> existence check in matched HAVING.
require.Contains(t, got, "countIf(has_error = true) > 0")
// aggregate condition -> HAVING on the matched aggregate alias.
require.Contains(t, got, "output_tokens")
}
// The resolver-unset (nil) fallback is covered in pkg/telemetryscopedtraces, which
// can construct that builder state directly.
// Trace-level and span-level predicates may not be OR-combined.
func TestBuild_TraceList_TraceOrSpanMixRejected(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000 OR gen_ai.request.model = 'x'"},
Limit: 10,
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "cannot be combined")
}
// An output-only aggregate (span_count / trace_duration_nano) can be displayed but not
// used in the aggregate filter or ORDER BY — it is not computable in the matched pass.
func TestBuild_TraceList_OutputOnlyAggregateRejected(t *testing.T) {
b := newTestBuilder(t)
// filter by span_count -> rejected
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "span_count > 3"},
}, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "span_count")
// order by trace_duration_nano -> rejected
_, err = b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace_duration_nano"}}, Direction: qbtypes.OrderDirectionDesc}},
}, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "unsupported order key")
}
// duration_nano no longer names an aggregate (the trace column is trace_duration_nano),
// so a bare filter on it is span-level like everywhere else in the product: the trace
// matches when any span exceeds the duration.
func TestBuild_TraceList_SpanDurationFilterIsSpanLevel(t *testing.T) {
keys := otelKeysMap()
keys["duration_nano"] = []*telemetrytypes.TelemetryFieldKey{{
Name: "duration_nano",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeNumber,
}}
b := newTestBuilderWithKeys(t, keys)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "duration_nano > 1000000"},
Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
require.Contains(t, got, "countIf(duration_nano > 1000000) > 0")
require.NotContains(t, got, "HAVING trace_duration_nano")
}
// A HAVING referencing a non-aggregate column is rejected.
func TestBuild_TraceList_Having_UnknownColumn(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Having: &qbtypes.Having{Expression: "service.name > 1"}, // not an aggregate column
Limit: 10,
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.Error(t, err)
}
// Ordering by an unknown key is rejected.
func TestBuild_TraceList_UnsupportedOrderKey(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Order: []qbtypes.OrderBy{
{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "http.request.method"}}, Direction: qbtypes.OrderDirectionDesc},
},
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "unsupported order key")
}
// With no limit set, the builder applies the default of 100.
func TestBuild_TraceList_DefaultLimit(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
}
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.NoError(t, err)
require.Contains(t, stmt.Query, "LIMIT ?")
require.Contains(t, stmt.Args, 100)
}
// Only trace list and span list (raw) are supported; distribution is not.
func TestBuild_UnsupportedRequestType(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{
{Expression: "count()"},
},
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeDistribution, query, nil)
require.ErrorIs(t, err, scopedtraces.ErrUnsupportedRequestType)
}
// A gate key ingested under several data types (e.g. string + number from a
// misbehaving SDK) contributes ALL variants to the mask, OR-combined — not just
// the first — matching the standard visitor's EXISTS handling.
func TestBuild_TraceList_MultiVariantGateKey(t *testing.T) {
keys := otelKeysMap()
keys[telemetrytypes.GenAIToolName] = append(keys[telemetrytypes.GenAIToolName], &telemetrytypes.TelemetryFieldKey{
Name: telemetrytypes.GenAIToolName,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeFloat64,
})
b := newTestBuilderWithKeys(t, keys)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces, Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
require.Contains(t, got, "mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_number, 'gen_ai.tool.name')")
}
// `trace.` parses as the trace field context and marks a trace-level aggregate; the
// legacy `tracefield.` spelling is explicitly rejected (filter and having alike), and
// an output-only aggregate under the context gets the targeted rejection rather than
// an unknown-span-field failure.
func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
b := newTestBuilder(t)
build := func(q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (*qbtypes.Statement, error) {
q.Signal, q.Limit = telemetrytypes.SignalTraces, 20
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, q, nil)
}
_, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"}})
require.NoError(t, err)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
require.Contains(t, err.Error(), `use the "trace." prefix`)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Having: &qbtypes.Having{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
require.Contains(t, err.Error(), `use the "trace." prefix`)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"}})
require.Error(t, err)
require.Contains(t, err.Error(), "cannot be used")
}
// Query variables in a trace-level condition are substituted into the HAVING (the
// span path binds them via PrepareWhereClause; the HAVING is a text rewrite).
func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
b := newTestBuilder(t)
build := func(expr string, vars map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: expr},
Limit: 20,
}, vars)
}
// scalar variable -> literal in HAVING
stmt, err := build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Value: 700}})
require.NoError(t, err)
require.Contains(t, stmt.Query, "HAVING output_tokens > 700")
// list variable with IN
stmt, err = build("trace.llm_call_count IN $counts",
map[string]qbtypes.VariableItem{"counts": {Value: []any{1, 2}}})
require.NoError(t, err)
require.Contains(t, stmt.Query, "HAVING llm_call_count IN")
// dynamic __all__ -> condition dropped, no HAVING at all
stmt, err = build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.DynamicVariableType, Value: "__all__"}})
require.NoError(t, err)
require.NotContains(t, stmt.Query, "HAVING")
// unresolved variable -> rejected, not compared as a literal
_, err = build("trace.output_tokens > $missing", map[string]qbtypes.VariableItem{"other": {Value: 1}})
require.Error(t, err)
}

View File

@@ -1188,27 +1188,6 @@ func enrichWithIntrinsicMetricKeys(keys map[string][]*telemetrytypes.TelemetryFi
return keys
}
// enrichWithGenAIKeys adds keys that can be queried for GenAI signals, even though they have not been ingested yet.
func enrichWithGenAIKeys(keys map[string][]*telemetrytypes.TelemetryFieldKey, selectors []*telemetrytypes.FieldKeySelector) map[string][]*telemetrytypes.TelemetryFieldKey {
for _, selector := range selectors {
if selector.Signal != telemetrytypes.SignalTraces && selector.Signal != telemetrytypes.SignalUnspecified {
continue
}
for name, def := range telemetrytypes.GenAIFieldDefinitions {
if len(keys[name]) > 0 {
continue // already resolved from ingested data
}
if !selectorMatchesIntrinsicField(selector, def) {
continue
}
keyCopy := def
keys[name] = []*telemetrytypes.TelemetryFieldKey{&keyCopy}
}
}
return keys
}
func selectorMatchesIntrinsicField(selector *telemetrytypes.FieldKeySelector, definition telemetrytypes.TelemetryFieldKey) bool {
if selector.FieldContext != telemetrytypes.FieldContextUnspecified && selector.FieldContext != definition.FieldContext {
return false
@@ -1294,9 +1273,6 @@ func (t *telemetryMetaStore) GetKeys(ctx context.Context, orgID valuer.UUID, fie
applyBackwardCompatibleKeys(mapOfKeys)
mapOfKeys = enrichWithIntrinsicMetricKeys(mapOfKeys, selectors)
if t.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableAIObservability, featuretypes.NewFlaggerEvaluationContext(orgID)) {
mapOfKeys = enrichWithGenAIKeys(mapOfKeys, selectors)
}
return mapOfKeys, complete, nil
}
@@ -1375,9 +1351,6 @@ func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, orgID valuer.UUID
applyBackwardCompatibleKeys(mapOfKeys)
mapOfKeys = enrichWithIntrinsicMetricKeys(mapOfKeys, fieldKeySelectors)
if t.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableAIObservability, featuretypes.NewFlaggerEvaluationContext(orgID)) {
mapOfKeys = enrichWithGenAIKeys(mapOfKeys, fieldKeySelectors)
}
return mapOfKeys, complete, nil
}

View File

@@ -1,208 +0,0 @@
package telemetryscopedtraces
import (
"context"
"fmt"
"strings"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// This file holds the Aggregate constructors a TraceScope's columns are declared
// with. All SQL rendering goes through the fieldMapper.
// Aggregate renders one column's SQL through the fieldMapper and lists the attribute
// keys it references so the builder can pre-fetch their metadata. Build one with the
// constructors below; the zero value is not usable.
type Aggregate struct {
keys []*telemetrytypes.TelemetryFieldKey
render func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (expr string, args []any, err error)
}
// IntrinsicSpanKey references an intrinsic span-index field (timestamp, name, …) by
// its canonical name; the field mapper resolves it to the physical column.
func IntrinsicSpanKey(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
}
}
// AggFunc is a ClickHouse aggregate function name.
type AggFunc string
const (
AggSum AggFunc = "sum"
AggMax AggFunc = "max"
AggMin AggFunc = "min"
)
// PickDirection selects the earliest (argMin) or latest (argMax) span by ordering.
type PickDirection int
const (
PickLatest PickDirection = iota
PickEarliest
)
// CountAll renders count().
func CountAll() Aggregate {
return Aggregate{render: func(context.Context, valuer.UUID, uint64, uint64, *fieldMapper) (string, []any, error) {
return "count()", nil, nil
}}
}
// FieldReduce renders <fn>(<field>) over a field-mapper-resolved column.
func FieldReduce(fn AggFunc, key *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
f, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", nil, err
}
return fmt.Sprintf("%s(%s)", fn, f), nil, nil
}}
}
// TraceDuration renders the full-trace wall duration: last span end minus first
// span start.
func TraceDuration(tsKey, durationKey *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
ts, err := m.FieldFor(ctx, orgID, startNs, endNs, tsKey)
if err != nil {
return "", nil, err
}
dur, err := m.FieldFor(ctx, orgID, startNs, endNs, durationKey)
if err != nil {
return "", nil, err
}
return fmt.Sprintf("(max(toUnixTimestamp64Nano(%s) + %s) - min(toUnixTimestamp64Nano(%s)))", ts, dur, ts), nil, nil
}}
}
// FieldAnyWhere renders anyIf(<field>, <cond>) — the field value from any span
// matching the condition.
func FieldAnyWhere(valueKey, condKey *telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, condValue any) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, err := m.FieldFor(ctx, orgID, startNs, endNs, valueKey)
if err != nil {
return "", nil, err
}
cond, args, err := m.ConditionFor(ctx, orgID, startNs, endNs, condKey, op, condValue)
return fmt.Sprintf("anyIf(%s, %s)", v, cond), args, err
}}
}
// AnyValue renders any(<value>) over a metadata-resolved attribute value.
func AnyValue(key *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) Aggregate {
return Aggregate{keys: keysOf(key), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, args, err := m.ValueFor(ctx, orgID, startNs, endNs, key, dt)
return fmt.Sprintf("any(%s)", v), args, err
}}
}
// CountExists renders countIf(<key> EXISTS) — counts spans carrying key.
func CountExists(key *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: keysOf(key), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
cond, args, err := m.ExistsFor(ctx, orgID, startNs, endNs, key)
return fmt.Sprintf("countIf(%s)", cond), args, err
}}
}
// CondCount renders countIf(<cond>) over a condition-builder-resolved predicate.
func CondCount(key *telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
cond, args, err := m.ConditionFor(ctx, orgID, startNs, endNs, key, op, value)
return fmt.Sprintf("countIf(%s)", cond), args, err
}}
}
// Reduce renders <fn>(<value>) over a resolved numeric attribute value.
func Reduce(fn AggFunc, valueKey *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: keysOf(valueKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, args, err := m.ValueFor(ctx, orgID, startNs, endNs, valueKey, telemetrytypes.FieldDataTypeFloat64)
return fmt.Sprintf("%s(%s)", fn, v), args, err
}}
}
// ScopedReduce renders <fn>If(<field>, <gate mask>) over a field-mapper-resolved column.
func ScopedReduce(fn AggFunc, key *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
f, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", nil, err
}
return fmt.Sprintf("%sIf(%s, %s)", fn, f, m.maskExpr), append([]any{}, m.maskArgs...), nil
}}
}
// ScopedToKeyColumn renders <fn>If(<field>, <scopeKey> EXISTS) — a span-index field
// aggregated over spans carrying scopeKey (e.g. max LLM latency).
func ScopedToKeyColumn(fn AggFunc, columnKey, scopeKey *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: keysOf(scopeKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
col, err := m.FieldFor(ctx, orgID, startNs, endNs, columnKey)
if err != nil {
return "", nil, err
}
cond, args, err := m.ExistsFor(ctx, orgID, startNs, endNs, scopeKey)
return fmt.Sprintf("%sIf(%s, %s)", fn, col, cond), args, err
}}
}
// PickBy renders argMinIf/argMaxIf(<value>, <orderField>, <value> EXISTS) — the value
// from the earliest/latest span that carries it.
func PickBy(valueKey *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType, orderKey *telemetrytypes.TelemetryFieldKey, dir PickDirection) Aggregate {
fn := "argMaxIf"
if dir == PickEarliest {
fn = "argMinIf"
}
return Aggregate{keys: keysOf(valueKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, vargs, err := m.ValueFor(ctx, orgID, startNs, endNs, valueKey, dt)
if err != nil {
return "", nil, err
}
order, err := m.FieldFor(ctx, orgID, startNs, endNs, orderKey)
if err != nil {
return "", nil, err
}
cond, cargs, err := m.ExistsFor(ctx, orgID, startNs, endNs, valueKey)
return fmt.Sprintf("%s(%s, %s, %s)", fn, v, order, cond), append(vargs, cargs...), err
}}
}
// UniqCount renders uniqIf(<value>, <value> EXISTS) — distinct count of an attribute.
func UniqCount(valueKey *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) Aggregate {
return Aggregate{keys: keysOf(valueKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, vargs, err := m.ValueFor(ctx, orgID, startNs, endNs, valueKey, dt)
if err != nil {
return "", nil, err
}
cond, cargs, err := m.ExistsFor(ctx, orgID, startNs, endNs, valueKey)
return fmt.Sprintf("uniqIf(%s, %s)", v, cond), append(vargs, cargs...), err
}}
}
// SumOfKeys renders coalesce(sum(<v1>), 0) + coalesce(sum(<v2>), 0) + … over several
// numeric attributes. Coalesced because a key absent from every span sums to NULL and
// NULL + n = NULL — a trace with only output tokens would otherwise total NULL.
func SumOfKeys(dt telemetrytypes.FieldDataType, valueKeys ...*telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: valueKeys, render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
parts := make([]string, 0, len(valueKeys))
var args []any
for _, k := range valueKeys {
v, vargs, err := m.ValueFor(ctx, orgID, startNs, endNs, k, dt)
if err != nil {
return "", nil, err
}
parts = append(parts, fmt.Sprintf("coalesce(sum(%s), 0)", v))
args = append(args, vargs...)
}
return strings.Join(parts, " + "), args, nil
}}
}
func keysOf(k *telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return []*telemetrytypes.TelemetryFieldKey{k}
}

View File

@@ -1,85 +0,0 @@
package telemetryscopedtraces
import (
"context"
"strings"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
// fieldMapper resolves aggregate-column SQL through the shared field mapper and
// condition builder, following their method shapes (FieldFor / ConditionFor / …) so
// column resolution reads like the other statement builders. keys is the fetched
// metadata for the keys the columns reference; the gate mask is set by the builder
// after resolveMask (Scoped* aggregates embed it). All returned expressions are
// escaped once, ready to embed in an outer builder.
type fieldMapper struct {
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
keys map[string][]*telemetrytypes.TelemetryFieldKey
maskExpr string
maskArgs []any
}
func newFieldMapper(fm qbtypes.FieldMapper, cb qbtypes.ConditionBuilder, keys map[string][]*telemetrytypes.TelemetryFieldKey) *fieldMapper {
return &fieldMapper{fm: fm, cb: cb, keys: keys}
}
// FieldFor returns the column expression for key via the field mapper.
func (r *fieldMapper) FieldFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
expr, err := r.fm.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", err
}
return sqlbuilder.Escape(expr), nil
}
// ConditionFor returns a boolean predicate for key via the condition builder
// (materialized column when present, else map access).
func (r *fieldMapper) ConditionFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
// The condition builder owns key resolution: hand it the raw key plus the full
// metadata map and it matches/synthesizes the candidates itself.
conds, _, err := r.cb.ConditionFor(ctx, orgID, startNs, endNs, key, r.keys, qbtypes.ConditionBuilderOptions{}, op, value, sb)
if err != nil {
return "", nil, err
}
if len(conds) == 0 {
return "", nil, nil
}
// One condition per candidate variant (a key can be ingested under several data
// types); OR them all, like the visitor does for EXISTS.
if len(conds) == 1 {
sb.Where(conds[0])
} else {
sb.Where(sb.Or(conds...))
}
expr, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
expr = strings.TrimPrefix(expr, "WHERE ")
return sqlbuilder.Escape(expr), args, nil
}
// ExistsFor returns the EXISTS predicate for key.
func (r *fieldMapper) ExistsFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, []any, error) {
return r.ConditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorExists, nil)
}
// ValueFor returns the value expression for an attribute key. The metadata variant
// is preferred because it carries Materialized — a provider's static definition
// never does, so a promoted attribute would otherwise fall back to map access.
func (r *fieldMapper) ValueFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) (string, []any, error) {
if cands := r.keys[key.Name]; len(cands) > 0 {
key = cands[0]
}
expr, err := r.fm.ColumnExpressionFor(ctx, orgID, startNs, endNs, key, dt, r.keys)
if err != nil {
return "", nil, err
}
// Escape before embedding in the outer builder: a materialized column name carries
// `$$` (from the dotted attribute name), which go-sqlbuilder's Build would otherwise
// unescape to a single `$` and reference the wrong column.
return sqlbuilder.Escape(expr), nil, nil
}

View File

@@ -1,66 +0,0 @@
package telemetryscopedtraces
import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// TraceScope is the configuration the scoped trace builder accepts: which spans are
// in scope and which per-trace columns the list computes. It only declares the gate
// and the columns; the builder resolves everything through the field mapper, so
// attribute access stays materialization-aware. A new span category only needs a
// new TraceScope.
type TraceScope struct {
// FilterExpression is the grammar-level (EXISTS) gate, used on the delegated
// span-list path.
FilterExpression string
// FieldKeys are the gate's keys, used to build the per-span mask
// (OR of resolved EXISTS conditions).
FieldKeys []*telemetrytypes.TelemetryFieldKey
// Columns are the per-trace output columns.
Columns []TraceColumn
// DefaultOrderAlias is sorted by (desc) when the query gives no order.
DefaultOrderAlias string
}
// TraceColumn is one per-trace output column.
type TraceColumn struct {
// Alias must not reuse a physical span-index column name (e.g. duration_nano):
// ClickHouse resolves bare identifiers to same-SELECT aliases first, so any
// expression referencing that column would silently bind to the alias.
Alias string
// Orderable columns can be used in ORDER BY and the aggregate filter. All-span
// aggregates (span_count, trace_duration_nano, …) are display-only and set false.
Orderable bool
// SpanLevel columns surface a real span/resource attribute (service.name,
// input/output messages); a filter on them is applied span-level, so they are
// excluded from the trace-level aliases.
SpanLevel bool
Expr Aggregate
}
// CommonTraceColumns are domain-neutral columns any trace list can reuse. All
// aggregate over every span, so none is Orderable.
func CommonTraceColumns() []TraceColumn {
ts := IntrinsicSpanKey("timestamp")
duration := IntrinsicSpanKey("duration_nano")
name := IntrinsicSpanKey("name")
parentSpanID := IntrinsicSpanKey("parent_span_id")
serviceName := &telemetrytypes.TelemetryFieldKey{
Name: "service.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
return []TraceColumn{
{Alias: "start_time", Expr: FieldReduce(AggMin, ts)},
{Alias: "end_time", Expr: FieldReduce(AggMax, ts)},
// Not plain "duration_nano": that name is the intrinsic span field, and an
// alias would shadow it — both in ClickHouse identifier resolution and in
// bare-name filter classification.
{Alias: "trace_duration_nano", Expr: TraceDuration(ts, duration)},
{Alias: "span_count", Expr: CountAll()},
{Alias: "root_span_name", Expr: FieldAnyWhere(name, parentSpanID, qbtypes.FilterOperatorEqual, "")},
{Alias: "service.name", SpanLevel: true, Expr: AnyValue(serviceName, telemetrytypes.FieldDataTypeString)},
}
}

View File

@@ -1,781 +0,0 @@
package telemetryscopedtraces
import (
"context"
"fmt"
"log/slog"
"sort"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
qbvariables "github.com/SigNoz/signoz/pkg/variables"
"github.com/huandu/go-sqlbuilder"
)
var (
ErrUnsupportedRequestType = errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported request type for the scoped trace builder")
)
// scopedTraceStatementBuilder builds a trace list scoped to one span category
// (e.g. gen_ai spans). The query shape is fixed; the TraceScope decides which spans
// are in scope and which per-trace columns to compute, so a new category only needs
// a new scope.
type scopedTraceStatementBuilder struct {
logger *slog.Logger
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
scope TraceScope
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
}
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*scopedTraceStatementBuilder)(nil)
// NewScopedTraceStatementBuilder wires the generic trace-list builder. The field
// mapper / condition builder are built here, not injected — the list always scans the
// telemetrytraces span index. traceStmtBuilder (the delegate for the span-list path)
// is injected because the provider already has the canonical instance.
func NewScopedTraceStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
scope TraceScope,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
fl flagger.Flagger,
) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
scopedSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryscopedtraces")
fm := telemetrytraces.NewFieldMapper()
cb := telemetrytraces.NewConditionBuilder(fm)
// Same resource-fingerprint prune as the standard trace builder — the list scans
// the same span index.
resourceFilterStmtBuilder := telemetryresourcefilter.New[qbtypes.TraceAggregation](
settings,
telemetrytraces.DBName,
telemetrytraces.TracesResourceV3TableName,
telemetrytypes.SignalTraces,
telemetrytypes.SourceUnspecified,
metadataStore,
nil,
fl,
)
return &scopedTraceStatementBuilder{
logger: scopedSettings.Logger(),
metadataStore: metadataStore,
fm: fm,
cb: cb,
scope: scope,
traceStmtBuilder: traceStmtBuilder,
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
}
}
func (b *scopedTraceStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
switch requestType {
case qbtypes.RequestTypeTrace:
return b.buildTraceListQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), query, variables)
case qbtypes.RequestTypeRaw:
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
default:
return nil, ErrUnsupportedRequestType
}
}
// buildDelegated ANDs the base gate into the user filter and delegates to the
// standard trace builder (the span-list / raw path).
func (b *scopedTraceStatementBuilder) buildDelegated(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
gate := b.scope.FilterExpression
expr := gate
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
expr = fmt.Sprintf("(%s) AND (%s)", gate, query.Filter.Expression)
}
// shallow copy; only Filter is replaced, caller's query untouched
gated := query
gated.Filter = &qbtypes.Filter{Expression: expr}
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
// buildTraceListQuery wires the CTE pipeline: one windowed pass picks the top-N
// traces, then a bucket-pruned pass enriches only those.
// Helpers appear in this file in the order they run. start/end are nanoseconds.
//
// RESOLVE (keys/columns → SQL via the field mapper)
// fetchKeys metadata for every key we reference
// resolveMask the "span is in scope" predicate (OR of EXISTS)
// resolveColumns per-trace column SQL
// resolveListOrders which columns to ORDER BY
// splitFilter span-level predicate + trace-level HAVING
//
// BUILD
// matched one windowed, mask-pruned GROUP BY trace_id scan fusing gate + span
// │ filter + HAVING + ORDER BY + LIMIT/OFFSET → the top-N trace_ids
// ▼
// ranked [start,end] bounds of those traces, from the small summary table
// ▼
// buckets the ts_bucket_start values they touch, to prune the next scan
// ▼
// enrichment every per-trace column for those traces over their full extent
// (not window-clipped), scanning only their buckets
//
// Only Orderable columns are computable in the mask-pruned matched pass, so only they
// can be ordered or filtered on; all-span columns (span_count, …) are output-only.
func (b *scopedTraceStatementBuilder) buildTraceListQuery(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
startBucket := start/querybuilder.NsToSeconds - querybuilder.BucketAdjustment
endBucket := end / querybuilder.NsToSeconds
limit := query.Limit
if limit <= 0 {
limit = 100
}
// Resolve keys and columns once; all attribute access goes through the field mapper.
keys, err := b.fetchKeys(ctx, orgID)
if err != nil {
return nil, err
}
mapper := newFieldMapper(b.fm, b.cb, keys)
maskExpr, maskArgs, err := b.resolveMask(ctx, orgID, start, end, mapper)
if err != nil {
return nil, err
}
mapper.maskExpr, mapper.maskArgs = maskExpr, maskArgs
resolved, err := b.resolveColumns(ctx, orgID, start, end, mapper)
if err != nil {
return nil, err
}
orders, err := b.resolveListOrders(query.Order, resolved)
if err != nil {
return nil, err
}
orderableSet := orderableAliasSet(resolved)
// If the filter references resource attributes, add a __resource_filter CTE and
// narrow the matched scan by resource_fingerprint; the span predicate drops those
// keys so they aren't applied twice.
resourceFrag, resourceArgs, resourcePred, err := b.maybeAttachResourceFilter(ctx, orgID, query, start, end, variables)
if err != nil {
return nil, err
}
// Split the user filter: span-level predicate + trace-level HAVING expression.
fp, err := b.splitFilter(ctx, orgID, query, b.aggregateAliasSet(), orderableSet, start, end, variables)
if err != nil {
return nil, err
}
// matched → ranked → buckets → enrichment
matchedFrag, matchedArgs, err := b.buildMatchedCTE(start, end, startBucket, endBucket, resolved, orders, orderableSet, maskExpr, maskArgs, fp, resourcePred, limit, query.Offset)
if err != nil {
return nil, err
}
rankedFrag, rankedArgs := b.buildRankedCTE(start, end)
bucketsFrag := buildBucketsCTE()
mainSQL, mainArgs := b.buildEnrichmentSelect(resolved, orders)
cteFragments := []string{matchedFrag, rankedFrag, bucketsFrag}
cteArgs := [][]any{matchedArgs, rankedArgs, nil}
// __resource_filter must precede `matched`, which references it.
if resourceFrag != "" {
cteFragments = append([]string{resourceFrag}, cteFragments...)
cteArgs = append([][]any{resourceArgs}, cteArgs...)
}
finalSQL := querybuilder.CombineCTEs(cteFragments) + mainSQL + " SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000"
finalArgs := querybuilder.PrependArgs(cteArgs, mainArgs)
return &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
Warnings: fp.warnings,
WarningsDocURL: fp.warningsURL,
}, nil
}
// maybeAttachResourceFilter builds the __resource_filter CTE (fingerprints matching
// the filter's resource conditions) and the predicate narrowing the span scan by
// resource_fingerprint; with no resource conditions it returns empty fragments.
//
// Unlike the standard trace builder there is deliberately no skip-fingerprint
// fallback: falling back would leave the resource conditions inside the OR'd
// span-filter bucket, which changes trace membership (any span from the resource +
// any gen_ai span, instead of a gen_ai span from the resource). Resource conditions
// always scope the whole matched scan.
func (b *scopedTraceStatementBuilder) maybeAttachResourceFilter(
ctx context.Context,
orgID valuer.UUID,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
variables map[string]qbtypes.VariableItem,
) (cteFrag string, cteArgs []any, fingerprintPred string, err error) {
stmt, err := b.resourceFilterStmtBuilder.Build(
ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables,
)
if err != nil {
return "", nil, "", err
}
if stmt == nil {
return "", nil, "", nil
}
return fmt.Sprintf("__resource_filter AS (%s)", stmt.Query), stmt.Args,
"resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)", nil
}
// ---------------------------------------------------------------------------
// RESOLVE — turn keys/columns into field-mapper-aware SQL
// ---------------------------------------------------------------------------
func (b *scopedTraceStatementBuilder) fetchKeys(ctx context.Context, orgID valuer.UUID) (map[string][]*telemetrytypes.TelemetryFieldKey, error) {
fields := b.resolverFieldKeys()
selectors := make([]*telemetrytypes.FieldKeySelector, 0, len(fields))
for _, k := range fields {
selectors = append(selectors, &telemetrytypes.FieldKeySelector{
Name: k.Name,
Signal: k.Signal,
FieldContext: k.FieldContext,
SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact,
})
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
return keys, err
}
func (b *scopedTraceStatementBuilder) resolverFieldKeys() []*telemetrytypes.TelemetryFieldKey {
seen := make(map[string]struct{})
var out []*telemetrytypes.TelemetryFieldKey
add := func(k *telemetrytypes.TelemetryFieldKey) {
if k == nil {
return
}
if _, dup := seen[k.Name]; dup {
return
}
seen[k.Name] = struct{}{}
out = append(out, k)
}
for _, k := range b.scope.FieldKeys {
add(k)
}
for _, c := range b.scope.Columns {
for _, k := range c.Expr.keys {
add(k)
}
}
return out
}
// resolveMask builds the per-span in-scope mask: OR of resolved EXISTS predicates
// over the base condition's field keys.
func (b *scopedTraceStatementBuilder) resolveMask(ctx context.Context, orgID valuer.UUID, start, end uint64, mapper *fieldMapper) (string, []any, error) {
fieldKeys := b.scope.FieldKeys
parts := make([]string, 0, len(fieldKeys))
var args []any
for _, key := range fieldKeys {
e, a, err := mapper.ExistsFor(ctx, orgID, start, end, key)
if err != nil {
return "", nil, err
}
parts = append(parts, e)
args = append(args, a...)
}
return "(" + strings.Join(parts, " OR ") + ")", args, nil
}
// resolvedColumn is a column resolved to SQL via the field mapper; expr is escaped
// once, ready to embed in an outer SELECT.
type resolvedColumn struct {
alias string
expr string
args []any
orderable bool
}
// resolveColumns turns the declarative columns into SQL through the resolver, so all
// attribute access goes through the field mapper / condition builder.
func (b *scopedTraceStatementBuilder) resolveColumns(ctx context.Context, orgID valuer.UUID, start, end uint64, mapper *fieldMapper) ([]resolvedColumn, error) {
cols := b.scope.Columns
out := make([]resolvedColumn, 0, len(cols))
for _, c := range cols {
expr, args, err := c.Expr.render(ctx, orgID, start, end, mapper)
if err != nil {
return nil, err
}
out = append(out, resolvedColumn{alias: c.Alias, expr: expr, args: args, orderable: c.Orderable})
}
return out, nil
}
// listOrder is a sort key resolved to a column alias + direction; both the matched
// CTE and the enrichment ORDER BY it.
type listOrder struct {
alias string
direction string
}
// resolveListOrders maps order keys to the resolved orderable columns; non-orderable
// columns are rejected. Defaults to the column provider's default order.
func (b *scopedTraceStatementBuilder) resolveListOrders(order []qbtypes.OrderBy, resolved []resolvedColumn) ([]listOrder, error) {
byAlias := make(map[string]resolvedColumn, len(resolved))
orderable := make([]string, 0, len(resolved))
for _, rc := range resolved {
byAlias[rc.alias] = rc
if rc.orderable {
orderable = append(orderable, rc.alias)
}
}
if len(order) == 0 {
return []listOrder{{alias: b.scope.DefaultOrderAlias, direction: "DESC"}}, nil
}
orders := make([]listOrder, 0, len(order))
for _, o := range order {
direction := "DESC"
if o.Direction == qbtypes.OrderDirectionAsc {
direction = "ASC"
}
rc, ok := byAlias[o.Key.Name]
if !ok || !rc.orderable {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"unsupported order key %q for the trace list; orderable keys: %s", o.Key.Name, strings.Join(orderable, ", "))
}
orders = append(orders, listOrder{alias: rc.alias, direction: direction})
}
return orders, nil
}
// filterParts is the user filter split into a span-level predicate (widens the
// matched WHERE prune and becomes a countIf existence check in HAVING) and a
// trace-level HAVING expression.
type filterParts struct {
spanPred string
spanArgs []any
hasSpanFilter bool
havingExpr string
warnings []string
warningsURL string
}
// splitFilter splits query.Filter into a span-level predicate and a trace-level
// HAVING expression (an explicit query.Having is ANDed onto the latter), then
// validates the trace-level part against the matched-pass aggregates.
func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID valuer.UUID, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], classifySet, orderableSet map[string]struct{}, start, end uint64, variables map[string]qbtypes.VariableItem) (filterParts, error) {
var fp filterParts
// The legacy tracefield. spelling parses identically to trace.; only the
// user-facing form is supported here.
if (query.Filter != nil && strings.Contains(query.Filter.Expression, "tracefield.")) ||
(query.Having != nil && strings.Contains(query.Having.Expression, "tracefield.")) {
return fp, errors.NewInvalidInputf(errors.CodeInvalidInput, "\"tracefield.\" is not supported; use the \"trace.\" prefix")
}
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
spanExpr, traceExpr, err := querybuilder.SplitFilterForAggregates(query.Filter.Expression, classifySet)
if err != nil {
return fp, err
}
fp.havingExpr = traceExpr
if strings.TrimSpace(spanExpr) != "" {
pred, args, warnings, url, err := b.resolveSpanPredicate(ctx, orgID, start, end, spanExpr, variables)
if err != nil {
return fp, err
}
// pred is empty when the span-level keys were all resource attributes
// already handled by the __resource_filter CTE.
if strings.TrimSpace(pred) != "" {
fp.spanPred, fp.spanArgs, fp.hasSpanFilter = pred, args, true
}
fp.warnings, fp.warningsURL = warnings, url
}
}
if query.Having != nil && strings.TrimSpace(query.Having.Expression) != "" {
if fp.havingExpr != "" {
fp.havingExpr = fmt.Sprintf("(%s) AND (%s)", fp.havingExpr, query.Having.Expression)
} else {
fp.havingExpr = query.Having.Expression
}
}
// The span predicate binds variables via PrepareWhereClause; the HAVING is a plain
// text rewrite, so substitute variables here (list/IN quoting, __all__ drops the
// condition) before validating.
if strings.TrimSpace(fp.havingExpr) != "" && len(variables) > 0 {
replaced, err := qbvariables.ReplaceVariablesInExpression(fp.havingExpr, variables)
if err != nil {
return fp, err
}
fp.havingExpr = replaced
}
if err := validateAggregateFilter(fp.havingExpr, orderableSet); err != nil {
return fp, err
}
return fp, nil
}
// resolveSpanPredicate resolves a span-level filter expression to a bare boolean
// SQL predicate + args via the field mapper.
func (b *scopedTraceStatementBuilder) resolveSpanPredicate(ctx context.Context, orgID valuer.UUID, start, end uint64, expr string, variables map[string]qbtypes.VariableItem) (string, []any, []string, string, error) {
selectors := querybuilder.QueryStringToKeysSelectors(expr)
for i := range selectors {
selectors[i].Signal = telemetrytypes.SignalTraces
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
if err != nil {
return "", nil, nil, "", err
}
prepared, err := querybuilder.PrepareWhereClause(expr, querybuilder.FilterExprVisitorOpts{
Context: ctx,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
FieldKeys: keys,
// resource conditions are always handled by the __resource_filter CTE
SkipResourceFilter: true,
Variables: variables,
StartNs: start,
EndNs: end,
})
if err != nil {
return "", nil, nil, "", err
}
if prepared.IsEmpty() {
return "", nil, nil, "", nil
}
sb := sqlbuilder.NewSelectBuilder()
sb.AddWhereClause(prepared.WhereClause)
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
pred := sql[strings.Index(sql, "WHERE ")+len("WHERE "):]
return sqlbuilder.Escape(pred), args, prepared.Warnings, prepared.WarningsDocURL, nil
}
// buildMatchedCTE builds `matched`: the single windowed GROUP BY trace_id scan that
// fuses gate + span filter + HAVING + ORDER BY + LIMIT/OFFSET, selecting only the
// aliases the ORDER BY / HAVING reference.
func (b *scopedTraceStatementBuilder) buildMatchedCTE(start, end, startBucket, endBucket uint64, resolved []resolvedColumn, orders []listOrder, orderableSet map[string]struct{}, maskExpr string, maskArgs []any, fp filterParts, resourcePred string, limit, offset int) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
// SELECT trace_id + only the aggregates ORDER BY / HAVING reference (as aliases).
needed := neededMatchedAliases(orders, fp.havingExpr, orderableSet)
selects := []string{"trace_id"}
for _, rc := range resolved {
if _, ok := needed[rc.alias]; !ok {
continue
}
colExpr, err := embedExpr(sb, rc.expr, rc.args)
if err != nil {
return "", nil, err
}
selects = append(selects, colExpr+" AS "+quoteAlias(rc.alias))
}
sb.Select(selects...)
sb.From(spanTable())
// WHERE: window + prune to in-scope spans, widened by the span filter so its
// spans survive for the countIf existence check below.
win := windowWhere(sb, start, end, startBucket, endBucket)
mask, err := embedExpr(sb, maskExpr, maskArgs)
if err != nil {
return "", nil, err
}
prune := "(" + mask
if fp.hasSpanFilter {
spanPred, err := embedExpr(sb, fp.spanPred, fp.spanArgs)
if err != nil {
return "", nil, err
}
prune += " OR " + spanPred
}
prune += ")"
where := append(win, prune)
if resourcePred != "" {
where = append(where, resourcePred)
}
sb.Where(where...)
sb.GroupBy("trace_id")
// HAVING: the gate/span existence checks are only needed when the WHERE was
// widened by a span filter; otherwise the mask alone already enforces the gate.
var having []string
if fp.hasSpanFilter {
havingMask, err := embedExpr(sb, maskExpr, maskArgs)
if err != nil {
return "", nil, err
}
havingPred, err := embedExpr(sb, fp.spanPred, fp.spanArgs)
if err != nil {
return "", nil, err
}
having = append(having, "countIf("+havingMask+") > 0")
having = append(having, "countIf("+havingPred+") > 0")
}
if strings.TrimSpace(fp.havingExpr) != "" {
hv, err := b.buildHaving(fp.havingExpr, orderableSet)
if err != nil {
return "", nil, err
}
if hv != "" {
// hv carries user text with values inlined by the rewriter; escape it so a
// literal $ can't be read as an interpolation marker at Build time. The
// countIf entries above hold live placeholders and must stay unescaped.
having = append(having, sqlbuilder.Escape(hv))
}
}
if len(having) > 0 {
sb.Having(strings.Join(having, " AND "))
}
sb.OrderBy(orderClause(orders)...)
sb.Limit(limit)
if offset > 0 {
sb.Offset(offset)
}
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("matched AS (%s)", sql), args, nil
}
// buildRankedCTE builds `ranked`: [start,end] bounds per matched trace, read from the
// small trace-summary table.
func (b *scopedTraceStatementBuilder) buildRankedCTE(start, end uint64) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("trace_id", "min(start) AS t_start", "max(end) AS t_end")
sb.From(summaryTable())
sb.Where(
"trace_id GLOBAL IN (SELECT trace_id FROM matched)",
"end >= fromUnixTimestamp64Nano("+sb.Var(start)+")",
"start < fromUnixTimestamp64Nano("+sb.Var(end)+")",
)
sb.GroupBy("trace_id")
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("ranked AS (%s)", sql), args
}
// buildBucketsCTE builds `buckets`: the ts_bucket_start values the matched traces
// span, so the enrichment scan is primary-key pruned. No args.
func buildBucketsCTE() string {
adj := querybuilder.BucketAdjustment // 30-min bucket width in seconds
return fmt.Sprintf("buckets AS (SELECT DISTINCT b AS ts_bucket FROM ranked "+
"ARRAY JOIN range("+
"toUInt64(intDiv(toUnixTimestamp(t_start), %d) * %d - %d), "+
"toUInt64(intDiv(toUnixTimestamp(t_end), %d) * %d + %d), "+
"%d) AS b)", adj, adj, adj, adj, adj, adj, adj)
}
// buildEnrichmentSelect builds the final SELECT: every per-trace column for the
// matched traces over their full extent, scanning only their buckets.
//
// Accepted discrepancy: matched ranks/paginates on window-clipped values (and, with a
// resource filter, only over fingerprint-matching spans), while this pass recomputes
// and ORDER BYs full-trace values — so a trace with activity outside the window or
// resource can sort differently than it ranked. Page membership is unaffected
// (LIMIT/OFFSET runs only in matched); rows still sort by the values the user sees.
// Ordering by matched's values instead would re-run the matched scan (ClickHouse
// re-executes a CTE per reference) without fixing the visible cross-page artifact.
func (b *scopedTraceStatementBuilder) buildEnrichmentSelect(resolved []resolvedColumn, orders []listOrder) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
selects, selectArgs := selectAllColumns(resolved)
sb.Select(selects...)
sb.From(spanTable())
sb.Where(
"ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)",
"trace_id GLOBAL IN (SELECT trace_id FROM ranked)",
)
sb.GroupBy("trace_id")
sb.OrderBy(orderClause(orders)...)
sql, builtArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return sql, append(append([]any{}, selectArgs...), builtArgs...)
}
// buildHaving rewrites a trace-level HAVING expression to the matched-pass column
// aliases. The rewriter matches raw key text, so the trace. form is mapped alongside
// the bare name (the legacy tracefield. spelling is rejected upfront in splitFilter).
func (b *scopedTraceStatementBuilder) buildHaving(havingExpr string, orderableSet map[string]struct{}) (string, error) {
columnMap := make(map[string]string, len(orderableSet)*2)
for a := range orderableSet {
columnMap[a] = quoteAlias(a)
columnMap[telemetrytypes.FieldContextTrace.StringValue()+"."+a] = quoteAlias(a)
}
return querybuilder.NewHavingExpressionRewriter().Rewrite(havingExpr, columnMap)
}
// ---------------------------------------------------------------------------
// Small shared SQL-builder utilities
// ---------------------------------------------------------------------------
// spanTable is the fully-qualified span index table.
func spanTable() string {
return fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.SpanIndexV3TableName)
}
// summaryTable is the fully-qualified trace-summary table.
func summaryTable() string {
return fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.TraceSummaryTableName)
}
// aggregateAliasSet is every trace-level column alias, used to classify filter keys
// as trace-level vs span-level. Derived from the scope's columns so a new column
// can't be forgotten; SpanLevel columns are filtered span-level, so skip them.
func (b *scopedTraceStatementBuilder) aggregateAliasSet() map[string]struct{} {
set := make(map[string]struct{}, len(b.scope.Columns))
for _, c := range b.scope.Columns {
if !c.SpanLevel {
set[c.Alias] = struct{}{}
}
}
return set
}
// orderableAliasSet is the subset of aliases computable in the matched pass — the
// only ones usable in ORDER BY and the aggregate filter.
func orderableAliasSet(resolved []resolvedColumn) map[string]struct{} {
set := make(map[string]struct{})
for _, rc := range resolved {
if rc.orderable {
set[rc.alias] = struct{}{}
}
}
return set
}
// neededMatchedAliases is the minimal alias set the matched pass must select: those
// in ORDER BY plus those in the aggregate HAVING. Everything else is left to the
// enrichment scan.
func neededMatchedAliases(orders []listOrder, havingExpr string, orderableSet map[string]struct{}) map[string]struct{} {
needed := make(map[string]struct{})
for _, o := range orders {
needed[o.alias] = struct{}{}
}
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; ok {
needed[name] = struct{}{}
}
}
return needed
}
// traceAggregateNames extracts the aggregate names a trace-level HAVING expression
// references. QueryStringToKeysSelectors emits an extra attribute-context fallback
// selector for context-prefixed keys (`trace.x` → attribute "trace.x"); only the
// unspecified- and trace-context selectors name aggregates.
func traceAggregateNames(havingExpr string) []string {
var names []string
for _, sel := range querybuilder.QueryStringToKeysSelectors(havingExpr) {
if sel.FieldContext == telemetrytypes.FieldContextUnspecified || sel.FieldContext == telemetrytypes.FieldContextTrace {
names = append(names, sel.Name)
}
}
return names
}
// validateAggregateFilter rejects a trace-level filter referencing an aggregate not
// computable in the matched pass (e.g. span_count, trace_duration_nano).
func validateAggregateFilter(havingExpr string, orderableSet map[string]struct{}) error {
if strings.TrimSpace(havingExpr) == "" {
return nil
}
allowed := make([]string, 0, len(orderableSet))
for a := range orderableSet {
allowed = append(allowed, a)
}
sort.Strings(allowed)
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregate %q cannot be used in the trace-list filter; filterable aggregates: %s", name, strings.Join(allowed, ", "))
}
}
return nil
}
// embedExpr inlines a resolved expr into sb, replacing each `?` placeholder with a
// builder Var so the args are tracked in appearance order. Resolved exprs carry
// values only as bound args, so every `?` is a placeholder; a count mismatch would
// silently shift args into the wrong slots — error out instead.
func embedExpr(sb *sqlbuilder.SelectBuilder, expr string, args []any) (string, error) {
if n := strings.Count(expr, "?"); n != len(args) {
return "", errors.NewInternalf(errors.CodeInternal,
"scoped trace builder: %d placeholders != %d args embedding %q", n, len(args), expr)
}
var out strings.Builder
ai := 0
for i := 0; i < len(expr); i++ {
if expr[i] == '?' {
out.WriteString(sb.Var(args[ai]))
ai++
continue
}
out.WriteByte(expr[i])
}
return out.String(), nil
}
// windowWhere binds the time-window predicates to sb and returns them so the caller
// can add its own predicates in the same Where call.
func windowWhere(sb *sqlbuilder.SelectBuilder, start, end, startBucket, endBucket uint64) []string {
return []string{
sb.GE("timestamp", fmt.Sprintf("%d", start)),
sb.L("timestamp", fmt.Sprintf("%d", end)),
sb.GE("ts_bucket_start", startBucket),
sb.LE("ts_bucket_start", endBucket),
}
}
// orderClause renders the ORDER BY terms plus the trace_id tiebreak.
func orderClause(orders []listOrder) []string {
out := make([]string, 0, len(orders)+1)
for _, o := range orders {
out = append(out, fmt.Sprintf("%s %s", quoteAlias(o.alias), o.direction))
}
return append(out, "trace_id DESC")
}
// selectAllColumns renders `expr AS alias` for every resolved column, args in select
// order.
func selectAllColumns(resolved []resolvedColumn) ([]string, []any) {
selects := []string{"trace_id"}
var args []any
for _, rc := range resolved {
selects = append(selects, rc.expr+" AS "+quoteAlias(rc.alias))
args = append(args, rc.args...)
}
return selects, args
}
// quoteAlias backticks an alias containing characters special to the SQL builder.
func quoteAlias(alias string) string {
if strings.ContainsAny(alias, ".$`") {
return "`" + alias + "`"
}
return alias
}

View File

@@ -1,30 +0,0 @@
package telemetryscopedtraces
import (
"strings"
"testing"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/require"
)
// The full-pipeline golden tests live in pkg/telemetryai, which exercises this
// builder through its production provider pair. The tests here cover only what
// needs the package internals.
// embedExpr treats every `?` byte as a placeholder; a count/args mismatch (an expr
// carrying a literal `?`, or a dropped arg) must fail loudly instead of silently
// shifting every subsequent arg into the wrong placeholder.
func TestEmbedExpr_PlaceholderArgMismatch(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
out, err := embedExpr(sb, "x = ? AND y = ?", []any{1, 2})
require.NoError(t, err)
require.Equal(t, 2, strings.Count(out, "$"), "both placeholders bound as builder vars")
_, err = embedExpr(sb, "x = ? AND y LIKE 'a?b'", []any{1})
require.Error(t, err, "literal ? in the expr must not pass as a placeholder")
_, err = embedExpr(sb, "x = ?", []any{1, 2})
require.Error(t, err, "extra args must not be silently dropped")
}

View File

@@ -16,10 +16,18 @@ import (
const (
LLMCostFeatureType agentConf.AgentFeatureType = "llm_pricing"
GenAIRequestModel = "gen_ai.request.model"
GenAIProviderName = "gen_ai.provider.name"
GenAIUsageInputTokens = "gen_ai.usage.input_tokens"
GenAIUsageOutputTokens = "gen_ai.usage.output_tokens"
GenAIUsageCacheReadInputTokens = "gen_ai.usage.cache_read.input_tokens"
GenAIUsageCacheCreationInputTokens = "gen_ai.usage.cache_creation.input_tokens"
SignozGenAICostInput = "_signoz.gen_ai.cost_input"
SignozGenAICostOutput = "_signoz.gen_ai.cost_output"
SignozGenAICostCacheRead = "_signoz.gen_ai.cost_cache_read"
SignozGenAICostCacheWrite = "_signoz.gen_ai.cost_cache_write"
SignozGenAITotalCost = "_signoz.gen_ai.total_cost"
)
var (

View File

@@ -4,7 +4,6 @@ import (
"bytes"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"gopkg.in/yaml.v3"
)
@@ -84,11 +83,11 @@ func buildProcessorConfig(rules []*LLMPricingRule) *LLMPricingRuleProcessorConfi
return &LLMPricingRuleProcessorConfig{
Attrs: LLMPricingRuleProcessorAttrs{
Model: telemetrytypes.GenAIRequestModel,
In: telemetrytypes.GenAIUsageInputTokens,
Out: telemetrytypes.GenAIUsageOutputTokens,
CacheRead: telemetrytypes.GenAIUsageCacheReadInputTokens,
CacheWrite: telemetrytypes.GenAIUsageCacheCreationInputTokens,
Model: GenAIRequestModel,
In: GenAIUsageInputTokens,
Out: GenAIUsageOutputTokens,
CacheRead: GenAIUsageCacheReadInputTokens,
CacheWrite: GenAIUsageCacheCreationInputTokens,
},
DefaultPricing: LLMPricingRuleProcessorDefaultPricing{
Rules: pricingRules,
@@ -98,7 +97,7 @@ func buildProcessorConfig(rules []*LLMPricingRule) *LLMPricingRuleProcessorConfi
Out: SignozGenAICostOutput,
CacheRead: SignozGenAICostCacheRead,
CacheWrite: SignozGenAICostCacheWrite,
Total: telemetrytypes.SignozGenAITotalCost,
Total: SignozGenAITotalCost,
},
}
}

View File

@@ -9,7 +9,6 @@ type QueryType struct {
var (
QueryTypeUnknown = QueryType{valuer.NewString("unknown")}
QueryTypeBuilder = QueryType{valuer.NewString("builder_query")}
QueryTypeBuilderAI = QueryType{valuer.NewString("builder_ai_query")}
QueryTypeFormula = QueryType{valuer.NewString("builder_formula")}
QueryTypeSubQuery = QueryType{valuer.NewString("builder_sub_query")}
QueryTypeJoin = QueryType{valuer.NewString("builder_join")}
@@ -22,7 +21,6 @@ var (
func (QueryType) Enum() []any {
return []any{
QueryTypeBuilder,
QueryTypeBuilderAI,
QueryTypeFormula,
// Not yet supported.
// QueryTypeSubQuery,

View File

@@ -62,13 +62,6 @@ type queryEnvelopeBuilder struct {
Spec builderQuerySpec `json:"spec" description:"The builder query specification."`
}
// queryEnvelopeBuilderAI is the OpenAPI schema for a builder_ai_query QueryEnvelope.
// The spec is always a traces builder query (the signal is implied by the type).
type queryEnvelopeBuilderAI struct {
Type QueryType `json:"type" required:"true" description:"The type of the query."`
Spec QueryBuilderQuery[TraceAggregation] `json:"spec" description:"The AI builder query specification."`
}
// queryEnvelopeFormula is the OpenAPI schema for a QueryEnvelope with type=builder_formula.
type queryEnvelopeFormula struct {
Type QueryType `json:"type" required:"true" description:"The type of the query."`
@@ -107,7 +100,6 @@ var _ jsonschema.OneOfExposer = QueryEnvelope{}
func (QueryEnvelope) JSONSchemaOneOf() []any {
return []any{
queryEnvelopeBuilder{},
queryEnvelopeBuilderAI{},
queryEnvelopeFormula{},
// queryEnvelopeJoin{}, // deferred — see commented queryEnvelopeJoin above
queryEnvelopeTraceOperator{},
@@ -128,7 +120,6 @@ func (QueryEnvelope) PrepareJSONSchema(s *jsonschema.Schema) error {
"propertyName": "type",
"mapping": map[string]string{
QueryTypeBuilder.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder",
QueryTypeBuilderAI.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI",
QueryTypeFormula.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula",
QueryTypeTraceOperator.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator",
QueryTypePromQL.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopePromQL",
@@ -159,16 +150,6 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
}
q.Spec = spec
case QueryTypeBuilderAI:
// A dedicated AI query is always a traces builder query; the signal is
// implied by the type, so pin it rather than requiring the caller to send it.
var spec QueryBuilderQuery[TraceAggregation]
if err := json.Unmarshal(shadow.Spec, &spec); err != nil {
return err
}
spec.Signal = telemetrytypes.SignalTraces
q.Spec = spec
case QueryTypeFormula:
var spec QueryBuilderFormula
if err := json.Unmarshal(shadow.Spec, &spec); err != nil {
@@ -213,7 +194,7 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
"unknown query type %q",
shadow.Type,
).WithAdditional(
"Valid query types are: builder_query, builder_ai_query, builder_sub_query, builder_formula, builder_join, builder_trace_operator, promql, clickhouse_sql",
"Valid query types are: builder_query, builder_sub_query, builder_formula, builder_join, builder_trace_operator, promql, clickhouse_sql",
).WithSuggestions(errors.NewValidReferences(errors.NounQueryTypes, QueryType{}.Enum()...))
}

View File

@@ -608,7 +608,7 @@ func (r *QueryRangeRequest) ValidateRequestScope() ([]ValidationOption, error) {
// Builder query names must be unique across the composite query.
queryNames := make(map[string]bool)
for _, envelope := range r.CompositeQuery.Queries {
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery || envelope.Type == QueryTypeBuilderAI {
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery {
name := envelope.GetQueryName()
if name != "" {
if queryNames[name] {
@@ -677,7 +677,7 @@ func (c *CompositeQuery) Validate(opts ...ValidationOption) error {
}
// Check name uniqueness for builder queries
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery || envelope.Type == QueryTypeBuilderAI {
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery {
name := envelope.GetQueryName()
if name != "" {
if queryNames[name] {
@@ -711,15 +711,6 @@ func validateQueryEnvelope(envelope QueryEnvelope, opts ...ValidationOption) err
"unknown query spec type",
)
}
case QueryTypeBuilderAI:
spec, ok := envelope.Spec.(QueryBuilderQuery[TraceAggregation])
if !ok {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid AI builder query spec",
)
}
return spec.Validate(opts...)
case QueryTypeFormula:
spec, ok := envelope.Spec.(QueryBuilderFormula)
if !ok {
@@ -789,7 +780,7 @@ func validateQueryEnvelope(envelope QueryEnvelope, opts ...ValidationOption) err
"unknown query type: %s",
envelope.Type,
).WithAdditional(
"Valid query types are: builder_query, builder_ai_query, builder_sub_query, builder_formula, builder_join, promql, clickhouse_sql, trace_operator",
"Valid query types are: builder_query, builder_sub_query, builder_formula, builder_join, promql, clickhouse_sql, trace_operator",
).WithSuggestions(errors.NewValidReferences(errors.NounQueryTypes, QueryType{}.Enum()...))
}
}

View File

@@ -0,0 +1,100 @@
package savedviewtypes
import (
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
// PanelType is the explore-page panel a saved view renders as. It is local
// to savedviewtypes (not the legacy pkg/query-service/model/v3.PanelType)
// following the same pattern ruletypes.PanelType uses alongside
// qbtypes.QueryEnvelope for alerts.
type PanelType struct {
valuer.String
}
var (
PanelTypeValue = PanelType{valuer.NewString("value")}
PanelTypeGraph = PanelType{valuer.NewString("graph")}
PanelTypeTable = PanelType{valuer.NewString("table")}
PanelTypeList = PanelType{valuer.NewString("list")}
PanelTypeTrace = PanelType{valuer.NewString("trace")}
)
// Enum implements jsonschema.Enum; returns the acceptable values for PanelType.
func (PanelType) Enum() []any {
return []any{
PanelTypeValue,
PanelTypeGraph,
PanelTypeTable,
PanelTypeList,
PanelTypeTrace,
}
}
func (p PanelType) Validate() error {
switch p {
case PanelTypeValue, PanelTypeGraph, PanelTypeTable, PanelTypeList, PanelTypeTrace:
return nil
default:
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "invalid panel type: %s", p.StringValue())
}
}
// QueryType records which query-builder mode (builder/ClickHouse SQL/PromQL)
// the view was last edited in -- it is a UI-tab selector, not a technical
// query-execution detail (that's qbtypes.QueryEnvelope.Type, which lives per
// query in Queries and can differ across entries). Local to savedviewtypes,
// mirroring ruletypes.QueryType.
type QueryType struct {
valuer.String
}
var (
QueryTypeBuilder = QueryType{valuer.NewString("builder")}
QueryTypeClickHouseSQL = QueryType{valuer.NewString("clickhouse_sql")}
QueryTypePromQL = QueryType{valuer.NewString("promql")}
)
// Enum implements jsonschema.Enum; returns the acceptable values for QueryType.
func (QueryType) Enum() []any {
return []any{
QueryTypeBuilder,
QueryTypeClickHouseSQL,
QueryTypePromQL,
}
}
func (q QueryType) Validate() error {
switch q {
case QueryTypeBuilder, QueryTypeClickHouseSQL, QueryTypePromQL:
return nil
default:
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "invalid query type: %s", q.StringValue())
}
}
// CompositeQuery is the query a saved view persists: a v5 query envelope
// list plus the panel/query type needed to render and edit it. Unlike the
// legacy v3.CompositeQuery it replaces here, this is v5-only by
// construction -- no builderQueries/chQueries/promQueries/unit/fillGaps.
// Saved views are always built from builder queries (never raw ClickHouse
// SQL or PromQL), so this covers every saved view.
type CompositeQuery struct {
PanelType PanelType `json:"panelType" required:"true"`
QueryType QueryType `json:"queryType" required:"true"`
Queries []qbtypes.QueryEnvelope `json:"queries" required:"true"`
}
func (q *CompositeQuery) Validate() error {
if err := q.PanelType.Validate(); err != nil {
return err
}
if err := q.QueryType.Validate(); err != nil {
return err
}
return (&qbtypes.CompositeQuery{Queries: q.Queries}).Validate()
}

View File

@@ -1,31 +1,170 @@
package savedviewtypes
import (
"encoding/json"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
type SavedView struct {
var (
ErrCodeSavedViewInvalidInput = errors.MustNewCode("saved_view_invalid_input")
ErrCodeSavedViewNotFound = errors.MustNewCode("saved_view_not_found")
)
type SourcePage struct {
valuer.String
}
var (
SourcePageTraces = SourcePage{valuer.NewString("traces")}
SourcePageLogs = SourcePage{valuer.NewString("logs")}
SourcePageMetrics = SourcePage{valuer.NewString("metrics")}
SourcePageMeter = SourcePage{valuer.NewString("meter")}
)
func (SourcePage) Enum() []any {
return []any{
SourcePageTraces,
SourcePageLogs,
SourcePageMetrics,
SourcePageMeter,
}
}
func (s SourcePage) Validate() error {
switch s {
case SourcePageTraces, SourcePageLogs, SourcePageMetrics, SourcePageMeter:
return nil
default:
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "invalid source page: %s", s.StringValue())
}
}
type GettableSavedView struct {
ID valuer.UUID `json:"id" required:"true"`
Name string `json:"name" required:"true"`
CreatedAt time.Time `json:"createdAt" required:"true"`
CreatedBy string `json:"createdBy" required:"true"`
UpdatedAt time.Time `json:"updatedAt" required:"true"`
UpdatedBy string `json:"updatedBy" required:"true"`
SourcePage SourcePage `json:"sourcePage" required:"true"`
CompositeQuery *CompositeQuery `json:"compositeQuery" required:"true"`
// ExtraData is JSON encoded data used by frontend to store additional data
ExtraData string `json:"extraData" required:"true"`
}
type PostableSavedView struct {
Name string `json:"name" required:"true"`
SourcePage SourcePage `json:"sourcePage" required:"true"`
CompositeQuery *CompositeQuery `json:"compositeQuery" required:"true"`
// ExtraData is JSON encoded data used by frontend to store additional data
ExtraData string `json:"extraData" required:"true"`
}
type UpdatableSavedView = PostableSavedView
func (p *PostableSavedView) Validate() error {
if err := p.SourcePage.Validate(); err != nil {
return err
}
if p.CompositeQuery == nil {
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "composite query is required")
}
return p.CompositeQuery.Validate()
}
type ListSavedViewsParams struct {
SourcePage SourcePage `query:"sourcePage"`
Name string `query:"name"`
}
func (p *ListSavedViewsParams) Validate() error {
if p.SourcePage.IsZero() {
return nil
}
return p.SourcePage.Validate()
}
// StorableSavedView has CompositeQuery stored JSON-encoded in Data.
type StorableSavedView struct {
bun.BaseModel `bun:"table:saved_views"`
types.Identifiable
types.TimeAuditable
types.UserAuditable
OrgID string `json:"orgId" bun:"org_id,notnull"`
Name string `json:"name" bun:"name,type:text,notnull"`
Category string `json:"category" bun:"category,type:text,notnull"`
SourcePage string `json:"sourcePage" bun:"source_page,type:text,notnull"`
Tags string `json:"tags" bun:"tags,type:text"`
Data string `json:"data" bun:"data,type:text,notnull"`
ExtraData string `json:"extraData" bun:"extra_data,type:text"`
OrgID string `json:"orgId" bun:"org_id,notnull"`
Name string `json:"name" bun:"name,type:text,notnull"`
SourcePage SourcePage `json:"sourcePage" bun:"source_page,type:text,notnull"`
Data string `json:"data" bun:"data,type:text,notnull"`
ExtraData string `json:"extraData" bun:"extra_data,type:text"`
// TODO: deprecated, remove it
Category string `json:"category" bun:"category,type:text,notnull"`
// TODO: deprecated, remove it
Tags string `json:"tags" bun:"tags,type:text"`
}
func NewStatsFromSavedViews(savedViews []*SavedView) map[string]any {
func NewStorableSavedView(orgID string, createdBy string, updatedBy string, view PostableSavedView) (*StorableSavedView, error) {
data, err := json.Marshal(view.CompositeQuery)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling explorer query data")
}
now := time.Now()
return &StorableSavedView{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
TimeAuditable: types.TimeAuditable{CreatedAt: now, UpdatedAt: now},
UserAuditable: types.UserAuditable{CreatedBy: createdBy, UpdatedBy: updatedBy},
OrgID: orgID,
Name: view.Name,
SourcePage: view.SourcePage,
Data: string(data),
ExtraData: view.ExtraData,
}, nil
}
func NewGettableSavedViewFromStorable(view *StorableSavedView) (*GettableSavedView, error) {
var compositeQuery CompositeQuery
if err := json.Unmarshal([]byte(view.Data), &compositeQuery); err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in unmarshalling explorer query data")
}
return &GettableSavedView{
ID: view.ID,
Name: view.Name,
CreatedAt: view.CreatedAt,
CreatedBy: view.CreatedBy,
UpdatedAt: view.UpdatedAt,
UpdatedBy: view.UpdatedBy,
SourcePage: view.SourcePage,
CompositeQuery: &compositeQuery,
ExtraData: view.ExtraData,
}, nil
}
func NewGettableSavedViewsFromStorable(views []*StorableSavedView) ([]*GettableSavedView, error) {
out := make([]*GettableSavedView, 0, len(views))
for _, view := range views {
gettable, err := NewGettableSavedViewFromStorable(view)
if err != nil {
return nil, err
}
out = append(out, gettable)
}
return out, nil
}
func NewStatsFromSavedViews(savedViews []*StorableSavedView) map[string]any {
stats := make(map[string]any)
for _, savedView := range savedViews {
key := "savedview.source." + strings.ToLower(string(savedView.SourcePage)) + ".count"
key := "savedview.source." + strings.ToLower(savedView.SourcePage.StringValue()) + ".count"
if _, ok := stats[key]; !ok {
stats[key] = int64(1)
} else {

View File

@@ -76,7 +76,6 @@ var (
"log": FieldContextLog,
"metric": FieldContextMetric,
"tracefield": FieldContextTrace,
"trace": FieldContextTrace,
}
)

View File

@@ -1,43 +0,0 @@
package telemetrytypes
// OpenTelemetry gen_ai semantic-convention attribute keys. Single source of truth
// shared by the AI query builder and the LLM pricing pipeline.
const (
GenAIRequestModel = "gen_ai.request.model"
GenAIToolName = "gen_ai.tool.name"
GenAIAgentName = "gen_ai.agent.name"
GenAIProviderName = "gen_ai.provider.name"
GenAIUsageInputTokens = "gen_ai.usage.input_tokens"
GenAIUsageOutputTokens = "gen_ai.usage.output_tokens"
GenAIUsageCacheReadInputTokens = "gen_ai.usage.cache_read.input_tokens"
GenAIUsageCacheCreationInputTokens = "gen_ai.usage.cache_creation.input_tokens"
GenAIInputMessages = "gen_ai.input.messages"
GenAIOutputMessages = "gen_ai.output.messages"
// SignozGenAITotalCost is not OTel semconv: it is the per-span total cost the
// SigNoz LLM pricing processor computes and attaches (see llmpricingruletypes).
SignozGenAITotalCost = "_signoz.gen_ai.total_cost"
)
// GenAIFieldDefinitions are the gen_ai semantic-convention span attributes the AI
// query builder relies on. They are surfaced by the metadata store for trace
// queries regardless of whether they have been ingested yet, so the AI gate/columns
// resolve on a fresh install (mirrors intrinsic metric keys). String keys are the
// gate; the usage keys are numeric.
var GenAIFieldDefinitions = map[string]TelemetryFieldKey{
GenAIRequestModel: {Name: GenAIRequestModel, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIToolName: {Name: GenAIToolName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIAgentName: {Name: GenAIAgentName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIProviderName: {Name: GenAIProviderName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIUsageInputTokens: {Name: GenAIUsageInputTokens, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},
GenAIUsageOutputTokens: {Name: GenAIUsageOutputTokens, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},
GenAIUsageCacheReadInputTokens: {Name: GenAIUsageCacheReadInputTokens, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},
GenAIUsageCacheCreationInputTokens: {Name: GenAIUsageCacheCreationInputTokens, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},
SignozGenAITotalCost: {Name: SignozGenAITotalCost, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},
GenAIInputMessages: {Name: GenAIInputMessages, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIOutputMessages: {Name: GenAIOutputMessages, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
}

View File

@@ -10,7 +10,6 @@ const wildcardSelector = "*"
var telemetryGrantQueryTypes = map[string]bool{
"builder_query": true,
"builder_ai_query": true,
"builder_sub_query": true,
"promql": false,
"clickhouse_sql": false,

View File

@@ -78,15 +78,12 @@ class BuilderQuery:
signal: str
name: str = "A"
source: str | None = None
query_type: str = "builder_query"
limit: int | None = None
offset: int | None = None
filter_expression: str | None = None
having_expression: str | None = None
select_fields: list[TelemetryFieldKey] | None = None
order: list[OrderBy] | None = None
aggregations: list[Aggregation | MetricAggregation] | None = None
group_by: list[TelemetryFieldKey] | None = None
step_interval: int | None = None
def to_dict(self) -> dict:
@@ -102,20 +99,16 @@ class BuilderQuery:
spec["offset"] = self.offset
if self.filter_expression:
spec["filter"] = {"expression": self.filter_expression}
if self.having_expression:
spec["having"] = {"expression": self.having_expression}
if self.select_fields:
spec["selectFields"] = [f.to_dict() for f in self.select_fields]
if self.order:
spec["order"] = [o.to_dict() if hasattr(o, "to_dict") else o for o in self.order]
if self.aggregations:
spec["aggregations"] = [agg.to_dict() if hasattr(agg, "to_dict") else agg for agg in self.aggregations]
if self.group_by:
spec["groupBy"] = [k.to_dict() for k in self.group_by]
if self.step_interval is not None:
spec["stepInterval"] = self.step_interval
return {"type": self.query_type, "spec": spec}
return {"type": "builder_query", "spec": spec}
@dataclass

View File

@@ -1,843 +0,0 @@
"""
Integration tests for query_type="builder_ai_query" over the traces signal.
Data shape (generic OTel gen_ai semantic conventions):
- a root span (no gen_ai attributes)
- an LLM span carrying gen_ai.request.model (str) and numeric usage attributes
(gen_ai.usage.input_tokens / output_tokens / cost) plus gen_ai.user.id
Each test tags its spans with a unique service.name and filters on it, so tests do
not interfere with each other's data.
"""
import json
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.querier import (
BuilderQuery,
OrderBy,
RequestType,
TelemetryFieldKey,
make_query_request,
)
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
def _ai_trace(
*,
now: datetime,
service: str,
user: str,
in_tokens: int | None,
out_tokens: int,
cost: float,
model: str = "gpt-4o-mini",
llm_duration_s: float = 1.0,
error: bool = False,
environment: str = "production",
) -> list[Traces]:
"""A minimal AI trace: root span + one LLM span with gen_ai attributes.
in_tokens=None omits the input-tokens attribute entirely (not zero)."""
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
llm_id = TraceIdGenerator.span_id()
resources = {"service.name": service, "deployment.environment": environment}
root = Traces(
timestamp=now - timedelta(seconds=5),
duration=timedelta(seconds=llm_duration_s + 0.1),
trace_id=trace_id,
span_id=root_id,
parent_span_id="",
name="POST /api/chat",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"http.request.method": "POST"},
)
attributes = {
"gen_ai.request.model": model,
"gen_ai.system": "openai",
"gen_ai.user.id": user,
# numeric values land in attributes_number
"gen_ai.usage.output_tokens": out_tokens,
"_signoz.gen_ai.total_cost": cost,
}
if in_tokens is not None:
attributes["gen_ai.usage.input_tokens"] = in_tokens
llm = Traces(
timestamp=now - timedelta(seconds=4),
duration=timedelta(seconds=llm_duration_s),
trace_id=trace_id,
span_id=llm_id,
parent_span_id=root_id,
name="chat gpt-4o-mini",
kind=TracesKind.SPAN_KIND_CLIENT,
status_code=(TracesStatusCode.STATUS_CODE_ERROR if error else TracesStatusCode.STATUS_CODE_OK),
resources=resources,
attributes=attributes,
)
return [root, llm]
def _non_ai_trace(*, now: datetime, service: str) -> list[Traces]:
"""A plain HTTP trace with no gen_ai attributes; must be excluded by the AI gate."""
trace_id = TraceIdGenerator.trace_id()
span_id = TraceIdGenerator.span_id()
return [
Traces(
timestamp=now - timedelta(seconds=4),
duration=timedelta(seconds=1),
trace_id=trace_id,
span_id=span_id,
parent_span_id="",
name="GET /health",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources={"service.name": service},
attributes={"http.request.method": "GET"},
)
]
def _window_ms(now: datetime) -> tuple[int, int]:
start_ms = int((now - timedelta(minutes=10)).timestamp() * 1000)
end_ms = int((now + timedelta(minutes=1)).timestamp() * 1000)
return start_ms, end_ms
def test_ai_list_excludes_non_ai(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
Trace-list panel (requestType="trace"): returns AI traces and excludes the
non-AI trace. Asserts on the raw response payload to stay agnostic to the exact
row schema.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-list"
ai = _ai_trace(now=now, service=service, user="alice", in_tokens=100, out_tokens=20, cost=0.5)
non_ai = _non_ai_trace(now=now, service=service)
ai_trace_id = ai[0].trace_id
non_ai_trace_id = non_ai[0].trace_id
insert_traces(ai + non_ai)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
body = json.dumps(response.json())
assert ai_trace_id in body, f"expected AI trace {ai_trace_id} in list response"
assert non_ai_trace_id not in body, f"non-AI trace {non_ai_trace_id} should be excluded by the gate"
def _ai_trace_mixed_spans(*, now: datetime, service: str, user: str) -> list[Traces]:
"""
Root + one LLM span + one tool span + one agent span. The gate matches all three
child spans, but only the LLM span carries gen_ai.request.model.
"""
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service, "deployment.environment": "production"}
def _span(name, kind, attributes, offset_s):
return Traces(
timestamp=now - timedelta(seconds=offset_s),
duration=timedelta(seconds=0.5),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name=name,
kind=kind,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes=attributes,
)
root = Traces(
timestamp=now - timedelta(seconds=5),
duration=timedelta(seconds=4),
trace_id=trace_id,
span_id=root_id,
parent_span_id="",
name="POST /api/chat",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"http.request.method": "POST"},
)
llm = _span(
"chat gpt-4o-mini",
TracesKind.SPAN_KIND_CLIENT,
{
"gen_ai.request.model": "gpt-4o-mini",
"gen_ai.system": "openai",
"gen_ai.user.id": user,
"gen_ai.usage.input_tokens": 100,
"gen_ai.usage.output_tokens": 20,
},
4,
)
tool = _span(
"execute_tool",
TracesKind.SPAN_KIND_INTERNAL,
{
"gen_ai.tool.name": "get_weather",
"gen_ai.tool.type": "function",
},
3,
)
agent = _span(
"agent.step",
TracesKind.SPAN_KIND_INTERNAL,
{
"gen_ai.agent.name": "chat-agent",
},
2,
)
return [root, llm, tool, agent]
def test_ai_list_having_aggregate_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
Aggregate filter written in the SAME filter box: the span-level predicate narrows
to the service, the trace-level `output_tokens > 100` keeps the large-token
trace and drops the small one (split internally into WHERE + HAVING). Both
spellings of a trace-level aggregate — bare and `trace.` — behave identically
(unit tests pin them to byte-identical SQL; this covers the wiring once
end-to-end). An output-only aggregate is rejected under either spelling.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-having"
small = _ai_trace(now=now, service=service, user="alice", in_tokens=10, out_tokens=20, cost=0.1)
large = _ai_trace(now=now, service=service, user="bob", in_tokens=10, out_tokens=500, cost=0.2)
small_id = small[0].trace_id
large_id = large[0].trace_id
insert_traces(small + large)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
for spelling in ("output_tokens", "trace.output_tokens"):
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}' AND {spelling} > 100",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, f"{spelling}: {response.text}"
body = json.dumps(response.json())
assert large_id in body, f"{spelling}: trace with 500 out-tokens should pass > 100"
assert small_id not in body, f"{spelling}: trace with 20 out-tokens should be filtered out by HAVING"
# output-only aggregate gets the targeted rejection.
bad = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression="trace.span_count > 3",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [bad.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
assert "cannot be used" in response.text
def test_ai_list_order_limit_offset(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Trace list honors order by (aggregate column) + limit + offset (pagination)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-order"
traces: list[Traces] = []
for out in (100, 200, 300, 400, 500):
traces += _ai_trace(now=now, service=service, user="u", in_tokens=10, out_tokens=out, cost=0.1)
insert_traces(traces)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
def page(offset: int) -> list[int]:
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
order=[OrderBy(key=TelemetryFieldKey(name="output_tokens"), direction="desc")],
limit=2,
offset=offset,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert resp.status_code == HTTPStatus.OK, resp.text
rows = resp.json()["data"]["data"]["results"][0]["rows"]
return [int(r["data"]["output_tokens"]) for r in rows]
assert page(0) == [500, 400], "first page: highest output_tokens, desc"
assert page(2) == [300, 200], "second page (offset 2): next two, desc"
def test_ai_span_list_limit(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Span list honors limit (delegated raw path): 6 gen_ai spans available, capped to 4."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-spanlimit"
insert_traces(_ai_trace_mixed_spans(now=now, service=service, user="a") + _ai_trace_mixed_spans(now=now, service=service, user="b"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
limit=4,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.RAW)
assert resp.status_code == HTTPStatus.OK, resp.text
rows = resp.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 4, f"limit should cap at 4 (6 gen_ai spans available), got {len(rows)}"
def test_ai_span_list_excludes_non_gen_ai_spans(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
Span list (requestType=raw): returns only the gen_ai spans (LLM/tool/agent); the
root span of the same trace (no gen_ai attributes) is excluded by the span-level gate.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-spanlist"
insert_traces(_ai_trace_mixed_spans(now=now, service=service, user="alice"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
select_fields=[TelemetryFieldKey(name="name", field_context="span")],
limit=50,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.RAW)
assert response.status_code == HTTPStatus.OK, response.text
rows = response.json()["data"]["data"]["results"][0]["rows"]
names = sorted(r["data"]["name"] for r in rows)
assert names == ["agent.step", "chat gpt-4o-mini", "execute_tool"], names
assert "POST /api/chat" not in names # root span excluded
def test_ai_list_having_or_aggregates(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
Two trace-level aggregates OR-ed within the filter box (regression guard for OR-group
whitespace handling): output_tokens > 100 OR input_tokens > 1000 keeps only the
large-output trace (input_tokens is 10 for both, so that branch never matches).
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-having-or"
small = _ai_trace(now=now, service=service, user="a", in_tokens=10, out_tokens=20, cost=0.1)
large = _ai_trace(now=now, service=service, user="b", in_tokens=10, out_tokens=500, cost=0.2)
small_id, large_id = small[0].trace_id, large[0].trace_id
insert_traces(small + large)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}' AND (output_tokens > 100 OR input_tokens > 1000)",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
body = json.dumps(response.json())
assert large_id in body
assert small_id not in body
def test_ai_list_resource_filter_isolates_by_fingerprint(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
A resource attribute in the filter is pulled into the __resource_filter fingerprint
CTE (see maybeAttachResourceFilter). Two traces on the same service but different
deployment.environment: `resource.deployment.environment = 'production'` must keep
the production trace and drop the staging one — the fingerprint prune isolates by
the resource, not by any span attribute.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-resfilter"
prod = _ai_trace(now=now, service=service, user="a", in_tokens=10, out_tokens=20, cost=0.1, environment="production")
stag = _ai_trace(now=now, service=service, user="b", in_tokens=10, out_tokens=20, cost=0.1, environment="staging")
prod_id, stag_id = prod[0].trace_id, stag[0].trace_id
insert_traces(prod + stag)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=(f"resource.service.name = '{service}' AND resource.deployment.environment = 'production'"),
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
body = json.dumps(response.json())
assert prod_id in body, "production trace should match the resource filter"
assert stag_id not in body, "staging trace should be excluded by the resource fingerprint prune"
def test_ai_list_rejects_aggregate_or_span_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
Aggregate (HAVING) columns may not be OR-ed with span-level keys in the trace
list; a span-OR-span filter is fine.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-orfilter"
# seed a trace so service.name resolves as a known key in this window (resource
# keys are discovered from ingested data).
insert_traces(_ai_trace(now=now, service=service, user="a", in_tokens=10, out_tokens=20, cost=0.1))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
# aggregate OR span -> rejected
bad = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
limit=10,
filter_expression=f"output_tokens > 1000 OR service.name = '{service}'",
)
response = make_query_request(signoz, token, start_ms, end_ms, [bad.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
assert "cannot be combined" in response.text
# span OR span -> accepted (result content doesn't matter; just not an error)
ok = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
limit=10,
filter_expression=f"service.name = '{service}' OR has_error = true",
)
response = make_query_request(signoz, token, start_ms, end_ms, [ok.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
def test_ai_list_nested_group_span_or_and_aggregate(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
A complex filter that mixes all three routing paths in one expression:
service.name = X AND (has_error = true OR gen_ai.request.model = 'gpt-4o') AND total_tokens > 100
The nested (span OR span) group must not flatten (precedence), the span predicates
go to WHERE as a trace-existence check, and the new `total_tokens` aggregate goes to
HAVING. Three traces isolate each discriminator:
- t_ok: gpt-4o, out=500 -> OR matches (model) AND total_tokens>100 -> IN
- t_or_miss: gpt-4o-mini, out=500 -> OR fails (no error, wrong model) -> OUT
- t_agg_miss: gpt-4o, out=20 -> OR matches but total_tokens<=100 -> OUT
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-nested"
t_ok = _ai_trace(now=now, service=service, user="a", model="gpt-4o", in_tokens=10, out_tokens=500, cost=0.1)
t_or_miss = _ai_trace(now=now, service=service, user="b", model="gpt-4o-mini", in_tokens=10, out_tokens=500, cost=0.1)
t_agg_miss = _ai_trace(now=now, service=service, user="c", model="gpt-4o", in_tokens=10, out_tokens=20, cost=0.1)
insert_traces(t_ok + t_or_miss + t_agg_miss)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=(f"service.name = '{service}' AND (has_error = true OR gen_ai.request.model = 'gpt-4o') AND total_tokens > 100"),
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
body = json.dumps(response.json())
assert t_ok[0].trace_id in body
assert t_or_miss[0].trace_id not in body, "nested (span OR span) group must exclude the wrong-model, no-error trace"
assert t_agg_miss[0].trace_id not in body, "HAVING total_tokens > 100 must exclude the low-token trace"
def test_ai_list_rejects_unknown_aggregate_key(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""A trace-level filter on an unknown aggregate name is rejected, not silently run."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
limit=10,
filter_expression="trace.bogus_tokens > 1",
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
def test_ai_list_rejects_order_by_span_attribute(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""Only gen_ai-scoped aggregates are orderable; ordering by a span/resource key errors."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
limit=5,
order=[OrderBy(key=TelemetryFieldKey(name="service.name"), direction="asc")],
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
assert "order key" in response.text
def test_ai_list_total_tokens_output_only(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
A trace whose LLM span carries only output tokens (no input-tokens attribute at
all) must still total: total_tokens is coalesce(sum(in),0)+coalesce(sum(out),0),
since sum over an absent attribute is NULL and NULL + n = NULL in ClickHouse.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-total-coalesce"
insert_traces(_ai_trace(now=now, service=service, user="a", in_tokens=None, out_tokens=300, cost=0.1))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 1, f"expected one trace, got: {rows}"
data = rows[0]["data"]
assert data["input_tokens"] is None, data # attribute absent -> NULL, not 0
assert data["output_tokens"] == 300, data
assert data["total_tokens"] == 300, f"total must coalesce the missing input side: {data}"
def test_ai_list_variable_in_aggregate_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A query variable in a trace-level condition is substituted into the HAVING."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-having-var"
small = _ai_trace(now=now, service=service, user="a", in_tokens=10, out_tokens=20, cost=0.1)
large = _ai_trace(now=now, service=service, user="b", in_tokens=10, out_tokens=500, cost=0.2)
small_id, large_id = small[0].trace_id, large[0].trace_id
insert_traces(small + large)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}' AND trace.output_tokens > $threshold",
limit=10,
)
response = make_query_request(
signoz,
token,
start_ms,
end_ms,
[query.to_dict()],
request_type="trace",
variables={"threshold": {"type": "custom", "value": 100}},
)
assert response.status_code == HTTPStatus.OK, response.text
body = json.dumps(response.json())
assert large_id in body
assert small_id not in body
def _ai_trace_two_llm(*, now: datetime, service: str) -> list[Traces]:
"""Root + two LLM spans at different times, each with distinct input/output messages."""
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service}
def _llm(offset_s: float, prompt: str, answer: str) -> Traces:
return Traces(
timestamp=now - timedelta(seconds=offset_s),
duration=timedelta(seconds=1),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="chat",
kind=TracesKind.SPAN_KIND_CLIENT,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={
"gen_ai.request.model": "gpt-4o-mini",
"gen_ai.input.messages": prompt,
"gen_ai.output.messages": answer,
},
)
root = Traces(
timestamp=now - timedelta(seconds=5),
duration=timedelta(seconds=4),
trace_id=trace_id,
span_id=root_id,
parent_span_id="",
name="POST /api/chat",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"http.request.method": "POST"},
)
# earlier call is the "first" (its input is the prompt), later call is the "last"
# (its output is the final answer).
first = _llm(4, "first prompt", "first answer")
last = _llm(2, "second prompt", "second answer")
return [root, first, last]
def test_ai_list_messages_first_input_last_output(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
`input` is the FIRST LLM span's prompt (argMin over timestamp) and `output` is the
LAST LLM span's answer (argMax) — the question -> final-answer preview.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-messages"
insert_traces(_ai_trace_two_llm(now=now, service=service))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
limit=10,
filter_expression=f"service.name = '{service}'",
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 1, f"expected one trace, got: {rows}"
data = rows[0]["data"]
assert data["input"] == "first prompt", f"input should be the earliest call's prompt: {data}"
assert data["output"] == "second answer", f"output should be the latest call's answer: {data}"
def _ai_trace_for_metrics(*, now: datetime, service: str) -> list[Traces]:
"""
Root + one errored LLM span (tokens/cost) + three tool spans (two 'get_weather',
one 'get_time') + one agent span, so the derived per-trace metrics have distinct
expected values. The agent span is in the gen_ai gate but carries no request.model,
so it must NOT count toward llm_call_count (only span_count / last_activity_time).
"""
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service}
def _tool(name: str, offset_s: float) -> Traces:
return Traces(
timestamp=now - timedelta(seconds=offset_s),
duration=timedelta(seconds=0.2),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="execute_tool",
kind=TracesKind.SPAN_KIND_INTERNAL,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"gen_ai.tool.name": name, "gen_ai.tool.type": "function"},
)
root = Traces(
timestamp=now - timedelta(seconds=5),
duration=timedelta(seconds=4),
trace_id=trace_id,
span_id=root_id,
parent_span_id="",
name="POST /api/chat",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"http.request.method": "POST"},
)
llm = Traces(
timestamp=now - timedelta(seconds=4),
duration=timedelta(seconds=2),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="chat gpt-4o-mini",
kind=TracesKind.SPAN_KIND_CLIENT,
status_code=TracesStatusCode.STATUS_CODE_ERROR, # -> has_error, drives error_count
resources=resources,
attributes={
"gen_ai.request.model": "gpt-4o-mini",
"gen_ai.usage.input_tokens": 100,
"gen_ai.usage.output_tokens": 20,
"_signoz.gen_ai.total_cost": 0.5,
},
)
agent = Traces(
timestamp=now - timedelta(seconds=1),
duration=timedelta(seconds=0.5),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="agent.step",
kind=TracesKind.SPAN_KIND_INTERNAL,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"gen_ai.agent.name": "chat-agent"},
)
return [root, llm, _tool("get_weather", 3), _tool("get_weather", 2.5), _tool("get_time", 2), agent]
def test_ai_list_enrichment_values(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""
End-to-end values of the derived per-trace columns (only integration can check that
ClickHouse computes uniqIf / sum+sum / countIf(predicate) correctly, not just that
the SQL is shaped right). One trace: root + 1 errored LLM + 3 tool spans
(get_weather x2, get_time x1) + 1 agent span. The tool and agent spans are in the
gen_ai gate but carry no request.model, so llm_call_count stays 1 while span_count
counts them all.
"""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-metrics"
insert_traces(_ai_trace_for_metrics(now=now, service=service))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = _window_ms(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
limit=10,
filter_expression=f"service.name = '{service}'",
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type="trace")
assert response.status_code == HTTPStatus.OK, response.text
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 1, f"expected one trace, got: {rows}"
data = rows[0]["data"]
assert data["span_count"] == 6, data # root + llm + 3 tools + agent
assert data["llm_call_count"] == 1, data # only the request.model span, not tool/agent
assert data["tool_call_count"] == 3, data # all three tool spans
assert data["distinct_tool_count"] == 2, data # get_weather, get_time
assert data["input_tokens"] == 100, data
assert data["output_tokens"] == 20, data
assert data["total_tokens"] == 120, data # input + output
assert data["estimated_total_cost"] == pytest.approx(0.5), data
assert data["error_count"] == 1, data # the errored LLM span
assert data["max_llm_duration_nano"] > 0, data # scoped max over LLM spans