Compare commits

...

30 Commits

Author SHA1 Message Date
nityanandagohain
093f9b41c4 fix: minor cleanup 2026-07-22 16:22:19 +05:30
nityanandagohain
596e128005 fix: updated openapi 2026-07-22 15:26:52 +05:30
nityanandagohain
2e1da92367 fix: remove source and change to builder ai query 2026-07-22 15:24:43 +05:30
nityanandagohain
1e8d72e8fa Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-21 17:08:34 +05:30
Vikrant Gupta
253ca7dd7e fix(session): validate ref and callback state against global allowed_origins (#12172)
* fix(session): use global external_url instead of ref param for SSO state

The sessions/context endpoint no longer reads the client-controlled ref
query param to build the SSO state and callback URLs. Each callback
authn provider now derives the site URL from the server-configured
global external_url, and siteURL is removed from the CallbackAuthN and
session Module interfaces.

* fix(session): validate ref and callback state against global allowed_origins

Instead of deriving SSO urls from the global external_url, keep the ref
roundtrip and validate its origin against the new optional
global.allowed_origins config. The callback state url is re-validated
before tokens are attached to it, closing the forged RelayState/state
exfiltration path. When allowed_origins is not configured, redirect
targets are not validated, preserving existing installs.

* fix(session): scope ref origin validation to sso auth domains

Move the allowed_origins check from the sessions/context handler to
getOrgSessionContext, right before the SSO login URL is built. A
disallowed ref no longer fails the whole request; only orgs with an
SSO-enabled auth domain get a per-org warning with password fallback.
2026-07-21 10:28:49 +00:00
Ashwin Bhatkal
cc45c1bef1 feat(dashboard-v2): share a dashboard link with the current variable selection (#12198)
* feat(share): support a page-specific extra option in the share dialog

* feat(dashboard-v2): share a dashboard link with the current variable selection
2026-07-21 07:08:20 +00:00
Abhi kumar
e70b3a0b52 chore: pr review fixes for 12137 (#12142) 2026-07-21 06:43:10 +00:00
Abhi kumar
f514af469e fix(dashboards-v2): panel editor Save/dirty, refresh retention, dropdown clipping & time-window fixes (#12146)
* fix(dashboards-v2): carry active time window across panel editor navigation

Opening, creating, or leaving the V2 panel editor now preserves the selected time
range (relative or custom) via URL params derived from Redux global time, so a
custom range picked in the editor isn't reset to the dashboard default on return.

Adds timeParamsFromGlobalTime + useTimeSearchParams; useOpenPanelEditor now takes an
options object ({ handoffState, search }) and appends the time params, and
useCreatePanel routes through it.

* fix(dashboards-v2): sync panel editor preview to the staged query on browser back/forward

The query builder reverts both currentQuery and stagedQuery via initQueryBuilderData on a
URL re-stage (chiefly browser Back/Forward), but the preview kept the last Run's result.
Commit the staged query into the draft whenever it re-stages outside an explicit Run, so the
preview follows. Live edits touch only currentQuery, so they still wait for Run; commitQuery
no-ops when unchanged.

* fix(dashboards-v2): stop query-builder dropdowns being clipped in the panel editor

The panel editor renders the query builder inside an overflow:hidden resizable pane, so
antd Select popups (group-by, order-by, having, metric name, aggregator, units) were cut
off. Make the shared QB filters defer to an ancestor ConfigProvider's popup container
(falling back to trigger.parentNode) via useSelectPopupContainer, and wrap the editor's
builder in a ConfigProvider that portals popups to document.body. Scoped to the editor
host, so the View modal keeps its own container and other surfaces are unchanged.

* fix(dashboards-v2): anchor panel editor dirty check to saved panel; retain query edits on refresh

The editor re-derived its dirty baseline from the incoming panel prop, which
is seeded from transient state — a stale URL compositeQuery after a refresh or
the View-mode handoff spec — instead of the persisted panel. So the draft was
compared against an already-edited baseline: after a refresh the panel showed
the saved query yet read dirty (inverted), and a View->Edit query change read
clean with Save disabled.

- Thread a savedPanel baseline (existingPanel) through PanelEditorPage ->
  PanelEditorContainer -> usePanelEditSession, distinct from the seed panel.
- usePanelEditorDraft compares isSpecDirty against savedPanel; the draft still
  seeds from the (possibly handed-off) panel.
- usePanelEditorQuerySync computes isQueryDirty as a V5-envelope comparison of
  the live query against the saved queries, routing both sides through the same
  fromPerses->toPerses round-trip so builder-added defaults absent from an older
  stored query never read an untouched panel as modified. Replaces the racy
  captured-first-stagedQuery baseline.
- Drop the mount forceReset on useShareBuilderUrl (both its reasons are now
  moot) so a refresh / browser Back-Forward hydrates the edited query from the
  URL and the staged-query effect syncs it into the draft — query edits survive
  a refresh instead of reverting to the saved query.

Add real-QueryBuilderProvider integration tests covering untouched-not-dirty
(incl. an older/minimal stored query), refresh retention, and handoff-dirty;
update the draft + query-sync unit tests.
2026-07-21 06:40:43 +00:00
Vikrant Gupta
806853798d feat(authz): add support for telemetry resource provisioning (#12168)
* feat(authz): provision telemetry roles with plaintext selectors and hashed tuples

Accept a user-facing telemetry selector string (<query_type>/<key>/<value>
with trailing wildcards), validate and canonicalize it, store it as-is in the
role JSON record, and hash it only at the OpenFGA boundary in Object(). Grant
and check both flow through Object() so the hashes match; the plaintext record
stays the readable source of truth for display and recreation.

Because the hash is one-way, role Update diffs at the tuple level via
DiffTuples instead of reconstructing transaction groups from stored tuples.

* refactor(authz): rename telemetry selector helpers and unexport hash

Rename grant_selector.go to selector.go, CanonicalizeTelemetryGrantSelector to
NewTelemetryGrantSelector, and CanonicalTelemetryGrantKey to NewTelemetryGrantKey.
Unexport telemetrySelectorHash since Object() is its only caller.

* fix(authz): expand telemetry ladder in the permission check API

The check API only probed the exact selector plus the full wildcard, so a
scoped grant like builder_query/service.name/* reported a concrete query as
unauthorized even though enforcement allowed it. Relocate the grant selector
ladder to telemetrytypes as NewTelemetryGrantSelectors so both enforcement and
the check API share it, and canonicalize plus fan out the ladder for telemetry
check transactions. Non-telemetry resources keep the exact-plus-wildcard probe.

* refactor(authz): move newCheckSelectors to the bottom of tuple.go

* fix(authz): reject key-scoped selectors for promql and clickhouse_sql

Only builder_query and builder_sub_query extract key-scoped selectors on the
check side, so a grant like clickhouse_sql/service.name/signoz could never
match anything. Track key-scope support per query type and reject the
<query_type>/<key>/<value> form for query types that only emit <query_type>/*.

* test(authz): use a valid promql selector in the check API test

The promql/service.name/service-a case became invalid input after key-scoped
promql/clickhouse_sql selectors were rejected, so the check endpoint returned
400 instead of 200. Use promql/* to keep exercising the different-query-type
denial with a valid selector.

* feat(authz): allow signoz.workspace.key.id as a telemetry grant key

Add signoz.workspace.key.id to the telemetry grant key allowlist so roles can
scope telemetry access by ingestion key, alongside service.name. Grant
validation and check-side extraction both pick it up through the shared
NewTelemetryGrantKey path; bare and resource.-prefixed spellings fold to the
same canonical key.

* Revert "feat(authz): allow signoz.workspace.key.id as a telemetry grant key"

This reverts commit c50d122b54.

* feat(authz): scope telemetry grants by signoz.workspace.key.id

Make signoz.workspace.key.id the sole telemetry grant key instead of
service.name, so roles scope telemetry access by ingestion key. The allowlist
is the only production change; grant validation and check-side extraction are
name-agnostic. Update the querierauthz integration suite and unit tests to the
new key.

* test(authz): update query_range_resources extractor tests to signoz.workspace.key.id

The grant-key switch left this extractor test filtering on and expecting
builder_query/service.name/... IDs, which now fall back to builder_query/*.
Rename the filters and expectations to signoz.workspace.key.id.
2026-07-21 06:13:13 +00:00
Jugal Kishore
f002b29685 fix(onboarding): list Temporal Cloud Metrics under metrics, not APM/Traces (#12203)
Temporal Cloud Metrics scrapes an OpenMetrics endpoint but only showed
under APM/Traces, bundled with the Go and TypeScript SDKs. Split it into
a standalone metrics card and trimmed the Temporal APM card to Go and
TypeScript.

Closes SigNoz/growth-pod#1122
2026-07-21 06:08:36 +00:00
Vinicius Lourenço
a5d4ef4498 fix(uplot): missing destroy uplot on unmount (#12182)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
2026-07-21 03:02:32 +00:00
Tushar Vats
967289ebc6 fix(metrics): resolve any filter key context instead of erroring (#12188)
* fix(metrics): resolve any filter key context instead of erroring

The metrics condition builder hard-errored ("key <name> not found") whenever a
filter key was absent from the metadata keys. This hit the full-text search
column (metric_name in the Metrics Explorer, labels in query_range) and every
explicit label context (resource./scope./attribute.), so a bare/partial filter
token 400'd the Metrics Explorer treemap and metric queries.

Metrics keeps every label in the `labels` JSON with no per-context storage, so
FieldFor already maps any key on its own (intrinsics incl. metric_name to their
column, every label context to a labels JSON extract). Use the key directly in
the len(keys)==0 branch so filters resolve and run instead of 400ing - matching
group-by, which already collapses all label contexts to labels.

Update the label-context test to expect the collapse (resource./scope. now
resolve like bare/attribute.), add unit coverage for the branch, and add a
queriermetrics test that a free-text filter no longer errors.

* fix(metrics): resolve free-text and unregistered filter keys instead of erroring

The metrics condition builder hard-errored ("key <name> not found") whenever a
filter key was absent from the metadata keys: the full-text search column
(metric_name on the Metrics Explorer, labels on query_range) and every explicit
label context (resource./scope.). So a bare/partial filter token 400'd the
Metrics Explorer treemap and metric query-builder filters.

Metrics has no per-context storage - every label lives in the `labels` JSON, and
FieldFor maps any key on its own (intrinsics incl. metric_name to their column,
every label context to a labels extract). Add an opt-in WithIgnoreNotFoundKeys
to the metrics condition builder that resolves such keys directly (matching
nothing) instead of erroring:

- query-builder (querier provider) and Metrics Explorer opt in, so free-text and
  unregistered contexts resolve - matching the statement builder's own fallback
  and group-by, which already collapses all label contexts to labels;
- infra-monitoring list APIs keep the default (strict), so a typo'd attribute
  still surfaces as a 400.

Intrinsics always resolve regardless of the flag. Adds unit coverage for both
modes and a queriermetrics test that a free-text filter no longer errors; the
label-context test now expects the collapse.

* chore: updated integration tests

* fix: dont assert warning == []

* chore: removed warning scaffolding for now

* fix: add back removed integration test

---------

Co-authored-by: nikhilmantri0902 <nikhil.mantri1999@gmail.com>
2026-07-21 01:48:54 +00:00
Srikanth Chekuri
bf0130a983 fix(clickhouseprometheus): resolve __name__ matchers against metric_name and anchor matcher regexes (#12154)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
The read client reduced every __name__ matcher to metric_name = <value>
(empty string when absent), so nameless selectors ({job="api"}), regex
names ({__name__=~".+"}), and negations silently returned empty. Map the
__name__ matcher onto the metric_name column per matcher type, drop the
condition when no __name__ matcher exists, and narrow the samples query
by the metric names the series lookup discovered.

Matcher regexes are now anchored (^(?:...)$) the way Prometheus compiles
them; ClickHouse match() is a substring search, so {job=~"api"} also
matched job="api-server" before.

The lookup and samples SQL is built with huandu/go-sqlbuilder (ClickHouse
flavor) as in telemetrymetrics and the v2 transpiler: the samples query
embeds the series lookup as a nested builder, with argument order merged
by the builder in render position instead of hand-numbered placeholders.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 18:59:34 +00:00
Tushar Vats
9d4349c9d1 fix(querybuilder): resolve data-type collisions for numeric intrinsic columns (#12176)
A span/log query that aggregates or filters a numeric intrinsic column
(e.g. duration_nano) failed with ClickHouse NO_COMMON_TYPE (386) when a
same-named, type-consistent attribute existed in the metadata: field-key
resolution unioned the intrinsic column with the attribute into a multiIf/OR
whose branches had incompatible types (UInt64 intrinsic vs Float64 attribute).
This broke /api/v1/span_percentile on affected tenants.

- fallback_expr: DataTypeCollisionHandledFieldName now handles the unspecified
  data type in the projection/aggregation path (operator Unknown), coercing the
  column so collision multiIf branches share a supertype. Comparison contexts
  are left bare so the column index stays usable.
- telemetrytraces/condition_builder: run collision handling for duration_nano
  (so a same-named string attribute is cast in comparisons) and coerce numeric
  duration values to int64, keeping the intrinsic comparison bare/index-friendly
  while preserving the duration-string QoL parsing.
- tests: add a type-consistent "collision" trace_noise variant; cover it in the
  percentile aggregation test and add a duration_nano QoL filter regression test.
2026-07-20 16:57:39 +00:00
Pandey
d095645d61 fix(querybuildertypesv5): round-trip unset stepInterval and empty field-key values (#12171)
* fix(querybuildertypesv5): omit unset stepInterval on the wire

Step is a struct (struct{ time.Duration }), so omitempty had no effect — an
unset stepInterval serialized as 0 instead of being omitted, so a typed
client reading a query back saw a 0 it never sent (create -> GET drift).

Tag stepInterval with ,omitzero so an unset value is dropped while a set
value still serializes (as seconds), on all three sites: builder query,
trace-operator, and secondary aggregation. Schema-invisible (no OpenAPI /
client change). source and the metric enums were already handled in #12164.

* fix(telemetrytypes): round-trip empty fieldContext/fieldDataType on field keys

A TelemetryFieldKey can deliberately leave fieldContext/fieldDataType empty
to match across any context / data type, but ,omitzero dropped that empty
value on serialize, so a typed client that sent "" read it back as absent
(create -> GET drift).

Make both fields always serialize and add the empty member to their Enum()s
so "" is a valid schema value that round-trips verbatim — the same approach
#12164 used for source. Signal keeps ,omitzero: its empty value is invalid
for the query/variable signal contexts that share the enum (adding "" there
breaks those consumers), and a field key's signal is not deliberately empty.

Regenerate the OpenAPI spec + client (fieldContext/fieldDataType enums gain
"") and update the ScalarData marshal test (column keys now echo the fields).

* fix(telemetrytypes): round-trip empty signal on field keys

Extend the field-key round-trip fix to Signal: add the empty member to
Signal.Enum() and make TelemetryFieldKey.Signal always serialize, so an
empty ("any") field-key signal round-trips as a valid value instead of
being dropped — matching the fieldContext/fieldDataType treatment.

The Signal enum is shared with query/variable signals, where "" is invalid.
Narrow the frontend's TelemetrySignal type to logs/traces/metrics (so the
variable/panel signal selectors stay exhaustive), label the empty member in
the panel type switcher's map, and fold an empty drilldown signal into "all".

Regenerate the OpenAPI spec + client (Signal enum gains ""), and update the
ScalarData marshal test and the querierlogs aggregation label assertions to
include the now-serialized empty signal.
2026-07-20 16:31:47 +00:00
nityanandagohain
cce080e1ae fix: add back the flag in metadata 2026-07-16 11:37:57 +05:30
nityanandagohain
deca060a4d Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-16 11:00:55 +05:30
nityanandagohain
03c7e524e7 fix: fix tests 2026-07-15 20:23:33 +05:30
nityanandagohain
815dc7d88b Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-15 20:10:05 +05:30
nityanandagohain
f50d9199fe fix: address comments 2026-07-15 19:47:39 +05:30
nityanandagohain
31efe177a4 fix: address comments 2026-07-14 18:53:30 +05:30
nityanandagohain
d502d12ac3 fix: update openapi 2026-07-10 14:27:07 +05:30
nityanandagohain
bd9f15a716 fix: update integration test 2026-07-10 14:21:16 +05:30
nityanandagohain
813ef988c9 fix: edge cases and correct cost key 2026-07-10 12:06:34 +05:30
nityanandagohain
40e6799285 fix: add resource fingerprint cte 2026-07-10 00:36:25 +05:30
nityanandagohain
1caa60a3cd fix: cleanup and more tests 2026-07-09 23:54:05 +05:30
nityanandagohain
3f781f0083 fix: more cleanup 2026-07-09 12:39:01 +05:30
nityanandagohain
6aec05cf7a fix: more tests 2026-07-09 08:45:22 +05:30
nityanandagohain
683a52f35a fix: take perf into consideration 2026-07-09 08:45:22 +05:30
nityanandagohain
e924fa1e62 feat: support llm trace list and span list 2026-07-09 08:45:20 +05:30
125 changed files with 5980 additions and 728 deletions

View File

@@ -53,6 +53,7 @@ jobs:
- queriermetrics
- querierscalar
- queriercommon
- querierai
- rawexportdata
- querierauthz
- role

View File

@@ -9,6 +9,11 @@ global:
# the path component (e.g. /signoz in https://example.com/signoz) is used
# as the base path for all HTTP routes (both API and web frontend).
external_url: <unset>
# origins (scheme://host[:port], no path) allowed as login redirect targets. include
# the origin the signoz ui is served on. when not configured, redirect targets are
# not validated.
# allowed_origins:
# - https://signoz.example.com
# the url where the SigNoz backend receives telemetry data (traces, metrics, logs) from instrumented applications.
ingestion_url: <unset>
# the url of the SigNoz MCP server. when unset, the MCP settings page is hidden in the frontend.

View File

@@ -6992,6 +6992,7 @@ components:
Querybuildertypesv5QueryEnvelope:
discriminator:
mapping:
builder_ai_query: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI'
builder_formula: '#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula'
builder_query: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder'
builder_trace_operator: '#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator'
@@ -7000,6 +7001,7 @@ components:
propertyName: type
oneOf:
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopePromQL'
@@ -7014,6 +7016,15 @@ components:
required:
- type
type: object
Querybuildertypesv5QueryEnvelopeBuilderAI:
properties:
spec:
$ref: '#/components/schemas/Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregation'
type:
$ref: '#/components/schemas/Querybuildertypesv5QueryType'
required:
- type
type: object
Querybuildertypesv5QueryEnvelopeClickHouseSQL:
properties:
spec:
@@ -7127,6 +7138,7 @@ components:
Querybuildertypesv5QueryType:
enum:
- builder_query
- builder_ai_query
- builder_formula
- builder_trace_operator
- clickhouse_sql
@@ -8558,6 +8570,7 @@ components:
- resource
- attribute
- body
- ""
type: string
TelemetrytypesFieldDataType:
enum:
@@ -8566,6 +8579,7 @@ components:
- float64
- int64
- number
- ""
type: string
TelemetrytypesGettableFieldKeys:
properties:
@@ -8597,6 +8611,7 @@ components:
- traces
- logs
- metrics
- ""
type: string
TelemetrytypesSource:
enum:

View File

@@ -223,17 +223,12 @@ func (provider *provider) Update(ctx context.Context, orgID valuer.UUID, updated
return err
}
existingGroups := authtypes.MustNewTransactionGroupsFromTuples(existingTuples)
additions, deletions := existingGroups.Diff(updatedRole.TransactionGroups)
additionTuples, err := authtypes.NewTuplesFromTransactionGroups(existingRole.Name, orgID, additions)
desiredTuples, err := authtypes.NewTuplesFromTransactionGroups(existingRole.Name, orgID, updatedRole.TransactionGroups)
if err != nil {
return err
}
deletionTuples, err := authtypes.NewTuplesFromTransactionGroups(existingRole.Name, orgID, deletions)
if err != nil {
return err
}
additionTuples, deletionTuples := authtypes.DiffTuples(existingTuples, desiredTuples)
err = provider.Write(ctx, additionTuples, deletionTuples)
if err != nil {

View File

@@ -240,17 +240,17 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
mock := mockStore.Mock()
// Mock the fingerprint query (for Prometheus label matching)
// args: $1=metric_name (the __name__ matcher maps onto the column)
mock.ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(fingerprintRows)
// Mock the samples query (for Prometheus metric data)
// args: metric_name IN (discovered names), subquery metric_name, start, end
mock.ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs(
"test_metric",
"test_metric",
"__name__",
"test_metric",
queryStart,
queryEnd,
).

View File

@@ -3479,6 +3479,7 @@ export enum TelemetrytypesFieldContextDTO {
resource = 'resource',
attribute = 'attribute',
body = 'body',
'' = '',
}
export enum TelemetrytypesFieldDataTypeDTO {
string = 'string',
@@ -3486,11 +3487,13 @@ export enum TelemetrytypesFieldDataTypeDTO {
float64 = 'float64',
int64 = 'int64',
number = 'number',
'' = '',
}
export enum TelemetrytypesSignalDTO {
traces = 'traces',
logs = 'logs',
metrics = 'metrics',
'' = '',
}
export interface Querybuildertypesv5GroupByKeyDTO {
/**
@@ -4343,6 +4346,18 @@ export interface Querybuildertypesv5QueryEnvelopeBuilderDTO {
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType;
}
export enum Querybuildertypesv5QueryEnvelopeBuilderAIDTOType {
builder_ai_query = 'builder_ai_query',
}
export interface Querybuildertypesv5QueryEnvelopeBuilderAIDTO {
spec?: Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregationDTO;
/**
* @type string
* @enum builder_ai_query
*/
type: Querybuildertypesv5QueryEnvelopeBuilderAIDTOType;
}
export interface Querybuildertypesv5QueryBuilderFormulaDTO {
/**
* @type boolean
@@ -4526,6 +4541,7 @@ export interface Querybuildertypesv5QueryEnvelopeClickHouseSQLDTO {
export type Querybuildertypesv5QueryEnvelopeDTO =
| Querybuildertypesv5QueryEnvelopeBuilderDTO
| Querybuildertypesv5QueryEnvelopeBuilderAIDTO
| Querybuildertypesv5QueryEnvelopeFormulaDTO
| Querybuildertypesv5QueryEnvelopeTraceOperatorDTO
| Querybuildertypesv5QueryEnvelopePromQLDTO
@@ -8370,6 +8386,7 @@ export interface Querybuildertypesv5QueryRangeResponseDTO {
export enum Querybuildertypesv5QueryTypeDTO {
builder_query = 'builder_query',
builder_ai_query = 'builder_ai_query',
builder_formula = 'builder_formula',
builder_trace_operator = 'builder_trace_operator',
clickhouse_sql = 'clickhouse_sql',

View File

@@ -20,7 +20,7 @@ import { Globe, Inbox, SquarePen } from '@signozhq/icons';
import AnnouncementsModal from './AnnouncementsModal';
import FeedbackModal from './FeedbackModal';
import ShareURLModal from './ShareURLModal';
import ShareURLModal, { type ShareURLExtraOption } from './ShareURLModal';
import './HeaderRightSection.styles.scss';
import { Typography } from '@signozhq/ui/typography';
@@ -29,12 +29,15 @@ interface HeaderRightSectionProps {
enableAnnouncements: boolean;
enableShare: boolean;
enableFeedback: boolean;
/** Optional page-specific toggle for the share dialog (e.g. "Include variables"). */
shareModalExtraOption?: ShareURLExtraOption;
}
function HeaderRightSection({
enableAnnouncements,
enableShare,
enableFeedback,
shareModalExtraOption,
}: HeaderRightSectionProps): JSX.Element | null {
const location = useLocation();
@@ -185,7 +188,7 @@ function HeaderRightSection({
rootClassName="header-section-popover-root"
className="shareable-link-popover"
placement="bottomRight"
content={<ShareURLModal />}
content={<ShareURLModal extraOption={shareModalExtraOption} />}
open={openShareURLModal}
destroyTooltipOnHide
arrow={false}

View File

@@ -24,7 +24,22 @@ const routesToBeSharedWithTime = [
ROUTES.METER_EXPLORER,
];
function ShareURLModal(): JSX.Element {
/**
* An optional, page-specific toggle in the share dialog (e.g. a dashboard's
* "Include variables"). When enabled, `apply` mutates the URL params that go into
* the shared link. Keeps this shared modal generic — the page owns what it adds.
*/
export interface ShareURLExtraOption {
label: string;
defaultEnabled?: boolean;
apply: (params: URLSearchParams) => void;
}
interface ShareURLModalProps {
extraOption?: ShareURLExtraOption;
}
function ShareURLModal({ extraOption }: ShareURLModalProps): JSX.Element {
const urlQuery = useUrlQuery();
const location = useLocation();
const { selectedTime } = useSelector<AppState, GlobalReducer>(
@@ -34,6 +49,9 @@ function ShareURLModal(): JSX.Element {
const [enableAbsoluteTime, setEnableAbsoluteTime] = useState(
selectedTime !== 'custom',
);
const [enableExtraOption, setEnableExtraOption] = useState(
extraOption?.defaultEnabled ?? false,
);
const startTime = urlQuery.get(QueryParams.startTime);
const endTime = urlQuery.get(QueryParams.endTime);
@@ -93,6 +111,11 @@ function ShareURLModal(): JSX.Element {
}
}
if (extraOption && enableExtraOption) {
extraOption.apply(urlQuery);
currentUrl = getAbsoluteUrl(`${location.pathname}?${urlQuery.toString()}`);
}
return currentUrl;
};
@@ -143,6 +166,20 @@ function ShareURLModal(): JSX.Element {
</>
)}
{extraOption && (
<div className="absolute-relative-time-toggler-container">
<Typography.Text className="absolute-relative-time-toggler-label">
{extraOption.label}
</Typography.Text>
<div className="absolute-relative-time-toggler">
<Switch
value={enableExtraOption}
onChange={(): void => setEnableExtraOption((prev) => !prev)}
/>
</div>
</div>
)}
<div className="share-link">
<div className="url-share-container">
<div className="url-share-container-header">

View File

@@ -5262,6 +5262,24 @@ const onboardingConfigWithLinks = [
],
},
},
{
dataSource: 'temporal-cloud-metrics',
label: 'Temporal Cloud Metrics',
imgUrl: temporalUrl,
tags: ['metrics'],
module: 'metrics',
relatedSearchKeywords: [
'metrics',
'integrations',
'temporal',
'temporal cloud',
'temporal cloud metrics',
'temporal metrics',
'openmetrics',
'prometheus',
],
link: '/docs/integrations/temporal-cloud-metrics/',
},
{
dataSource: 'temporal',
label: 'Temporal',
@@ -5273,9 +5291,6 @@ const onboardingConfigWithLinks = [
'application performance monitoring',
'integrations',
'temporal',
'temporal cloud',
'temporal logs',
'temporal metrics',
'temporal traces',
'traces',
'tracing',
@@ -5284,12 +5299,6 @@ const onboardingConfigWithLinks = [
desc: 'What are you using ?',
type: 'select',
options: [
{
key: 'temporal-cloud',
label: 'Cloud Metrics',
imgUrl: temporalUrl,
link: '/docs/integrations/temporal-cloud-metrics/',
},
{
key: 'temporal-golang',
label: 'Go',

View File

@@ -22,7 +22,7 @@ import {
import { MetricAggregation } from 'types/api/v5/queryRange';
import { DataSource } from 'types/common/queryBuilder';
import { ExtendedSelectOption } from 'types/common/select';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { selectStyle } from '../QueryBuilderSearch/config';
import OptionRenderer from '../QueryBuilderSearch/OptionRenderer';
@@ -39,6 +39,7 @@ export const AggregatorFilter = memo(function AggregatorFilter({
signalSource,
setAttributeKeys,
}: AgregatorFilterProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const queryClient = useQueryClient();
const [optionsData, setOptionsData] = useState<ExtendedSelectOption[]>([]);
@@ -289,7 +290,7 @@ export const AggregatorFilter = memo(function AggregatorFilter({
return (
<AutoComplete
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
placeholder={getPlaceholder()}
style={selectStyle}
filterOption={false}

View File

@@ -2,7 +2,7 @@ import { Select, SelectProps, Space } from 'antd';
import { Typography } from '@signozhq/ui/typography';
import { getCategorySelectOptionByName } from 'container/NewWidget/RightContainer/alertFomatCategories';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { categoryToSupport } from './config';
import { selectStyles } from './styles';
@@ -13,6 +13,7 @@ function BuilderUnitsFilter({
onChange,
yAxisUnit,
}: IBuilderUnitsFilterProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const { currentQuery, handleOnUnitsChange } = useQueryBuilder();
const selectedValue = yAxisUnit || currentQuery?.unit;
@@ -36,7 +37,7 @@ function BuilderUnitsFilter({
Y-axis unit
</Typography.Text>
<Select
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
style={selectStyles}
onChange={onChangeHandler}
value={selectedValue}

View File

@@ -9,12 +9,13 @@ import {
} from 'lib/query/transformQueryBuilderData';
import { Having, HavingForm } from 'types/api/queryBuilder/queryBuilderData';
import { SelectOption } from 'types/common/select';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { getHavingObject, isValidHavingValue } from '../../utils';
import { HavingFilterProps, HavingTagRenderProps } from './types';
function HavingFilter({ formula, onChange }: HavingFilterProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const { having } = formula;
const [searchText, setSearchText] = useState<string>('');
const [localValues, setLocalValues] = useState<string[]>([]);
@@ -171,7 +172,7 @@ function HavingFilter({ formula, onChange }: HavingFilterProps): JSX.Element {
return (
<Select
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
autoClearSearchValue={false}
mode="multiple"
onSearch={handleSearch}

View File

@@ -2,7 +2,7 @@ import { useMemo } from 'react';
import { Select, Spin } from 'antd';
import { useGetAggregateKeys } from 'hooks/queryBuilder/useGetAggregateKeys';
import { MetricAggregateOperator } from 'types/common/queryBuilder';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { selectStyle } from '../../QueryBuilderSearch/config';
import { OrderByProps } from './types';
@@ -13,6 +13,7 @@ function OrderByFilter({
onChange,
query,
}: OrderByProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const {
debouncedSearchText,
createOptions,
@@ -64,7 +65,7 @@ function OrderByFilter({
return (
<Select
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
mode="tags"
style={selectStyle}
onSearch={handleSearchKeys}

View File

@@ -21,7 +21,7 @@ import { isEqual, uniqWith } from 'lodash-es';
import { BaseAutocompleteData } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { DataSource } from 'types/common/queryBuilder';
import { SelectOption } from 'types/common/select';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { selectStyle } from '../QueryBuilderSearch/config';
import OptionRenderer from '../QueryBuilderSearch/OptionRenderer';
@@ -33,6 +33,7 @@ export const GroupByFilter = memo(function GroupByFilter({
disabled,
signalSource,
}: GroupByFilterProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const queryClient = useQueryClient();
const [searchText, setSearchText] = useState<string>('');
const [optionsData, setOptionsData] = useState<
@@ -174,7 +175,7 @@ export const GroupByFilter = memo(function GroupByFilter({
return (
<Select
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
mode="tags"
style={selectStyle}
onSearch={handleSearchKeys}

View File

@@ -16,7 +16,7 @@ import {
import { Having, HavingForm } from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import { SelectOption } from 'types/common/select';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { getHavingObject, isValidHavingValue } from '../utils';
// ** Types
@@ -27,6 +27,7 @@ export function HavingFilter({
query,
onChange,
}: HavingFilterProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const { having } = query;
const [searchText, setSearchText] = useState<string>('');
const [options, setOptions] = useState<SelectOption<string, string>[]>([]);
@@ -231,7 +232,7 @@ export function HavingFilter({
return (
<>
<Select
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
autoClearSearchValue={false}
mode="multiple"
onSearch={handleSearch}

View File

@@ -14,7 +14,7 @@ import {
import { IBuilderQuery } from 'types/api/queryBuilder/queryBuilderData';
import { MetricAggregation } from 'types/api/v5/queryRange';
import { ExtendedSelectOption } from 'types/common/select';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { selectStyle } from '../QueryBuilderSearch/config';
import OptionRenderer from '../QueryBuilderSearch/OptionRenderer';
@@ -85,6 +85,7 @@ export const MetricNameSelector = memo(function MetricNameSelector({
signalSource,
'data-testid': dataTestId,
}: MetricNameSelectorProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const currentMetricName =
(query.aggregations?.[0] as MetricAggregation)?.metricName ||
query.aggregateAttribute?.key ||
@@ -272,7 +273,7 @@ export const MetricNameSelector = memo(function MetricNameSelector({
return (
<AutoComplete
className="metric-name-selector"
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
style={selectStyle}
filterOption={false}
placeholder={placeholder}

View File

@@ -3,7 +3,7 @@ import { Select, Spin } from 'antd';
import { useGetAggregateKeys } from 'hooks/queryBuilder/useGetAggregateKeys';
import { DataSource, MetricAggregateOperator } from 'types/common/queryBuilder';
import { getParsedAggregationOptionsForOrderBy } from 'utils/aggregationConverter';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { selectStyle } from '../QueryBuilderSearch/config';
import { OrderByFilterProps } from './OrderByFilter.interfaces';
@@ -16,6 +16,7 @@ export function OrderByFilter({
entityVersion,
isNewQueryV2 = false,
}: OrderByFilterProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const {
debouncedSearchText,
selectedValue,
@@ -78,7 +79,7 @@ export function OrderByFilter({
return (
<Select
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
mode="tags"
style={selectStyle}
onSearch={handleSearchKeys}

View File

@@ -50,7 +50,7 @@ import {
} from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import { getUserOperatingSystem, UserOperatingSystem } from 'utils/getUserOS';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { v4 as uuid } from 'uuid';
import { FeatureKeys } from '../../../../constants/features';
@@ -95,6 +95,7 @@ function QueryBuilderSearch({
disableNavigationShortcuts,
entity,
}: QueryBuilderSearchProps): JSX.Element {
const getPopupContainer = useSelectPopupContainer();
const { pathname } = useLocation();
const isLogsExplorerPage = useMemo(
() => pathname === ROUTES.LOGS_EXPLORER,
@@ -397,7 +398,7 @@ function QueryBuilderSearch({
<Select
data-testid={'qb-search-select'}
ref={selectRef}
getPopupContainer={popupContainer}
getPopupContainer={getPopupContainer}
transitionName=""
choiceTransitionName=""
virtual={false}

View File

@@ -50,7 +50,7 @@ import {
TagFilter,
} from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import { popupContainer } from 'utils/selectPopupContainer';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { v4 as uuid } from 'uuid';
import { selectStyle } from '../QueryBuilderSearch/config';
@@ -157,6 +157,8 @@ function QueryBuilderSearchV2(
selectProps,
} = props;
const getPopupContainer = useSelectPopupContainer();
const { registerShortcut, deregisterShortcut } = useKeyboardHotkeys();
const { handleRunQuery, currentQuery } = useQueryBuilder();
@@ -989,7 +991,7 @@ function QueryBuilderSearchV2(
{...selectProps}
data-testid={'qb-search-select'}
ref={selectRef}
{...(hasPopupContainer ? { getPopupContainer: popupContainer } : {})}
{...(hasPopupContainer ? { getPopupContainer } : {})}
{...(maxTagCount ? { maxTagCount } : {})}
key={queryTags.join('.')}
virtual={false}

View File

@@ -126,6 +126,15 @@ export default function UPlotChart({
}
}, [isDataEmpty, destroyPlot]);
/**
* Destroy the plot on unmount. Without this, uPlot's window-level
* `dppxchange` listener keeps the instance (and its whole detached DOM
* subtree) alive after the component is gone.
*/
const destroyPlotRef = useRef(destroyPlot);
destroyPlotRef.current = destroyPlot;
useEffect(() => (): void => destroyPlotRef.current(), []);
/**
* Handle initialization and prop changes
*/

View File

@@ -327,6 +327,32 @@ describe('UPlotChart', () => {
expect(firstInstance.destroy).toHaveBeenCalled();
expect(instances).toHaveLength(2);
});
it('destroys the instance and notifies callbacks on unmount', () => {
const plotRef = jest.fn();
const onDestroy = jest.fn();
const { unmount } = render(
<UPlotChart
config={createMockConfig()}
data={validData}
width={600}
height={400}
plotRef={plotRef}
onDestroy={onDestroy}
/>,
{ wrapper: Wrapper },
);
const firstInstance = instances[0];
plotRef.mockClear();
unmount();
expect(onDestroy).toHaveBeenCalledWith(firstInstance);
expect(firstInstance.destroy).toHaveBeenCalledTimes(1);
expect(plotRef).toHaveBeenCalledWith(null);
});
});
describe('spanGaps data transformation', () => {

View File

@@ -16,7 +16,13 @@ import { sortBy } from 'lodash-es';
export type VariableType = 'QUERY' | 'CUSTOM' | 'TEXT' | 'DYNAMIC';
/** Telemetry signal — the generated enum (traces / logs / metrics). */
export type TelemetrySignal = TelemetrytypesSignalDTO;
// A query/variable signal is only logs/traces/metrics. TelemetrytypesSignalDTO
// also carries the empty "any" value used on field keys, which is not a valid
// query/variable signal, so exclude it here.
export type TelemetrySignal =
| TelemetrytypesSignalDTO.logs
| TelemetrytypesSignalDTO.traces
| TelemetrytypesSignalDTO.metrics;
/**
* Signal selected in the dynamic-variable editor. `'all'` is UI-only (the

View File

@@ -17,6 +17,9 @@ const SIGNAL_LABEL: Record<TelemetrytypesSignalDTO, string> = {
[TelemetrytypesSignalDTO.logs]: 'logs',
[TelemetrytypesSignalDTO.traces]: 'traces',
[TelemetrytypesSignalDTO.metrics]: 'metrics',
// The empty "any" signal only appears on field keys, never on a panel query;
// mapped for exhaustiveness.
[TelemetrytypesSignalDTO['']]: '',
};
/**

View File

@@ -1,5 +1,6 @@
import { ChevronDown } from '@signozhq/icons';
import { ColorPicker } from 'antd';
import { ThresholdColor } from 'pages/DashboardPageV2/DashboardContainer/Panels/types/threshold';
import styles from './ThresholdsSection.module.scss';
@@ -11,11 +12,11 @@ interface ThresholdColorSelectProps {
// Named presets from the SigNoz palette (cherry / amber / forest / robin). They surface
// as quick swatches in the picker; the full picker below covers any custom color.
const PRESETS: { label: string; value: string }[] = [
{ label: 'Red', value: '#F1575F' },
{ label: 'Orange', value: '#F5B225' },
{ label: 'Green', value: '#2BB673' },
{ label: 'Blue', value: '#4E74F8' },
const PRESETS: { label: string; value: ThresholdColor }[] = [
{ label: 'Red', value: ThresholdColor.RED },
{ label: 'Orange', value: ThresholdColor.ORANGE },
{ label: 'Green', value: ThresholdColor.GREEN },
{ label: 'Blue', value: ThresholdColor.BLUE },
];
/**

View File

@@ -12,6 +12,7 @@ import {
AnyThreshold,
ThresholdVariant,
} from 'pages/DashboardPageV2/DashboardContainer/Panels/types/sections';
import { ThresholdColor } from 'pages/DashboardPageV2/DashboardContainer/Panels/types/threshold';
import type { TableColumnOption } from '../../../hooks/useTableColumns';
import type { SectionEditorContext } from '../../sectionContext';
@@ -22,7 +23,7 @@ import TableThresholdRow from './rows/TableThresholdRow';
import styles from './ThresholdsSection.module.scss';
// New thresholds default to red (the first palette preset); the user recolors per rule.
const DEFAULT_THRESHOLD_COLOR = '#F1575F';
const DEFAULT_THRESHOLD_COLOR = ThresholdColor.RED;
// Add-button testId per variant — kept stable so existing E2E/unit selectors hold.
const ADD_TESTID: Record<ThresholdVariant, string> = {

View File

@@ -73,6 +73,25 @@ describe('usePanelEditorDraft', () => {
expect(result.current.isSpecDirty).toBe(false);
});
it('flags spec-dirty when the seed differs from the saved baseline (View handoff)', () => {
// The editor opens on a handed-off, already-edited spec (`seed`) but compares
// against the persisted panel (`saved`) — so it starts dirty, not clean.
const seed = panel('Memory', 'usage');
const saved = panel('CPU', 'usage');
const { result } = renderHook(() => usePanelEditorDraft(seed, saved));
expect(result.current.isSpecDirty).toBe(true);
});
it('is clean when the seed matches the saved baseline', () => {
const { result } = renderHook(() =>
usePanelEditorDraft(panel('CPU', 'usage'), panel('CPU', 'usage')),
);
expect(result.current.isSpecDirty).toBe(false);
});
it('reset restores the spec and clears dirty after an edit', () => {
const { result } = renderHook(() => usePanelEditorDraft(panel()));

View File

@@ -0,0 +1,201 @@
import { renderHook, waitFor } from '@testing-library/react';
import type {
DashboardtypesPanelDTO,
DashboardtypesQueryDTO,
} from 'api/generated/services/sigNoz.schemas';
import { QueryParams } from 'constants/query';
import { initialQueriesMap, PANEL_TYPES } from 'constants/queryBuilder';
import type { Query } from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import { AllTheProviders } from 'tests/test-utils';
import { toPerses } from '../../../queryV5/persesQueryAdapters';
import { usePanelEditorQuerySync } from '../usePanelEditorQuerySync';
// Exercises the REAL query builder provider (not mocks) so the dirty check is
// verified against the builder's actual re-serialization — the "always dirty"
// regression only reproduces with the real normalization in the loop.
const panelType = PANEL_TYPES.TIME_SERIES;
function makeSavedQueries(): DashboardtypesQueryDTO[] {
const base: Query = {
...initialQueriesMap[DataSource.METRICS],
builder: {
...initialQueriesMap[DataSource.METRICS].builder,
queryData: [
{
...initialQueriesMap[DataSource.METRICS].builder.queryData[0],
legend: 'cpu',
},
],
},
};
return toPerses(base, panelType);
}
function makePanel(queries: DashboardtypesQueryDTO[]): DashboardtypesPanelDTO {
return {
kind: 'Panel',
spec: {
display: { name: 'CPU' },
plugin: { kind: 'signoz/TimeSeriesPanel', spec: {} },
queries,
},
} as unknown as DashboardtypesPanelDTO;
}
describe('usePanelEditorQuerySync (real query builder)', () => {
it('an untouched existing panel is NOT query-dirty on mount', async () => {
const saved = makeSavedQueries();
const { result } = renderHook(
() =>
usePanelEditorQuerySync({
draft: makePanel(saved),
panelType,
setSpec: jest.fn(),
refetch: jest.fn(),
savedQueries: saved,
}),
{ wrapper: AllTheProviders },
);
// The builder force-resets to the saved query asynchronously; once settled the
// live query must serialize back to the saved queries → clean.
await waitFor(() => expect(result.current.isQueryDirty).toBe(false));
// And stays clean (no late re-stage flips it dirty).
expect(result.current.isQueryDirty).toBe(false);
});
it('an untouched panel with a minimal/older stored query is NOT dirty (drift fix)', async () => {
// An older saved query carries only a few fields; the builder re-emits many more
// (source, stepInterval, filter, spaceAggregation, …). Comparing raw would read
// this as always-dirty; the round-tripped baseline normalizes both sides.
const minimalSaved: DashboardtypesQueryDTO[] = [
{
kind: 'time_series',
spec: {
plugin: {
kind: 'signoz/CompositeQuery',
spec: {
queries: [
{
type: 'builder_query',
spec: {
name: 'A',
signal: 'metrics',
aggregations: [
{ metricName: 'system_cpu_time', timeAggregation: 'avg' },
],
},
},
],
},
},
},
},
] as unknown as DashboardtypesQueryDTO[];
const { result } = renderHook(
() =>
usePanelEditorQuerySync({
draft: makePanel(minimalSaved),
panelType,
setSpec: jest.fn(),
refetch: jest.fn(),
savedQueries: minimalSaved,
}),
{ wrapper: AllTheProviders },
);
await waitFor(() => expect(result.current.isQueryDirty).toBe(false));
expect(result.current.isQueryDirty).toBe(false);
});
it('retains an in-editor query edit carried in the URL across a refresh (and reads dirty)', async () => {
// Simulate a refresh mid-edit: the saved panel is unchanged, but the URL still
// carries the last-run (edited) query. The builder must hydrate from the URL —
// not discard it — so the edit survives, and it must read dirty against saved.
const saved = makeSavedQueries();
const editedInUrl: Query = {
...initialQueriesMap[DataSource.METRICS],
id: 'edited-in-url',
builder: {
...initialQueriesMap[DataSource.METRICS].builder,
queryData: [
{
...initialQueriesMap[DataSource.METRICS].builder.queryData[0],
legend: 'cpu-edited-in-url',
},
],
},
};
const params = new URLSearchParams();
params.set(
QueryParams.compositeQuery,
encodeURIComponent(JSON.stringify(editedInUrl)),
);
const setSpec = jest.fn();
const wrapper = ({
children,
}: {
children: React.ReactNode;
}): JSX.Element => (
<AllTheProviders initialRoute={`/?${params.toString()}`}>
{children}
</AllTheProviders>
);
const { result } = renderHook(
() =>
usePanelEditorQuerySync({
// The draft/preview open on the saved query…
draft: makePanel(saved),
panelType,
setSpec,
refetch: jest.fn(),
savedQueries: saved,
}),
{ wrapper },
);
// The URL edit is retained → dirty, and it's synced into the draft so the
// preview follows (setSpec called with the edited query).
await waitFor(() => expect(result.current.isQueryDirty).toBe(true));
expect(setSpec).toHaveBeenCalled();
});
it('is query-dirty when the draft carries an edit the saved panel does not (View handoff)', async () => {
const saved = makeSavedQueries();
const editedBase: Query = {
...initialQueriesMap[DataSource.METRICS],
builder: {
...initialQueriesMap[DataSource.METRICS].builder,
queryData: [
{
...initialQueriesMap[DataSource.METRICS].builder.queryData[0],
legend: 'cpu-edited',
},
],
},
};
const editedQueries = toPerses(editedBase, panelType);
const { result } = renderHook(
() =>
usePanelEditorQuerySync({
// The builder seeds from the draft (the handed-off edit)…
draft: makePanel(editedQueries),
panelType,
setSpec: jest.fn(),
refetch: jest.fn(),
// …but the baseline is the persisted panel.
savedQueries: saved,
}),
{ wrapper: AllTheProviders },
);
await waitFor(() => expect(result.current.isQueryDirty).toBe(true));
});
});

View File

@@ -95,6 +95,7 @@ describe('usePanelEditorQuerySync', () => {
draft?: DashboardtypesPanelDTO;
setSpec?: jest.Mock;
refetch?: jest.Mock;
savedQueries?: DashboardtypesPanelSpecDTO['queries'];
} = {},
): {
result: {
@@ -119,20 +120,22 @@ describe('usePanelEditorQuerySync', () => {
panelType: PANEL_TYPES.TIME_SERIES,
setSpec,
refetch,
savedQueries: opts.savedQueries,
}),
);
return { result, setSpec, refetch, rerender };
}
it('force-resets the builder to the saved queries on mount (discards stale URL)', () => {
it('seeds the builder from the draft queries on mount (URL query, when present, wins)', () => {
setup();
expect(mockFromPerses).toHaveBeenCalledWith(
SAVED_QUERIES,
PANEL_TYPES.TIME_SERIES,
);
// No forceReset: useShareBuilderUrl resets to the seed only when the URL carries
// no query, so an in-editor edit in the URL survives a refresh.
expect(mockUseShareBuilderUrl).toHaveBeenCalledWith({
defaultValue: SEED_V1,
forceReset: true,
});
});
@@ -342,44 +345,127 @@ describe('usePanelEditorQuerySync', () => {
});
});
describe('query dirty + save', () => {
it('compares the live query against the builder baseline (first staged query), not the raw seed', () => {
mockGetIsQueryModified.mockReturnValue(true);
const { result } = setup();
describe('staged-query re-sync (browser back/forward)', () => {
it('commits the staged query into the draft when it re-stages', () => {
const state = builderState();
mockUseQueryBuilder.mockImplementation(() => state);
// Baseline is the builder's own normalized staged query — immune to the
// raw-seed vs builder-normalized serialization drift.
expect(mockGetIsQueryModified).toHaveBeenCalledWith(
expect.anything(),
STAGED_V1,
const { setSpec, rerender } = setup();
setSpec.mockClear();
// Browser Back re-stages a different query via initQueryBuilderData; the
// preview must follow it instead of keeping the last Run's result.
mockGetIsQueryModified.mockReturnValue(true);
state.stagedQuery = {
id: 'restaged',
queryType: 'builder',
} as unknown as Query;
rerender();
expect(setSpec).toHaveBeenCalledWith({
...makeDraft().spec,
queries: CONVERTED_QUERIES,
});
});
it('does not commit when only the live query changes (no re-stage)', () => {
const state = builderState({
currentQuery: { id: 'a', queryType: 'builder' } as Query,
});
mockUseQueryBuilder.mockImplementation(() => state);
mockGetIsQueryModified.mockReturnValue(true);
const { setSpec, rerender } = setup();
setSpec.mockClear();
// Live edit: currentQuery changes, staged query + structure unchanged.
state.currentQuery = { id: 'b', queryType: 'builder' } as Query;
rerender();
expect(setSpec).not.toHaveBeenCalled();
});
});
describe('query dirty + save', () => {
// isQueryDirty compares the live query to the SAVED queries at the V5 envelope
// level (toQueryEnvelopes is mocked identity). Drive it via an input-sensitive
// toPerses so the envelope comparison — not getIsQueryModified — decides.
const SAVED_BASELINE = [{ id: 'saved-baseline' }] as unknown as NonNullable<
DashboardtypesPanelSpecDTO['queries']
>;
const EDITED_ENVELOPES = [
{ id: 'edited-envelopes' },
] as unknown as NonNullable<DashboardtypesPanelSpecDTO['queries']>;
const editedQuery = { id: 'edited', queryType: 'builder' } as Query;
const unchangedQuery = { id: 'unchanged', queryType: 'builder' } as Query;
beforeEach(() => {
mockToPerses.mockImplementation((query: Query) =>
query?.id === 'edited' ? EDITED_ENVELOPES : SAVED_BASELINE,
);
});
it('is query-dirty when the live query no longer serializes to the saved queries', () => {
mockUseQueryBuilder.mockReturnValue(
builderState({ currentQuery: editedQuery }),
);
const { result } = setup({ savedQueries: SAVED_BASELINE });
expect(result.current.isQueryDirty).toBe(true);
});
it('is not query-dirty when the live query matches the baseline', () => {
mockGetIsQueryModified.mockReturnValue(false);
const { result } = setup();
it('is not query-dirty when the live query still serializes to the saved queries', () => {
mockUseQueryBuilder.mockReturnValue(
builderState({ currentQuery: unchangedQuery }),
);
const { result } = setup({ savedQueries: SAVED_BASELINE });
expect(result.current.isQueryDirty).toBe(false);
});
it('buildSaveSpec bakes the live query in when dirty', () => {
mockGetIsQueryModified.mockReturnValue(true);
const { result } = setup();
mockUseQueryBuilder.mockReturnValue(
builderState({ currentQuery: editedQuery }),
);
const { result } = setup({ savedQueries: SAVED_BASELINE });
const { spec } = makeDraft();
expect(result.current.buildSaveSpec(spec)).toStrictEqual({
...spec,
queries: CONVERTED_QUERIES,
queries: EDITED_ENVELOPES,
});
});
it('buildSaveSpec returns the spec untouched when the query is unchanged', () => {
mockGetIsQueryModified.mockReturnValue(false);
const { result } = setup();
mockUseQueryBuilder.mockReturnValue(
builderState({ currentQuery: unchangedQuery }),
);
const { result } = setup({ savedQueries: SAVED_BASELINE });
const { spec } = makeDraft();
expect(result.current.buildSaveSpec(spec)).toBe(spec);
});
it('anchors the baseline to savedQueries, not the draft the builder seeds from (View handoff / refresh)', () => {
// The draft carries the View-mode edit (the builder seeds from it), but the
// baseline is the persisted panel: a live query equal to the edited draft
// still reads dirty against the saved queries.
mockUseQueryBuilder.mockReturnValue(
builderState({ currentQuery: editedQuery }),
);
const draft = makeDraft(EDITED_ENVELOPES);
const { result } = setup({ draft, savedQueries: SAVED_BASELINE });
expect(result.current.isQueryDirty).toBe(true);
});
it('falls back to the seed query as the baseline when there are no saved queries (new panel)', () => {
mockUseQueryBuilder.mockReturnValue(
builderState({ currentQuery: unchangedQuery }),
);
const { result } = setup();
expect(result.current.isQueryDirty).toBe(false);
});
});
});

View File

@@ -23,6 +23,12 @@ import { usePanelTypeSwitch } from './usePanelTypeSwitch';
interface UsePanelEditSessionArgs {
panel: DashboardtypesPanelDTO;
panelId: string;
/**
* The persisted panel the dirty check compares against. Distinct from `panel` (the
* seed), which may carry unsaved edits handed off from View mode. Omit for a new
* panel or the drilldown modal, where the seed is the baseline.
*/
savedPanel?: DashboardtypesPanelDTO;
/** Per-view time window (epoch ms); omit to follow the dashboard's global window. */
time?: PanelQueryTimeOverride;
/** Serialize the live builder query into the spec on save even if unchanged (new panels). */
@@ -67,12 +73,15 @@ export interface UsePanelEditSessionReturn {
export function usePanelEditSession({
panel,
panelId,
savedPanel,
time,
alwaysSerializeQuery = false,
seedQuerySignal = false,
}: UsePanelEditSessionArgs): UsePanelEditSessionReturn {
const { draft, spec, setSpec, isSpecDirty, reset } =
usePanelEditorDraft(panel);
const { draft, spec, setSpec, isSpecDirty, reset } = usePanelEditorDraft(
panel,
savedPanel,
);
const panelKind = draft.spec.plugin.kind;
const panelDefinition = getPanelDefinition(panelKind);
@@ -93,6 +102,7 @@ export function usePanelEditSession({
refetch: query.refetch,
alwaysSerializeQuery,
signal: seedQuerySignal ? defaultSignal : undefined,
savedQueries: savedPanel?.spec.queries,
});
const { onChangePanelKind } = usePanelTypeSwitch({

View File

@@ -13,9 +13,14 @@ import type { PanelEditorDraftApi } from '../types';
* preview renders it through the dashboard's renderer registry and the save hook
* patches it without conversion. Everything the config pane edits flows through the
* single `spec`/`setSpec` pair.
*
* `savedPanel` is the persisted panel the dirty check compares against — distinct from
* `initialPanel` (the seed), which may carry unsaved edits handed off from View mode.
* Defaults to the seed when there's no separate saved baseline (a new panel).
*/
export function usePanelEditorDraft(
initialPanel: DashboardtypesPanelDTO,
savedPanel: DashboardtypesPanelDTO = initialPanel,
): PanelEditorDraftApi {
const [draft, setDraft] = useState<DashboardtypesPanelDTO>(initialPanel);
@@ -35,9 +40,9 @@ export function usePanelEditorDraft(
() =>
!isEqual(
{ ...draft, spec: { ...draft.spec, queries: null } },
{ ...initialPanel, spec: { ...initialPanel.spec, queries: null } },
{ ...savedPanel, spec: { ...savedPanel.spec, queries: null } },
),
[draft, initialPanel],
[draft, savedPanel],
);
return {

View File

@@ -1,7 +1,8 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useCallback, useEffect, useMemo, useRef } from 'react';
import type {
DashboardtypesPanelDTO,
DashboardtypesPanelSpecDTO,
DashboardtypesQueryDTO,
TelemetrytypesSignalDTO,
} from 'api/generated/services/sigNoz.schemas';
import { initialQueriesMap, PANEL_TYPES } from 'constants/queryBuilder';
@@ -27,6 +28,12 @@ interface UsePanelEditorQuerySyncArgs {
alwaysSerializeQuery?: boolean;
/** Signal to seed a new panel's builder with — the kind's first supported signal. */
signal?: TelemetrytypesSignalDTO;
/**
* The persisted panel's queries — the dirty baseline. Distinct from `draft.spec.queries`,
* which the builder seeds from and may carry unsaved edits handed off from View mode. Omit
* for a new panel, where the seed query is the baseline.
*/
savedQueries?: DashboardtypesQueryDTO[];
}
interface UsePanelEditorQuerySyncApi {
@@ -53,43 +60,31 @@ export function usePanelEditorQuerySync({
refetch,
alwaysSerializeQuery = false,
signal,
savedQueries,
}: UsePanelEditorQuerySyncArgs): UsePanelEditorQuerySyncApi {
const { currentQuery, stagedQuery, handleRunQuery } = useQueryBuilder();
// Saved queries, captured once: seed the builder and serve as the restore target.
const savedQueries = draft.spec.queries;
const draftQueries = draft.spec.queries;
// A new panel has no saved query: seed from the kind's first supported signal
// instead of letting `fromPerses` fall back to the metrics default (which List
// doesn't support).
// A new panel has no saved query: seed from the kind's first supported signal rather
// than `fromPerses`'s metrics default (which List doesn't support).
const seedQuery = useMemo(
() =>
savedQueries.length === 0 && signal
draftQueries.length === 0 && signal
? initialQueriesMap[signal]
: fromPerses(savedQueries, panelType),
[savedQueries, panelType, signal],
: fromPerses(draftQueries, panelType),
[draftQueries, panelType, signal],
);
// Force-reset the builder to the SAVED panel on first render only, discarding a
// stale URL query from a prior edit (else the QB/preview diverge and the dirty
// baseline is captured from the URL). After mount the URL syncs normally.
const isInitialRenderRef = useRef(true);
useShareBuilderUrl({
defaultValue: seedQuery,
forceReset: isInitialRenderRef.current,
});
useEffect(() => {
isInitialRenderRef.current = false;
}, []);
// No forceReset: seed the builder only when the URL carries no query, so an
// in-editor edit in the URL survives a refresh / browser Back-Forward.
useShareBuilderUrl({ defaultValue: seedQuery });
// Commit the live query into the draft (what the preview fetches). The dirty
// check compares against the SAVED query (`seedQuery`), not the URL-synced
// staged query, which can carry stale state across a refresh and read a real
// switch as "unchanged". Returns whether the draft changed.
// Commit the live query into the draft (what the preview fetches).
const commitQuery = useCallback(
(query: Query): boolean => {
const next = getIsQueryModified(query, seedQuery)
? toPerses(query, panelType)
: savedQueries;
: draftQueries;
// No-op guard at the V5 envelope level: equivalent wrappers (bare
// `signoz/BuilderQuery` vs `signoz/CompositeQuery`) unwrap to the same
// envelopes, so a structural compare would falsely dirty the draft.
@@ -100,7 +95,7 @@ export function usePanelEditorQuerySync({
setSpec({ ...draft.spec, queries: next });
return true;
},
[seedQuery, panelType, savedQueries, draft.spec, setSpec],
[seedQuery, panelType, draftQueries, draft.spec, setSpec],
);
// Latest query/commit, read by the structural-change effect without re-subscribing.
@@ -110,7 +105,7 @@ export function usePanelEditorQuerySync({
queryRef.current = currentQuery;
// Re-commit on a query-type/datasource switch so the preview refetches. Skip
// mount: the draft already holds the saved queries the builder is reset to.
// mount: the initial query is synced into the draft by the staged-query effect below.
const dataSources = useMemo(
() => (currentQuery.builder?.queryData ?? []).map((q) => q.dataSource),
[currentQuery.builder],
@@ -136,6 +131,15 @@ export function usePanelEditorQuerySync({
// eslint-disable-next-line react-hooks/exhaustive-deps -- structural change only
}, [currentQuery.queryType, dataSourceSignature]);
// Follow the staged (executed) query into the draft on a URL re-stage (mount
// hydration, browser Back/Forward) so the preview matches. Live edits touch only
// currentQuery, so they still wait for Run; commitQuery no-ops when unchanged.
useEffect(() => {
if (stagedQuery) {
commitRef.current(stagedQuery);
}
}, [stagedQuery]);
// Stage & Run / ⌘↵: stage, commit, and re-fetch when unchanged so it can be re-run.
const runQuery = useCallback((): void => {
handleRunQuery();
@@ -144,20 +148,29 @@ export function usePanelEditorQuerySync({
}
}, [handleRunQuery, commitQuery, currentQuery, refetch]);
// Dirty baseline: the builder's OWN normalized saved query (first non-null
// `stagedQuery` after the mount reset) — comparing builder-normalized to
// builder-normalized avoids serialization drift reading an untouched query as
// modified. In state (not a ref) so capture re-triggers `isQueryDirty`; captured
// once and never moved by Stage & Run, so it stays anchored to saved.
const [queryBaseline, setQueryBaseline] = useState<Query | null>(null);
useEffect(() => {
if (queryBaseline === null && stagedQuery) {
setQueryBaseline(stagedQuery);
}
}, [queryBaseline, stagedQuery]);
const isQueryDirty =
queryBaseline !== null && getIsQueryModified(currentQuery, queryBaseline);
// Dirty = the live query no longer serializes to the SAVED panel's query, compared at
// the V5 envelope level. Anchoring to `savedQueries` (not the builder-seed) keeps a
// handed-off / URL-restored edit reading as dirty; routing both sides through the same
// `fromPerses → toPerses` round-trip stops builder-added defaults (absent from an older
// stored query) reading an untouched panel as modified. New panel: fall back to seed.
const baselineEnvelopes = useMemo(
() =>
toQueryEnvelopes(
toPerses(
savedQueries ? fromPerses(savedQueries, panelType) : seedQuery,
panelType,
),
),
[savedQueries, seedQuery, panelType],
);
const isQueryDirty = useMemo(
() =>
!isEqual(
toQueryEnvelopes(toPerses(currentQuery, panelType)),
baselineEnvelopes,
),
[currentQuery, panelType, baselineEnvelopes],
);
const buildSaveSpec = useCallback(
(spec: DashboardtypesPanelSpecDTO): DashboardtypesPanelSpecDTO =>

View File

@@ -6,6 +6,7 @@ import {
useDefaultLayout,
} from '@signozhq/ui/resizable';
import { toast } from '@signozhq/ui/sonner';
import { ConfigProvider } from 'antd';
import {
type DashboardtypesPanelDTO,
TelemetrytypesSignalDTO,
@@ -41,10 +42,22 @@ import styles from './PanelEditor.module.scss';
import logEvent from '@/api/common/logEvent';
import { DashboardEvents } from '../../constants/events';
// The query builder sits in an `overflow:hidden` resizable pane, so its Select
// popups (group-by, order-by, having, …) clip when they open into the short pane.
// Portal them to the document body; the query-builder filters honor this via
// `useSelectPopupContainer`. Scoped to the full-page editor — the View modal keeps
// its own `ConfigProvider` so popups stay inside the focus-trapped dialog.
const getBodyPopupContainer = (): HTMLElement => document.body;
interface PanelEditorContainerProps {
dashboardId: string;
panelId: string;
panel: DashboardtypesPanelDTO;
/**
* The persisted panel the dirty check compares against. Distinct from `panel` (the
* seed), which may carry unsaved edits handed off from View mode. Omit for a new panel.
*/
savedPanel?: DashboardtypesPanelDTO;
/** Creating a new panel (seeded default) vs editing an existing one. */
isNew?: boolean;
/** Target section for a new panel; falls back to the last/new section. */
@@ -68,6 +81,7 @@ function PanelEditorContainer({
dashboardId,
panelId,
panel,
savedPanel,
isNew = false,
layoutIndex,
isEditable,
@@ -91,6 +105,7 @@ function PanelEditorContainer({
} = usePanelEditSession({
panel,
panelId,
savedPanel,
alwaysSerializeQuery: isNew,
seedQuerySignal: true,
});
@@ -288,22 +303,24 @@ function PanelEditorContainer({
</ResizablePanel>
<ResizableHandle withHandle className={styles.handle} />
<ResizablePanel minSize="35%" maxSize="45%" defaultSize="40%">
<PanelEditorQueryBuilder
panelKind={panelKind}
signal={listSignal}
isLoadingQueries={isFetching}
onStageRunQuery={runQuery}
onCancelQuery={cancelQuery}
footer={
isListPanel ? (
<ListColumnsEditor
spec={spec}
onChangeSpec={setSpec}
signal={listSignal}
/>
) : undefined
}
/>
<ConfigProvider getPopupContainer={getBodyPopupContainer}>
<PanelEditorQueryBuilder
panelKind={panelKind}
signal={listSignal}
isLoadingQueries={isFetching}
onStageRunQuery={runQuery}
onCancelQuery={cancelQuery}
footer={
isListPanel ? (
<ListColumnsEditor
spec={spec}
onChangeSpec={setSpec}
signal={listSignal}
/>
) : undefined
}
/>
</ConfigProvider>
</ResizablePanel>
</ResizablePanelGroup>
</div>

View File

@@ -22,6 +22,22 @@ export interface ComparisonThresholdShape {
format?: DashboardtypesThresholdFormatDTO;
}
/** SigNoz threshold palette; single source of truth for the hex values. */
export enum ThresholdColor {
RED = '#F1575F',
ORANGE = '#F5B225',
GREEN = '#2BB673',
BLUE = '#4E74F8',
}
/** Palette ordered most-dangerous first (preset order + alert-severity ranking). */
export const THRESHOLD_COLOR_DANGER_ORDER: ThresholdColor[] = [
ThresholdColor.RED,
ThresholdColor.ORANGE,
ThresholdColor.GREEN,
ThresholdColor.BLUE,
];
/** Comparison operators a threshold can use, as evaluable symbols. */
export type ThresholdComparisonOperator = '>' | '<' | '>=' | '<=' | '=' | '!=';

View File

@@ -104,7 +104,9 @@ function ViewPanelModalContent({
logEvent(DashboardEvents.SWITCH_TO_EDIT_MODE, {
panelId: panelId,
});
openPanelEditor(panelId, { editSpec: buildSaveSpec(draft.spec) });
openPanelEditor(panelId, {
handoffState: { editSpec: buildSaveSpec(draft.spec) },
});
};
return (

View File

@@ -105,7 +105,8 @@ export function useDrilldownDashboardVariables({
type: 'DYNAMIC',
multiSelect: true,
dynamicAttribute: fieldName,
dynamicSignal: signal ?? DYNAMIC_SIGNAL_ALL,
// `||` (not `??`): an empty "any" signal maps to All, same as unset.
dynamicSignal: signal || DYNAMIC_SIGNAL_ALL,
};
try {
await patchAsync(

View File

@@ -5,11 +5,13 @@ import type {
DashboardtypesPanelDTO,
DashboardtypesPanelPluginDTO,
} from 'api/generated/services/sigNoz.schemas';
import { normalizeOperator } from 'container/CreateAlertV2/context/conditionNormalizers';
import {
AlertThresholdMatchType,
AlertThresholdOperator,
Threshold,
} from 'container/CreateAlertV2/context/types';
import { THRESHOLD_COLOR_DANGER_ORDER } from 'pages/DashboardPageV2/DashboardContainer/Panels/types/threshold';
import type { MetricAggregation } from 'types/api/v5/queryRange';
import type { Query } from 'types/api/queryBuilder/queryBuilderData';
import { ReduceOperators } from 'types/common/queryBuilder';
@@ -21,14 +23,6 @@ export interface PanelAlertPrefill {
threshold?: Threshold;
}
// Most-dangerous first, matching the panel editor palette; unknown colors sort last.
const THRESHOLD_COLOR_DANGER_ORDER = [
'#f1575f',
'#f5b225',
'#2bb673',
'#4e74f8',
];
interface NormalizedPanelThreshold {
color: string;
value: number;
@@ -93,8 +87,12 @@ function readPanelThresholds(
}
}
// Match case-insensitively (picker emits lowercase hex); unknown colors sort last.
function colorRank(color: string): number {
const index = THRESHOLD_COLOR_DANGER_ORDER.indexOf(color.toLowerCase());
const target = color.toLowerCase();
const index = THRESHOLD_COLOR_DANGER_ORDER.findIndex(
(paletteColor) => paletteColor.toLowerCase() === target,
);
return index === -1 ? THRESHOLD_COLOR_DANGER_ORDER.length : index;
}
@@ -106,22 +104,17 @@ function pickHighestDanger(
)[0];
}
// The alert UI has no inclusive operator; collapse "or equal" onto its strict variant.
function panelOperatorToAlertOperator(
operator: DashboardtypesComparisonOperatorDTO | undefined,
): AlertThresholdOperator | undefined {
switch (operator) {
case 'above':
case 'above_or_equal':
return AlertThresholdOperator.IS_ABOVE;
case 'below':
return normalizeOperator('above');
case 'below_or_equal':
return AlertThresholdOperator.IS_BELOW;
case 'equal':
return AlertThresholdOperator.IS_EQUAL_TO;
case 'not_equal':
return AlertThresholdOperator.IS_NOT_EQUAL_TO;
return normalizeOperator('below');
default:
return undefined;
return normalizeOperator(operator);
}
}

View File

@@ -2,6 +2,7 @@ import { memo } from 'react';
import HeaderRightSection from 'components/HeaderRightSection/HeaderRightSection';
import DashboardPageBreadcrumbs from './DashboardPageBreadcrumbs';
import { useShareVariablesOption } from './useShareVariablesOption';
import styles from './DashboardPageHeader.module.scss';
@@ -14,10 +15,16 @@ function DashboardPageHeader({
title,
image,
}: DashboardPageHeaderProps): JSX.Element {
const shareVariablesOption = useShareVariablesOption();
return (
<div className={styles.dashboardPageHeader}>
<DashboardPageBreadcrumbs title={title} image={image} />
<HeaderRightSection enableAnnouncements={false} enableShare enableFeedback />
<HeaderRightSection
enableAnnouncements={false}
enableShare
enableFeedback
shareModalExtraOption={shareVariablesOption}
/>
</div>
);
}

View File

@@ -0,0 +1,41 @@
import { useMemo } from 'react';
import type { ShareURLExtraOption } from 'components/HeaderRightSection/ShareURLModal';
import type { SelectedVariableValue } from '../../VariablesBar/selectionTypes';
import {
ALL_SELECTED,
variablesUrlParser,
} from '../../VariablesBar/utils/variablesUrlState';
import { selectVariableValues } from '../../store/slices/variableSelectionSlice';
import { useDashboardStore } from '../../store/useDashboardStore';
/**
* The share-dialog "Include variables" option: serializes the current variable
* selection into the `?variables=` param (ALL encoded as the sentinel) so a shared
* link reproduces it for the recipient — who hydrates it into local storage on load,
* after which the param is cleared (see useSeedVariableSelection). Returns undefined
* when there is nothing selected to share.
*/
export function useShareVariablesOption(): ShareURLExtraOption | undefined {
const dashboardId = useDashboardStore((s) => s.dashboardId);
const selections = useDashboardStore(selectVariableValues(dashboardId ?? ''));
return useMemo(() => {
const names = Object.keys(selections);
if (names.length === 0) {
return undefined;
}
const urlShape: Record<string, SelectedVariableValue> = {};
names.forEach((name) => {
const selection = selections[name];
urlShape[name] = selection.allSelected ? ALL_SELECTED : selection.value;
});
const serialized = variablesUrlParser.serialize(urlShape);
return {
label: 'Include variables',
apply: (params): void => {
params.set('variables', serialized);
},
};
}, [selections]);
}

View File

@@ -0,0 +1,65 @@
import { act, renderHook } from '@testing-library/react';
import { NANO_SECOND_MULTIPLIER } from 'store/globalTime';
import { useCreatePanel } from '../useCreatePanel';
const mockSafeNavigate = jest.fn();
jest.mock('hooks/useSafeNavigate', () => ({
useSafeNavigate: (): { safeNavigate: jest.Mock } => ({
safeNavigate: mockSafeNavigate,
}),
}));
let mockGlobalTime = {
selectedTime: '30m',
minTime: 0,
maxTime: 0,
};
jest.mock('react-redux', () => ({
useSelector: (selector: (state: unknown) => unknown): unknown =>
selector({ globalTime: mockGlobalTime }),
}));
jest.mock('../../store/useDashboardStore', () => ({
useDashboardStore: (selector: (state: unknown) => unknown): unknown =>
selector({ dashboardId: 'dash-1' }),
}));
describe('useCreatePanel', () => {
beforeEach(() => {
jest.clearAllMocks();
mockGlobalTime = { selectedTime: '30m', minTime: 0, maxTime: 0 };
});
it('carries the relative time window onto the new-panel route', () => {
mockGlobalTime = { selectedTime: '6h', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useCreatePanel());
act(() => {
result.current.createPanel('timeSeries' as never, 2);
});
const [url] = mockSafeNavigate.mock.calls[0];
expect(url).toContain('/dashboard/dash-1/panel/new');
expect(url).toContain('panelKind=timeSeries');
expect(url).toContain('layoutIndex=2');
expect(url).toContain('relativeTime=6h');
});
it('carries a custom absolute window and never a stray relativeTime', () => {
mockGlobalTime = {
selectedTime: 'custom',
minTime: 1000 * NANO_SECOND_MULTIPLIER,
maxTime: 2000 * NANO_SECOND_MULTIPLIER,
};
const { result } = renderHook(() => useCreatePanel());
act(() => {
result.current.createPanel('timeSeries' as never, 2);
});
const [url] = mockSafeNavigate.mock.calls[0];
expect(url).toContain('startTime=1000');
expect(url).toContain('endTime=2000');
expect(url).not.toContain('relativeTime');
expect(url).not.toContain('&&');
});
});

View File

@@ -0,0 +1,95 @@
import { renderHook } from '@testing-library/react';
import { NANO_SECOND_MULTIPLIER } from 'store/globalTime';
import { useOpenPanelEditor } from '../useOpenPanelEditor';
const mockSafeNavigate = jest.fn();
jest.mock('hooks/useSafeNavigate', () => ({
useSafeNavigate: (): { safeNavigate: jest.Mock } => ({
safeNavigate: mockSafeNavigate,
}),
}));
let mockGlobalTime = {
selectedTime: '30m',
minTime: 0,
maxTime: 0,
};
jest.mock('react-redux', () => ({
useSelector: (selector: (state: unknown) => unknown): unknown =>
selector({ globalTime: mockGlobalTime }),
}));
jest.mock('../../store/useDashboardStore', () => ({
useDashboardStore: (selector: (state: unknown) => unknown): unknown =>
selector({ dashboardId: 'dash-1' }),
}));
describe('useOpenPanelEditor', () => {
beforeEach(() => {
jest.clearAllMocks();
mockGlobalTime = { selectedTime: '30m', minTime: 0, maxTime: 0 };
});
it('carries the relative time window into the editor route', () => {
mockGlobalTime = { selectedTime: '6h', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useOpenPanelEditor());
result.current('panel-9');
expect(mockSafeNavigate).toHaveBeenCalledWith(
'/dashboard/dash-1/panel/panel-9?relativeTime=6h',
undefined,
);
});
it('carries a custom absolute window as a start/end ms pair', () => {
mockGlobalTime = {
selectedTime: 'custom',
minTime: 1000 * NANO_SECOND_MULTIPLIER,
maxTime: 2000 * NANO_SECOND_MULTIPLIER,
};
const { result } = renderHook(() => useOpenPanelEditor());
result.current('panel-9');
const [url] = mockSafeNavigate.mock.calls[0];
expect(url).toContain('startTime=1000');
expect(url).toContain('endTime=2000');
// A custom range must not also carry relativeTime (it would win on the editor).
expect(url).not.toContain('relativeTime');
});
it('omits the query string for an uninitialized custom window', () => {
mockGlobalTime = { selectedTime: 'custom', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useOpenPanelEditor());
result.current('panel-9');
expect(mockSafeNavigate).toHaveBeenCalledWith(
'/dashboard/dash-1/panel/panel-9',
undefined,
);
});
it('forwards handoff state as router location state', () => {
mockGlobalTime = { selectedTime: '1h', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useOpenPanelEditor());
const handoffState = { editSpec: { title: 'x' } } as never;
result.current('panel-9', { handoffState });
expect(mockSafeNavigate).toHaveBeenCalledWith(
'/dashboard/dash-1/panel/panel-9?relativeTime=1h',
{ state: handoffState },
);
});
it('merges search with the time window (leading ? tolerated)', () => {
mockGlobalTime = { selectedTime: '6h', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useOpenPanelEditor());
result.current('new', { search: '?panelKind=timeSeries&layoutIndex=2' });
const [url] = mockSafeNavigate.mock.calls[0];
expect(url).toContain('/dashboard/dash-1/panel/new?');
expect(url).toContain('panelKind=timeSeries');
expect(url).toContain('layoutIndex=2');
expect(url).toContain('relativeTime=6h');
});
});

View File

@@ -0,0 +1,43 @@
import { renderHook } from '@testing-library/react';
import { NANO_SECOND_MULTIPLIER } from 'store/globalTime';
import { useTimeSearchParams } from '../useTimeSearchParams';
let mockGlobalTime = {
selectedTime: '30m',
minTime: 0,
maxTime: 0,
};
jest.mock('react-redux', () => ({
useSelector: (selector: (state: unknown) => unknown): unknown =>
selector({ globalTime: mockGlobalTime }),
}));
describe('useTimeSearchParams', () => {
it('returns a relativeTime query string for a relative selection', () => {
mockGlobalTime = { selectedTime: '6h', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useTimeSearchParams());
expect(result.current).toBe('relativeTime=6h');
});
it('returns an absolute ms pair for a custom selection', () => {
mockGlobalTime = {
selectedTime: 'custom',
minTime: 1000 * NANO_SECOND_MULTIPLIER,
maxTime: 2000 * NANO_SECOND_MULTIPLIER,
};
const { result } = renderHook(() => useTimeSearchParams());
expect(result.current).toContain('startTime=1000');
expect(result.current).toContain('endTime=2000');
expect(result.current).not.toContain('relativeTime');
});
it('returns an empty string for an uninitialized custom window', () => {
mockGlobalTime = { selectedTime: 'custom', minTime: 0, maxTime: 0 };
const { result } = renderHook(() => useTimeSearchParams());
expect(result.current).toBe('');
});
});

View File

@@ -1,11 +1,8 @@
import { useCallback, useState } from 'react';
import { generatePath } from 'react-router-dom';
import ROUTES from 'constants/routes';
import { useSafeNavigate } from 'hooks/useSafeNavigate';
import { newPanelSearch, NEW_PANEL_ID } from '../PanelEditor/newPanelRoute';
import type { PanelKind } from '../Panels/types/panelKind';
import { useDashboardStore } from '../store/useDashboardStore';
import { useOpenPanelEditor } from './useOpenPanelEditor';
interface UseCreatePanelResult {
isPickerOpen: boolean;
@@ -24,8 +21,7 @@ interface UseCreatePanelResult {
* until save.
*/
export function useCreatePanel(): UseCreatePanelResult {
const { safeNavigate } = useSafeNavigate();
const dashboardId = useDashboardStore((s) => s.dashboardId);
const openPanelEditor = useOpenPanelEditor();
const [isPickerOpen, setIsPickerOpen] = useState(false);
// Captured on open, consumed on select.
@@ -43,15 +39,12 @@ export function useCreatePanel(): UseCreatePanelResult {
const createPanel = useCallback(
(panelKind: PanelKind, targetIndex?: number): void => {
setIsPickerOpen(false);
const path = generatePath(ROUTES.DASHBOARD_PANEL_EDITOR, {
dashboardId,
panelId: NEW_PANEL_ID,
});
const target = targetIndex ?? layoutIndex;
// Variable selection is read from the persisted store, not the URL.
safeNavigate(`${path}${newPanelSearch(panelKind, target)}`);
openPanelEditor(NEW_PANEL_ID, {
search: newPanelSearch(panelKind, target),
});
},
[safeNavigate, dashboardId, layoutIndex],
[openPanelEditor, layoutIndex],
);
return {

View File

@@ -5,30 +5,39 @@ import { useSafeNavigate } from 'hooks/useSafeNavigate';
import type { PanelEditorHandoffState } from '../PanelEditor/panelEditorHandoff';
import { useDashboardStore } from '../store/useDashboardStore';
import { useTimeSearchParams } from './useTimeSearchParams';
/**
* Returns a callback that opens the V2 panel editor by navigating to its full-page route
* (`/dashboard/:dashboardId/panel/:panelId`). The dashboard id comes from the store, so any
* caller can open the editor with just the panel id. Variable selection is read from the
* persisted store (localStorage), not carried in the URL. The optional `handoffState` is
* passed as router location state — the View modal uses it to hand its drilldown-edited spec
* off to the editor (view → edit) so the editor opens on those edits rather than the saved
* panel.
*/
interface OpenPanelEditorOptions {
handoffState?: PanelEditorHandoffState;
/** Extra query merged into the editor URL (leading `?` optional). */
search?: string;
}
/** Opens the V2 panel editor, carrying the active time window in the URL. */
export function useOpenPanelEditor(): (
panelId: string,
handoffState?: PanelEditorHandoffState,
options?: OpenPanelEditorOptions,
) => void {
const { safeNavigate } = useSafeNavigate();
const timeSearch = useTimeSearchParams();
const dashboardId = useDashboardStore((s) => s.dashboardId);
return useCallback(
(panelId: string, handoffState?: PanelEditorHandoffState): void => {
(panelId: string, options?: OpenPanelEditorOptions): void => {
const path = generatePath(ROUTES.DASHBOARD_PANEL_EDITOR, {
dashboardId,
panelId,
});
const params = new URLSearchParams(options?.search);
new URLSearchParams(timeSearch).forEach((value, key) => {
params.set(key, value);
});
const search = params.toString();
safeNavigate(
generatePath(ROUTES.DASHBOARD_PANEL_EDITOR, { dashboardId, panelId }),
handoffState ? { state: handoffState } : undefined,
search ? `${path}?${search}` : path,
options?.handoffState ? { state: options.handoffState } : undefined,
);
},
[safeNavigate, dashboardId],
[safeNavigate, dashboardId, timeSearch],
);
}

View File

@@ -0,0 +1,20 @@
import { useMemo } from 'react';
// eslint-disable-next-line no-restricted-imports -- global time still lives in redux
import { useSelector } from 'react-redux';
import { AppState } from 'store/reducers';
import type { GlobalReducer } from 'types/reducer/globalTime';
import { timeParamsFromGlobalTime } from '../utils/timeUrlParams';
/** Active time window as a query string (no leading `?`), or `''` when unset. */
export function useTimeSearchParams(): string {
const { selectedTime, minTime, maxTime } = useSelector<
AppState,
GlobalReducer
>((state) => state.globalTime);
return useMemo(
() => timeParamsFromGlobalTime({ selectedTime, minTime, maxTime }).toString(),
[selectedTime, minTime, maxTime],
);
}

View File

@@ -0,0 +1,51 @@
import { NANO_SECOND_MULTIPLIER } from 'store/globalTime';
import { timeParamsFromGlobalTime } from '../timeUrlParams';
describe('timeParamsFromGlobalTime', () => {
it('emits relativeTime for a relative selection', () => {
const params = timeParamsFromGlobalTime({
selectedTime: '6h',
minTime: 0,
maxTime: 0,
});
expect(params.get('relativeTime')).toBe('6h');
// Mutually exclusive: no absolute pair alongside a relative range.
expect(params.has('startTime')).toBe(false);
expect(params.has('endTime')).toBe(false);
});
it('emits an absolute ms pair for a custom selection (converting from ns)', () => {
const params = timeParamsFromGlobalTime({
selectedTime: 'custom',
minTime: 1000 * NANO_SECOND_MULTIPLIER,
maxTime: 2000 * NANO_SECOND_MULTIPLIER,
});
expect(params.get('startTime')).toBe('1000');
expect(params.get('endTime')).toBe('2000');
// A custom range must not carry a relativeTime that would win on the editor.
expect(params.has('relativeTime')).toBe(false);
});
it('carries a custom shorthand relative selection verbatim', () => {
const params = timeParamsFromGlobalTime({
selectedTime: '13m',
minTime: 0,
maxTime: 0,
});
expect(params.get('relativeTime')).toBe('13m');
});
it('emits nothing for an uninitialized custom window', () => {
const params = timeParamsFromGlobalTime({
selectedTime: 'custom',
minTime: 0,
maxTime: 0,
});
expect(params.toString()).toBe('');
});
});

View File

@@ -0,0 +1,39 @@
import { QueryParams } from 'constants/query';
import { NANO_SECOND_MULTIPLIER } from 'store/globalTime';
import type { GlobalReducer } from 'types/reducer/globalTime';
type GlobalTimeSelection = Pick<
GlobalReducer,
'selectedTime' | 'minTime' | 'maxTime'
>;
/**
* Time-window URL params for the active selection. Derived from Redux (what the picker and
* panel queries read), not the URL: the legacy react-router and newer nuqs time writers fall
* out of sync, leaving a stale `relativeTime` that `DateTimeSelectionV2` prefers over an
* absolute range. Redux keeps them mutually exclusive (custom → start/end ms; else relativeTime).
*/
export function timeParamsFromGlobalTime({
selectedTime,
minTime,
maxTime,
}: GlobalTimeSelection): URLSearchParams {
const params = new URLSearchParams();
if (selectedTime === 'custom') {
if (minTime > 0 && maxTime > 0) {
params.set(
QueryParams.startTime,
String(Math.floor(minTime / NANO_SECOND_MULTIPLIER)),
);
params.set(
QueryParams.endTime,
String(Math.floor(maxTime / NANO_SECOND_MULTIPLIER)),
);
}
} else {
params.set(QueryParams.relativeTime, selectedTime);
}
return params;
}

View File

@@ -21,6 +21,7 @@ import {
parseNewPanelLayoutIndex,
} from '../DashboardContainer/PanelEditor/newPanelRoute';
import { useSyncVariablesForSuggestions } from '../DashboardContainer/hooks/useSyncVariablesForSuggestions';
import { useTimeSearchParams } from '../DashboardContainer/hooks/useTimeSearchParams';
import { createDefaultPanel } from '../DashboardContainer/patchOps';
import { useDashboardStore } from '../DashboardContainer/store/useDashboardStore';
import { useSeedVariableSelection } from '../DashboardContainer/VariablesBar/hooks/useSeedVariableSelection';
@@ -38,6 +39,7 @@ function PanelEditorPage(): JSX.Element {
}>();
const { search, state } = useLocation();
const { safeNavigate } = useSafeNavigate();
const timeSearch = useTimeSearchParams();
// Edits handed off from the View modal's drilldown — open the editor on these
// instead of the saved panel. Lost on refresh/new-tab, which falls back to saved.
@@ -105,10 +107,11 @@ function PanelEditorPage(): JSX.Element {
const layoutIndex = parseNewPanelLayoutIndex(search);
const backToDashboard = useCallback((): void => {
// Drop editor-only URL state (chiefly `compositeQuery`); the dashboard reads its
// variable selection from the persisted store, and time lives in Redux.
safeNavigate(generatePath(ROUTES.DASHBOARD, { dashboardId }));
}, [safeNavigate, dashboardId]);
// Drop editor-only URL state (variables come from the persisted store), but carry
// time so a custom range picked in the editor isn't reset to the dashboard default.
const path = generatePath(ROUTES.DASHBOARD, { dashboardId });
safeNavigate(timeSearch ? `${path}?${timeSearch}` : path);
}, [safeNavigate, dashboardId, timeSearch]);
if (isLoading) {
return <Spinner tip="Loading dashboard..." />;
@@ -137,6 +140,7 @@ function PanelEditorPage(): JSX.Element {
dashboardId={dashboardId}
panelId={panelId}
panel={panel}
savedPanel={existingPanel}
isNew={!!newKind}
layoutIndex={layoutIndex}
isEditable={isEditable}

View File

@@ -1,5 +1,20 @@
import { SelectProps } from 'antd';
import { ConfigProvider, SelectProps } from 'antd';
// eslint-disable-next-line no-restricted-imports
import { useContext } from 'react';
export const popupContainer: SelectProps['getPopupContainer'] = (
trigger,
): HTMLElement => trigger.parentNode;
/**
* Popup container for query-builder Selects. Prefers a container supplied by an
* ancestor antd `ConfigProvider` (set by hosts that render the builder inside a
* clipped/portaled surface — e.g. the panel editor's `overflow:hidden` resizable
* pane, or the View modal's focus-trapped dialog) and otherwise falls back to
* `trigger.parentNode`, the app-wide default. No `ConfigProvider` container is set
* app-wide, so surfaces that don't opt in keep the legacy behavior unchanged.
*/
export function useSelectPopupContainer(): SelectProps['getPopupContainer'] {
const { getPopupContainer } = useContext(ConfigProvider.ConfigContext);
return getPopupContainer ?? popupContainer;
}

View File

@@ -12,13 +12,15 @@ import (
var (
ErrCodeInvalidGlobalConfig = errors.MustNewCode("invalid_global_config")
ErrCodeOriginNotAllowed = errors.MustNewCode("origin_not_allowed")
)
type Config struct {
ExternalURL *url.URL `mapstructure:"external_url"`
IngestionURL *url.URL `mapstructure:"ingestion_url"`
MCPURL *url.URL `mapstructure:"mcp_url"`
AIAssistantURL *url.URL `mapstructure:"ai_assistant_url"`
ExternalURL *url.URL `mapstructure:"external_url"`
AllowedOrigins []*url.URL `mapstructure:"allowed_origins"`
IngestionURL *url.URL `mapstructure:"ingestion_url"`
MCPURL *url.URL `mapstructure:"mcp_url"`
AIAssistantURL *url.URL `mapstructure:"ai_assistant_url"`
}
func NewConfigFactory() factory.ConfigFactory {
@@ -49,9 +51,33 @@ func (c Config) Validate() error {
}
}
for _, origin := range c.AllowedOrigins {
if origin == nil || origin.Scheme == "" || origin.Host == "" {
return errors.NewInvalidInputf(ErrCodeInvalidGlobalConfig, "global::allowed_origins entries must be of the form scheme://host[:port], got %q", origin)
}
if origin.Path != "" && origin.Path != "/" {
return errors.NewInvalidInputf(ErrCodeInvalidGlobalConfig, "global::allowed_origins entries must not contain a path, got %q", origin)
}
}
return nil
}
func (c Config) IsOriginAllowed(u *url.URL) bool {
if len(c.AllowedOrigins) == 0 {
return true
}
for _, origin := range c.AllowedOrigins {
if strings.EqualFold(origin.Scheme, u.Scheme) && strings.EqualFold(origin.Host, u.Host) {
return true
}
}
return false
}
func (c Config) ExternalPath() string {
if c.ExternalURL == nil || c.ExternalURL.Path == "" || c.ExternalURL.Path == "/" {
return ""

View File

@@ -123,6 +123,26 @@ func TestValidate(t *testing.T) {
config: Config{ExternalURL: &url.URL{Path: "signoz"}},
fail: true,
},
{
name: "ValidAllowedOrigin",
config: Config{AllowedOrigins: []*url.URL{{Scheme: "https", Host: "signoz.example.com"}}},
fail: false,
},
{
name: "AllowedOriginWithoutScheme",
config: Config{AllowedOrigins: []*url.URL{{Host: "signoz.example.com"}}},
fail: true,
},
{
name: "AllowedOriginWithoutHost",
config: Config{AllowedOrigins: []*url.URL{{Scheme: "https"}}},
fail: true,
},
{
name: "AllowedOriginWithPath",
config: Config{AllowedOrigins: []*url.URL{{Scheme: "https", Host: "signoz.example.com", Path: "/login"}}},
fail: true,
},
}
for _, tc := range testCases {
@@ -137,3 +157,96 @@ func TestValidate(t *testing.T) {
})
}
}
func TestIsOriginAllowedWhenUnconfigured(t *testing.T) {
testCases := []struct {
name string
config Config
}{
{
name: "Empty",
config: Config{},
},
{
name: "ExternalURLDoesNotActivateValidation",
config: Config{ExternalURL: &url.URL{Scheme: "https", Host: "signoz.example.com"}},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
u, err := url.Parse("https://anything.example.com/login")
assert.NoError(t, err)
assert.True(t, tc.config.IsOriginAllowed(u))
})
}
}
func TestIsOriginAllowed(t *testing.T) {
config := Config{
AllowedOrigins: []*url.URL{
{Scheme: "https", Host: "signoz.example.com"},
{Scheme: "http", Host: "localhost:3301"},
},
}
testCases := []struct {
name string
input string
expected bool
}{
{
name: "ConfiguredOrigin",
input: "https://signoz.example.com/login",
expected: true,
},
{
name: "ConfiguredOriginWithQuery",
input: "http://localhost:3301/login?next=/dashboards",
expected: true,
},
{
name: "CaseInsensitiveHost",
input: "https://SigNoz.Example.Com/login",
expected: true,
},
{
name: "UnknownHost",
input: "https://attacker.example.com/login",
expected: false,
},
{
name: "SchemeMismatch",
input: "http://signoz.example.com/login",
expected: false,
},
{
name: "PortMismatch",
input: "https://signoz.example.com:8443/login",
expected: false,
},
{
name: "SuffixConfusion",
input: "https://evilsignoz.example.com/login",
expected: false,
},
{
name: "UserInfoConfusion",
input: "https://signoz.example.com@attacker.example.com/login",
expected: false,
},
{
name: "SchemeRelative",
input: "//attacker.example.com/login",
expected: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
u, err := url.Parse(tc.input)
assert.NoError(t, err)
assert.Equal(t, tc.expected, config.IsOriginAllowed(u))
})
}
}

View File

@@ -213,18 +213,18 @@ func (module *module) discoverModels(ctx context.Context, orgID valuer.UUID) ([]
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: fmt.Sprintf("%s EXISTS", llmpricingruletypes.GenAIRequestModel)},
Filter: &qbtypes.Filter{Expression: fmt.Sprintf("%s EXISTS", telemetrytypes.GenAIRequestModel)},
Aggregations: []qbtypes.TraceAggregation{
{Expression: "count()", Alias: "spanCount"},
},
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: llmpricingruletypes.GenAIRequestModel,
Name: telemetrytypes.GenAIRequestModel,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: llmpricingruletypes.GenAIProviderName,
Name: telemetrytypes.GenAIProviderName,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
@@ -254,9 +254,9 @@ func (module *module) discoverModels(ctx context.Context, orgID valuer.UUID) ([]
switch c.Type {
case qbtypes.ColumnTypeGroup:
switch c.Name {
case llmpricingruletypes.GenAIRequestModel:
case telemetrytypes.GenAIRequestModel:
modelIdx = i
case llmpricingruletypes.GenAIProviderName:
case telemetrytypes.GenAIProviderName:
providerIdx = i
}
case qbtypes.ColumnTypeAggregation:

View File

@@ -12,6 +12,7 @@ import (
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/modules/authdomain"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/session"
@@ -23,26 +24,28 @@ import (
)
type module struct {
settings factory.ScopedProviderSettings
authNs map[authtypes.AuthNProvider]authn.AuthN
userSetter user.Setter
userGetter user.Getter
authDomain authdomain.Module
tokenizer tokenizer.Tokenizer
orgGetter organization.Getter
authz authz.AuthZ
settings factory.ScopedProviderSettings
authNs map[authtypes.AuthNProvider]authn.AuthN
userSetter user.Setter
userGetter user.Getter
authDomain authdomain.Module
tokenizer tokenizer.Tokenizer
orgGetter organization.Getter
authz authz.AuthZ
globalConfig global.Config
}
func NewModule(providerSettings factory.ProviderSettings, authNs map[authtypes.AuthNProvider]authn.AuthN, userSetter user.Setter, userGetter user.Getter, authDomain authdomain.Module, tokenizer tokenizer.Tokenizer, orgGetter organization.Getter, authz authz.AuthZ) session.Module {
func NewModule(providerSettings factory.ProviderSettings, authNs map[authtypes.AuthNProvider]authn.AuthN, userSetter user.Setter, userGetter user.Getter, authDomain authdomain.Module, tokenizer tokenizer.Tokenizer, orgGetter organization.Getter, authz authz.AuthZ, globalConfig global.Config) session.Module {
return &module{
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/session/implsession"),
authNs: authNs,
userSetter: userSetter,
userGetter: userGetter,
authDomain: authDomain,
tokenizer: tokenizer,
orgGetter: orgGetter,
authz: authz,
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/session/implsession"),
authNs: authNs,
userSetter: userSetter,
userGetter: userGetter,
authDomain: authDomain,
tokenizer: tokenizer,
orgGetter: orgGetter,
authz: authz,
globalConfig: globalConfig,
}
}
@@ -140,6 +143,10 @@ func (module *module) CreateCallbackAuthNSession(ctx context.Context, authNProvi
return "", err
}
if callbackIdentity.State.URL.Host != "" && !module.globalConfig.IsOriginAllowed(callbackIdentity.State.URL) {
return "", errors.Newf(errors.TypeForbidden, global.ErrCodeOriginNotAllowed, "state redirect %q is not an allowed origin", callbackIdentity.State.URL.String())
}
authDomain, err := module.authDomain.GetByOrgIDAndID(ctx, callbackIdentity.OrgID, callbackIdentity.State.DomainID)
if err != nil {
return "", err
@@ -217,6 +224,10 @@ func (module *module) getOrgSessionContext(ctx context.Context, org *types.Organ
return nil, err
}
if !module.globalConfig.IsOriginAllowed(siteURL) {
return nil, errors.Newf(errors.TypeInvalidInput, global.ErrCodeOriginNotAllowed, "ref %q is not an allowed origin", siteURL.String())
}
loginURL, err := provider.LoginURL(ctx, siteURL, authDomain)
if err != nil {
return nil, err

View File

@@ -56,19 +56,21 @@ func (c *captureClient) Read(ctx context.Context, query *prompb.Query, _ bool) (
}
}
var metricName string
// Without executing the series lookup, only an exact-name selector's
// metric name is known.
var metricNames []string
for _, matcher := range query.Matchers {
if matcher.Name == "__name__" {
metricName = matcher.Value
if matcher.Name == "__name__" && matcher.Type == prompb.LabelMatcher_EQ {
metricNames = []string{matcher.Value}
}
}
// Build the executing path's queries, but only record them.
subQuery, args, err := c.queryToClickhouseQuery(ctx, query, metricName, true)
sub, err := seriesLookupQuery(query, true)
if err != nil {
return nil, err
}
samplesQuery, samplesArgs := buildSamplesQuery(int64(query.StartTimestampMs), int64(query.EndTimestampMs), metricName, subQuery, args)
samplesQuery, samplesArgs := buildSamplesQuery(int64(query.StartTimestampMs), int64(query.EndTimestampMs), metricNames, sub)
c.recorder.record(samplesQuery, samplesArgs)
return storage.EmptySeriesSet(), nil

View File

@@ -4,8 +4,7 @@ import (
"context"
"fmt"
"math"
"strconv"
"strings"
"sort"
"sync"
"github.com/SigNoz/signoz/pkg/errors"
@@ -15,6 +14,7 @@ import (
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/cespare/xxhash/v2"
"github.com/huandu/go-sqlbuilder"
promValue "github.com/prometheus/prometheus/model/value"
"github.com/prometheus/prometheus/prompb"
"github.com/prometheus/prometheus/storage"
@@ -56,19 +56,13 @@ func (client *client) Read(ctx context.Context, query *prompb.Query, sortSeries
}
}
var metricName string
for _, matcher := range query.Matchers {
if matcher.Name == "__name__" {
metricName = matcher.Value
}
}
clickhouseQuery, args, err := client.queryToClickhouseQuery(ctx, query, metricName, false)
lookup, err := seriesLookupQuery(query, false)
if err != nil {
return nil, err
}
lookupSQL, lookupArgs := lookup.BuildWithFlavor(sqlbuilder.ClickHouse)
fingerprints, err := client.getFingerprintsFromClickhouseQuery(ctx, clickhouseQuery, args)
fingerprints, metricNames, err := client.getFingerprintsFromClickhouseQuery(ctx, lookupSQL, lookupArgs)
if err != nil {
return nil, err
}
@@ -76,13 +70,14 @@ func (client *client) Read(ctx context.Context, query *prompb.Query, sortSeries
return remote.FromQueryResult(sortSeries, new(prompb.QueryResult)), nil
}
clickhouseSubQuery, args, err := client.queryToClickhouseQuery(ctx, query, metricName, true)
sub, err := seriesLookupQuery(query, true)
if err != nil {
return nil, err
}
samplesSQL, samplesArgs := buildSamplesQuery(int64(query.StartTimestampMs), int64(query.EndTimestampMs), metricNames, sub)
res := new(prompb.QueryResult)
timeseries, err := client.querySamples(ctx, int64(query.StartTimestampMs), int64(query.EndTimestampMs), fingerprints, metricName, clickhouseSubQuery, args)
timeseries, err := client.querySamples(ctx, samplesSQL, samplesArgs, fingerprints)
if err != nil {
return nil, err
}
@@ -126,86 +121,115 @@ func (c *client) ReadMultiple(ctx context.Context, queries []*prompb.Query, sort
return storage.NewMergeSeriesSet(sets, 0, storage.ChainedSeriesMerge), nil
}
func (client *client) queryToClickhouseQuery(_ context.Context, query *prompb.Query, metricName string, subQuery bool) (string, []any, error) {
var clickHouseQuery string
var conditions []string
var argCount = 0
var selectString = "fingerprint, any(labels)"
// anchorRegex makes a pattern fully anchored, the way Prometheus compiles
// matcher regexes; ClickHouse's match() would otherwise substring-match.
func anchorRegex(pattern string) string {
return "^(?:" + pattern + ")$"
}
// seriesLookupQuery builds the time-series lookup. It returns a builder so
// the samples query can embed it as a subquery with the args merged in
// render order by the builder instead of hand-numbered placeholders.
func seriesLookupQuery(query *prompb.Query, subQuery bool) (*sqlbuilder.SelectBuilder, error) {
sb := sqlbuilder.NewSelectBuilder()
if subQuery {
argCount = 1
selectString = "fingerprint"
sb.Select("fingerprint")
} else {
sb.Select("fingerprint", "any(labels)")
}
start, end, tableName := getStartAndEndAndTableName(query.StartTimestampMs, query.EndTimestampMs)
sb.From(databaseName + "." + tableName)
var args []any
conditions = append(conditions, fmt.Sprintf("metric_name = $%d", argCount+1))
conditions = append(conditions, "temporality IN ['Cumulative', 'Unspecified']")
sb.Where("temporality IN ['Cumulative', 'Unspecified']")
// Inclusive upper bound: registration rows are hour-floored by the
// exporter, so a series first registered in the hour starting exactly at
// `end` would otherwise be invisible while its samples (<= end) are in
// range.
conditions = append(conditions, fmt.Sprintf("unix_milli >= %d AND unix_milli <= %d", start, end))
sb.Where(fmt.Sprintf("unix_milli >= %d AND unix_milli <= %d", start, end))
args = append(args, metricName)
for _, m := range query.Matchers {
if m.Name == "__name__" {
// __name__ maps onto the metric_name column per matcher type;
// reducing regex/negated/absent name matchers to one equality
// made such selectors silently return empty.
switch m.Type {
case prompb.LabelMatcher_EQ:
sb.Where(sb.E("metric_name", m.Value))
case prompb.LabelMatcher_NEQ:
sb.Where(sb.NE("metric_name", m.Value))
case prompb.LabelMatcher_RE:
sb.Where(fmt.Sprintf("match(metric_name, %s)", sb.Var(anchorRegex(m.Value))))
case prompb.LabelMatcher_NRE:
sb.Where(fmt.Sprintf("not match(metric_name, %s)", sb.Var(anchorRegex(m.Value))))
default:
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported or invalid matcher type: %s", m.Type.String())
}
continue
}
switch m.Type {
case prompb.LabelMatcher_EQ:
conditions = append(conditions, fmt.Sprintf("JSONExtractString(labels, $%d) = $%d", argCount+2, argCount+3))
sb.Where(fmt.Sprintf("JSONExtractString(labels, %s) = %s", sb.Var(m.Name), sb.Var(m.Value)))
case prompb.LabelMatcher_NEQ:
conditions = append(conditions, fmt.Sprintf("JSONExtractString(labels, $%d) != $%d", argCount+2, argCount+3))
sb.Where(fmt.Sprintf("JSONExtractString(labels, %s) != %s", sb.Var(m.Name), sb.Var(m.Value)))
case prompb.LabelMatcher_RE:
conditions = append(conditions, fmt.Sprintf("match(JSONExtractString(labels, $%d), $%d)", argCount+2, argCount+3))
sb.Where(fmt.Sprintf("match(JSONExtractString(labels, %s), %s)", sb.Var(m.Name), sb.Var(anchorRegex(m.Value))))
case prompb.LabelMatcher_NRE:
conditions = append(conditions, fmt.Sprintf("not match(JSONExtractString(labels, $%d), $%d)", argCount+2, argCount+3))
sb.Where(fmt.Sprintf("not match(JSONExtractString(labels, %s), %s)", sb.Var(m.Name), sb.Var(anchorRegex(m.Value))))
default:
return "", nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported or invalid matcher type: %s", m.Type.String())
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported or invalid matcher type: %s", m.Type.String())
}
args = append(args, m.Name, m.Value)
argCount += 2
}
whereClause := strings.Join(conditions, " AND ")
clickHouseQuery = fmt.Sprintf(`SELECT %s FROM %s.%s WHERE %s GROUP BY fingerprint`, selectString, databaseName, tableName, whereClause)
return clickHouseQuery, args, nil
sb.GroupBy("fingerprint")
return sb, nil
}
func (client *client) getFingerprintsFromClickhouseQuery(ctx context.Context, query string, args []any) (map[uint64][]prompb.Label, error) {
func (client *client) getFingerprintsFromClickhouseQuery(ctx context.Context, query string, args []any) (map[uint64][]prompb.Label, []string, error) {
ctx = client.withClickhousePrometheusContext(ctx, "getFingerprintsFromClickhouseQuery")
rows, err := client.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, err
return nil, nil, err
}
defer rows.Close()
fingerprints := make(map[uint64][]prompb.Label)
nameSet := make(map[string]struct{})
var fingerprint uint64
var labelString string
for rows.Next() {
if err = rows.Scan(&fingerprint, &labelString); err != nil {
return nil, err
return nil, nil, err
}
labels, _, err := unmarshalLabels(labelString)
labels, metricName, err := unmarshalLabels(labelString)
if err != nil {
return nil, err
return nil, nil, err
}
fingerprints[fingerprint] = labels
if metricName != "" {
nameSet[metricName] = struct{}{}
}
}
if err := rows.Err(); err != nil {
return nil, err
return nil, nil, err
}
return fingerprints, nil
metricNames := make([]string, 0, len(nameSet))
for name := range nameSet {
metricNames = append(metricNames, name)
}
sort.Strings(metricNames)
return fingerprints, metricNames, nil
}
// buildSamplesQuery renders the samples SQL (and args) that fetches data
// points for the series selected by subQuery.
// buildSamplesQuery renders the samples SQL for the series selected by
// subQuery. The metric_name condition exists only for primary-key pruning;
// the fingerprint filter already selects the right rows.
//
// Time bounds are inclusive on both ends because that is Prometheus's
// storage contract: Select(mint, maxt) returns [start, end] and the engine
@@ -215,27 +239,29 @@ func (client *client) getFingerprintsFromClickhouseQuery(ctx context.Context, qu
// its own model — toStartOfInterval buckets covering [t, t+step), where a
// sample at `end` falls in an unrendered bucket and end-exclusive ranges
// tile exactly across cached time slices.
func buildSamplesQuery(start int64, end int64, metricName string, subQuery string, args []any) (string, []any) {
argCount := len(args)
func buildSamplesQuery(start int64, end int64, metricNames []string, sub *sqlbuilder.SelectBuilder) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", "fingerprint", "unix_milli", "value", "flags")
sb.From(databaseName + "." + distributedSamplesV4)
query := fmt.Sprintf(`
SELECT metric_name, fingerprint, unix_milli, value, flags
FROM %s.%s
WHERE metric_name = $1 AND fingerprint GLOBAL IN (%s) AND unix_milli >= $%s AND unix_milli <= $%s ORDER BY fingerprint, unix_milli;`,
databaseName, distributedSamplesV4, subQuery, strconv.Itoa(argCount+2), strconv.Itoa(argCount+3))
query = strings.TrimSpace(query)
if len(metricNames) > 0 {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
sb.Where(sb.In("metric_name", names...))
}
sb.Where(fmt.Sprintf("fingerprint GLOBAL IN (%s)", sb.Var(sub)))
sb.Where(sb.GTE("unix_milli", start), sb.LTE("unix_milli", end))
sb.OrderBy("fingerprint", "unix_milli")
allArgs := append([]any{metricName}, args...)
allArgs = append(allArgs, start, end)
return query, allArgs
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
func (client *client) querySamples(ctx context.Context, start int64, end int64, fingerprints map[uint64][]prompb.Label, metricName string, subQuery string, args []any) ([]*prompb.TimeSeries, error) {
func (client *client) querySamples(ctx context.Context, query string, args []any, fingerprints map[uint64][]prompb.Label) ([]*prompb.TimeSeries, error) {
ctx = client.withClickhousePrometheusContext(ctx, "querySamples")
query, allArgs := buildSamplesQuery(start, end, metricName, subQuery, args)
rows, err := client.telemetryStore.ClickhouseDB().Query(ctx, query, allArgs...)
rows, err := client.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, err
}
@@ -243,6 +269,7 @@ func (client *client) querySamples(ctx context.Context, start int64, end int64,
var res []*prompb.TimeSeries
var ts *prompb.TimeSeries
var metricName string
var fingerprint, prevFingerprint uint64
var timestampMs, prevTimestamp int64
var value float64

View File

@@ -11,6 +11,7 @@ import (
"github.com/DATA-DOG/go-sqlmock"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/huandu/go-sqlbuilder"
"github.com/prometheus/prometheus/prompb"
"github.com/stretchr/testify/assert"
)
@@ -29,7 +30,7 @@ func TestClient_QuerySamples(t *testing.T) {
start int64
end int64
fingerprints map[uint64][]prompb.Label
metricName string
metricNames []string
subQuery string
args []any
setupMock func(mock cmock.ClickConnMockCommon, args ...any)
@@ -52,7 +53,7 @@ func TestClient_QuerySamples(t *testing.T) {
{Name: "instance", Value: "localhost:9091"},
},
},
metricName: "cpu_usage",
metricNames: []string{"cpu_usage"},
subQuery: "SELECT metric_name, fingerprint, unix_milli, value, flags",
expectedTimeSeries: 2,
expectError: false,
@@ -97,10 +98,10 @@ func TestClient_QuerySamples(t *testing.T) {
telemetryStore := telemetrystoretest.New(telemetrystore.Config{Provider: "clickhouse"}, sqlmock.QueryMatcherRegexp)
readClient := client{telemetryStore: telemetryStore}
if tt.setupMock != nil {
tt.setupMock(telemetryStore.Mock(), tt.metricName, tt.start, tt.end)
tt.setupMock(telemetryStore.Mock(), "cpu_usage", tt.start, tt.end)
}
result, err := readClient.querySamples(ctx, tt.start, tt.end, tt.fingerprints, tt.metricName, tt.subQuery, tt.args)
result, err := readClient.querySamples(ctx, tt.subQuery, []any{"cpu_usage", tt.start, tt.end}, tt.fingerprints)
if tt.expectError {
assert.Error(t, err)
@@ -115,101 +116,6 @@ func TestClient_QuerySamples(t *testing.T) {
}
}
func TestClient_getFingerprintsFromClickhouseQuery(t *testing.T) {
cols := []cmock.ColumnType{
{Name: "fingerprint", Type: "UInt64"},
{Name: "labels", Type: "String"},
}
sortLabels := func(ls []prompb.Label) {
sort.Slice(ls, func(i, j int) bool {
if ls[i].Name == ls[j].Name {
return ls[i].Value < ls[j].Value
}
return ls[i].Name < ls[j].Name
})
}
tests := []struct {
name string
start, end int64
metricName string
subQuery string
args []any
setupMock func(m cmock.ClickConnMockCommon, args ...any)
want map[uint64][]prompb.Label
wantErr bool
}{
{
name: "happy-path - two fingerprints",
start: 1000,
end: 2000,
metricName: "cpu_usage",
subQuery: `SELECT fingerprint,labels`,
// args slice is empty here, but testcase still owns it
args: []any{},
setupMock: func(m cmock.ClickConnMockCommon, args ...any) {
rows := [][]any{
{uint64(123), `{"t1":"s1","t2":"s2"}`},
{uint64(234), `{"t1":"s1","t2":"s2","empty":""}`},
}
m.ExpectQuery(`SELECT fingerprint,labels`).WithArgs(args...).WillReturnRows(
cmock.NewRows(cols, rows),
)
},
// No synthetic fingerprint label (#8563), empty-valued labels
// dropped: both fingerprints present one labelset for
// querySamples to merge.
want: map[uint64][]prompb.Label{
123: {
{Name: "t1", Value: "s1"},
{Name: "t2", Value: "s2"},
},
234: {
{Name: "t1", Value: "s1"},
{Name: "t2", Value: "s2"},
},
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
ctx := context.Background()
store := telemetrystoretest.New(
telemetrystore.Config{Provider: "clickhouse"},
sqlmock.QueryMatcherRegexp,
)
if tc.setupMock != nil {
tc.setupMock(store.Mock(), tc.args...)
}
c := client{telemetryStore: store}
got, err := c.getFingerprintsFromClickhouseQuery(ctx, tc.subQuery, tc.args)
if tc.wantErr {
require.Error(t, err)
require.Nil(t, got)
return
}
require.NoError(t, err)
require.Equal(t, len(tc.want), len(got), "fingerprint map length mismatch")
for fp, expLabels := range tc.want {
gotLabels, ok := got[fp]
require.Truef(t, ok, "missing fingerprint %d", fp)
sortLabels(expLabels)
sortLabels(gotLabels)
assert.Equalf(t, expLabels, gotLabels, "labels mismatch for fingerprint %d", fp)
}
})
}
}
// Regression for the duplicate-series class behind #8563: fingerprints
// sharing one labelset must come back as one merged series, the higher
// fingerprint winning equal timestamps.
@@ -251,7 +157,7 @@ func TestClient_QuerySamplesMergesIdenticalLabelSets(t *testing.T) {
WillReturnRows(cmock.NewRows(cols, values))
readClient := client{telemetryStore: telemetryStore}
result, err := readClient.querySamples(ctx, 1000, 3000, fingerprints, "requests", "SELECT metric_name, fingerprint, unix_milli, value, flags", nil)
result, err := readClient.querySamples(ctx, "SELECT metric_name, fingerprint, unix_milli, value, flags", []any{"requests", int64(1000), int64(3000)}, fingerprints)
require.NoError(t, err)
assert.Equal(t, []*prompb.TimeSeries{
@@ -272,6 +178,188 @@ func TestClient_QuerySamplesMergesIdenticalLabelSets(t *testing.T) {
}, result)
}
func TestClient_getFingerprintsFromClickhouseQuery(t *testing.T) {
cols := []cmock.ColumnType{
{Name: "fingerprint", Type: "UInt64"},
{Name: "labels", Type: "String"},
}
sortLabels := func(ls []prompb.Label) {
sort.Slice(ls, func(i, j int) bool {
if ls[i].Name == ls[j].Name {
return ls[i].Value < ls[j].Value
}
return ls[i].Name < ls[j].Name
})
}
tests := []struct {
name string
start, end int64
metricName string
subQuery string
args []any
setupMock func(m cmock.ClickConnMockCommon, args ...any)
want map[uint64][]prompb.Label
wantNames []string
wantErr bool
}{
{
name: "happy-path - two fingerprints",
start: 1000,
end: 2000,
metricName: "cpu_usage",
subQuery: `SELECT fingerprint,labels`,
// args slice is empty here, but testcase still owns it
args: []any{},
setupMock: func(m cmock.ClickConnMockCommon, args ...any) {
rows := [][]any{
{uint64(123), `{"__name__":"cpu_usage","t1":"s1","t2":"s2"}`},
{uint64(234), `{"__name__":"cpu_usage","t1":"s1","t2":"s2","empty":""}`},
}
m.ExpectQuery(`SELECT fingerprint,labels`).WithArgs(args...).WillReturnRows(
cmock.NewRows(cols, rows),
)
},
// No synthetic fingerprint label (#8563), empty-valued labels
// dropped: both fingerprints present one labelset for
// querySamples to merge.
want: map[uint64][]prompb.Label{
123: {
{Name: "__name__", Value: "cpu_usage"},
{Name: "t1", Value: "s1"},
{Name: "t2", Value: "s2"},
},
234: {
{Name: "__name__", Value: "cpu_usage"},
{Name: "t1", Value: "s1"},
{Name: "t2", Value: "s2"},
},
},
wantNames: []string{"cpu_usage"},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
ctx := context.Background()
store := telemetrystoretest.New(
telemetrystore.Config{Provider: "clickhouse"},
sqlmock.QueryMatcherRegexp,
)
if tc.setupMock != nil {
tc.setupMock(store.Mock(), tc.args...)
}
c := client{telemetryStore: store}
got, gotNames, err := c.getFingerprintsFromClickhouseQuery(ctx, tc.subQuery, tc.args)
if tc.wantErr {
require.Error(t, err)
require.Nil(t, got)
return
}
require.NoError(t, err)
assert.Equal(t, tc.wantNames, gotNames, "discovered metric names mismatch")
require.Equal(t, len(tc.want), len(got), "fingerprint map length mismatch")
for fp, expLabels := range tc.want {
gotLabels, ok := got[fp]
require.Truef(t, ok, "missing fingerprint %d", fp)
sortLabels(expLabels)
sortLabels(gotLabels)
assert.Equalf(t, expLabels, gotLabels, "labels mismatch for fingerprint %d", fp)
}
})
}
}
// Regression for nameless/regex-name selectors silently returning empty:
// the old code reduced every __name__ matcher to `metric_name = <value>`
// (empty string when absent). Regexes must come out anchored — Prometheus
// matcher semantics, while ClickHouse match() substring-matches.
func TestQueryToClickhouseQueryNameMatchers(t *testing.T) {
query := func(matchers ...*prompb.LabelMatcher) *prompb.Query {
return &prompb.Query{StartTimestampMs: 0, EndTimestampMs: 1000, Matchers: matchers}
}
tests := []struct {
name string
query *prompb.Query
contains []string
absent []string
args []any
}{
{
name: "exact name",
query: query(&prompb.LabelMatcher{Type: prompb.LabelMatcher_EQ, Name: "__name__", Value: "cpu_usage"}),
contains: []string{"metric_name = ?"},
args: []any{"cpu_usage"},
},
{
name: "regex name is anchored",
query: query(&prompb.LabelMatcher{Type: prompb.LabelMatcher_RE, Name: "__name__", Value: ".+"}),
contains: []string{"match(metric_name, ?)"},
args: []any{"^(?:.+)$"},
},
{
name: "nameless selector has no metric_name condition",
query: query(
&prompb.LabelMatcher{Type: prompb.LabelMatcher_EQ, Name: "job", Value: "api"},
&prompb.LabelMatcher{Type: prompb.LabelMatcher_NRE, Name: "group", Value: "can.*"},
),
contains: []string{
"JSONExtractString(labels, ?) = ?",
"not match(JSONExtractString(labels, ?), ?)",
},
absent: []string{"metric_name"},
args: []any{"job", "api", "group", "^(?:can.*)$"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
lookup, err := seriesLookupQuery(tt.query, false)
require.NoError(t, err)
sql, args := lookup.BuildWithFlavor(sqlbuilder.ClickHouse)
for _, want := range tt.contains {
assert.Contains(t, sql, want)
}
for _, notWant := range tt.absent {
assert.NotContains(t, sql, notWant)
}
assert.Equal(t, tt.args, args)
})
}
}
// The samples query narrows by the metric names the lookup discovered and
// embeds the series lookup as a subquery, the builder merging its args in
// render order.
func TestBuildSamplesQueryMetricNames(t *testing.T) {
sub := sqlbuilder.NewSelectBuilder()
sub.Select("fingerprint")
sub.From("t")
sub.Where(sub.E("k", "v"))
sql, args := buildSamplesQuery(5, 9, []string{"a_total", "b_total"}, sub)
assert.Contains(t, sql, "metric_name IN (?, ?)")
assert.Contains(t, sql, "fingerprint GLOBAL IN (SELECT fingerprint FROM t WHERE k = ?)")
assert.Contains(t, sql, "unix_milli >= ? AND unix_milli <= ?")
assert.Equal(t, []any{"a_total", "b_total", "v", int64(5), int64(9)}, args)
sub2 := sqlbuilder.NewSelectBuilder()
sub2.Select("fingerprint")
sub2.From("t")
sql, args = buildSamplesQuery(5, 9, nil, sub2)
assert.NotContains(t, sql, "metric_name IN")
assert.Equal(t, []any{int64(5), int64(9)}, args)
}
// Hash grouping must stay order-insensitive (stored JSON key order is not
// canonical across fingerprints), and a 64-bit hash collision between
// distinct labelsets must not merge them — splitByLabelSet is that guard.

View File

@@ -242,7 +242,7 @@ func (handler *handler) ReplaceVariables(rw http.ResponseWriter, req *http.Reque
errs := []error{}
for idx, item := range queryRangeRequest.CompositeQuery.Queries {
if item.Type == qbtypes.QueryTypeBuilder {
if item.Type == qbtypes.QueryTypeBuilder || item.Type == qbtypes.QueryTypeBuilderAI {
switch spec := item.Spec.(type) {
case qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]:
if spec.Filter != nil && spec.Filter.Expression != "" {

View File

@@ -249,6 +249,7 @@ func (q *querier) buildPreviewProviders(
func rendersStandaloneStatement(t qbtypes.QueryType) bool {
switch t {
case qbtypes.QueryTypeBuilder,
qbtypes.QueryTypeBuilderAI,
qbtypes.QueryTypePromQL,
qbtypes.QueryTypeClickHouseSQL,
qbtypes.QueryTypeTraceOperator:

View File

@@ -42,6 +42,7 @@ type querier struct {
metadataStore telemetrytypes.MetadataStore
promEngine prometheus.Prometheus
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation]
@@ -61,6 +62,7 @@ func New(
metadataStore telemetrytypes.MetadataStore,
promEngine prometheus.Prometheus,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
@@ -82,6 +84,7 @@ func New(
metadataStore: metadataStore,
promEngine: promEngine,
traceStmtBuilder: traceStmtBuilder,
aiTraceStmtBuilder: aiTraceStmtBuilder,
logStmtBuilder: logStmtBuilder,
auditStmtBuilder: auditStmtBuilder,
metricStmtBuilder: metricStmtBuilder,
@@ -232,6 +235,16 @@ func (q *querier) buildQueries(
}
queries[traceOpQuery.Name] = toq
steps[traceOpQuery.Name] = traceOpQuery.StepInterval
case qbtypes.QueryTypeBuilderAI:
spec, ok := query.Spec.(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation])
if !ok {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid AI builder query spec %T", query.Spec)
}
spec.ShiftBy = extractShiftFromBuilderQuery(spec)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, req.RequestType)
bq := newBuilderQuery(q.logger, q.telemetryStore, orgID, q.aiTraceStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
queries[spec.Name] = bq
steps[spec.Name] = spec.StepInterval
case qbtypes.QueryTypeBuilder:
switch spec := query.Spec.(type) {
case qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]:
@@ -298,6 +311,11 @@ func (q *querier) populateQBEvent(event *qbtypes.QBEvent, queries []qbtypes.Quer
case qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]:
event.MetricsUsed = true
}
case qbtypes.QueryTypeBuilderAI:
filter := query.GetFilter()
event.FilterApplied = event.FilterApplied || (filter != nil && filter.Expression != "")
event.GroupByApplied = event.GroupByApplied || len(query.GetGroupBy()) > 0
event.TracesUsed = true
case qbtypes.QueryTypePromQL:
event.MetricsUsed = true
case qbtypes.QueryTypeTraceOperator:
@@ -860,7 +878,9 @@ func (q *querier) createRangedQuery(_ valuer.UUID, originalQuery qbtypes.Query,
specCopy := qt.spec.Copy()
specCopy.ShiftBy = extractShiftFromBuilderQuery(specCopy)
adjustedTimeRange := adjustTimeRangeForShift(specCopy, timeRange, qt.kind)
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, q.traceStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
// Reuse the statement builder the original query was created with, so an AI
// query keeps its AI builder without re-deriving it from the spec.
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, qt.stmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
case *builderQuery[qbtypes.LogAggregation]:
specCopy := qt.spec.Copy()
@@ -1217,6 +1237,8 @@ func (q *querier) adjustStepInterval(queries []qbtypes.QueryEnvelope, start, end
if qe.GetStepInterval().Seconds() == 0 {
qe.SetStepInterval(secondsStep(metricRecommended))
}
case qbtypes.QueryTypeBuilderAI:
clampStep(qe, traceLogRecommended, traceLogMin, &warnings)
case qbtypes.QueryTypeTraceOperator:
clampStep(qe, traceLogRecommended, traceLogMin, &warnings)
}

View File

@@ -49,6 +49,7 @@ func TestQueryRange_MetricTypeMissing(t *testing.T) {
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
@@ -121,6 +122,7 @@ func TestQueryRange_MetricTypeFromStore(t *testing.T) {
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
&mockMetricStmtBuilder{}, // metricStmtBuilder

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryai"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
@@ -92,6 +93,17 @@ func newProvider(
cfg.SkipResourceFingerprint.Threshold,
)
// AI trace statement builder (builder_ai_query). The gen_ai gate/column keys are
// surfaced by the metadata store itself (enrichWithGenAIKeys), so queries work
// before any gen_ai metadata is ingested — no per-builder decoration needed.
// The standard trace builder doubles as the delegate for the span-list path.
aiTraceStmtBuilder := telemetryai.NewAITraceStatementBuilder(
settings,
telemetryMetadataStore,
traceStmtBuilder,
flagger,
)
// Create trace operator statement builder
traceOperatorStmtBuilder := telemetrytraces.NewTraceOperatorStatementBuilder(
settings,
@@ -181,6 +193,7 @@ func newProvider(
telemetryMetadataStore,
prometheus,
traceStmtBuilder,
aiTraceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,

View File

@@ -237,7 +237,7 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
// Mock the fingerprint query (for Prometheus label matching)
mock.ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(fingerprintRows)
// Mock the samples query (for Prometheus metric data)
@@ -245,8 +245,6 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
WithArgs(
"test_metric",
"test_metric",
"__name__",
"test_metric",
queryStart,
queryEnd,
).

View File

@@ -925,20 +925,18 @@ func TestPromRuleUnitCombinations(t *testing.T) {
}
samplesRows := cmock.NewRows(samplesCols, samplesData)
// args: $1=metric_name, $2=label_name, $3=label_value
// args: $1=metric_name (the __name__ matcher maps onto the column)
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(fingerprintRows)
// args: $1=metric_name (outer), $2=metric_name (subquery), $3=label_name, $4=label_value, $5=start, $6=end
// args: $1=metric_name IN (discovered names), $2=metric_name (subquery), $3=start, $4=end
telemetryStore.Mock().
ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs(
"test_metric",
"test_metric",
"__name__",
"test_metric",
queryStart,
queryEnd,
).
@@ -1063,7 +1061,7 @@ func TestPromRuleNoData(t *testing.T) {
// no rows == no data
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(fingerprintRows)
promProvider := prometheustest.New(context.Background(), instrumentationtest.New().ToProviderSettings(), prometheus.Config{Timeout: 2 * time.Minute}, telemetryStore)
@@ -1273,7 +1271,7 @@ func TestMultipleThresholdPromRule(t *testing.T) {
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(fingerprintRows)
telemetryStore.Mock().
@@ -1281,8 +1279,6 @@ func TestMultipleThresholdPromRule(t *testing.T) {
WithArgs(
"test_metric",
"test_metric",
"__name__",
"test_metric",
queryStart,
queryEnd,
).
@@ -1439,12 +1435,12 @@ func TestPromRule_NoData(t *testing.T) {
labelsJSON := `{"__name__":"test_metric"}`
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(cmock.NewRows(fingerprintCols, [][]any{{fingerprint, labelsJSON}}))
telemetryStore.Mock().
ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs("test_metric", "test_metric", "__name__", "test_metric", queryStart, queryEnd).
WithArgs("test_metric", "test_metric", queryStart, queryEnd).
WillReturnRows(cmock.NewRows(samplesCols, [][]any{}))
promProvider := prometheustest.New(
@@ -1575,11 +1571,11 @@ func TestPromRule_NoData_AbsentFor(t *testing.T) {
queryStart1, queryEnd1 := calcQueryRange(t1)
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(cmock.NewRows(fingerprintCols, [][]any{{fingerprint, labelsJSON}}))
telemetryStore.Mock().
ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs("test_metric", "test_metric", "__name__", "test_metric", queryStart1, queryEnd1).
WithArgs("test_metric", "test_metric", queryStart1, queryEnd1).
WillReturnRows(cmock.NewRows(samplesCols, [][]any{
// Data points in the past relative to t1
{"test_metric", fingerprint, baseTime.UnixMilli(), 100.0, uint32(0)},
@@ -1591,11 +1587,11 @@ func TestPromRule_NoData_AbsentFor(t *testing.T) {
queryStart2, queryEnd2 := calcQueryRange(t2)
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(cmock.NewRows(fingerprintCols, [][]any{{fingerprint, labelsJSON}}))
telemetryStore.Mock().
ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs("test_metric", "test_metric", "__name__", "test_metric", queryStart2, queryEnd2).
WithArgs("test_metric", "test_metric", queryStart2, queryEnd2).
WillReturnRows(cmock.NewRows(samplesCols, [][]any{})) // empty - no data
promProvider := prometheustest.New(
@@ -1752,11 +1748,11 @@ func TestPromRuleEval_RequireMinPoints(t *testing.T) {
telemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &queryMatcherAny{})
telemetryStore.Mock().
ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric", "__name__", "test_metric").
WithArgs("test_metric").
WillReturnRows(cmock.NewRows(fingerprintCols, fingerprintData))
telemetryStore.Mock().
ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs("test_metric", "test_metric", "__name__", "test_metric", queryStart, queryEnd).
WithArgs("test_metric", "test_metric", queryStart, queryEnd).
WillReturnRows(cmock.NewRows(samplesCols, samplesData))
promProvider := prometheustest.New(
context.Background(),

View File

@@ -48,6 +48,7 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
metricStmtBuilder,
@@ -101,6 +102,7 @@ func prepareQuerierForLogs(t *testing.T, telemetryStore telemetrystore.Telemetry
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
logStmtBuilder, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
@@ -150,6 +152,7 @@ func prepareQuerierForTraces(t *testing.T, telemetryStore telemetrystore.Telemet
metadataStore,
nil, // prometheus
traceStmtBuilder, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder

View File

@@ -181,6 +181,15 @@ func DataTypeCollisionHandledFieldName(key *telemetrytypes.TelemetryFieldKey, va
// dynamic array elements will be default casted to string
tblFieldName, value = castString(tblFieldName), toStrings(v)
}
case telemetrytypes.FieldDataTypeUnspecified:
if operator == qbtypes.FilterOperatorUnknown {
switch value.(type) {
case float32, float64, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, json.Number:
tblFieldName = accurateCastFloat(tblFieldName)
case string:
tblFieldName = castString(tblFieldName)
}
}
}
return tblFieldName, value
}

View File

@@ -0,0 +1,154 @@
package querybuilder
import (
"strings"
"github.com/SigNoz/signoz/pkg/errors"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/antlr4-go/antlr/v4"
)
// SplitFilterForAggregates partitions a single filter expression into a span-level
// part (a WHERE over spans) and a trace-level part (a HAVING over per-trace
// aggregates), splitting on the top-level AND.
//
// A key is trace-level when it carries the trace field context (`trace.completion_tokens`)
// or, with no context, its bare name is in aggregateNames. Any other explicit context
// (`span.`, `resource.`, …) is span-level. Trace-level and span-level keys may be
// AND-combined (they run at different query stages) but not OR-combined; an OR that
// mixes the two is an error.
//
// Syntax errors are ignored here — each part is re-parsed downstream (PrepareWhereClause
// for the span part, the HAVING rewriter for the trace part), which surface them.
func SplitFilterForAggregates(query string, aggregateNames map[string]struct{}) (spanExpr string, havingExpr string, err error) {
if strings.TrimSpace(query) == "" {
return "", "", nil
}
s := filterSplitter{query: []rune(query), aggregateNames: aggregateNames}
s.visit(parseFilterQuery(query))
if s.mixed {
return "", "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level and span-level filters cannot be combined within an OR/NOT group; separate them with a top-level AND")
}
return strings.Join(s.span, " AND "), strings.Join(s.having, " AND "), nil
}
func parseFilterQuery(query string) antlr.Tree {
lexer := grammar.NewFilterQueryLexer(antlr.NewInputStream(query))
lexer.RemoveErrorListeners()
parser := grammar.NewFilterQueryParser(antlr.NewCommonTokenStream(lexer, 0))
parser.RemoveErrorListeners()
return parser.Query()
}
// filterSplitter walks the parse tree once, flattening the top-level AND chain and
// routing each atom (a comparison, a NOT expression, or a whole multi-branch OR group)
// to the span or having bucket by the class of the keys it references.
type filterSplitter struct {
query []rune
aggregateNames map[string]struct{}
span []string
having []string
mixed bool
}
func (s *filterSplitter) visit(node antlr.Tree) {
switch n := node.(type) {
case *grammar.QueryContext:
if n.Expression() != nil {
s.visit(n.Expression())
}
case *grammar.ExpressionContext:
if n.OrExpression() != nil {
s.visit(n.OrExpression())
}
case *grammar.OrExpressionContext:
// a single branch is just an AND chain; multiple branches are a real OR, kept
// whole so a class-mixing OR can be rejected.
if ands := n.AllAndExpression(); len(ands) == 1 {
s.visit(ands[0])
} else {
s.route(n)
}
case *grammar.AndExpressionContext:
for _, u := range n.AllUnaryExpression() {
s.visit(u)
}
case *grammar.UnaryExpressionContext:
if n.NOT() != nil {
s.route(n)
} else if n.Primary() != nil {
s.visit(n.Primary())
}
case *grammar.PrimaryContext:
if n.OrExpression() != nil { // parenthesized sub-expression
s.visit(n.OrExpression())
} else {
s.route(n)
}
}
}
// route classifies an atom and appends its original source text to the right bucket.
func (s *filterSplitter) route(atom antlr.ParserRuleContext) {
isTrace, isSpan := classifyKeys(atom, s.aggregateNames)
if isTrace && isSpan {
s.mixed = true
return
}
text := atomSourceText(s.query, atom)
// A multi-branch OR group's source slice excludes its enclosing parens (they belong
// to the parent Primary). Re-wrap it so rejoining a bucket with " AND " cannot invert
// OR/AND precedence, e.g. `a AND (b OR c)` must not flatten to `a AND b OR c`.
if or, ok := atom.(*grammar.OrExpressionContext); ok && len(or.AllAndExpression()) > 1 {
text = "(" + text + ")"
}
if isTrace {
s.having = append(s.having, text)
} else {
s.span = append(s.span, text)
}
}
// classifyKeys reports whether a subtree references trace-level and/or span-level keys.
// A key is trace-level when it carries the trace field context or, with no context,
// its name is a known aggregate; an unknown name under the trace context stays
// trace-level so the aggregate validation rejects it with a targeted error. Any other
// explicit context (`span.`, `resource.`, …) is span-level.
func classifyKeys(node antlr.Tree, aggregateNames map[string]struct{}) (isTrace, isSpan bool) {
kc, ok := node.(*grammar.KeyContext)
if ok {
key := telemetrytypes.GetFieldKeyFromKeyText(kc.GetText())
switch key.FieldContext {
case telemetrytypes.FieldContextTrace:
isTrace = true
case telemetrytypes.FieldContextUnspecified:
_, isTrace = aggregateNames[key.Name]
isSpan = !isTrace
default:
isSpan = true
}
return
}
for i := 0; i < node.GetChildCount(); i++ {
t, s := classifyKeys(node.GetChild(i), aggregateNames)
isTrace = isTrace || t
isSpan = isSpan || s
}
return
}
// atomSourceText returns the original source substring for an atom, preserving
// whitespace. The token stream drops skipped whitespace, which would glue word
// operators (OR/AND/NOT) to their operands, so slice the input by token offsets.
// ANTLR offsets are rune indices (InputStream holds []rune), hence the rune slice.
func atomSourceText(query []rune, atom antlr.ParserRuleContext) string {
start, stop := atom.GetStart(), atom.GetStop()
if start == nil || stop == nil || start.GetStart() < 0 || stop.GetStop() >= len(query) || stop.GetStop() < start.GetStart() {
return atom.GetText()
}
return string(query[start.GetStart() : stop.GetStop()+1])
}

View File

@@ -0,0 +1,167 @@
package querybuilder
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestSplitFilterForAggregates(t *testing.T) {
agg := map[string]struct{}{"completion_tokens": {}, "span_count": {}, "prompt_tokens": {}}
type tc struct {
name string
query string
span string // expected span-level (WHERE) part; "" => empty
having string // expected trace-level (HAVING) part; "" => empty
wantErr bool
}
cases := []tc{
// --- empty input ---------------------------------------------------------
{
name: "empty",
},
{
name: "whitespace only",
query: " ",
},
// --- single class --------------------------------------------------------
{
name: "span only",
query: "service.name = 'x'",
span: "service.name = 'x'",
},
{
name: "agg only bare",
query: "completion_tokens > 1000",
having: "completion_tokens > 1000",
},
{
// the user-facing `trace.` prefix marks a trace-level aggregate.
name: "agg only trace prefix",
query: "trace.completion_tokens > 1000",
having: "trace.completion_tokens > 1000",
},
{
// an unknown name under the trace context still routes trace-level, so the
// aggregate validation rejects it with a targeted error instead of the span
// path failing on an unknown field.
name: "unknown aggregate under trace context stays trace-level",
query: "trace.not_an_aggregate > 1000",
having: "trace.not_an_aggregate > 1000",
},
{
// ANTLR token offsets are rune indices; slicing must not shift after a
// multi-byte char (this used to truncate 1000 → 100).
name: "unicode value before the split",
query: "service.name = 'héllo' AND completion_tokens > 1000",
span: "service.name = 'héllo'",
having: "completion_tokens > 1000",
},
// --- top-level AND splits across the two buckets -------------------------
{
name: "span AND agg",
query: "service.name = 'x' AND completion_tokens > 1000",
span: "service.name = 'x'",
having: "completion_tokens > 1000",
},
{
// order within a bucket is preserved; the two span atoms join with AND.
name: "span AND span AND agg",
query: "service.name = 'x' AND kind_string = 'Internal' AND completion_tokens > 1000",
span: "service.name = 'x' AND kind_string = 'Internal'",
having: "completion_tokens > 1000",
},
{
// a parenthesized top-level AND still splits across the two buckets.
name: "parenthesized span AND agg",
query: "(service.name = 'x' AND completion_tokens > 1000)",
span: "service.name = 'x'",
having: "completion_tokens > 1000",
},
// --- OR groups are re-wrapped in parens so a later AND-join can't invert
// precedence (`a AND (b OR c)` must not flatten to `a AND b OR c`) ------
{
name: "agg OR agg",
query: "completion_tokens > 1000 OR span_count > 3",
having: "(completion_tokens > 1000 OR span_count > 3)",
},
{
name: "span OR span",
query: "service.name = 'x' OR kind_string = 'Internal'",
span: "(service.name = 'x' OR kind_string = 'Internal')",
},
{
name: "span AND (span OR span)",
query: "service.name = 'x' AND (kind_string = 'Internal' OR kind_string = 'Client')",
span: "service.name = 'x' AND (kind_string = 'Internal' OR kind_string = 'Client')",
},
{
name: "agg AND (agg OR agg)",
query: "prompt_tokens > 5 AND (completion_tokens > 1000 OR span_count > 3)",
having: "prompt_tokens > 5 AND (completion_tokens > 1000 OR span_count > 3)",
},
{
// the OR group routes to span, the trailing aggregate to having.
name: "span AND (span OR span) AND agg",
query: "a.b = 'x' AND (c.d = 'y' OR e.f = 'z') AND completion_tokens > 1000",
span: "a.b = 'x' AND (c.d = 'y' OR e.f = 'z')",
having: "completion_tokens > 1000",
},
// --- a nested AND group flattens across the buckets (no spurious parens) --
{
name: "(span AND agg) AND agg",
query: "(service.name = 'x' AND completion_tokens > 1000) AND prompt_tokens > 5",
span: "service.name = 'x'",
having: "completion_tokens > 1000 AND prompt_tokens > 5",
},
// --- NOT wrapping a single-class group is routed whole to that class ------
{
name: "not agg",
query: "NOT (completion_tokens > 1000)",
having: "NOT (completion_tokens > 1000)",
},
{
name: "not span",
query: "NOT (service.name = 'x')",
span: "NOT (service.name = 'x')",
},
// --- class-mixing is rejected in an OR group, a NOT group, or a nested OR -
{
name: "agg OR span rejected",
query: "completion_tokens > 1000 OR service.name = 'x'",
wantErr: true,
},
{
name: "not mixed rejected",
query: "NOT (completion_tokens > 1000 AND service.name = 'x')",
wantErr: true,
},
{
name: "span AND (agg OR span) rejected",
query: "service.name = 'x' AND (completion_tokens > 1000 OR kind_string = 'Client')",
wantErr: true,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
span, having, err := SplitFilterForAggregates(c.query, agg)
if c.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
require.Equal(t, c.span, span, "span part")
require.Equal(t, c.having, having, "having part")
})
}
}

View File

@@ -18,6 +18,19 @@ func NewHavingExpressionRewriter() *HavingExpressionRewriter {
}
}
// Rewrite rewrites and validates a HAVING expression against a caller-supplied
// column map (user-facing name -> SQL identifier/expression). Values are inlined, so
// the result is a bare SQL boolean expression with no bound args. Used by callers
// that project their own aggregate columns (e.g. the AI trace list) rather than the
// query's Aggregations.
func (r *HavingExpressionRewriter) Rewrite(expression string, columnMap map[string]string) (string, error) {
if len(strings.TrimSpace(expression)) == 0 {
return "", nil
}
r.columnMap = columnMap
return r.rewriteAndValidate(expression)
}
// RewriteForTraces rewrites and validates the HAVING expression for a traces query.
func (r *HavingExpressionRewriter) RewriteForTraces(expression string, aggregations []qbtypes.TraceAggregation) (string, error) {
if len(strings.TrimSpace(expression)) == 0 {

View File

@@ -3,7 +3,6 @@ package querybuilder
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
@@ -14,38 +13,8 @@ import (
"github.com/tidwall/gjson"
)
var telemetryGrantKeys = map[string]struct{}{
"service.name": {},
}
const telemetryValueSafeBytes = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._-"
func EscapeTelemetryValue(value string) string {
var escaped strings.Builder
for _, character := range []byte(value) {
if strings.IndexByte(telemetryValueSafeBytes, character) >= 0 {
escaped.WriteByte(character)
continue
}
escaped.WriteString(fmt.Sprintf("%%%02X", character))
}
return escaped.String()
}
func TelemetrySelector(_ context.Context, resource coretypes.Resource, id string, _ valuer.UUID) ([]coretypes.Selector, error) {
values := []string{id}
segments := strings.Split(id, "/")
for level := len(segments) - 1; level >= 1; level-- {
value := strings.Join(segments[:level], "/") + "/" + coretypes.WildCardSelectorString
if value == id {
continue
}
values = append(values, value)
}
if id != coretypes.WildCardSelectorString {
values = append(values, coretypes.WildCardSelectorString)
}
values := telemetrytypes.NewTelemetryGrantSelectors(id)
selectors := make([]coretypes.Selector, 0, len(values))
for _, value := range values {
@@ -113,6 +82,10 @@ func resourcesForQuery(query gjson.Result, variables map[string]qbtypes.Variable
switch queryType {
case qbtypes.QueryTypeBuilder.StringValue(), qbtypes.QueryTypeSubQuery.StringValue():
return resourcesForBuilderQuery(queryType, query.Get("spec"), variables)
case qbtypes.QueryTypeBuilderAI.StringValue():
// An AI builder query is always a traces query; the signal is implied by the
// type (and may be absent from the payload), so pin the resource directly.
return builderQueryResourceRefs(queryType, coretypes.ResourceTelemetryResourceTraces, query.Get("spec"), variables)
case qbtypes.QueryTypePromQL.StringValue():
return []coretypes.ResourceWithID{{Resource: coretypes.ResourceTelemetryResourceMetrics, ID: typeWildcard}}, nil
case qbtypes.QueryTypeClickHouseSQL.StringValue():
@@ -134,7 +107,10 @@ func resourcesForBuilderQuery(queryType string, spec gjson.Result, variables map
if err != nil {
return nil, err
}
return builderQueryResourceRefs(queryType, resource, spec, variables)
}
func builderQueryResourceRefs(queryType string, resource coretypes.Resource, spec gjson.Result, variables map[string]qbtypes.VariableItem) ([]coretypes.ResourceWithID, error) {
ids, err := builderQuerySelectors(queryType, spec.Get("filter.expression").String(), variables)
if err != nil {
return nil, err
@@ -187,14 +163,14 @@ func builderQuerySelectors(queryType, expression string, variables map[string]qb
continue
}
key, ok := canonicalTelemetryGrantKey(condition.Key)
key, ok := telemetrytypes.NewTelemetryGrantKey(condition.Key)
if !ok {
continue
}
if condition.Operator == "=" || condition.Operator == "IN" {
for _, value := range condition.Values {
ids = append(ids, queryType+"/"+key+"/"+EscapeTelemetryValue(value))
ids = append(ids, queryType+"/"+key+"/"+value)
}
}
}
@@ -205,16 +181,3 @@ func builderQuerySelectors(queryType, expression string, variables map[string]qb
return ids, nil
}
func canonicalTelemetryGrantKey(keyText string) (string, bool) {
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(keyText)
if fieldKey.FieldContext != telemetrytypes.FieldContextUnspecified && fieldKey.FieldContext != telemetrytypes.FieldContextResource {
return "", false
}
if _, ok := telemetryGrantKeys[fieldKey.Name]; !ok {
return "", false
}
return fieldKey.Name, true
}

View File

@@ -2,7 +2,6 @@ package querybuilder
import (
"context"
"strings"
"testing"
"github.com/SigNoz/signoz/pkg/types/coretypes"
@@ -22,33 +21,33 @@ func TestQueryRangeResources(t *testing.T) {
expected []coretypes.ResourceWithID
}{
{
name: "top level service equality",
body: builderQueryBody("logs", "service.name = 'checkout' AND status = 500"),
name: "top level key equality",
body: builderQueryBody("logs", "signoz.workspace.key.id = 'checkout' AND status = 500"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/checkout"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "resource prefixed service key",
body: builderQueryBody("traces", "resource.service.name = 'checkout'"),
name: "resource prefixed key",
body: builderQueryBody("traces", "resource.signoz.workspace.key.id = 'checkout'"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_query/service.name/checkout"},
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "in atom requires every value",
body: builderQueryBody("logs", "service.name IN ('b', 'a')"),
body: builderQueryBody("logs", "signoz.workspace.key.id IN ('b', 'a')"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/a"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/b"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/a"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/b"},
},
},
{
name: "multiple equality atoms each require a grant",
body: builderQueryBody("logs", "service.name = 'b' AND service.name = 'a'"),
body: builderQueryBody("logs", "signoz.workspace.key.id = 'b' AND signoz.workspace.key.id = 'a'"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/a"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/b"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/a"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/b"},
},
},
{
@@ -59,38 +58,52 @@ func TestQueryRangeResources(t *testing.T) {
},
},
{
name: "service atom under or does not qualify",
body: builderQueryBody("logs", "service.name = 'a' OR status = 500"),
name: "key atom under or does not qualify",
body: builderQueryBody("logs", "signoz.workspace.key.id = 'a' OR status = 500"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/*"},
},
},
{
name: "negated service atom does not qualify",
body: builderQueryBody("logs", "NOT service.name = 'a'"),
name: "negated key atom does not qualify",
body: builderQueryBody("logs", "NOT signoz.workspace.key.id = 'a'"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/*"},
},
},
{
name: "service inequality does not qualify",
body: builderQueryBody("logs", "service.name != 'a'"),
name: "key inequality does not qualify",
body: builderQueryBody("logs", "signoz.workspace.key.id != 'a'"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/*"},
},
},
{
name: "unsafe value bytes are escaped",
body: builderQueryBody("logs", "service.name = 'check out/2'"),
name: "value with spaces and slashes stays plaintext in the id",
body: builderQueryBody("logs", "signoz.workspace.key.id = 'check out/2'"),
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/check%20out%2F2"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/check out/2"},
},
},
{
name: "audit source maps to audit logs resource",
body: `{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"logs","source":"audit","filter":{"expression":"service.name = 'a'"}}}]}}`,
body: `{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"logs","source":"audit","filter":{"expression":"signoz.workspace.key.id = 'a'"}}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceAuditLogs, ID: "builder_query/service.name/a"},
{Resource: coretypes.ResourceTelemetryResourceAuditLogs, ID: "builder_query/signoz.workspace.key.id/a"},
},
},
{
name: "ai builder query maps to traces resource without a signal",
body: `{"compositeQuery":{"queries":[{"type":"builder_ai_query","spec":{"filter":{"expression":"signoz.workspace.key.id = 'checkout'"}}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_ai_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "ai builder query without filter is wildcard",
body: `{"compositeQuery":{"queries":[{"type":"builder_ai_query","spec":{}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_ai_query/*"},
},
},
{
@@ -117,23 +130,23 @@ func TestQueryRangeResources(t *testing.T) {
},
{
name: "trace operator rides on its referenced queries",
body: `{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"name":"A","signal":"traces","disabled":true,"filter":{"expression":"service.name = 'checkout'"}}},{"type":"builder_query","spec":{"name":"B","signal":"traces","disabled":true,"filter":{"expression":"service.name = 'checkout' AND has_error = true"}}},{"type":"builder_trace_operator","spec":{"name":"T1","expression":"A => B","returnSpansFrom":"A"}}]}}`,
body: `{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"name":"A","signal":"traces","disabled":true,"filter":{"expression":"signoz.workspace.key.id = 'checkout'"}}},{"type":"builder_query","spec":{"name":"B","signal":"traces","disabled":true,"filter":{"expression":"signoz.workspace.key.id = 'checkout' AND has_error = true"}}},{"type":"builder_trace_operator","spec":{"name":"T1","expression":"A => B","returnSpansFrom":"A"}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_query/service.name/checkout"},
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "variable substitution qualifies",
body: `{"variables":{"svc":{"value":"checkout"}},"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"logs","filter":{"expression":"service.name = $svc"}}}]}}`,
body: `{"variables":{"key":{"value":"checkout"}},"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"logs","filter":{"expression":"signoz.workspace.key.id = $key"}}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/checkout"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "duplicate queries dedupe",
body: `{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"logs","filter":{"expression":"service.name = 'a'"}}},{"type":"builder_query","spec":{"signal":"logs","filter":{"expression":"service.name='a'"}}}]}}`,
body: `{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"logs","filter":{"expression":"signoz.workspace.key.id = 'a'"}}},{"type":"builder_query","spec":{"signal":"logs","filter":{"expression":"signoz.workspace.key.id='a'"}}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/service.name/a"},
{Resource: coretypes.ResourceTelemetryResourceLogs, ID: "builder_query/signoz.workspace.key.id/a"},
},
},
}
@@ -151,7 +164,7 @@ func TestQueryRangeResourcesErrors(t *testing.T) {
bodies := []string{
`{"compositeQuery":{"queries":[]}}`,
`{}`,
builderQueryBody("logs", "service.name = "),
builderQueryBody("logs", "signoz.workspace.key.id = "),
`{"compositeQuery":{"queries":[{"type":"builder_query","spec":{"signal":"unknown"}}]}}`,
`{"compositeQuery":{"queries":[{"type":"unknown_type"}]}}`,
}
@@ -175,10 +188,10 @@ func TestTelemetrySelector(t *testing.T) {
return values
}
assert.Equal(t, []string{"builder_query/service.name/a", "builder_query/service.name/*", "builder_query/*", "*"}, selectorValues("builder_query/service.name/a"))
assert.Equal(t, []string{"builder_query/*", "*"}, selectorValues("builder_query/*"))
assert.Equal(t, []string{"promql/*", "*"}, selectorValues("promql/*"))
_, err := TelemetrySelector(context.Background(), coretypes.ResourceTelemetryResourceLogs, strings.Repeat("a", 256), orgID)
assert.Error(t, err)
assert.Equal(t, []string{"builder_query/signoz.workspace.key.id/a", "builder_query/signoz.workspace.key.id/*", "builder_query/*", "*"}, selectorValues("builder_query/signoz.workspace.key.id/a"))
assert.Equal(t, []string{"builder_query/*", "*"}, selectorValues("builder_query"))
assert.Equal(t, []string{"promql/*", "*"}, selectorValues("promql"))
assert.Equal(t, []string{"*"}, selectorValues("*"))
// a value containing "/" stays one logical segment (SplitN 3).
assert.Equal(t, []string{"builder_query/signoz.workspace.key.id/a/b", "builder_query/signoz.workspace.key.id/*", "builder_query/*", "*"}, selectorValues("builder_query/signoz.workspace.key.id/a/b"))
}

View File

@@ -149,7 +149,7 @@ func NewModules(
TraceFunnel: impltracefunnel.NewModule(impltracefunnel.NewStore(sqlstore)),
RawDataExport: implrawdataexport.NewModule(querier),
AuthDomain: authDomainModule,
Session: implsession.NewModule(providerSettings, authNs, userSetter, userGetter, authDomainModule, tokenizer, orgGetter, authz),
Session: implsession.NewModule(providerSettings, authNs, userSetter, userGetter, authDomainModule, tokenizer, orgGetter, authz, config.Global),
SpanPercentile: implspanpercentile.NewModule(querier, providerSettings),
Services: implservices.NewModule(querier, telemetryStore),
MetricsExplorer: implmetricsexplorer.NewModule(telemetryStore, telemetryMetadataStore, cache, ruleStore, dashboard, fl, providerSettings, config.MetricsExplorer),

View File

@@ -0,0 +1,100 @@
package telemetryai
import (
"strings"
scopedtraces "github.com/SigNoz/signoz/pkg/telemetryscopedtraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// genAIBaseConditionProvider: an AI trace has >=1 gen_ai LLM, tool, or agent span.
type genAIBaseConditionProvider struct {
keys []string
}
var _ scopedtraces.BaseConditionProvider = (*genAIBaseConditionProvider)(nil)
func newGenAIBaseConditionProvider() scopedtraces.BaseConditionProvider {
return &genAIBaseConditionProvider{
keys: []string{telemetrytypes.GenAIRequestModel, telemetrytypes.GenAIToolName, telemetrytypes.GenAIAgentName},
}
}
func (p *genAIBaseConditionProvider) FilterExpression() string {
parts := make([]string, 0, len(p.keys))
for _, k := range p.keys {
parts = append(parts, k+" EXISTS")
}
return strings.Join(parts, " OR ")
}
func (p *genAIBaseConditionProvider) FieldKeys() []*telemetrytypes.TelemetryFieldKey {
keys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(p.keys))
for _, name := range p.keys {
keys = append(keys, &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
})
}
return keys
}
// genAIColumnProvider adds AI/LLM per-trace metrics on top of the common columns.
type genAIColumnProvider struct{}
var _ scopedtraces.ColumnProvider = (*genAIColumnProvider)(nil)
func newGenAIColumnProvider() scopedtraces.ColumnProvider {
return &genAIColumnProvider{}
}
func (genAIColumnProvider) Columns() []scopedtraces.TraceColumn {
defs := telemetrytypes.GenAIFieldDefinitions
reqModel := defs[telemetrytypes.GenAIRequestModel]
toolName := defs[telemetrytypes.GenAIToolName]
inTok := defs[telemetrytypes.GenAIUsageInputTokens]
outTok := defs[telemetrytypes.GenAIUsageOutputTokens]
cost := defs[telemetrytypes.SignozGenAITotalCost]
inMsg := defs[telemetrytypes.GenAIInputMessages]
outMsg := defs[telemetrytypes.GenAIOutputMessages]
str := telemetrytypes.FieldDataTypeString
return append(scopedtraces.CommonTraceColumns(),
// LLM calls only (request model present), not the full gate.
scopedtraces.TraceColumn{Alias: "llm_call_count", Orderable: true, Expr: scopedtraces.CountExists(&reqModel)},
scopedtraces.TraceColumn{Alias: "tool_call_count", Orderable: true, Expr: scopedtraces.CountExists(&toolName)},
scopedtraces.TraceColumn{Alias: "distinct_tool_count", Orderable: true, Expr: scopedtraces.UniqCount(&toolName, str)},
// tokens live only on LLM spans, so a plain sum needs no gate scoping.
scopedtraces.TraceColumn{Alias: "input_tokens", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &inTok)},
scopedtraces.TraceColumn{Alias: "output_tokens", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &outTok)},
scopedtraces.TraceColumn{Alias: "total_tokens", Orderable: true, Expr: scopedtraces.SumOfKeys(telemetrytypes.FieldDataTypeFloat64, &inTok, &outTok)},
// per-span cost attached by the SigNoz LLM pricing processor.
scopedtraces.TraceColumn{Alias: "estimated_cost_usd", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &cost)},
// slowest single LLM call in the trace.
scopedtraces.TraceColumn{Alias: "max_llm_latency_ns", Orderable: true, Expr: scopedtraces.ScopedToKeyColumn(scopedtraces.AggMax, scopedtraces.IntrinsicSpanKey("duration_nano"), &reqModel)},
// errors across the whole trace (any span), so display-only.
scopedtraces.TraceColumn{Alias: "error_count", Expr: scopedtraces.CondCount(scopedtraces.IntrinsicSpanKey("has_error"), qbtypes.FilterOperatorEqual, true)},
// timestamp of the last gen_ai span (LLM/tool/agent), hence gate-scoped.
scopedtraces.TraceColumn{Alias: "last_activity_time", Orderable: true, Expr: scopedtraces.ScopedReduce(scopedtraces.AggMax, scopedtraces.IntrinsicSpanKey("timestamp"))},
// previews: first call's input (the prompt), last call's output (the answer).
scopedtraces.TraceColumn{Alias: "input", SpanLevel: true, Expr: scopedtraces.PickBy(&inMsg, str, scopedtraces.IntrinsicSpanKey("timestamp"), scopedtraces.PickEarliest)},
scopedtraces.TraceColumn{Alias: "output", SpanLevel: true, Expr: scopedtraces.PickBy(&outMsg, str, scopedtraces.IntrinsicSpanKey("timestamp"), scopedtraces.PickLatest)},
)
}
func (genAIColumnProvider) DefaultOrderAlias() string { return "last_activity_time" }
func (p genAIColumnProvider) AggregateAliases() []string {
// Derived from Columns() so a new column can't be forgotten; SpanLevel columns
// are filtered span-level, so skip them.
cols := p.Columns()
aliases := make([]string, 0, len(cols))
for _, c := range cols {
if !c.SpanLevel {
aliases = append(aliases, c.Alias)
}
}
return aliases
}

View File

@@ -0,0 +1,21 @@
package telemetryai
import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
scopedtraces "github.com/SigNoz/signoz/pkg/telemetryscopedtraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// NewAITraceStatementBuilder wires the generic scoped-trace builder with the gen_ai
// gate and AI columns. This package holds only gen_ai domain knowledge; the query
// topology lives in telemetryscopedtraces.
func NewAITraceStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
fl flagger.Flagger,
) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
return scopedtraces.NewScopedTraceStatementBuilder(settings, metadataStore, newGenAIBaseConditionProvider(), newGenAIColumnProvider(), traceStmtBuilder, fl)
}

View File

@@ -0,0 +1,987 @@
package telemetryai
import (
"context"
"fmt"
"strings"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
scopedtraces "github.com/SigNoz/signoz/pkg/telemetryscopedtraces"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
// otelKeysMap seeds the OpenTelemetry gen_ai semantic-convention keys the AI
// queries reference, so the metadata-backed field resolution succeeds in tests.
func otelKeysMap() map[string][]*telemetrytypes.TelemetryFieldKey {
strKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
numKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeFloat64,
}
}
m := make(map[string][]*telemetrytypes.TelemetryFieldKey)
// gen_ai semconv keys sourced from the single source of truth, mirroring what the
// production metadata store surfaces via enrichWithGenAIKeys.
for name, def := range telemetrytypes.GenAIFieldDefinitions {
keyCopy := def
m[name] = []*telemetrytypes.TelemetryFieldKey{&keyCopy}
}
// Extra keys these tests reference that aren't gen_ai semconv definitions.
m["gen_ai.user.id"] = []*telemetrytypes.TelemetryFieldKey{strKey("gen_ai.user.id")}
m["_signoz.gen_ai.total_cost"] = []*telemetrytypes.TelemetryFieldKey{numKey("_signoz.gen_ai.total_cost")}
m["gen_ai.usage.cached_input_tokens"] = []*telemetrytypes.TelemetryFieldKey{numKey("gen_ai.usage.cached_input_tokens")}
m["has_error"] = []*telemetrytypes.TelemetryFieldKey{{
Name: "has_error",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeBool,
}}
// service.name carries the resource-column evolutions like production metadata, so
// the rendered value expression prefers the JSON resource column over the legacy
// map (matching the standard traces builder tests).
m["service.name"] = []*telemetrytypes.TelemetryFieldKey{{
Name: "service.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
Evolutions: resourceEvolutions(),
}}
return m
}
// resourceEvolutions is the canonical resource-column timeline: the legacy
// resources_string map at epoch 0 and the JSON resource column released inside the
// test window (mirrors telemetrytraces' mockEvolutionData).
func resourceEvolutions() []*telemetrytypes.EvolutionEntry {
return []*telemetrytypes.EvolutionEntry{
{
Signal: telemetrytypes.SignalTraces,
ColumnName: "resources_string",
ColumnType: "Map(LowCardinality(String), String)",
FieldContext: telemetrytypes.FieldContextResource,
FieldName: "__all__",
ReleaseTime: time.Unix(0, 0),
},
{
Signal: telemetrytypes.SignalTraces,
ColumnName: "resource",
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextResource,
FieldName: "__all__",
ReleaseTime: time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC),
},
}
}
// standard test window (ms), matching the traces builder tests.
const (
testStartMs = uint64(1747947419000)
testEndMs = uint64(1747983448000)
)
func newTestBuilder(t *testing.T) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
return newTestBuilderWithKeys(t, otelKeysMap())
}
// newTestBuilderWithKeys mirrors the production wiring in signozquerier's provider.
// The gen_ai keys are seeded via keysMap here; in production the metadata store
// surfaces them itself (enrichWithGenAIKeys).
func newTestBuilderWithKeys(t *testing.T, keysMap map[string][]*telemetrytypes.TelemetryFieldKey) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
t.Helper()
settings := instrumentationtest.New().ToProviderSettings()
fm := telemetrytraces.NewFieldMapper()
cb := telemetrytraces.NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = keysMap
fl := flaggertest.New(t)
// In production the metadata store enriches gen_ai keys (enrichWithGenAIKeys);
// here the mock is seeded directly via keysMap.
metadataStore := telemetrytypes.MetadataStore(mockMetadataStore)
rewriter := querybuilder.NewAggExprRewriter(settings, nil, fm, cb, fl)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
settings,
metadataStore,
fm,
cb,
rewriter,
nil,
fl,
false,
100000,
)
return NewAITraceStatementBuilder(
settings,
metadataStore,
traceStmtBuilder,
fl,
)
}
// ---------------------------------------------------------------------------
// Full-query golden tests
//
// Each pins the WHOLE generated statement, with bound args inlined into the `?`
// placeholders, as ONE self-contained literal — so a failure diff shows the entire
// query and the expected SQL can be copied straight into a ClickHouse client. The
// `want` strings are formatted for readability; the comparison is whitespace- and
// backtick-insensitive (see normalizeSQL), so only the SQL tokens themselves matter.
//
// The four trace-list goldens cover the corners of how `matched` is assembled —
// {no span filter, span filter} × {no aggregate filter, aggregate filter} — plus a
// mixed filter + multi-key order, plus the delegated span list. Note `matched` selects
// only the aggregates ORDER BY / HAVING reference; the rest appear only in enrichment.
//
// Run `go test ./pkg/telemetryai/ -run TestBuild_FullSQL -v` to also print each query.
// ---------------------------------------------------------------------------
// renderSQL substitutes bound args into the `?` placeholders so the whole statement
// reads as one literal SQL string.
func renderSQL(t *testing.T, stmt *qbtypes.Statement) string {
t.Helper()
var b strings.Builder
argi := 0
for i := 0; i < len(stmt.Query); i++ {
if stmt.Query[i] == '?' {
require.Less(t, argi, len(stmt.Args), "more ? than args in query")
b.WriteString(formatArg(stmt.Args[argi]))
argi++
continue
}
b.WriteByte(stmt.Query[i])
}
require.Equal(t, len(stmt.Args), argi, "arg count does not match number of placeholders")
return b.String()
}
func formatArg(a any) string {
if s, ok := a.(string); ok {
return "'" + s + "'"
}
return fmt.Sprintf("%v", a)
}
// normalizeSQL makes the comparison insensitive to formatting: it drops identifier
// backticks, collapses whitespace runs to a single space, and removes spaces directly
// inside parentheses. This lets the golden strings be freely indented/wrapped (and
// written as Go raw literals, which cannot contain backticks) — only the SQL tokens
// and their order matter.
func normalizeSQL(s string) string {
s = strings.Join(strings.Fields(strings.ReplaceAll(s, "`", "")), " ")
s = strings.ReplaceAll(s, "( ", "(")
s = strings.ReplaceAll(s, " )", ")")
return s
}
func requireSQLEqual(t *testing.T, want string, stmt *qbtypes.Statement) {
t.Helper()
got := renderSQL(t, stmt)
t.Logf("\n%s", got)
require.Equal(t, normalizeSQL(want), normalizeSQL(got))
}
// No filter: matched selects only the default order key (last_activity_time), WHERE is
// just window + gate mask, no HAVING.
func TestBuild_FullSQL_TraceList_NoFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces, Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Promotion: a materialized gen_ai attribute must resolve to its materialized column
// everywhere it appears — gate mask, countIf/scoped existence, and value columns —
// while un-promoted attributes stay in the attributes map, so one query mixes both
// forms. Here gen_ai.request.model and gen_ai.usage.input_tokens are materialized:
// the gate/llm_call_count/max_llm_latency use `..._exists`, input_tokens/total_tokens
// use the materialized value column, and tool/output_tokens/cost/messages stay in the map.
func TestBuild_FullSQL_TraceList_MaterializedColumns(t *testing.T) {
keys := otelKeysMap()
for _, name := range []string{"gen_ai.request.model", "gen_ai.usage.input_tokens"} {
for _, k := range keys[name] {
k.Materialized = true
}
}
b := newTestBuilderWithKeys(t, keys)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces, Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(attribute_string_gen_ai$$request$$model_exists) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, attribute_string_gen_ai$$request$$model_exists) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span-level AND trace-level filter, order by the aggregate, pagination. matched selects
// only output_tokens (the sole aggregate referenced by both ORDER BY and HAVING) — not
// input_tokens/llm_call_count/last_activity_time. The span predicate widens the WHERE
// prune and becomes a countIf(...) > 0 existence check alongside the gate countIf.
func TestBuild_FullSQL_TraceList_SpanAndTraceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND output_tokens > 1000"},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "output_tokens"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 10, Offset: 30,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
OR (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model')))
GROUP BY trace_id
HAVING countIf((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) > 0
AND countIf((attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))) > 0
AND output_tokens > 1000
ORDER BY output_tokens DESC, trace_id DESC
LIMIT 10 OFFSET 30
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY output_tokens DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Aggregate-only filter (no span filter). WHERE prune is NOT widened, there is no
// gate/span countIf, just the aggregate HAVING. `trace.output_tokens` rewrites to the
// output_tokens alias. matched selects output_tokens (HAVING) + last_activity_time (default order).
func TestBuild_FullSQL_TraceList_AggregateFilterOnly(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"},
Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
HAVING output_tokens > 1000
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span-only filter (no aggregate filter). WHERE is widened; HAVING has the gate + span
// countIf pair but no trailing aggregate. `has_error = true` resolves to a
// materialized-column predicate (not a map access). matched selects only the default order key.
func TestBuild_FullSQL_TraceList_SpanFilterOnly(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "has_error = true"},
Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
OR has_error = true)
GROUP BY trace_id
HAVING countIf((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) > 0
AND countIf(has_error = true) > 0
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Resource filter: a resource attribute in the filter is pulled into a __resource_filter
// CTE (fingerprints matching the resource condition), and the `matched` scan is narrowed
// by `resource_fingerprint GLOBAL IN (…)`. The resource key is dropped from the span
// predicate (skipResourceFilter), so here there is no span-level existence check — the
// prune stays the gate mask and the whole match is scoped to the resource fingerprints.
func TestBuild_FullSQL_TraceList_ResourceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "resource.service.name = 'checkout'"},
Limit: 20,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH __resource_filter AS (
SELECT fingerprint
FROM signoz_traces.distributed_traces_v3_resource
WHERE (simpleJSONExtractString(labels, 'service.name') = 'checkout' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"checkout%')
AND seen_at_ts_bucket_start >= 1747945619
AND seen_at_ts_bucket_start <= 1747983448
GROUP BY fingerprint
),
matched AS (
SELECT trace_id,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Mixed filter (two span predicates AND'd into one existence check + an aggregate) with
// a two-key order on different aggregates than the filter. matched selects input_tokens
// + last_activity_time (ORDER BY) and output_tokens (HAVING) — three of four; llm_call_count is not.
func TestBuild_FullSQL_TraceList_MixedFiltersMultiOrder(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o' AND has_error = true AND output_tokens > 500"},
Order: []qbtypes.OrderBy{
{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "input_tokens"}}, Direction: qbtypes.OrderDirectionDesc},
{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "last_activity_time"}}, Direction: qbtypes.OrderDirectionAsc},
},
Limit: 15,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
OR ((attributes_string['gen_ai.request.model'] = 'gpt-4o' AND mapContains(attributes_string, 'gen_ai.request.model')) AND has_error = true))
GROUP BY trace_id
HAVING countIf((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) > 0
AND countIf(((attributes_string['gen_ai.request.model'] = 'gpt-4o' AND mapContains(attributes_string, 'gen_ai.request.model')) AND has_error = true)) > 0
AND output_tokens > 500
ORDER BY input_tokens DESC, last_activity_time ASC, trace_id DESC
LIMIT 15
),
ranked AS (
SELECT trace_id, min(start) AS t_start, max(end) AS t_end
FROM signoz_traces.distributed_trace_summary
WHERE trace_id GLOBAL IN (SELECT trace_id FROM matched)
AND end >= fromUnixTimestamp64Nano(1747947419000000000)
AND start < fromUnixTimestamp64Nano(1747983448000000000)
GROUP BY trace_id
),
buckets AS (
SELECT DISTINCT b AS ts_bucket
FROM ranked
ARRAY JOIN range(toUInt64(intDiv(toUnixTimestamp(t_start), 1800) * 1800 - 1800), toUInt64(intDiv(toUnixTimestamp(t_end), 1800) * 1800 + 1800), 1800) AS b
)
SELECT trace_id,
min(timestamp) AS start_time,
max(timestamp) AS end_time,
(max(toUnixTimestamp64Nano(timestamp) + duration_nano) - min(toUnixTimestamp64Nano(timestamp))) AS trace_duration_nano,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS service.name,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, '_signoz.gen_ai.total_cost'), toFloat64(attributes_number['_signoz.gen_ai.total_cost']), NULL)) AS estimated_cost_usd,
maxIf(duration_nano, mapContains(attributes_string, 'gen_ai.request.model')) AS max_llm_latency_ns,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3
WHERE ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)
AND trace_id GLOBAL IN (SELECT trace_id FROM ranked)
GROUP BY trace_id
ORDER BY input_tokens DESC, last_activity_time ASC, trace_id DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span list (requestType raw): delegated to the traces builder with the gate ANDed
// into the user filter, so only gen_ai spans matching the filter come back. Standard
// span columns, single SELECT (no CTE pipeline).
func TestBuild_FullSQL_SpanList_Raw(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini'"},
Limit: 10,
}, nil)
require.NoError(t, err)
requireSQLEqual(t, `
SELECT timestamp AS __SELECT_KEY_0_timestamp, trace_id AS __SELECT_KEY_1_trace_id, span_id AS __SELECT_KEY_2_span_id,
trace_state AS __SELECT_KEY_3_trace_state, parent_span_id AS __SELECT_KEY_4_parent_span_id, flags AS __SELECT_KEY_5_flags,
name AS __SELECT_KEY_6_name, kind AS __SELECT_KEY_7_kind, kind_string AS __SELECT_KEY_8_kind_string, duration_nano AS __SELECT_KEY_9_duration_nano,
status_code AS __SELECT_KEY_10_status_code, status_message AS __SELECT_KEY_11_status_message,
status_code_string AS __SELECT_KEY_12_status_code_string, events AS __SELECT_KEY_13_events, links AS __SELECT_KEY_14_links,
response_status_code AS __SELECT_KEY_15_response_status_code, external_http_url AS __SELECT_KEY_16_external_http_url,
http_url AS __SELECT_KEY_17_http_url, external_http_method AS __SELECT_KEY_18_external_http_method,
http_method AS __SELECT_KEY_19_http_method, http_host AS __SELECT_KEY_20_http_host, db_name AS __SELECT_KEY_21_db_name,
db_operation AS __SELECT_KEY_22_db_operation, has_error AS __SELECT_KEY_23_has_error, is_remote AS __SELECT_KEY_24_is_remote,
attributes_string, attributes_number, attributes_bool, resources_string
FROM signoz_traces.distributed_signoz_index_v3
WHERE (((mapContains(attributes_string, 'gen_ai.request.model')
OR mapContains(attributes_string, 'gen_ai.tool.name')
OR mapContains(attributes_string, 'gen_ai.agent.name')))
AND ((attributes_string['gen_ai.request.model'] = 'gpt-4o-mini'
AND mapContains(attributes_string, 'gen_ai.request.model'))))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
LIMIT 10
`, stmt)
}
// ---------------------------------------------------------------------------
// Behavior / branch tests not covered by the goldens above
// ---------------------------------------------------------------------------
// A filter mixing a resource attribute with a span-level and an aggregate condition:
// the resource key routes into __resource_filter (fingerprint prune), the span key stays
// as a countIf existence check, and the aggregate becomes a HAVING — all AND-combined.
// service.name (resource context) comes from otelKeysMap.
func TestBuild_TraceList_ResourcePlusSpanPlusAggregateFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "resource.service.name = 'checkout' AND has_error = true AND output_tokens > 1000"},
Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
// resource condition -> fingerprint CTE + prune, not filtered on the span index
// (the service.name output column still reads the resource map, hence the = form).
require.Contains(t, got, "__resource_filter AS (")
require.Contains(t, got, "resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
require.NotContains(t, got, "resources_string['service.name'] = 'checkout'")
// span condition -> existence check in matched HAVING.
require.Contains(t, got, "countIf(has_error = true) > 0")
// aggregate condition -> HAVING on the matched aggregate alias.
require.Contains(t, got, "output_tokens")
}
// The resolver-unset (nil) fallback is covered in pkg/telemetryscopedtraces, which
// can construct that builder state directly.
// Trace-level and span-level predicates may not be OR-combined.
func TestBuild_TraceList_TraceOrSpanMixRejected(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000 OR gen_ai.request.model = 'x'"},
Limit: 10,
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "cannot be combined")
}
// An output-only aggregate (span_count / trace_duration_nano) can be displayed but not
// used in the aggregate filter or ORDER BY — it is not computable in the matched pass.
func TestBuild_TraceList_OutputOnlyAggregateRejected(t *testing.T) {
b := newTestBuilder(t)
// filter by span_count -> rejected
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "span_count > 3"},
}, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "span_count")
// order by trace_duration_nano -> rejected
_, err = b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace_duration_nano"}}, Direction: qbtypes.OrderDirectionDesc}},
}, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "unsupported order key")
}
// duration_nano no longer names an aggregate (the trace column is trace_duration_nano),
// so a bare filter on it is span-level like everywhere else in the product: the trace
// matches when any span exceeds the duration.
func TestBuild_TraceList_SpanDurationFilterIsSpanLevel(t *testing.T) {
keys := otelKeysMap()
keys["duration_nano"] = []*telemetrytypes.TelemetryFieldKey{{
Name: "duration_nano",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeNumber,
}}
b := newTestBuilderWithKeys(t, keys)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "duration_nano > 1000000"},
Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
require.Contains(t, got, "countIf(duration_nano > 1000000) > 0")
require.NotContains(t, got, "HAVING trace_duration_nano")
}
// A HAVING referencing a non-aggregate column is rejected.
func TestBuild_TraceList_Having_UnknownColumn(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Having: &qbtypes.Having{Expression: "service.name > 1"}, // not an aggregate column
Limit: 10,
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.Error(t, err)
}
// Ordering by an unknown key is rejected.
func TestBuild_TraceList_UnsupportedOrderKey(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Order: []qbtypes.OrderBy{
{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "http.request.method"}}, Direction: qbtypes.OrderDirectionDesc},
},
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.Error(t, err)
require.Contains(t, err.Error(), "unsupported order key")
}
// With no limit set, the builder applies the default of 100.
func TestBuild_TraceList_DefaultLimit(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
}
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, query, nil)
require.NoError(t, err)
require.Contains(t, stmt.Query, "LIMIT ?")
require.Contains(t, stmt.Args, 100)
}
// Only trace list and span list (raw) are supported; distribution is not.
func TestBuild_UnsupportedRequestType(t *testing.T) {
b := newTestBuilder(t)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{
{Expression: "count()"},
},
}
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeDistribution, query, nil)
require.ErrorIs(t, err, scopedtraces.ErrUnsupportedRequestType)
}
// A gate key ingested under several data types (e.g. string + number from a
// misbehaving SDK) contributes ALL variants to the mask, OR-combined — not just
// the first — matching the standard visitor's EXISTS handling.
func TestBuild_TraceList_MultiVariantGateKey(t *testing.T) {
keys := otelKeysMap()
keys[telemetrytypes.GenAIToolName] = append(keys[telemetrytypes.GenAIToolName], &telemetrytypes.TelemetryFieldKey{
Name: telemetrytypes.GenAIToolName,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeFloat64,
})
b := newTestBuilderWithKeys(t, keys)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces, Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
require.Contains(t, got, "mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_number, 'gen_ai.tool.name')")
}
// `trace.` parses as the trace field context and marks a trace-level aggregate; the
// legacy `tracefield.` spelling is explicitly rejected (filter and having alike), and
// an output-only aggregate under the context gets the targeted rejection rather than
// an unknown-span-field failure.
func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
b := newTestBuilder(t)
build := func(q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (*qbtypes.Statement, error) {
q.Signal, q.Limit = telemetrytypes.SignalTraces, 20
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, q, nil)
}
_, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"}})
require.NoError(t, err)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
require.Contains(t, err.Error(), `use the "trace." prefix`)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Having: &qbtypes.Having{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
require.Contains(t, err.Error(), `use the "trace." prefix`)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"}})
require.Error(t, err)
require.Contains(t, err.Error(), "cannot be used")
}
// Query variables in a trace-level condition are substituted into the HAVING (the
// span path binds them via PrepareWhereClause; the HAVING is a text rewrite).
func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
b := newTestBuilder(t)
build := func(expr string, vars map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: expr},
Limit: 20,
}, vars)
}
// scalar variable -> literal in HAVING
stmt, err := build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Value: 700}})
require.NoError(t, err)
require.Contains(t, stmt.Query, "HAVING output_tokens > 700")
// list variable with IN
stmt, err = build("trace.llm_call_count IN $counts",
map[string]qbtypes.VariableItem{"counts": {Value: []any{1, 2}}})
require.NoError(t, err)
require.Contains(t, stmt.Query, "HAVING llm_call_count IN")
// dynamic __all__ -> condition dropped, no HAVING at all
stmt, err = build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.DynamicVariableType, Value: "__all__"}})
require.NoError(t, err)
require.NotContains(t, stmt.Query, "HAVING")
// unresolved variable -> rejected, not compared as a literal
_, err = build("trace.output_tokens > $missing", map[string]qbtypes.VariableItem{"other": {Value: 1}})
require.Error(t, err)
}

View File

@@ -1188,6 +1188,27 @@ func enrichWithIntrinsicMetricKeys(keys map[string][]*telemetrytypes.TelemetryFi
return keys
}
// enrichWithGenAIKeys adds keys that can be queried for GenAI signals, even though they have not been ingested yet.
func enrichWithGenAIKeys(keys map[string][]*telemetrytypes.TelemetryFieldKey, selectors []*telemetrytypes.FieldKeySelector) map[string][]*telemetrytypes.TelemetryFieldKey {
for _, selector := range selectors {
if selector.Signal != telemetrytypes.SignalTraces && selector.Signal != telemetrytypes.SignalUnspecified {
continue
}
for name, def := range telemetrytypes.GenAIFieldDefinitions {
if len(keys[name]) > 0 {
continue // already resolved from ingested data
}
if !selectorMatchesIntrinsicField(selector, def) {
continue
}
keyCopy := def
keys[name] = []*telemetrytypes.TelemetryFieldKey{&keyCopy}
}
}
return keys
}
func selectorMatchesIntrinsicField(selector *telemetrytypes.FieldKeySelector, definition telemetrytypes.TelemetryFieldKey) bool {
if selector.FieldContext != telemetrytypes.FieldContextUnspecified && selector.FieldContext != definition.FieldContext {
return false
@@ -1273,6 +1294,9 @@ func (t *telemetryMetaStore) GetKeys(ctx context.Context, orgID valuer.UUID, fie
applyBackwardCompatibleKeys(mapOfKeys)
mapOfKeys = enrichWithIntrinsicMetricKeys(mapOfKeys, selectors)
if t.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableAIObservability, featuretypes.NewFlaggerEvaluationContext(orgID)) {
mapOfKeys = enrichWithGenAIKeys(mapOfKeys, selectors)
}
return mapOfKeys, complete, nil
}
@@ -1351,6 +1375,9 @@ func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, orgID valuer.UUID
applyBackwardCompatibleKeys(mapOfKeys)
mapOfKeys = enrichWithIntrinsicMetricKeys(mapOfKeys, fieldKeySelectors)
if t.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableAIObservability, featuretypes.NewFlaggerEvaluationContext(orgID)) {
mapOfKeys = enrichWithGenAIKeys(mapOfKeys, fieldKeySelectors)
}
return mapOfKeys, complete, nil
}

View File

@@ -162,18 +162,28 @@ func (c *conditionBuilder) ConditionFor(
return nil, nil, err
}
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
keys := querybuilder.MatchingFieldKeys(key, fieldKeys)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)
}
if len(keys) == 0 {
return nil, warnings, querybuilder.NewKeyNotFoundError(key.Name)
if _, isColumn := timeSeriesV4Columns[key.Name]; isColumn {
keys = []*telemetrytypes.TelemetryFieldKey{key}
} else {
if len(fieldKeys[key.Name]) == 0 {
warnings = append(warnings, fmt.Sprintf("label `%s` not found in metadata; check the label name for typos", key.Name))
}
keys = []*telemetrytypes.TelemetryFieldKey{
telemetrytypes.NewTelemetryFieldKey(key.Name, telemetrytypes.FieldContextAttribute, key.FieldDataType),
}
if key.FieldContext != telemetrytypes.FieldContextUnspecified {
keys = append(keys, telemetrytypes.NewTelemetryFieldKey(
key.FieldContext.StringValue()+"."+key.Name, telemetrytypes.FieldContextAttribute, key.FieldDataType))
}
}
}
conds := make([]string, 0, len(keys))
for _, k := range keys {
cond, err := c.conditionForKey(ctx, orgID, startNs, endNs, k, operator, value, sb)
cond, err := c.conditionFor(ctx, orgID, startNs, endNs, k, operator, value, sb)
if err != nil {
return nil, nil, err
}
@@ -181,21 +191,3 @@ func (c *conditionBuilder) ConditionFor(
}
return conds, warnings, nil
}
func (c *conditionBuilder) conditionForKey(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) (string, error) {
condition, err := c.conditionFor(ctx, orgID, startNs, endNs, key, operator, value, sb)
if err != nil {
return "", err
}
return condition, nil
}

View File

@@ -307,3 +307,86 @@ func TestConditionForMultipleKeys(t *testing.T) {
})
}
}
func TestConditionForKeyNotInMetadata(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
key telemetrytypes.TelemetryFieldKey
fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey
operator qbtypes.FilterOperator
value any
expectedSQL []string
expectWarn bool
}{
{
name: "intrinsic metric_name full-text resolves without warning",
key: telemetrytypes.TelemetryFieldKey{Name: "metric_name", FieldContext: telemetrytypes.FieldContextMetric},
fieldKeys: map[string][]*telemetrytypes.TelemetryFieldKey{},
operator: qbtypes.FilterOperatorRegexp,
value: "k8s",
expectedSQL: []string{"match(metric_name, ?)"},
expectWarn: false,
},
{
name: "unknown label resolves to labels extract with a typo warning",
key: telemetrytypes.TelemetryFieldKey{Name: "foo", FieldContext: telemetrytypes.FieldContextUnspecified},
fieldKeys: map[string][]*telemetrytypes.TelemetryFieldKey{},
operator: qbtypes.FilterOperatorEqual,
value: "bar",
expectedSQL: []string{"JSONExtractString(labels, 'foo') = ?"},
expectWarn: true,
},
{
name: "context prefix that may be part of the name tries both readings",
key: telemetrytypes.TelemetryFieldKey{Name: "a.b.c", FieldContext: telemetrytypes.FieldContextScope},
fieldKeys: map[string][]*telemetrytypes.TelemetryFieldKey{},
operator: qbtypes.FilterOperatorEqual,
value: "x",
expectedSQL: []string{"JSONExtractString(labels, 'a.b.c') = ?", "JSONExtractString(labels, 'scope.a.b.c') = ?"},
expectWarn: true,
},
{
name: "unresolved metric-context name is treated as a label prefix",
key: telemetrytypes.TelemetryFieldKey{Name: "foo", FieldContext: telemetrytypes.FieldContextMetric},
fieldKeys: map[string][]*telemetrytypes.TelemetryFieldKey{},
operator: qbtypes.FilterOperatorEqual,
value: "bar",
expectedSQL: []string{"JSONExtractString(labels, 'foo') = ?", "JSONExtractString(labels, 'metric.foo') = ?"},
expectWarn: true,
},
{
name: "known label under a mismatched context collapses without warning",
key: telemetrytypes.TelemetryFieldKey{Name: "region", FieldContext: telemetrytypes.FieldContextResource},
fieldKeys: map[string][]*telemetrytypes.TelemetryFieldKey{
"region": {{Name: "region", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString}},
},
operator: qbtypes.FilterOperatorEqual,
value: "us",
expectedSQL: []string{"JSONExtractString(labels, 'region') = ?"},
expectWarn: false,
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
cond, warnings, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, &tc.key, tc.fieldKeys, qbtypes.ConditionBuilderOptions{}, tc.operator, tc.value, sb)
require.NoError(t, err)
sb.Where(cond...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
for _, want := range tc.expectedSQL {
assert.Contains(t, sql, want)
}
if tc.expectWarn {
assert.NotEmpty(t, warnings)
} else {
assert.Empty(t, warnings)
}
})
}
}

View File

@@ -0,0 +1,111 @@
package telemetryscopedtraces
import (
"context"
"strings"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
// CommonTraceColumns are domain-neutral columns any trace list can reuse. All
// aggregate over every span, so none is Orderable.
func CommonTraceColumns() []TraceColumn {
ts := IntrinsicSpanKey("timestamp")
duration := IntrinsicSpanKey("duration_nano")
name := IntrinsicSpanKey("name")
parentSpanID := IntrinsicSpanKey("parent_span_id")
serviceName := &telemetrytypes.TelemetryFieldKey{
Name: "service.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
return []TraceColumn{
{Alias: "start_time", Expr: FieldReduce(AggMin, ts)},
{Alias: "end_time", Expr: FieldReduce(AggMax, ts)},
// Not plain "duration_nano": that name is the intrinsic span field, and an
// alias would shadow it — both in ClickHouse identifier resolution and in
// bare-name filter classification.
{Alias: "trace_duration_nano", Expr: TraceDuration(ts, duration)},
{Alias: "span_count", Expr: CountAll()},
{Alias: "root_span_name", Expr: FieldAnyWhere(name, parentSpanID, qbtypes.FilterOperatorEqual, "")},
{Alias: "service.name", SpanLevel: true, Expr: AnyValue(serviceName, telemetrytypes.FieldDataTypeString)},
}
}
// fieldMapper resolves aggregate-column SQL through the shared field mapper and
// condition builder, following their method shapes (FieldFor / ConditionFor / …) so
// column resolution reads like the other statement builders. keys is the fetched
// metadata for the keys the columns reference; the gate mask is set by the builder
// after resolveMask (Scoped* aggregates embed it). All returned expressions are
// escaped once, ready to embed in an outer builder.
type fieldMapper struct {
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
keys map[string][]*telemetrytypes.TelemetryFieldKey
maskExpr string
maskArgs []any
}
func newFieldMapper(fm qbtypes.FieldMapper, cb qbtypes.ConditionBuilder, keys map[string][]*telemetrytypes.TelemetryFieldKey) *fieldMapper {
return &fieldMapper{fm: fm, cb: cb, keys: keys}
}
// FieldFor returns the column expression for key via the field mapper.
func (r *fieldMapper) FieldFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
expr, err := r.fm.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", err
}
return sqlbuilder.Escape(expr), nil
}
// ConditionFor returns a boolean predicate for key via the condition builder
// (materialized column when present, else map access).
func (r *fieldMapper) ConditionFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
// The condition builder owns key resolution: hand it the raw key plus the full
// metadata map and it matches/synthesizes the candidates itself.
conds, _, err := r.cb.ConditionFor(ctx, orgID, startNs, endNs, key, r.keys, qbtypes.ConditionBuilderOptions{}, op, value, sb)
if err != nil {
return "", nil, err
}
if len(conds) == 0 {
return "", nil, nil
}
// One condition per candidate variant (a key can be ingested under several data
// types); OR them all, like the visitor does for EXISTS.
if len(conds) == 1 {
sb.Where(conds[0])
} else {
sb.Where(sb.Or(conds...))
}
expr, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
expr = strings.TrimPrefix(expr, "WHERE ")
return sqlbuilder.Escape(expr), args, nil
}
// ExistsFor returns the EXISTS predicate for key.
func (r *fieldMapper) ExistsFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, []any, error) {
return r.ConditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorExists, nil)
}
// ValueFor returns the value expression for an attribute key. The metadata variant
// is preferred because it carries Materialized — a provider's static definition
// never does, so a promoted attribute would otherwise fall back to map access.
func (r *fieldMapper) ValueFor(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) (string, []any, error) {
if cands := r.keys[key.Name]; len(cands) > 0 {
key = cands[0]
}
expr, err := r.fm.ColumnExpressionFor(ctx, orgID, startNs, endNs, key, dt, r.keys)
if err != nil {
return "", nil, err
}
// Escape before embedding in the outer builder: a materialized column name carries
// `$$` (from the dotted attribute name), which go-sqlbuilder's Build would otherwise
// unescape to a single `$` and reference the wrong column.
return sqlbuilder.Escape(expr), nil, nil
}

View File

@@ -0,0 +1,247 @@
package telemetryscopedtraces
import (
"context"
"fmt"
"strings"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// This file is the extension surface of the scoped trace builder: the two contracts a
// span category implements (base condition + columns) and the Aggregate constructors
// the columns are declared with. All SQL rendering goes through the fieldMapper.
// BaseConditionProvider defines which spans are in scope. It only declares the gate
// (a filter expression + its field keys); the builder resolves the keys through the
// field mapper, so attribute access stays materialization-aware.
type BaseConditionProvider interface {
// FilterExpression is the grammar-level (EXISTS) gate, used on the delegated
// span-list path.
FilterExpression() string
// FieldKeys are the gate's keys, used to build the per-span mask
// (OR of resolved EXISTS conditions).
FieldKeys() []*telemetrytypes.TelemetryFieldKey
}
// ColumnProvider supplies the columns a trace list computes.
type ColumnProvider interface {
Columns() []TraceColumn
// DefaultOrderAlias is sorted by (desc) when the query gives no order.
DefaultOrderAlias() string
// AggregateAliases are the computed per-trace column names, used to classify a
// filter key as trace-level vs span-level. Excludes SpanLevel columns.
AggregateAliases() []string
}
// TraceColumn is one per-trace output column.
type TraceColumn struct {
// Alias must not reuse a physical span-index column name (e.g. duration_nano):
// ClickHouse resolves bare identifiers to same-SELECT aliases first, so any
// expression referencing that column would silently bind to the alias.
Alias string
// Orderable columns can be used in ORDER BY and the aggregate filter. All-span
// aggregates (span_count, trace_duration_nano, …) are display-only and set false.
Orderable bool
// SpanLevel columns surface a real span/resource attribute (service.name,
// input/output messages); a filter on them is applied span-level, so they are
// excluded from AggregateAliases.
SpanLevel bool
Expr Aggregate
}
// Aggregate renders one column's SQL through the fieldMapper and lists the attribute
// keys it references so the builder can pre-fetch their metadata. Build one with the
// constructors below; the zero value is not usable.
type Aggregate struct {
keys []*telemetrytypes.TelemetryFieldKey
render func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (expr string, args []any, err error)
}
// IntrinsicSpanKey references an intrinsic span-index field (timestamp, name, …) by
// its canonical name; the field mapper resolves it to the physical column.
func IntrinsicSpanKey(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
}
}
// AggFunc is a ClickHouse aggregate function name.
type AggFunc string
const (
AggSum AggFunc = "sum"
AggMax AggFunc = "max"
AggMin AggFunc = "min"
)
// PickDirection selects the earliest (argMin) or latest (argMax) span by ordering.
type PickDirection int
const (
PickLatest PickDirection = iota
PickEarliest
)
// CountAll renders count().
func CountAll() Aggregate {
return Aggregate{render: func(context.Context, valuer.UUID, uint64, uint64, *fieldMapper) (string, []any, error) {
return "count()", nil, nil
}}
}
// FieldReduce renders <fn>(<field>) over a field-mapper-resolved column.
func FieldReduce(fn AggFunc, key *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
f, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", nil, err
}
return fmt.Sprintf("%s(%s)", fn, f), nil, nil
}}
}
// TraceDuration renders the full-trace wall duration: last span end minus first
// span start.
func TraceDuration(tsKey, durationKey *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
ts, err := m.FieldFor(ctx, orgID, startNs, endNs, tsKey)
if err != nil {
return "", nil, err
}
dur, err := m.FieldFor(ctx, orgID, startNs, endNs, durationKey)
if err != nil {
return "", nil, err
}
return fmt.Sprintf("(max(toUnixTimestamp64Nano(%s) + %s) - min(toUnixTimestamp64Nano(%s)))", ts, dur, ts), nil, nil
}}
}
// FieldAnyWhere renders anyIf(<field>, <cond>) — the field value from any span
// matching the condition.
func FieldAnyWhere(valueKey, condKey *telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, condValue any) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, err := m.FieldFor(ctx, orgID, startNs, endNs, valueKey)
if err != nil {
return "", nil, err
}
cond, args, err := m.ConditionFor(ctx, orgID, startNs, endNs, condKey, op, condValue)
return fmt.Sprintf("anyIf(%s, %s)", v, cond), args, err
}}
}
// AnyValue renders any(<value>) over a metadata-resolved attribute value.
func AnyValue(key *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) Aggregate {
return Aggregate{keys: keysOf(key), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, args, err := m.ValueFor(ctx, orgID, startNs, endNs, key, dt)
return fmt.Sprintf("any(%s)", v), args, err
}}
}
// CountExists renders countIf(<key> EXISTS) — counts spans carrying key.
func CountExists(key *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: keysOf(key), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
cond, args, err := m.ExistsFor(ctx, orgID, startNs, endNs, key)
return fmt.Sprintf("countIf(%s)", cond), args, err
}}
}
// CondCount renders countIf(<cond>) over a condition-builder-resolved predicate.
func CondCount(key *telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
cond, args, err := m.ConditionFor(ctx, orgID, startNs, endNs, key, op, value)
return fmt.Sprintf("countIf(%s)", cond), args, err
}}
}
// Reduce renders <fn>(<value>) over a resolved numeric attribute value.
func Reduce(fn AggFunc, valueKey *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: keysOf(valueKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, args, err := m.ValueFor(ctx, orgID, startNs, endNs, valueKey, telemetrytypes.FieldDataTypeFloat64)
return fmt.Sprintf("%s(%s)", fn, v), args, err
}}
}
// ScopedReduce renders <fn>If(<field>, <gate mask>) over a field-mapper-resolved column.
func ScopedReduce(fn AggFunc, key *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
f, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", nil, err
}
return fmt.Sprintf("%sIf(%s, %s)", fn, f, m.maskExpr), append([]any{}, m.maskArgs...), nil
}}
}
// ScopedToKeyColumn renders <fn>If(<field>, <scopeKey> EXISTS) — a span-index field
// aggregated over spans carrying scopeKey (e.g. max LLM latency).
func ScopedToKeyColumn(fn AggFunc, columnKey, scopeKey *telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: keysOf(scopeKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
col, err := m.FieldFor(ctx, orgID, startNs, endNs, columnKey)
if err != nil {
return "", nil, err
}
cond, args, err := m.ExistsFor(ctx, orgID, startNs, endNs, scopeKey)
return fmt.Sprintf("%sIf(%s, %s)", fn, col, cond), args, err
}}
}
// PickBy renders argMinIf/argMaxIf(<value>, <orderField>, <value> EXISTS) — the value
// from the earliest/latest span that carries it.
func PickBy(valueKey *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType, orderKey *telemetrytypes.TelemetryFieldKey, dir PickDirection) Aggregate {
fn := "argMaxIf"
if dir == PickEarliest {
fn = "argMinIf"
}
return Aggregate{keys: keysOf(valueKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, vargs, err := m.ValueFor(ctx, orgID, startNs, endNs, valueKey, dt)
if err != nil {
return "", nil, err
}
order, err := m.FieldFor(ctx, orgID, startNs, endNs, orderKey)
if err != nil {
return "", nil, err
}
cond, cargs, err := m.ExistsFor(ctx, orgID, startNs, endNs, valueKey)
return fmt.Sprintf("%s(%s, %s, %s)", fn, v, order, cond), append(vargs, cargs...), err
}}
}
// UniqCount renders uniqIf(<value>, <value> EXISTS) — distinct count of an attribute.
func UniqCount(valueKey *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) Aggregate {
return Aggregate{keys: keysOf(valueKey), render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
v, vargs, err := m.ValueFor(ctx, orgID, startNs, endNs, valueKey, dt)
if err != nil {
return "", nil, err
}
cond, cargs, err := m.ExistsFor(ctx, orgID, startNs, endNs, valueKey)
return fmt.Sprintf("uniqIf(%s, %s)", v, cond), append(vargs, cargs...), err
}}
}
// SumOfKeys renders coalesce(sum(<v1>), 0) + coalesce(sum(<v2>), 0) + … over several
// numeric attributes. Coalesced because a key absent from every span sums to NULL and
// NULL + n = NULL — a trace with only output tokens would otherwise total NULL.
func SumOfKeys(dt telemetrytypes.FieldDataType, valueKeys ...*telemetrytypes.TelemetryFieldKey) Aggregate {
return Aggregate{keys: valueKeys, render: func(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, m *fieldMapper) (string, []any, error) {
parts := make([]string, 0, len(valueKeys))
var args []any
for _, k := range valueKeys {
v, vargs, err := m.ValueFor(ctx, orgID, startNs, endNs, k, dt)
if err != nil {
return "", nil, err
}
parts = append(parts, fmt.Sprintf("coalesce(sum(%s), 0)", v))
args = append(args, vargs...)
}
return strings.Join(parts, " + "), args, nil
}}
}
func keysOf(k *telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return []*telemetrytypes.TelemetryFieldKey{k}
}

View File

@@ -0,0 +1,781 @@
package telemetryscopedtraces
import (
"context"
"fmt"
"log/slog"
"sort"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
qbvariables "github.com/SigNoz/signoz/pkg/variables"
"github.com/huandu/go-sqlbuilder"
)
var (
ErrUnsupportedRequestType = errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported request type for the scoped trace builder")
)
// scopedTraceStatementBuilder builds a trace list scoped to one span category
// (e.g. gen_ai spans). The query shape is fixed; BaseConditionProvider decides which
// spans are in scope and ColumnProvider decides the per-trace columns, so a new
// category only needs a new pair of providers.
type scopedTraceStatementBuilder struct {
logger *slog.Logger
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
baseCond BaseConditionProvider
columnProvider ColumnProvider
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
}
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*scopedTraceStatementBuilder)(nil)
// NewScopedTraceStatementBuilder wires the generic trace-list builder. The field
// mapper / condition builder are built here, not injected — the list always scans the
// telemetrytraces span index. traceStmtBuilder (the delegate for the span-list path)
// is injected because the provider already has the canonical instance.
func NewScopedTraceStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
baseCond BaseConditionProvider,
columnProvider ColumnProvider,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
fl flagger.Flagger,
) qbtypes.StatementBuilder[qbtypes.TraceAggregation] {
scopedSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryscopedtraces")
fm := telemetrytraces.NewFieldMapper()
cb := telemetrytraces.NewConditionBuilder(fm)
// Same resource-fingerprint prune as the standard trace builder — the list scans
// the same span index.
resourceFilterStmtBuilder := telemetryresourcefilter.New[qbtypes.TraceAggregation](
settings,
telemetrytraces.DBName,
telemetrytraces.TracesResourceV3TableName,
telemetrytypes.SignalTraces,
telemetrytypes.SourceUnspecified,
metadataStore,
nil,
fl,
)
return &scopedTraceStatementBuilder{
logger: scopedSettings.Logger(),
metadataStore: metadataStore,
fm: fm,
cb: cb,
baseCond: baseCond,
columnProvider: columnProvider,
traceStmtBuilder: traceStmtBuilder,
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
}
}
func (b *scopedTraceStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
switch requestType {
case qbtypes.RequestTypeTrace:
return b.buildTraceListQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), query, variables)
case qbtypes.RequestTypeRaw:
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
default:
return nil, ErrUnsupportedRequestType
}
}
// buildDelegated ANDs the base gate into the user filter and delegates to the
// standard trace builder (the span-list / raw path).
func (b *scopedTraceStatementBuilder) buildDelegated(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
gate := b.baseCond.FilterExpression()
expr := gate
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
expr = fmt.Sprintf("(%s) AND (%s)", gate, query.Filter.Expression)
}
// shallow copy; only Filter is replaced, caller's query untouched
gated := query
gated.Filter = &qbtypes.Filter{Expression: expr}
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
// buildTraceListQuery wires the CTE pipeline: one windowed pass picks the top-N
// traces, then a bucket-pruned pass enriches only those.
// Helpers appear in this file in the order they run. start/end are nanoseconds.
//
// RESOLVE (keys/columns → SQL via the field mapper)
// fetchKeys metadata for every key we reference
// resolveMask the "span is in scope" predicate (OR of EXISTS)
// resolveColumns per-trace column SQL
// resolveListOrders which columns to ORDER BY
// splitFilter span-level predicate + trace-level HAVING
//
// BUILD
// matched one windowed, mask-pruned GROUP BY trace_id scan fusing gate + span
// │ filter + HAVING + ORDER BY + LIMIT/OFFSET → the top-N trace_ids
// ▼
// ranked [start,end] bounds of those traces, from the small summary table
// ▼
// buckets the ts_bucket_start values they touch, to prune the next scan
// ▼
// enrichment every per-trace column for those traces over their full extent
// (not window-clipped), scanning only their buckets
//
// Only Orderable columns are computable in the mask-pruned matched pass, so only they
// can be ordered or filtered on; all-span columns (span_count, …) are output-only.
func (b *scopedTraceStatementBuilder) buildTraceListQuery(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
startBucket := start/querybuilder.NsToSeconds - querybuilder.BucketAdjustment
endBucket := end / querybuilder.NsToSeconds
limit := query.Limit
if limit <= 0 {
limit = 100
}
// Resolve keys and columns once; all attribute access goes through the field mapper.
keys, err := b.fetchKeys(ctx, orgID)
if err != nil {
return nil, err
}
mapper := newFieldMapper(b.fm, b.cb, keys)
maskExpr, maskArgs, err := b.resolveMask(ctx, orgID, start, end, mapper)
if err != nil {
return nil, err
}
mapper.maskExpr, mapper.maskArgs = maskExpr, maskArgs
resolved, err := b.resolveColumns(ctx, orgID, start, end, mapper)
if err != nil {
return nil, err
}
orders, err := b.resolveListOrders(query.Order, resolved)
if err != nil {
return nil, err
}
orderableSet := orderableAliasSet(resolved)
// If the filter references resource attributes, add a __resource_filter CTE and
// narrow the matched scan by resource_fingerprint; the span predicate drops those
// keys so they aren't applied twice.
resourceFrag, resourceArgs, resourcePred, err := b.maybeAttachResourceFilter(ctx, orgID, query, start, end, variables)
if err != nil {
return nil, err
}
// Split the user filter: span-level predicate + trace-level HAVING expression.
fp, err := b.splitFilter(ctx, orgID, query, b.aggregateAliasSet(), orderableSet, start, end, variables)
if err != nil {
return nil, err
}
// matched → ranked → buckets → enrichment
matchedFrag, matchedArgs, err := b.buildMatchedCTE(start, end, startBucket, endBucket, resolved, orders, orderableSet, maskExpr, maskArgs, fp, resourcePred, limit, query.Offset)
if err != nil {
return nil, err
}
rankedFrag, rankedArgs := b.buildRankedCTE(start, end)
bucketsFrag := buildBucketsCTE()
mainSQL, mainArgs := b.buildEnrichmentSelect(resolved, orders)
cteFragments := []string{matchedFrag, rankedFrag, bucketsFrag}
cteArgs := [][]any{matchedArgs, rankedArgs, nil}
// __resource_filter must precede `matched`, which references it.
if resourceFrag != "" {
cteFragments = append([]string{resourceFrag}, cteFragments...)
cteArgs = append([][]any{resourceArgs}, cteArgs...)
}
finalSQL := querybuilder.CombineCTEs(cteFragments) + mainSQL + " SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000"
finalArgs := querybuilder.PrependArgs(cteArgs, mainArgs)
return &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
Warnings: fp.warnings,
WarningsDocURL: fp.warningsURL,
}, nil
}
// maybeAttachResourceFilter builds the __resource_filter CTE (fingerprints matching
// the filter's resource conditions) and the predicate narrowing the span scan by
// resource_fingerprint; with no resource conditions it returns empty fragments.
//
// Unlike the standard trace builder there is deliberately no skip-fingerprint
// fallback: falling back would leave the resource conditions inside the OR'd
// span-filter bucket, which changes trace membership (any span from the resource +
// any gen_ai span, instead of a gen_ai span from the resource). Resource conditions
// always scope the whole matched scan.
func (b *scopedTraceStatementBuilder) maybeAttachResourceFilter(
ctx context.Context,
orgID valuer.UUID,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
variables map[string]qbtypes.VariableItem,
) (cteFrag string, cteArgs []any, fingerprintPred string, err error) {
stmt, err := b.resourceFilterStmtBuilder.Build(
ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables,
)
if err != nil {
return "", nil, "", err
}
if stmt == nil {
return "", nil, "", nil
}
return fmt.Sprintf("__resource_filter AS (%s)", stmt.Query), stmt.Args,
"resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)", nil
}
// ---------------------------------------------------------------------------
// RESOLVE — turn keys/columns into field-mapper-aware SQL
// ---------------------------------------------------------------------------
func (b *scopedTraceStatementBuilder) fetchKeys(ctx context.Context, orgID valuer.UUID) (map[string][]*telemetrytypes.TelemetryFieldKey, error) {
fields := b.resolverFieldKeys()
selectors := make([]*telemetrytypes.FieldKeySelector, 0, len(fields))
for _, k := range fields {
selectors = append(selectors, &telemetrytypes.FieldKeySelector{
Name: k.Name,
Signal: k.Signal,
FieldContext: k.FieldContext,
SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact,
})
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
return keys, err
}
func (b *scopedTraceStatementBuilder) resolverFieldKeys() []*telemetrytypes.TelemetryFieldKey {
seen := make(map[string]struct{})
var out []*telemetrytypes.TelemetryFieldKey
add := func(k *telemetrytypes.TelemetryFieldKey) {
if k == nil {
return
}
if _, dup := seen[k.Name]; dup {
return
}
seen[k.Name] = struct{}{}
out = append(out, k)
}
for _, k := range b.baseCond.FieldKeys() {
add(k)
}
for _, c := range b.columnProvider.Columns() {
for _, k := range c.Expr.keys {
add(k)
}
}
return out
}
// resolveMask builds the per-span in-scope mask: OR of resolved EXISTS predicates
// over the base condition's field keys.
func (b *scopedTraceStatementBuilder) resolveMask(ctx context.Context, orgID valuer.UUID, start, end uint64, mapper *fieldMapper) (string, []any, error) {
fieldKeys := b.baseCond.FieldKeys()
parts := make([]string, 0, len(fieldKeys))
var args []any
for _, key := range fieldKeys {
e, a, err := mapper.ExistsFor(ctx, orgID, start, end, key)
if err != nil {
return "", nil, err
}
parts = append(parts, e)
args = append(args, a...)
}
return "(" + strings.Join(parts, " OR ") + ")", args, nil
}
// resolvedColumn is a column resolved to SQL via the field mapper; expr is escaped
// once, ready to embed in an outer SELECT.
type resolvedColumn struct {
alias string
expr string
args []any
orderable bool
}
// resolveColumns turns the declarative columns into SQL through the resolver, so all
// attribute access goes through the field mapper / condition builder.
func (b *scopedTraceStatementBuilder) resolveColumns(ctx context.Context, orgID valuer.UUID, start, end uint64, mapper *fieldMapper) ([]resolvedColumn, error) {
cols := b.columnProvider.Columns()
out := make([]resolvedColumn, 0, len(cols))
for _, c := range cols {
expr, args, err := c.Expr.render(ctx, orgID, start, end, mapper)
if err != nil {
return nil, err
}
out = append(out, resolvedColumn{alias: c.Alias, expr: expr, args: args, orderable: c.Orderable})
}
return out, nil
}
// listOrder is a sort key resolved to a column alias + direction; both the matched
// CTE and the enrichment ORDER BY it.
type listOrder struct {
alias string
direction string
}
// resolveListOrders maps order keys to the resolved orderable columns; non-orderable
// columns are rejected. Defaults to the column provider's default order.
func (b *scopedTraceStatementBuilder) resolveListOrders(order []qbtypes.OrderBy, resolved []resolvedColumn) ([]listOrder, error) {
byAlias := make(map[string]resolvedColumn, len(resolved))
orderable := make([]string, 0, len(resolved))
for _, rc := range resolved {
byAlias[rc.alias] = rc
if rc.orderable {
orderable = append(orderable, rc.alias)
}
}
if len(order) == 0 {
return []listOrder{{alias: b.columnProvider.DefaultOrderAlias(), direction: "DESC"}}, nil
}
orders := make([]listOrder, 0, len(order))
for _, o := range order {
direction := "DESC"
if o.Direction == qbtypes.OrderDirectionAsc {
direction = "ASC"
}
rc, ok := byAlias[o.Key.Name]
if !ok || !rc.orderable {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"unsupported order key %q for the trace list; orderable keys: %s", o.Key.Name, strings.Join(orderable, ", "))
}
orders = append(orders, listOrder{alias: rc.alias, direction: direction})
}
return orders, nil
}
// filterParts is the user filter split into a span-level predicate (widens the
// matched WHERE prune and becomes a countIf existence check in HAVING) and a
// trace-level HAVING expression.
type filterParts struct {
spanPred string
spanArgs []any
hasSpanFilter bool
havingExpr string
warnings []string
warningsURL string
}
// splitFilter splits query.Filter into a span-level predicate and a trace-level
// HAVING expression (an explicit query.Having is ANDed onto the latter), then
// validates the trace-level part against the matched-pass aggregates.
func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID valuer.UUID, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], classifySet, orderableSet map[string]struct{}, start, end uint64, variables map[string]qbtypes.VariableItem) (filterParts, error) {
var fp filterParts
// The legacy tracefield. spelling parses identically to trace.; only the
// user-facing form is supported here.
if (query.Filter != nil && strings.Contains(query.Filter.Expression, "tracefield.")) ||
(query.Having != nil && strings.Contains(query.Having.Expression, "tracefield.")) {
return fp, errors.NewInvalidInputf(errors.CodeInvalidInput, "\"tracefield.\" is not supported; use the \"trace.\" prefix")
}
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
spanExpr, traceExpr, err := querybuilder.SplitFilterForAggregates(query.Filter.Expression, classifySet)
if err != nil {
return fp, err
}
fp.havingExpr = traceExpr
if strings.TrimSpace(spanExpr) != "" {
pred, args, warnings, url, err := b.resolveSpanPredicate(ctx, orgID, start, end, spanExpr, variables)
if err != nil {
return fp, err
}
// pred is empty when the span-level keys were all resource attributes
// already handled by the __resource_filter CTE.
if strings.TrimSpace(pred) != "" {
fp.spanPred, fp.spanArgs, fp.hasSpanFilter = pred, args, true
}
fp.warnings, fp.warningsURL = warnings, url
}
}
if query.Having != nil && strings.TrimSpace(query.Having.Expression) != "" {
if fp.havingExpr != "" {
fp.havingExpr = fmt.Sprintf("(%s) AND (%s)", fp.havingExpr, query.Having.Expression)
} else {
fp.havingExpr = query.Having.Expression
}
}
// The span predicate binds variables via PrepareWhereClause; the HAVING is a plain
// text rewrite, so substitute variables here (list/IN quoting, __all__ drops the
// condition) before validating.
if strings.TrimSpace(fp.havingExpr) != "" && len(variables) > 0 {
replaced, err := qbvariables.ReplaceVariablesInExpression(fp.havingExpr, variables)
if err != nil {
return fp, err
}
fp.havingExpr = replaced
}
if err := validateAggregateFilter(fp.havingExpr, orderableSet); err != nil {
return fp, err
}
return fp, nil
}
// resolveSpanPredicate resolves a span-level filter expression to a bare boolean
// SQL predicate + args via the field mapper.
func (b *scopedTraceStatementBuilder) resolveSpanPredicate(ctx context.Context, orgID valuer.UUID, start, end uint64, expr string, variables map[string]qbtypes.VariableItem) (string, []any, []string, string, error) {
selectors := querybuilder.QueryStringToKeysSelectors(expr)
for i := range selectors {
selectors[i].Signal = telemetrytypes.SignalTraces
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
if err != nil {
return "", nil, nil, "", err
}
prepared, err := querybuilder.PrepareWhereClause(expr, querybuilder.FilterExprVisitorOpts{
Context: ctx,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
FieldKeys: keys,
// resource conditions are always handled by the __resource_filter CTE
SkipResourceFilter: true,
Variables: variables,
StartNs: start,
EndNs: end,
})
if err != nil {
return "", nil, nil, "", err
}
if prepared.IsEmpty() {
return "", nil, nil, "", nil
}
sb := sqlbuilder.NewSelectBuilder()
sb.AddWhereClause(prepared.WhereClause)
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
pred := sql[strings.Index(sql, "WHERE ")+len("WHERE "):]
return sqlbuilder.Escape(pred), args, prepared.Warnings, prepared.WarningsDocURL, nil
}
// buildMatchedCTE builds `matched`: the single windowed GROUP BY trace_id scan that
// fuses gate + span filter + HAVING + ORDER BY + LIMIT/OFFSET, selecting only the
// aliases the ORDER BY / HAVING reference.
func (b *scopedTraceStatementBuilder) buildMatchedCTE(start, end, startBucket, endBucket uint64, resolved []resolvedColumn, orders []listOrder, orderableSet map[string]struct{}, maskExpr string, maskArgs []any, fp filterParts, resourcePred string, limit, offset int) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
// SELECT trace_id + only the aggregates ORDER BY / HAVING reference (as aliases).
needed := neededMatchedAliases(orders, fp.havingExpr, orderableSet)
selects := []string{"trace_id"}
for _, rc := range resolved {
if _, ok := needed[rc.alias]; !ok {
continue
}
colExpr, err := embedExpr(sb, rc.expr, rc.args)
if err != nil {
return "", nil, err
}
selects = append(selects, colExpr+" AS "+quoteAlias(rc.alias))
}
sb.Select(selects...)
sb.From(spanTable())
// WHERE: window + prune to in-scope spans, widened by the span filter so its
// spans survive for the countIf existence check below.
win := windowWhere(sb, start, end, startBucket, endBucket)
mask, err := embedExpr(sb, maskExpr, maskArgs)
if err != nil {
return "", nil, err
}
prune := "(" + mask
if fp.hasSpanFilter {
spanPred, err := embedExpr(sb, fp.spanPred, fp.spanArgs)
if err != nil {
return "", nil, err
}
prune += " OR " + spanPred
}
prune += ")"
where := append(win, prune)
if resourcePred != "" {
where = append(where, resourcePred)
}
sb.Where(where...)
sb.GroupBy("trace_id")
// HAVING: the gate/span existence checks are only needed when the WHERE was
// widened by a span filter; otherwise the mask alone already enforces the gate.
var having []string
if fp.hasSpanFilter {
havingMask, err := embedExpr(sb, maskExpr, maskArgs)
if err != nil {
return "", nil, err
}
havingPred, err := embedExpr(sb, fp.spanPred, fp.spanArgs)
if err != nil {
return "", nil, err
}
having = append(having, "countIf("+havingMask+") > 0")
having = append(having, "countIf("+havingPred+") > 0")
}
if strings.TrimSpace(fp.havingExpr) != "" {
hv, err := b.buildHaving(fp.havingExpr, orderableSet)
if err != nil {
return "", nil, err
}
if hv != "" {
// hv carries user text with values inlined by the rewriter; escape it so a
// literal $ can't be read as an interpolation marker at Build time. The
// countIf entries above hold live placeholders and must stay unescaped.
having = append(having, sqlbuilder.Escape(hv))
}
}
if len(having) > 0 {
sb.Having(strings.Join(having, " AND "))
}
sb.OrderBy(orderClause(orders)...)
sb.Limit(limit)
if offset > 0 {
sb.Offset(offset)
}
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("matched AS (%s)", sql), args, nil
}
// buildRankedCTE builds `ranked`: [start,end] bounds per matched trace, read from the
// small trace-summary table.
func (b *scopedTraceStatementBuilder) buildRankedCTE(start, end uint64) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("trace_id", "min(start) AS t_start", "max(end) AS t_end")
sb.From(summaryTable())
sb.Where(
"trace_id GLOBAL IN (SELECT trace_id FROM matched)",
"end >= fromUnixTimestamp64Nano("+sb.Var(start)+")",
"start < fromUnixTimestamp64Nano("+sb.Var(end)+")",
)
sb.GroupBy("trace_id")
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("ranked AS (%s)", sql), args
}
// buildBucketsCTE builds `buckets`: the ts_bucket_start values the matched traces
// span, so the enrichment scan is primary-key pruned. No args.
func buildBucketsCTE() string {
adj := querybuilder.BucketAdjustment // 30-min bucket width in seconds
return fmt.Sprintf("buckets AS (SELECT DISTINCT b AS ts_bucket FROM ranked "+
"ARRAY JOIN range("+
"toUInt64(intDiv(toUnixTimestamp(t_start), %d) * %d - %d), "+
"toUInt64(intDiv(toUnixTimestamp(t_end), %d) * %d + %d), "+
"%d) AS b)", adj, adj, adj, adj, adj, adj, adj)
}
// buildEnrichmentSelect builds the final SELECT: every per-trace column for the
// matched traces over their full extent, scanning only their buckets.
//
// Accepted discrepancy: matched ranks/paginates on window-clipped values (and, with a
// resource filter, only over fingerprint-matching spans), while this pass recomputes
// and ORDER BYs full-trace values — so a trace with activity outside the window or
// resource can sort differently than it ranked. Page membership is unaffected
// (LIMIT/OFFSET runs only in matched); rows still sort by the values the user sees.
// Ordering by matched's values instead would re-run the matched scan (ClickHouse
// re-executes a CTE per reference) without fixing the visible cross-page artifact.
func (b *scopedTraceStatementBuilder) buildEnrichmentSelect(resolved []resolvedColumn, orders []listOrder) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
selects, selectArgs := selectAllColumns(resolved)
sb.Select(selects...)
sb.From(spanTable())
sb.Where(
"ts_bucket_start GLOBAL IN (SELECT ts_bucket FROM buckets)",
"trace_id GLOBAL IN (SELECT trace_id FROM ranked)",
)
sb.GroupBy("trace_id")
sb.OrderBy(orderClause(orders)...)
sql, builtArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return sql, append(append([]any{}, selectArgs...), builtArgs...)
}
// buildHaving rewrites a trace-level HAVING expression to the matched-pass column
// aliases. The rewriter matches raw key text, so the trace. form is mapped alongside
// the bare name (the legacy tracefield. spelling is rejected upfront in splitFilter).
func (b *scopedTraceStatementBuilder) buildHaving(havingExpr string, orderableSet map[string]struct{}) (string, error) {
columnMap := make(map[string]string, len(orderableSet)*2)
for a := range orderableSet {
columnMap[a] = quoteAlias(a)
columnMap[telemetrytypes.FieldContextTrace.StringValue()+"."+a] = quoteAlias(a)
}
return querybuilder.NewHavingExpressionRewriter().Rewrite(havingExpr, columnMap)
}
// ---------------------------------------------------------------------------
// Small shared SQL-builder utilities
// ---------------------------------------------------------------------------
// spanTable is the fully-qualified span index table.
func spanTable() string {
return fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.SpanIndexV3TableName)
}
// summaryTable is the fully-qualified trace-summary table.
func summaryTable() string {
return fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.TraceSummaryTableName)
}
// aggregateAliasSet is every trace-level column alias, used to classify filter keys
// as trace-level vs span-level.
func (b *scopedTraceStatementBuilder) aggregateAliasSet() map[string]struct{} {
set := make(map[string]struct{}, len(b.columnProvider.AggregateAliases()))
for _, a := range b.columnProvider.AggregateAliases() {
set[a] = struct{}{}
}
return set
}
// orderableAliasSet is the subset of aliases computable in the matched pass — the
// only ones usable in ORDER BY and the aggregate filter.
func orderableAliasSet(resolved []resolvedColumn) map[string]struct{} {
set := make(map[string]struct{})
for _, rc := range resolved {
if rc.orderable {
set[rc.alias] = struct{}{}
}
}
return set
}
// neededMatchedAliases is the minimal alias set the matched pass must select: those
// in ORDER BY plus those in the aggregate HAVING. Everything else is left to the
// enrichment scan.
func neededMatchedAliases(orders []listOrder, havingExpr string, orderableSet map[string]struct{}) map[string]struct{} {
needed := make(map[string]struct{})
for _, o := range orders {
needed[o.alias] = struct{}{}
}
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; ok {
needed[name] = struct{}{}
}
}
return needed
}
// traceAggregateNames extracts the aggregate names a trace-level HAVING expression
// references. QueryStringToKeysSelectors emits an extra attribute-context fallback
// selector for context-prefixed keys (`trace.x` → attribute "trace.x"); only the
// unspecified- and trace-context selectors name aggregates.
func traceAggregateNames(havingExpr string) []string {
var names []string
for _, sel := range querybuilder.QueryStringToKeysSelectors(havingExpr) {
if sel.FieldContext == telemetrytypes.FieldContextUnspecified || sel.FieldContext == telemetrytypes.FieldContextTrace {
names = append(names, sel.Name)
}
}
return names
}
// validateAggregateFilter rejects a trace-level filter referencing an aggregate not
// computable in the matched pass (e.g. span_count, trace_duration_nano).
func validateAggregateFilter(havingExpr string, orderableSet map[string]struct{}) error {
if strings.TrimSpace(havingExpr) == "" {
return nil
}
allowed := make([]string, 0, len(orderableSet))
for a := range orderableSet {
allowed = append(allowed, a)
}
sort.Strings(allowed)
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregate %q cannot be used in the trace-list filter; filterable aggregates: %s", name, strings.Join(allowed, ", "))
}
}
return nil
}
// embedExpr inlines a resolved expr into sb, replacing each `?` placeholder with a
// builder Var so the args are tracked in appearance order. Resolved exprs carry
// values only as bound args, so every `?` is a placeholder; a count mismatch would
// silently shift args into the wrong slots — error out instead.
func embedExpr(sb *sqlbuilder.SelectBuilder, expr string, args []any) (string, error) {
if n := strings.Count(expr, "?"); n != len(args) {
return "", errors.NewInternalf(errors.CodeInternal,
"scoped trace builder: %d placeholders != %d args embedding %q", n, len(args), expr)
}
var out strings.Builder
ai := 0
for i := 0; i < len(expr); i++ {
if expr[i] == '?' {
out.WriteString(sb.Var(args[ai]))
ai++
continue
}
out.WriteByte(expr[i])
}
return out.String(), nil
}
// windowWhere binds the time-window predicates to sb and returns them so the caller
// can add its own predicates in the same Where call.
func windowWhere(sb *sqlbuilder.SelectBuilder, start, end, startBucket, endBucket uint64) []string {
return []string{
sb.GE("timestamp", fmt.Sprintf("%d", start)),
sb.L("timestamp", fmt.Sprintf("%d", end)),
sb.GE("ts_bucket_start", startBucket),
sb.LE("ts_bucket_start", endBucket),
}
}
// orderClause renders the ORDER BY terms plus the trace_id tiebreak.
func orderClause(orders []listOrder) []string {
out := make([]string, 0, len(orders)+1)
for _, o := range orders {
out = append(out, fmt.Sprintf("%s %s", quoteAlias(o.alias), o.direction))
}
return append(out, "trace_id DESC")
}
// selectAllColumns renders `expr AS alias` for every resolved column, args in select
// order.
func selectAllColumns(resolved []resolvedColumn) ([]string, []any) {
selects := []string{"trace_id"}
var args []any
for _, rc := range resolved {
selects = append(selects, rc.expr+" AS "+quoteAlias(rc.alias))
args = append(args, rc.args...)
}
return selects, args
}
// quoteAlias backticks an alias containing characters special to the SQL builder.
func quoteAlias(alias string) string {
if strings.ContainsAny(alias, ".$`") {
return "`" + alias + "`"
}
return alias
}

View File

@@ -0,0 +1,30 @@
package telemetryscopedtraces
import (
"strings"
"testing"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/require"
)
// The full-pipeline golden tests live in pkg/telemetryai, which exercises this
// builder through its production provider pair. The tests here cover only what
// needs the package internals.
// embedExpr treats every `?` byte as a placeholder; a count/args mismatch (an expr
// carrying a literal `?`, or a dropped arg) must fail loudly instead of silently
// shifting every subsequent arg into the wrong placeholder.
func TestEmbedExpr_PlaceholderArgMismatch(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
out, err := embedExpr(sb, "x = ? AND y = ?", []any{1, 2})
require.NoError(t, err)
require.Equal(t, 2, strings.Count(out, "$"), "both placeholders bound as builder vars")
_, err = embedExpr(sb, "x = ? AND y LIKE 'a?b'", []any{1})
require.Error(t, err, "literal ? in the expr must not pass as a placeholder")
_, err = embedExpr(sb, "x = ?", []any{1, 2})
require.Error(t, err, "extra args must not be silently dropped")
}

View File

@@ -49,23 +49,24 @@ func (c *conditionBuilder) conditionFor(
// TODO(srikanthccv): maybe extend this to every possible attribute
if key.Name == "duration_nano" || key.Name == "durationNano" { // QoL improvement
if strDuration, ok := value.(string); ok {
duration, err := time.ParseDuration(strDuration)
if err == nil {
switch v := value.(type) {
case string:
if duration, err := time.ParseDuration(v); err == nil {
value = duration.Nanoseconds()
} else if f, err := strconv.ParseFloat(v, 64); err == nil {
value = int64(f)
} else {
duration, err := strconv.ParseFloat(strDuration, 64)
if err == nil {
value = duration
} else {
return "", errors.WrapInvalidInputf(err, errors.CodeInvalidInput, "invalid duration value: %s", strDuration)
}
return "", errors.WrapInvalidInputf(err, errors.CodeInvalidInput, "invalid duration value: %s", v)
}
case float64:
value = int64(v)
case float32:
value = int64(v)
}
} else {
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(key, value, fieldExpression, operator)
}
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(key, value, fieldExpression, operator)
// regular operators
switch operator {
// regular operators

View File

@@ -367,7 +367,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(response_status_code <> '', response_status_code, NULL)) AS `__GROUP_BY_KEY_0_responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> 0, duration_nano, NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_responseStatusCode` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(response_status_code <> '', response_status_code, NULL)) AS `__GROUP_BY_KEY_0_responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> 0, duration_nano, NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_responseStatusCode`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_responseStatusCode` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_responseStatusCode`",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(response_status_code <> '', response_status_code, NULL)) AS `__GROUP_BY_KEY_0_responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> 0, accurateCastOrNull(duration_nano, 'Float64'), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_responseStatusCode` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(response_status_code <> '', response_status_code, NULL)) AS `__GROUP_BY_KEY_0_responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> 0, accurateCastOrNull(duration_nano, 'Float64'), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_responseStatusCode`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_responseStatusCode` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_responseStatusCode`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,

View File

@@ -343,7 +343,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND toFloat64(response_status_code) < ?), A_AND_B AS (SELECT l.* FROM A AS l INNER JOIN B AS r ON l.trace_id = r.trace_id) SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, avg(multiIf(duration_nano <> 0, duration_nano, mapContains(attributes_number, 'duration_nano'), toFloat64(attributes_number['duration_nano']), NULL)) AS __result_0 FROM A_AND_B GROUP BY `service.name` ORDER BY __result_0 desc SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND toFloat64(response_status_code) < ?), A_AND_B AS (SELECT l.* FROM A AS l INNER JOIN B AS r ON l.trace_id = r.trace_id) SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, avg(multiIf(duration_nano <> 0, accurateCastOrNull(duration_nano, 'Float64'), mapContains(attributes_number, 'duration_nano'), toFloat64(attributes_number['duration_nano']), NULL)) AS __result_0 FROM A_AND_B GROUP BY `service.name` ORDER BY __result_0 desc SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), float64(400)},
},
expectedErr: nil,

View File

@@ -6,6 +6,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -106,10 +107,6 @@ func NewGettableTransaction(results []*TransactionWithAuthorization) []*Gettable
return gettableTransactions
}
func (groups TransactionGroups) Diff(desired TransactionGroups) (additions, deletions TransactionGroups) {
return desired.subtract(groups), groups.subtract(desired)
}
func (groups TransactionGroups) Value() (driver.Value, error) {
data, err := json.Marshal(groups)
if err != nil {
@@ -168,51 +165,6 @@ func (transaction *Transaction) TransactionKey() string {
return transaction.Relation.StringValue() + ":" + transaction.Object.Resource.Type.StringValue() + ":" + transaction.Object.Resource.Kind.String()
}
func (groups TransactionGroups) subtract(other TransactionGroups) TransactionGroups {
otherSelectors := other.selectorSet()
order := make([]string, 0)
grouped := make(map[string]*TransactionGroup)
for _, group := range groups {
for _, selector := range group.ObjectGroup.Selectors {
if _, ok := otherSelectors[group.selectorKey(selector)]; ok {
continue
}
groupKey := group.Relation.StringValue() + "|" + group.ObjectGroup.Resource.String()
out, ok := grouped[groupKey]
if !ok {
out = &TransactionGroup{Relation: group.Relation, ObjectGroup: coretypes.ObjectGroup{Resource: group.ObjectGroup.Resource, Selectors: make([]coretypes.Selector, 0)}}
grouped[groupKey] = out
order = append(order, groupKey)
}
out.ObjectGroup.Selectors = append(out.ObjectGroup.Selectors, selector)
}
}
result := make(TransactionGroups, 0, len(order))
for _, key := range order {
result = append(result, grouped[key])
}
return result
}
func (groups TransactionGroups) selectorSet() map[string]struct{} {
set := make(map[string]struct{})
for _, group := range groups {
for _, selector := range group.ObjectGroup.Selectors {
set[group.selectorKey(selector)] = struct{}{}
}
}
return set
}
func (group *TransactionGroup) selectorKey(selector coretypes.Selector) string {
return group.Relation.StringValue() + "|" + group.ObjectGroup.Resource.String() + "|" + selector.String()
}
func newTransactionGroup(raw rawTransactionGroup, index int) (*TransactionGroup, error) {
verb, err := coretypes.NewVerb(raw.Relation)
if err != nil {
@@ -236,6 +188,13 @@ func newTransactionGroup(raw rawTransactionGroup, index int) (*TransactionGroup,
selectors := make([]coretypes.Selector, 0, len(raw.ObjectGroup.Selectors))
for selectorIndex, rawSelector := range raw.ObjectGroup.Selectors {
if resourceType.Equals(coretypes.TypeTelemetryResource) {
rawSelector, err = telemetrytypes.NewTelemetryGrantSelector(rawSelector)
if err != nil {
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "transactionGroups[%d].objectGroup.selectors[%d]: %s", index, selectorIndex, err.Error())
}
}
selector, err := resourceType.Selector(rawSelector)
if err != nil {
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "transactionGroups[%d].objectGroup.selectors[%d]: %s", index, selectorIndex, err.Error())

View File

@@ -3,6 +3,7 @@ package authtypes
import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
openfgav1 "github.com/openfga/api/proto/openfga/v1"
)
@@ -68,6 +69,38 @@ func NewTuplesFromTransactionGroups(name string, orgID valuer.UUID, transactionG
return tuples, nil
}
func DiffTuples(existing, desired []*openfgav1.TupleKey) (additions, deletions []*openfgav1.TupleKey) {
key := func(tuple *openfgav1.TupleKey) string {
return tuple.GetUser() + "|" + tuple.GetRelation() + "|" + tuple.GetObject()
}
existingSet := make(map[string]struct{}, len(existing))
for _, tuple := range existing {
existingSet[key(tuple)] = struct{}{}
}
desiredSet := make(map[string]struct{}, len(desired))
for _, tuple := range desired {
desiredSet[key(tuple)] = struct{}{}
}
additions = make([]*openfgav1.TupleKey, 0)
for _, tuple := range desired {
if _, ok := existingSet[key(tuple)]; !ok {
additions = append(additions, tuple)
}
}
deletions = make([]*openfgav1.TupleKey, 0)
for _, tuple := range existing {
if _, ok := desiredSet[key(tuple)]; !ok {
deletions = append(deletions, tuple)
}
}
return additions, deletions
}
func MustNewTransactionGroupsFromTuples(tuples []*openfgav1.TupleKey) TransactionGroups {
objectsByRelation := make(map[string][]*coretypes.Object)
@@ -109,17 +142,29 @@ func NewTuplesFromTransactionsWithCorrelations(transactions []*Transaction, subj
return nil, nil, err
}
selectorStrings, err := newCheckSelectors(txn.Object.Resource.Type, txn.Object.Selector)
if err != nil {
return nil, nil, err
}
selectors := make([]coretypes.Selector, 0, len(selectorStrings))
for _, selectorString := range selectorStrings {
selector, err := txn.Object.Resource.Type.Selector(selectorString)
if err != nil {
return nil, nil, err
}
selectors = append(selectors, selector)
}
txnID := txn.ID.StringValue()
txnTuples := NewTuples(resource, subject, txn.Relation, []coretypes.Selector{txn.Object.Selector}, orgID)
tuples[txnID] = txnTuples[0]
if txn.Object.Selector.String() != coretypes.WildCardSelectorString {
wildcardSelector := txn.Object.Resource.Type.MustSelector(coretypes.WildCardSelectorString)
wildcardTuples := NewTuples(resource, subject, txn.Relation, []coretypes.Selector{wildcardSelector}, orgID)
for index, tuple := range NewTuples(resource, subject, txn.Relation, selectors, orgID) {
if index == 0 {
tuples[txnID] = tuple
continue
}
correlationID := valuer.GenerateUUID().StringValue()
tuples[correlationID] = wildcardTuples[0]
tuples[correlationID] = tuple
correlations[txnID] = append(correlations[txnID], correlationID)
}
}
@@ -214,3 +259,21 @@ func NewTransactionWithAuthorizationFromBatchResults(
return output
}
func newCheckSelectors(resourceType coretypes.Type, selector coretypes.Selector) ([]string, error) {
if resourceType.Equals(coretypes.TypeTelemetryResource) {
canonical, err := telemetrytypes.NewTelemetryGrantSelector(selector.String())
if err != nil {
return nil, err
}
return telemetrytypes.NewTelemetryGrantSelectors(canonical), nil
}
selectorStrings := []string{selector.String()}
if selector.String() != coretypes.WildCardSelectorString {
selectorStrings = append(selectorStrings, coretypes.WildCardSelectorString)
}
return selectorStrings, nil
}

View File

@@ -23,5 +23,5 @@ var (
TypeRole = Type{valuer.NewString("role"), regexp.MustCompile(`^([a-z-]{1,50}|\*)$`), []Verb{VerbAssignee, VerbCreate, VerbList, VerbRead, VerbUpdate, VerbDelete, VerbAttach, VerbDetach}}
TypeOrganization = Type{valuer.NewString("organization"), regexp.MustCompile(`^(^[0-9a-f]{8}(?:\-[0-9a-f]{4}){3}-[0-9a-f]{12}$|\*)$`), []Verb{VerbRead, VerbUpdate}}
TypeMetaResource = Type{valuer.NewString("metaresource"), regexp.MustCompile(`^(^[0-9a-f]{8}(?:\-[0-9a-f]{4}){3}-[0-9a-f]{12}$|\*)$`), []Verb{VerbCreate, VerbList, VerbRead, VerbUpdate, VerbDelete, VerbAttach, VerbDetach}}
TypeTelemetryResource = Type{valuer.NewString("telemetryresource"), regexp.MustCompile(`^(\*|[a-z_]{1,32}(/(\*|[A-Za-z0-9._%-]{1,128})){0,2})$`), []Verb{VerbRead}}
TypeTelemetryResource = Type{valuer.NewString("telemetryresource"), regexp.MustCompile(`^.{1,512}$`), []Verb{VerbRead}}
)

View File

@@ -1,6 +1,9 @@
package coretypes
import (
"crypto/sha256"
"encoding/hex"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -26,7 +29,18 @@ func (resourceTelemetryResource *resourceTelemetryResource) Prefix(orgID valuer.
}
func (resourceTelemetryResource *resourceTelemetryResource) Object(orgID valuer.UUID, selector string) string {
return resourceTelemetryResource.Prefix(orgID) + "/" + selector
if selector == WildCardSelectorString {
return resourceTelemetryResource.Prefix(orgID) + "/" + selector
}
return resourceTelemetryResource.Prefix(orgID) + "/" + telemetrySelectorHash(selector)
}
// Must stay stable: grant-time and check-time tuple objects both hash the selector
// here, so changing this invalidates every stored telemetry grant tuple.
func telemetrySelectorHash(selector string) string {
sum := sha256.Sum256([]byte(selector))
return hex.EncodeToString(sum[:16])
}
func (resourceTelemetryResource *resourceTelemetryResource) Scope(verb Verb) string {

View File

@@ -16,18 +16,10 @@ import (
const (
LLMCostFeatureType agentConf.AgentFeatureType = "llm_pricing"
GenAIRequestModel = "gen_ai.request.model"
GenAIProviderName = "gen_ai.provider.name"
GenAIUsageInputTokens = "gen_ai.usage.input_tokens"
GenAIUsageOutputTokens = "gen_ai.usage.output_tokens"
GenAIUsageCacheReadInputTokens = "gen_ai.usage.cache_read.input_tokens"
GenAIUsageCacheCreationInputTokens = "gen_ai.usage.cache_creation.input_tokens"
SignozGenAICostInput = "_signoz.gen_ai.cost_input"
SignozGenAICostOutput = "_signoz.gen_ai.cost_output"
SignozGenAICostCacheRead = "_signoz.gen_ai.cost_cache_read"
SignozGenAICostCacheWrite = "_signoz.gen_ai.cost_cache_write"
SignozGenAITotalCost = "_signoz.gen_ai.total_cost"
)
var (

View File

@@ -4,6 +4,7 @@ import (
"bytes"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"gopkg.in/yaml.v3"
)
@@ -83,11 +84,11 @@ func buildProcessorConfig(rules []*LLMPricingRule) *LLMPricingRuleProcessorConfi
return &LLMPricingRuleProcessorConfig{
Attrs: LLMPricingRuleProcessorAttrs{
Model: GenAIRequestModel,
In: GenAIUsageInputTokens,
Out: GenAIUsageOutputTokens,
CacheRead: GenAIUsageCacheReadInputTokens,
CacheWrite: GenAIUsageCacheCreationInputTokens,
Model: telemetrytypes.GenAIRequestModel,
In: telemetrytypes.GenAIUsageInputTokens,
Out: telemetrytypes.GenAIUsageOutputTokens,
CacheRead: telemetrytypes.GenAIUsageCacheReadInputTokens,
CacheWrite: telemetrytypes.GenAIUsageCacheCreationInputTokens,
},
DefaultPricing: LLMPricingRuleProcessorDefaultPricing{
Rules: pricingRules,
@@ -97,7 +98,7 @@ func buildProcessorConfig(rules []*LLMPricingRule) *LLMPricingRuleProcessorConfi
Out: SignozGenAICostOutput,
CacheRead: SignozGenAICostCacheRead,
CacheWrite: SignozGenAICostCacheWrite,
Total: SignozGenAITotalCost,
Total: telemetrytypes.SignozGenAITotalCost,
},
}
}

View File

@@ -613,7 +613,7 @@ func (o OrderBy) Copy() OrderBy {
type SecondaryAggregation struct {
// stepInterval of the query
// if not set, it will use the step interval of the primary aggregation
StepInterval Step `json:"stepInterval,omitempty"`
StepInterval Step `json:"stepInterval,omitzero"`
// expression to aggregate. example: count(), sum(item_price), countIf(day > 10)
Expression string `json:"expression"`
// if any, it will be used as the alias of the aggregation in the result

View File

@@ -16,7 +16,7 @@ type QueryBuilderQuery[T any] struct {
Name string `json:"name"`
// stepInterval of the query
StepInterval Step `json:"stepInterval,omitempty"`
StepInterval Step `json:"stepInterval,omitzero"`
// signal to query
Signal telemetrytypes.Signal `json:"signal,omitempty"`

View File

@@ -709,3 +709,40 @@ func TestQueryBuilderQuery_MetricAggregation_MarshalJSONEnumRoundTrip(t *testing
})
}
}
// stepInterval is a struct-backed Step, so omitempty had no effect and an unset
// value serialized as 0; ,omitzero omits it when unset while still echoing a set
// value (as seconds), keeping the create -> GET round-trip stable.
func TestQueryBuilderQuery_StepInterval_MarshalRoundTrip(t *testing.T) {
testCases := []struct {
name string
query QueryBuilderQuery[LogAggregation]
present []string
absent []string
}{
{
name: "UnsetStepIntervalIsOmitted",
query: QueryBuilderQuery[LogAggregation]{Name: "A", Signal: telemetrytypes.SignalLogs},
absent: []string{`"stepInterval"`},
},
{
name: "SetStepIntervalIsSerializedAsSeconds",
query: QueryBuilderQuery[LogAggregation]{Name: "A", Signal: telemetrytypes.SignalLogs, StepInterval: Step{60 * time.Second}},
present: []string{`"stepInterval":60`},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
expected, err := json.Marshal(testCase.query)
require.NoError(t, err)
for _, fragment := range testCase.present {
assert.Contains(t, string(expected), fragment)
}
for _, fragment := range testCase.absent {
assert.NotContains(t, string(expected), fragment)
}
})
}
}

View File

@@ -9,6 +9,7 @@ type QueryType struct {
var (
QueryTypeUnknown = QueryType{valuer.NewString("unknown")}
QueryTypeBuilder = QueryType{valuer.NewString("builder_query")}
QueryTypeBuilderAI = QueryType{valuer.NewString("builder_ai_query")}
QueryTypeFormula = QueryType{valuer.NewString("builder_formula")}
QueryTypeSubQuery = QueryType{valuer.NewString("builder_sub_query")}
QueryTypeJoin = QueryType{valuer.NewString("builder_join")}
@@ -21,6 +22,7 @@ var (
func (QueryType) Enum() []any {
return []any{
QueryTypeBuilder,
QueryTypeBuilderAI,
QueryTypeFormula,
// Not yet supported.
// QueryTypeSubQuery,

View File

@@ -62,6 +62,13 @@ type queryEnvelopeBuilder struct {
Spec builderQuerySpec `json:"spec" description:"The builder query specification."`
}
// queryEnvelopeBuilderAI is the OpenAPI schema for a builder_ai_query QueryEnvelope.
// The spec is always a traces builder query (the signal is implied by the type).
type queryEnvelopeBuilderAI struct {
Type QueryType `json:"type" required:"true" description:"The type of the query."`
Spec QueryBuilderQuery[TraceAggregation] `json:"spec" description:"The AI builder query specification."`
}
// queryEnvelopeFormula is the OpenAPI schema for a QueryEnvelope with type=builder_formula.
type queryEnvelopeFormula struct {
Type QueryType `json:"type" required:"true" description:"The type of the query."`
@@ -100,6 +107,7 @@ var _ jsonschema.OneOfExposer = QueryEnvelope{}
func (QueryEnvelope) JSONSchemaOneOf() []any {
return []any{
queryEnvelopeBuilder{},
queryEnvelopeBuilderAI{},
queryEnvelopeFormula{},
// queryEnvelopeJoin{}, // deferred — see commented queryEnvelopeJoin above
queryEnvelopeTraceOperator{},
@@ -120,6 +128,7 @@ func (QueryEnvelope) PrepareJSONSchema(s *jsonschema.Schema) error {
"propertyName": "type",
"mapping": map[string]string{
QueryTypeBuilder.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder",
QueryTypeBuilderAI.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI",
QueryTypeFormula.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula",
QueryTypeTraceOperator.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator",
QueryTypePromQL.StringValue(): "#/components/schemas/Querybuildertypesv5QueryEnvelopePromQL",
@@ -150,6 +159,16 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
}
q.Spec = spec
case QueryTypeBuilderAI:
// A dedicated AI query is always a traces builder query; the signal is
// implied by the type, so pin it rather than requiring the caller to send it.
var spec QueryBuilderQuery[TraceAggregation]
if err := json.Unmarshal(shadow.Spec, &spec); err != nil {
return err
}
spec.Signal = telemetrytypes.SignalTraces
q.Spec = spec
case QueryTypeFormula:
var spec QueryBuilderFormula
if err := json.Unmarshal(shadow.Spec, &spec); err != nil {
@@ -194,7 +213,7 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
"unknown query type %q",
shadow.Type,
).WithAdditional(
"Valid query types are: builder_query, builder_sub_query, builder_formula, builder_join, builder_trace_operator, promql, clickhouse_sql",
"Valid query types are: builder_query, builder_ai_query, builder_sub_query, builder_formula, builder_join, builder_trace_operator, promql, clickhouse_sql",
).WithSuggestions(errors.NewValidReferences(errors.NounQueryTypes, QueryType{}.Enum()...))
}

View File

@@ -130,7 +130,7 @@ func TestScalarData_MarshalJSON(t *testing.T) {
{4.0, 5.0, 6.0},
},
},
expected: `{"queryName":"test_query","columns":[{"name":"value","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[[1,2,3],[4,5,6]]}`,
expected: `{"queryName":"test_query","columns":[{"name":"value","signal":"","fieldContext":"","fieldDataType":"","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[[1,2,3],[4,5,6]]}`,
},
{
name: "scalar data with NaN",
@@ -149,7 +149,7 @@ func TestScalarData_MarshalJSON(t *testing.T) {
{math.Inf(1), 5.0, math.Inf(-1)},
},
},
expected: `{"queryName":"test_query","columns":[{"name":"value","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[[1,"NaN",3],["Inf",5,"-Inf"]]}`,
expected: `{"queryName":"test_query","columns":[{"name":"value","signal":"","fieldContext":"","fieldDataType":"","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[[1,"NaN",3],["Inf",5,"-Inf"]]}`,
},
{
name: "scalar data with mixed types",
@@ -168,7 +168,7 @@ func TestScalarData_MarshalJSON(t *testing.T) {
{nil, math.Inf(1), 3.14, false},
},
},
expected: `{"queryName":"test_query","columns":[{"name":"mixed","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[["string",42,"NaN",true],[null,"Inf",3.14,false]]}`,
expected: `{"queryName":"test_query","columns":[{"name":"mixed","signal":"","fieldContext":"","fieldDataType":"","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[["string",42,"NaN",true],[null,"Inf",3.14,false]]}`,
},
{
name: "scalar data with nested structures",
@@ -189,7 +189,7 @@ func TestScalarData_MarshalJSON(t *testing.T) {
},
},
},
expected: `{"queryName":"test_query","columns":[{"name":"nested","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[[{"count":10,"value":"NaN"},[1,"Inf",3]]]}`,
expected: `{"queryName":"test_query","columns":[{"name":"nested","signal":"","fieldContext":"","fieldDataType":"","queryName":"test_query","aggregationIndex":0,"meta":{},"columnType":"aggregation"}],"data":[[{"count":10,"value":"NaN"},[1,"Inf",3]]]}`,
},
{
name: "empty scalar data",

View File

@@ -51,7 +51,7 @@ type QueryBuilderTraceOperator struct {
Order []OrderBy `json:"order,omitzero"`
Aggregations []TraceAggregation `json:"aggregations,omitzero"`
StepInterval Step `json:"stepInterval,omitempty"`
StepInterval Step `json:"stepInterval,omitzero"`
GroupBy []GroupByKey `json:"groupBy,omitzero"`
// having clause to apply to the aggregated query results

View File

@@ -608,7 +608,7 @@ func (r *QueryRangeRequest) ValidateRequestScope() ([]ValidationOption, error) {
// Builder query names must be unique across the composite query.
queryNames := make(map[string]bool)
for _, envelope := range r.CompositeQuery.Queries {
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery {
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery || envelope.Type == QueryTypeBuilderAI {
name := envelope.GetQueryName()
if name != "" {
if queryNames[name] {
@@ -677,7 +677,7 @@ func (c *CompositeQuery) Validate(opts ...ValidationOption) error {
}
// Check name uniqueness for builder queries
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery {
if envelope.Type == QueryTypeBuilder || envelope.Type == QueryTypeSubQuery || envelope.Type == QueryTypeBuilderAI {
name := envelope.GetQueryName()
if name != "" {
if queryNames[name] {
@@ -711,6 +711,15 @@ func validateQueryEnvelope(envelope QueryEnvelope, opts ...ValidationOption) err
"unknown query spec type",
)
}
case QueryTypeBuilderAI:
spec, ok := envelope.Spec.(QueryBuilderQuery[TraceAggregation])
if !ok {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid AI builder query spec",
)
}
return spec.Validate(opts...)
case QueryTypeFormula:
spec, ok := envelope.Spec.(QueryBuilderFormula)
if !ok {
@@ -780,7 +789,7 @@ func validateQueryEnvelope(envelope QueryEnvelope, opts ...ValidationOption) err
"unknown query type: %s",
envelope.Type,
).WithAdditional(
"Valid query types are: builder_query, builder_sub_query, builder_formula, builder_join, promql, clickhouse_sql, trace_operator",
"Valid query types are: builder_query, builder_ai_query, builder_sub_query, builder_formula, builder_join, promql, clickhouse_sql, trace_operator",
).WithSuggestions(errors.NewValidReferences(errors.NounQueryTypes, QueryType{}.Enum()...))
}
}

View File

@@ -30,12 +30,16 @@ const (
)
type TelemetryFieldKey struct {
Name string `json:"name" validate:"required" required:"true"`
Description string `json:"description,omitempty"`
Unit string `json:"unit,omitempty"`
Signal Signal `json:"signal,omitzero"`
FieldContext FieldContext `json:"fieldContext,omitzero"`
FieldDataType FieldDataType `json:"fieldDataType,omitzero"`
Name string `json:"name" validate:"required" required:"true"`
Description string `json:"description,omitempty"`
Unit string `json:"unit,omitempty"`
// signal/fieldContext/fieldDataType always serialize (empty included): the empty
// value is a first-class "unspecified / any" selection a client can set, so it
// must round-trip verbatim rather than be dropped. Their Enum()s include the
// empty member so the "" is a valid schema value.
Signal Signal `json:"signal"`
FieldContext FieldContext `json:"fieldContext"`
FieldDataType FieldDataType `json:"fieldDataType"`
JSONPlan JSONAccessPlan `json:"-"`
Indexes []TelemetryFieldKeySkipIndex `json:"-"`

View File

@@ -76,6 +76,7 @@ var (
"log": FieldContextLog,
"metric": FieldContextMetric,
"tracefield": FieldContextTrace,
"trace": FieldContextTrace,
}
)
@@ -185,5 +186,6 @@ func (FieldContext) Enum() []any {
FieldContextAttribute,
// FieldContextEvent,
FieldContextBody,
FieldContextUnspecified,
}
}

View File

@@ -190,6 +190,7 @@ func (FieldDataType) Enum() []any {
FieldDataTypeFloat64,
FieldDataTypeInt64,
FieldDataTypeNumber,
FieldDataTypeUnspecified,
// FieldDataTypeArrayString,
// FieldDataTypeArrayFloat64,
// FieldDataTypeArrayBool,

Some files were not shown because too many files have changed in this diff Show More