Compare commits

..

42 Commits

Author SHA1 Message Date
nityanandagohain
d6d09549f0 fix: add NewFactory 2026-07-29 19:14:35 +05:30
nityanandagohain
d71c0c938a fix: remove accidentally added file 2026-07-29 17:01:11 +05:30
nityanandagohain
e911223b1d Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-29 16:45:11 +05:30
Tushar Vats
4e45620b72 refactor(statementbuilder): drop Builders bundle; fold factories into statement_builder.go (#12330)
Remove the statementbuilder.Builders aggregate. The querier chain (querier.New,
signozquerier.NewFactory, NewQuerierProviderFactories) now takes the six per-signal
statement builders individually, and signoz.go::newQueryStack returns them directly
via a plain multi-return. The statementbuilder package is now config-only.

Remove each <signal>statementbuilder/new.go and fold its NewFactory into that
package's statement_builder.go (traces' NewOperatorFactory into
trace_operator_statement_builder.go), giving the layout: struct -> NewFactory ->
New<X>QueryStatementBuilder.
2026-07-29 10:46:55 +00:00
Naman Verma
a9506f1354 fix: add meter source where needed during dashboard migration (#12322)
* fix: add meter source during migration

* chore: move source field handling to wrapinv5envelope

* fix: add migration to backfill source for migrated dashboards

* chore: shorten comments
2026-07-29 10:39:41 +00:00
nityanandagohain
cb3cc3de56 fix: remove comment 2026-07-29 16:04:49 +05:30
nityanandagohain
017e62814a Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-29 15:45:35 +05:30
Tushar Vats
e2e7caf1ca refactor(querier): 3-layer per-signal query architecture (#12304)
Split the five telemetry<signal> packages, which mixed three concerns, into
three per-signal layers with a cycle-free dependency direction:

- telemetryschema/<signal>telemetryschema — primitives (const + table
  selection, field_mapper, condition_builder, trace helpers); leaf layer.
- statementbuilder/<signal>statementbuilder — SQL generation. The parent
  statementbuilder package is contract-only (the Builders bundle + Config);
  each sub-package exposes a factory.ProviderFactory[..., statementbuilder.Config]
  whose New internalizes FieldMapper/ConditionBuilder/AggExprRewriter and reads
  SkipResourceFingerprint. Traces exposes two factories (query + operator).
  telemetryresourcefilter moves here as statementbuilder/resourcefilter.
- telemetrymetadata — key/value resolution; NewTelemetryMetaStore collapses
  from 24 args to (settings, telemetrystore, flagger), sourcing table names
  from the schema constants.

Centralize query-stack assembly in signoz.go via newQueryStack: build the
single metadata store, run each per-signal statement-builder factory, assemble
the statementbuilder.Builders bundle, and build the bucket cache — once. This
is the only place that imports the concrete sub-packages (so the edge runs
subs -> parent, cycle-free), and it removes the duplicate metadata store that
signozquerier used to build, leaving signozquerier a thin
querier.New(*statementbuilder.Builders) adapter.

Also:
- statementbuilder.Config owns SkipResourceFingerprint (moved off
  querier.Config). YAML key moves querier.skip_resource_fingerprint ->
  statementbuilder.skip_resource_fingerprint.
- Querier interface moves into querier.go (interfaces.go removed); BucketCache
  -> bucket_cache.go, Handler -> api.go.
- Add pkg/querier/queriertest.MockQuerier.
2026-07-29 09:24:08 +00:00
Nityananda Gohain
f10f526249 Merge branch 'main' into issue_5601 2026-07-29 14:14:37 +05:30
Srikanth Chekuri
bca2370862 test(promql): add upstream promqltest conformance corpus and suite (#12156)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
Freezes an absolute-truth oracle from prometheus@v0.311.3's own
promql/promqltest testdata: a generator command (scripts/promqltestcorpus, go run .)
evaluates upstream's load scripts with the vendored reference engine and
writes 755 cases to a committed corpus; a new integration suite replays
the ingestion and asserts /api/v5/query_range responses against it. The
known-divergences ledger is enforced exactly in both directions and is
empty: the serving path matches the reference engine on every case.

Also folds in the last serving-path fix the corpus surfaced: the v5
output filter dropped every __-prefixed label, mangling labelsets that
legitimately carry one (e.g. __address__); it now strips only known
storage keys (__temporality__, __scope./__resource. prefixes).

The metrics fixture writes registration rows per (series, hour bucket)
with hour-floored timestamps, matching the exporter; the queriermetrics
dormant-metric warning test now uses genuinely dormant data (production-
shaped registration shares an hour bucket with recently-stale data).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 03:54:22 +00:00
Gaurav Tewari
591837152e fix(warning-popover): background color to l2 & message alignment (#12302)
Some checks failed
build-staging / prepare (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
* fix: update css styles

* feat: add warning popover

* chore: revert bg color change

* feat: update styles

* chore: css conventios

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-07-28 17:16:05 +00:00
Pandey
2c5b4184c4 feat(querier): log user-authored clickhouse sql after running them through the parser (#12301)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* fix(querier): restrict user-authored clickhouse sql to read-only selects

Validate every user-authored ClickHouse statement before it runs: exactly one
statement, SELECT only, no table functions and no readonly override. Validation
runs on the rendered statement, since substituted variable values are user input
too.

Apply it to both entry points that reach the telemetry store with user SQL, the
clickhouse_sql query type in query_range and the dashboard variables query, and
replace the substring blacklist in the latter, which ran before substitution.

Also run these statements with readonly = 2 as a backstop. The connection is
shared with write paths, so it is opt-in per query and writers never set it.

* fix(querier): address review on clickhouse sql validation

Rename the validator to ValidateReadOnlySelect and collapse the multi-line error
constructions onto single lines.

Carry the parse failure in the message rather than as a wrapped cause. Neither
renderer surfaces the cause: render.Error reads only the message off the base
error, and RespondError reads Error(), which returns the cause alone and drops
the message. Both now show the same text with a 400.

Add unit tests covering statement kinds, table functions nested in joins, CTEs,
subqueries and unions, and readonly overrides.

* fix(querier): reject internal databases in user-authored clickhouse sql

Reading system or information_schema exposes grants, users and server metadata,
and ClickHouse read-only mode does not prevent it, so reject any table reference
into them.

Update the dashboard variables test for the new rejection message and cover both
a statement smuggled through a variable value and a system table read.

* fix(querier): reject unterminated block comments before parsing

The parser loops forever on an unterminated block comment, so validating a query
containing one would spin a request goroutine at full CPU instead of rejecting
it. Detect it up front, skipping comment markers that sit inside string literals.

Cover the parser panicking on a settings clause with no value in its own test,
which asserts the input still panics the parser so the case cannot quietly stop
exercising the recover.

* fix(querier): drop the block comment guard now the parser handles it

The parser no longer loops on an unterminated block comment, so the hand-rolled
scan that worked around it is redundant, and it was the riskier of the two: it
duplicated the lexer's handling of string literals and could have rejected a
legitimate query. Leave the parser as the single source of truth.

The panic case likewise no longer panics, so its test can no longer cover the
recover and is removed. The recover stays as insurance, since this reaches
user-authored SQL and the parser has regressed this way before.

Cover INTERSECT and EXCEPT, which the parser only started accepting in this
version and which reach a second query through the same rules.

* fix(querier): log clickhouse sql validation failures instead of rejecting

The parser's grammar has gaps against SQL that ClickHouse accepts, so rejecting
whatever it cannot read would break working dashboards and alerts. Sampling a
week of production clickhouse_sql found roughly one query in eighteen tripping
one of three gaps, none of them reading anything a telemetry query should not.

Log the failure with the rendered query instead, so the gaps can be told apart
from statements that genuinely break the rules before anything is enforced.

Wrap the parse error rather than formatting it in, so a caller can recover the
parser's *ParseError and read the position off it. The tests use that to pin the
construct each gap stops at, alongside the rewrite that the parser does accept.

* chore(querier): tighten the clickhouse sql validation comments

Condense the comments to single lines, move the recover note inside the defer it
explains, separate the visitor cases, and shorten the log message. Report how
many statements were found when rejecting a multi-statement query.

* fix(querier): log invalid clickhouse sql on the dashboard variables path too

The two callers disagreed: query_range logged and carried on, while dashboard
variables rejected. Given the parser trips on roughly one real query in eighteen,
that path could refuse a legitimate variable query, and being a rejection it left
nothing behind to show it had happened.

Both now go through LogIfStatementIsNotValid. prepareQuery is back to rendering
and nothing else, so the cases that expected it to police the statement move to
where the rules actually live.

* fix(querier): drop the read-only clickhouse session

Setting readonly on the session was defence in depth for a validator that now
only logs, so it guarded nothing while adding a context key and a settings branch
to a connection that is shared with every write path.

Restore the dashboard variables handler to what it was and call the validation
alongside, rather than reworking the rendering to accommodate it.

* chore(querybuilder): drop the redundant suffix from the failing case names

The test they sit in is already named _Fail.
2026-07-28 14:54:42 +00:00
Gaurav Tewari
0b69f5f677 fix: smooth Only/Toggle hover reveal in quick-filters checkbox (#12248)
The Only and Toggle buttons were revealed by overlapping hover triggers
(`.value` for Toggle, `.checkbox-value-section` for Only), so hovering the
label matched both rules and showed the two buttons side by side, shifting
the row layout — the visible hover "jerk".

- Scope the Toggle trigger to the checkbox (`.check-box:hover ~ ...`) so it
  no longer overlaps the label region; Only and Toggle are now mutually
  exclusive by cursor position.
- Stack both buttons in a single grid cell (`.value-actions`) so revealing
  one swaps in place instead of shifting the row.
- Add a fade + slide entry/exit animation (opacity + translateX with
  `@starting-style` and `display` `allow-discrete`), disabled under
  prefers-reduced-motion.

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-07-28 12:21:39 +00:00
Abhi kumar
a28b95da35 fix(dashboard-v2): keep new-panel Save enabled after running the query (#12311)
* fix(dashboard-v2): keep new-panel Save enabled after running the query

A new panel's Save button re-disabled after Stage & Run: the query-dirty
baseline was derived from the live draft.spec.queries, which the run commits
the current query into, so the baseline drifted onto the just-run query and
isQueryDirty flipped back to false. Freeze the seed used for the new-panel
baseline so a run can't reset it. Existing panels are unaffected (they anchor
to savedQueries).

* fix(dashboard-v2): always allow Save and surface the real save-failure message

Two follow-ups to the new-panel Save regression:

- Decouple the Save button from the dirty check so it is never disabled on
  account of "no changes"; Save is gated only by read-only / in-flight. The
  dirty flag still drives the discard-on-close confirmation.
- On a failed save, show the server's error message in the toast description
  instead of only the generic "Failed to save panel".

* feat(dashboard-v2): show the "Unsaved Changes" badge only when the panel is dirty

Gate the panel-editor header badge on isDirty so it appears only when there
are unsaved edits, and import Badge from the @signozhq/ui/badge subpath.

* chore: replaced toast with the error modal

* fix: dashboard delete calling v1 api
2026-07-28 11:44:55 +00:00
Ashwin Bhatkal
b4b8a9fda0 fix(api): guard ErrorResponseHandlerV2 against non-envelope error bodies (#12209)
* fix(api): guard ErrorResponseHandlerV2 against non-envelope error bodies

The handler assumed every non-2xx response carried the V2 error envelope
and read response.data.error.code directly. During a deployment the
gateway returns a 5xx with an HTML/empty body, so response.data.error is
undefined and the handler threw its own TypeError, masking the real
failure and crashing the caller.

Type the inbound body as unknown and narrow it with an isErrorV2Resp type
guard; when the body isn't a V2 envelope, synthesize an APIError from the
HTTP status instead of throwing. Callers already pass
AxiosError<ErrorV2Resp>, which stays assignable to AxiosError<unknown>.

* refactor(api): keep ErrorV2Resp signature, launder response.data via unknown

Restore the AxiosError<ErrorV2Resp> signature so it documents that this is
the V2 error handler, and confine the runtime uncertainty to the one spot
that matters: launder response.data through a local unknown binding before
narrowing it with the isErrorV2Resp guard.

* fix(api): use UPSTREAM_UNAVAILABLE code for non-envelope error bodies

Address review: when the response body isn't a V2 envelope (e.g. a gateway
5xx during a deploy), throw a stable UPSTREAM_UNAVAILABLE code instead of
the stringified HTTP status, and trim the guard comment.

* fix(api): use UPSTREAM_UNAVAILABLE fallback code in convertToApiError

When the response body carries no error code, fall back to a stable
UPSTREAM_UNAVAILABLE code instead of a stringified HTTP status, mirroring
the ErrorResponseHandlerV2 fallback.

* test(dashboard-v2): update panelStatus fallback code to UPSTREAM_UNAVAILABLE

convertToApiError now returns UPSTREAM_UNAVAILABLE (not a stringified
status) when the response carries no error code; update the panelStatus
fallback assertion to match.

* fix(api): guard generated-API handler + strengthen V2 handler tests

- Guard the deprecated ErrorResponseHandlerForGeneratedAPIs against a
  non-envelope response body (gateway 5xx with HTML/empty body), mirroring
  ErrorResponseHandlerV2 — falls back to UPSTREAM_UNAVAILABLE instead of
  throwing on response.data.error.code.
- Parametrize the V2 handler tests into an { error, expected } table and
  assert the sub-error 'errors' messages, which several UI surfaces rely on.

* revert(api): drop generated-API handler guard from this PR

The deprecated ErrorResponseHandlerForGeneratedAPIs guard broke toAPIError's
defaultMessage fallback (which relied on the handler crashing), regressing the
error UX in ServiceAccount/Roles screens. Moved to a stacked PR + tracked in
engineering-pod#5761. Keeps this PR scoped to ErrorResponseHandlerV2 +
convertToApiError, and the strengthened V2 handler tests remain.

* fix(api): guard deprecated generated-API handler against non-envelope bodies (#12228)
2026-07-28 11:31:26 +00:00
nityanandagohain
a5350682c3 fix: remove tracefield. explicit rejection 2026-07-28 16:54:26 +05:30
nityanandagohain
a7b74005ea fix: address comments 2026-07-28 16:43:20 +05:30
Ashwin Bhatkal
de59c123e1 fix(metrics-explorer): fetch related dashboards from the v3 API (#12312)
The metric details drawer resolved "N dashboards" through
`GET /api/v2/metrics/dashboards`, whose backend walks the raw `data["widgets"]`
array and so only matches v1-schema dashboards. Since #12249 migrated stored
dashboards to the Perses v6 schema and made `/dashboard/:id` render V2
unconditionally, that endpoint has nothing left to match and the popover
silently shows no dashboards.

Switch to `useGetMetricDashboardsV2` (`GET /api/v3/metrics/dashboards`, added in
#11784), which parses the Perses spec. Response items change from
`MetricDashboard` (`widgetId`/`widgetName`) to `DashboardPanelRef`
(`panelId`/`panelName`); the popover only reads `dashboardId`/`dashboardName`,
so the render path is unchanged.

Alerts stay on `/api/v2/metrics/alerts` — there is no v3 counterpart, and it
reads the rule store rather than dashboard schema, so it is unaffected.
2026-07-28 10:33:56 +00:00
Nityananda Gohain
787e1be974 Merge branch 'main' into issue_5601 2026-07-28 15:55:47 +05:30
nityanandagohain
19ee51be66 fix: address comments 2026-07-28 15:50:28 +05:30
Abhi kumar
9aeb9a8240 fix(drilldown): carry the field type through a breakout (#12309)
getBreakoutQuery rebuilt the group-by as `{ key, type }`, dropping `dataType`.
The breakout query then goes out with no field type for the backend to work
from, and a drilldown on the breakout's own result has nothing to branch on —
so a numeric column's filter value gets quoted like a string.

Carrying it through needed the type to be nameable, and it wasn't:
QueryKeyDataSuggestionsProps declared `fieldDataType` as the antlr key-type
enum (`string | number | boolean`), which that endpoint never returns. Five
call sites already cast it back to FieldDataType to get work done.

- declare the suggestions response's `fieldDataType` as FieldDataType, and drop
  the casts that existed only to undo the wrong declaration
- convert where a reported type becomes a query-builder one, so the crossing is
  a function rather than a cast. The table is keyed `Record<FieldDataType, …>`,
  so a type added to the API union fails to compile here instead of going
  missing at runtime
- carry `dataType` through getBreakoutQuery; the object now satisfies
  BaseAutocompleteData on its own, so that cast goes too

Depends on the operator-lookup fix in #12298: a numeric breakout now carries
`float64`, and before that change the menu threw on anything the operator map
didn't have.
2026-07-28 07:38:29 +00:00
Pandey
44828b4185 chore(deps): bump clickhouse-sql-parser to v0.5.2 (#12303)
Some checks failed
build-staging / js-build (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* chore(deps): bump clickhouse-sql-parser to v0.5.2

The parser hangs in an infinite loop on an unterminated block comment and panics
on a settings clause with no value, both reachable from user-authored SQL. v0.5.2
fixes those, along with Accept and Walk missing AST children, which anything
walking the tree to inspect a query depends on.

String() on Expr is gone in favour of FormatSQL, so the callers that rendered a
node back to SQL now go through the Format helper, which formats compactly the
way String() did.

* test(querierlogs): cover the rate aggregation family

rate and rate_sum divide by the query window rather than the step, and nothing
exercised that path, so a change to how the aggregation expression is rendered
would have gone unnoticed. Assert both against the inserted logs.

The window is now passed to the request helper instead of relying on its default,
since these are the only assertions in the file that depend on it.

* test(querierlogs): match the response rounding for rate expectations

Scalar responses round floats to three significant figures below one, and the
rates are the only values in this test that do not divide evenly, so compare
against the rounded form rather than the exact quotient.

* test(queriertraces): cover the rate aggregation family

Traces derives the scalar rate window separately from logs, so a divergence
between the two would go unnoticed with only the logs case. rate_sum here is
taken over the intrinsic duration column, which also puts it on the other side
of the response rounding boundary from the logs test.
2026-07-27 20:30:16 +00:00
nityanandagohain
b2c5af428e fix: refactor as requested 2026-07-24 12:07:37 +05:30
nityanandagohain
093f9b41c4 fix: minor cleanup 2026-07-22 16:22:19 +05:30
nityanandagohain
596e128005 fix: updated openapi 2026-07-22 15:26:52 +05:30
nityanandagohain
2e1da92367 fix: remove source and change to builder ai query 2026-07-22 15:24:43 +05:30
nityanandagohain
1e8d72e8fa Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-21 17:08:34 +05:30
nityanandagohain
cce080e1ae fix: add back the flag in metadata 2026-07-16 11:37:57 +05:30
nityanandagohain
deca060a4d Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-16 11:00:55 +05:30
nityanandagohain
03c7e524e7 fix: fix tests 2026-07-15 20:23:33 +05:30
nityanandagohain
815dc7d88b Merge remote-tracking branch 'origin/main' into issue_5601 2026-07-15 20:10:05 +05:30
nityanandagohain
f50d9199fe fix: address comments 2026-07-15 19:47:39 +05:30
nityanandagohain
31efe177a4 fix: address comments 2026-07-14 18:53:30 +05:30
nityanandagohain
d502d12ac3 fix: update openapi 2026-07-10 14:27:07 +05:30
nityanandagohain
bd9f15a716 fix: update integration test 2026-07-10 14:21:16 +05:30
nityanandagohain
813ef988c9 fix: edge cases and correct cost key 2026-07-10 12:06:34 +05:30
nityanandagohain
40e6799285 fix: add resource fingerprint cte 2026-07-10 00:36:25 +05:30
nityanandagohain
1caa60a3cd fix: cleanup and more tests 2026-07-09 23:54:05 +05:30
nityanandagohain
3f781f0083 fix: more cleanup 2026-07-09 12:39:01 +05:30
nityanandagohain
6aec05cf7a fix: more tests 2026-07-09 08:45:22 +05:30
nityanandagohain
683a52f35a fix: take perf into consideration 2026-07-09 08:45:22 +05:30
nityanandagohain
e924fa1e62 feat: support llm trace list and span list 2026-07-09 08:45:20 +05:30
195 changed files with 138282 additions and 1493 deletions

View File

@@ -53,7 +53,9 @@ jobs:
- queriermetrics
- querierscalar
- queriercommon
- querierai
- rawexportdata
- promqlconformance
- querierauthz
- role
- rootuser

View File

@@ -2,9 +2,9 @@ package main
import (
"github.com/SigNoz/signoz/pkg/metercollector"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
"github.com/SigNoz/signoz/pkg/types/zeustypes"
)
@@ -25,8 +25,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterLogSize,
Unit: zeustypes.MeterUnitBytes,
Aggregation: zeustypes.MeterAggregationSum,
DBName: telemetrylogs.DBName,
TableName: telemetrylogs.LogsV2LocalTableName,
DBName: logstelemetryschema.DBName,
TableName: logstelemetryschema.LogsV2LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultLogsRetentionDays,
},
},
@@ -36,8 +36,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterSpanSize,
Unit: zeustypes.MeterUnitBytes,
Aggregation: zeustypes.MeterAggregationSum,
DBName: telemetrytraces.DBName,
TableName: telemetrytraces.SpanIndexV3LocalTableName,
DBName: tracestelemetryschema.DBName,
TableName: tracestelemetryschema.SpanIndexV3LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultTracesRetentionDays,
},
},
@@ -47,8 +47,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterDatapointCount,
Unit: zeustypes.MeterUnitCount,
Aggregation: zeustypes.MeterAggregationSum,
DBName: telemetrymetrics.DBName,
TableName: telemetrymetrics.SamplesV4LocalTableName,
DBName: metricstelemetryschema.DBName,
TableName: metricstelemetryschema.SamplesV4LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultMetricsRetentionDays,
},
},

View File

@@ -6899,6 +6899,7 @@ components:
Querybuildertypesv5QueryEnvelope:
discriminator:
mapping:
builder_ai_query: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI'
builder_formula: '#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula'
builder_query: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder'
builder_trace_operator: '#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator'
@@ -6907,6 +6908,7 @@ components:
propertyName: type
oneOf:
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilder'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeBuilderAI'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeFormula'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopeTraceOperator'
- $ref: '#/components/schemas/Querybuildertypesv5QueryEnvelopePromQL'
@@ -6921,6 +6923,15 @@ components:
required:
- type
type: object
Querybuildertypesv5QueryEnvelopeBuilderAI:
properties:
spec:
$ref: '#/components/schemas/Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregation'
type:
$ref: '#/components/schemas/Querybuildertypesv5QueryType'
required:
- type
type: object
Querybuildertypesv5QueryEnvelopeClickHouseSQL:
properties:
spec:
@@ -7034,6 +7045,7 @@ components:
Querybuildertypesv5QueryType:
enum:
- builder_query
- builder_ai_query
- builder_formula
- builder_trace_operator
- clickhouse_sql

View File

@@ -16,7 +16,7 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/metercollector"
"github.com/SigNoz/signoz/pkg/modules/retention"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/licensetypes"
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
@@ -153,7 +153,7 @@ func (provider *Provider) Collect(
func buildOriginQuery(meterName string) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("toInt64(ifNull(min(unix_milli), 0))")
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
sb.Where(sb.Equal("metric_name", meterName))
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
@@ -171,7 +171,7 @@ func buildQuery(meterName string, segment *retentiontypes.RetentionPolicySegment
sb := sqlbuilder.NewSelectBuilder()
sb.Select(selects...)
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
sb.Where(
sb.Equal("metric_name", meterName),
sb.GTE("unix_milli", segment.StartMs),

View File

@@ -8,16 +8,16 @@ import (
sqlbuilder "github.com/huandu/go-sqlbuilder"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/metricreductionruletypes"
)
var (
reductionRulesTable = telemetrymetrics.DBName + "." + telemetrymetrics.ReductionRulesTableName
metadataTable = telemetrymetrics.DBName + "." + telemetrymetrics.AttributesMetadataTableName
bufferSeriesTable = telemetrymetrics.DBName + "." + telemetrymetrics.TimeseriesV4BufferTableName
reductionRulesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.ReductionRulesTableName
metadataTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.AttributesMetadataTableName
bufferSeriesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.TimeseriesV4BufferTableName
)
const timeSeriesBucketMilli = int64(time.Hour / time.Millisecond)
@@ -192,7 +192,7 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", "uniq(fingerprint)")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
conds := []string{
sb.In("metric_name", names...),
sb.GE("unix_milli", startMs),
@@ -229,8 +229,8 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
// reduced sample tables.
func (c *clickhouse) reducedSeriesCount(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[string]uint64, error) {
out := make(map[string]uint64, len(metricNames))
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
counts, err := c.reducedSeriesCountForTable(ctx, telemetrymetrics.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
counts, err := c.reducedSeriesCountForTable(ctx, metricstelemetryschema.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
@@ -300,9 +300,9 @@ func (c *clickhouse) RankByVolume(ctx context.Context, metricNames []string, eff
direction = "DESC"
}
ingestedTable := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName
reducedLast := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedLastTableName
reducedSum := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedSumTableName
ingestedTable := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName
reducedLast := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedLastTableName
reducedSum := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedSumTableName
sb := sqlbuilder.NewSelectBuilder()
sb.Select("base.metric_name AS metric_name", "ifNull(i.cnt, 0) AS ingested", "ifNull(d.cnt, 0) AS reduced")
@@ -352,16 +352,16 @@ func (c *clickhouse) SampleVolumeByMetric(ctx context.Context, metricNames []str
}
ctx = c.withThreads(ctx)
ingested, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
ingested, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
last, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
last, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
sum, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
sum, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
@@ -419,7 +419,7 @@ func (c *clickhouse) TotalVolume(ctx context.Context, startMs, endMs int64) (uin
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint)", "count()")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -470,7 +470,7 @@ func (c *clickhouse) SampleTimeseries(ctx context.Context, ruledMetrics []string
func (c *clickhouse) totalSamplesByBucket(ctx context.Context, startMs, endMs int64) (map[int64]uint64, error) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "count()")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
sb.GroupBy("bucket")
@@ -485,7 +485,7 @@ func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNam
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "count()")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
@@ -499,7 +499,7 @@ func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNam
// reduced 60s rows are versioned by computed_at, so count distinct buckets.
func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
out := make(map[int64]uint64)
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
@@ -507,7 +507,7 @@ func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNam
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "uniq(reduced_fingerprint, unix_milli)")
sb.From(telemetrymetrics.DBName + "." + table)
sb.From(metricstelemetryschema.DBName + "." + table)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))

View File

@@ -2,6 +2,29 @@ import { RenderErrorResponseDTO } from 'api/generated/services/sigNoz.schemas';
import { AxiosError } from 'axios';
import APIError from 'types/api/error';
// The wire shape these handlers can actually rely on. The generated
// RenderErrorResponseDTO marks code/message/url/errors as required, but the
// server omits any of them even on valid errors (e.g. a 400 with just a
// message), so a present `error` object is all the guard can guarantee.
type ErrorEnvelope = {
error: {
code?: string;
message?: string;
url?: string;
errors?: { message?: string }[];
};
};
function isErrorEnvelope(data: unknown): data is ErrorEnvelope {
return (
typeof data === 'object' &&
data !== null &&
'error' in data &&
typeof (data as ErrorEnvelope).error === 'object' &&
(data as ErrorEnvelope).error !== null
);
}
// @deprecated Use convertToApiError instead
export function ErrorResponseHandlerForGeneratedAPIs(
error: AxiosError<RenderErrorResponseDTO>,
@@ -10,15 +33,29 @@ export function ErrorResponseHandlerForGeneratedAPIs(
// The request was made and the server responded with a status code
// that falls out of the range of 2xx
if (response) {
// The body isn't guaranteed to be an error envelope — e.g. a gateway 5xx
// with an HTML/empty body during a deploy. Verify the shape before reading
// it; otherwise synthesize a consistent error from the status.
const data: unknown = response.data;
if (isErrorEnvelope(data)) {
const { code, message, url, errors } = data.error;
throw new APIError({
httpStatusCode: response.status || 500,
error: {
code: code ?? '',
message: message ?? '',
url: url ?? '',
errors: (errors ?? []).map((e) => ({ message: e.message ?? '' })),
},
});
}
throw new APIError({
httpStatusCode: response.status || 500,
error: {
code: response.data.error.code,
message: response.data.error.message,
url: response.data.error.url ?? '',
errors: (response.data.error.errors ?? []).map((e) => ({
message: e.message ?? '',
})),
code: 'UPSTREAM_UNAVAILABLE',
message: error.message || 'Something went wrong',
url: '',
errors: [],
},
});
}
@@ -62,9 +99,7 @@ export function convertToApiError(
return new APIError({
httpStatusCode: response?.status || error.status || 500,
error: {
code:
errorData?.code ||
String(response?.status || error.code || 'unknown_error'),
code: errorData?.code || 'UPSTREAM_UNAVAILABLE',
message:
errorData?.message ||
response?.statusText ||

View File

@@ -2,19 +2,38 @@ import { AxiosError } from 'axios';
import { ErrorV2Resp } from 'types/api';
import APIError from 'types/api/error';
function isErrorV2Resp(data: unknown): data is ErrorV2Resp {
return (
typeof data === 'object' &&
data !== null &&
'error' in data &&
typeof (data as ErrorV2Resp).error?.code === 'string'
);
}
// reference - https://axios-http.com/docs/handling_errors
export function ErrorResponseHandlerV2(error: AxiosError<ErrorV2Resp>): never {
const { response, request } = error;
// The request was made and the server responded with a status code
// that falls out of the range of 2xx
if (response) {
// response.data isn't guaranteed to be a V2 envelope (e.g. a gateway 5xx
// with an HTML/empty body during a deploy), so verify the shape first.
const data: unknown = response.data;
if (isErrorV2Resp(data)) {
const { code, message, url, errors } = data.error;
throw new APIError({
httpStatusCode: response.status || 500,
error: { code, message, url, errors },
});
}
throw new APIError({
httpStatusCode: response.status || 500,
error: {
code: response.data.error.code,
message: response.data.error.message,
url: response.data.error.url,
errors: response.data.error.errors,
code: 'UPSTREAM_UNAVAILABLE',
message: error.message || 'Something went wrong',
url: '',
errors: [],
},
});
}

View File

@@ -0,0 +1,126 @@
import { ErrorResponseHandlerV2 } from 'api/ErrorResponseHandlerV2';
import { AxiosError } from 'axios';
import { ErrorV2Resp } from 'types/api';
import APIError from 'types/api/error';
function asAxiosError(partial: Partial<AxiosError>): AxiosError<ErrorV2Resp> {
return partial as AxiosError<ErrorV2Resp>;
}
// ErrorResponseHandlerV2 always throws — capture the APIError so assertions stay
// unconditional.
function runHandler(error: AxiosError<ErrorV2Resp>): APIError {
try {
ErrorResponseHandlerV2(error);
} catch (thrown) {
return thrown as APIError;
}
throw new Error('expected ErrorResponseHandlerV2 to throw');
}
type ExpectedError = {
httpStatusCode: number;
code: string;
message: string;
errors: { message: string }[];
};
// One row per response shape the handler must normalize. New shapes (with
// different bodies) can be added here without a new test block.
const cases: {
name: string;
error: AxiosError<ErrorV2Resp>;
expected: ExpectedError;
}[] = [
{
name: 'well-formed V2 error envelope',
error: asAxiosError({
message: 'Request failed with status code 400',
response: {
status: 400,
data: {
error: {
code: 'bad_request',
message: 'Invalid dashboard payload',
url: 'https://signoz.io/docs',
errors: [{ message: 'name is required' }, { message: 'name too long' }],
},
},
} as AxiosError['response'],
}),
expected: {
httpStatusCode: 400,
code: 'bad_request',
message: 'Invalid dashboard payload',
errors: [{ message: 'name is required' }, { message: 'name too long' }],
},
},
{
// Regression: during a deployment the gateway returns a 5xx with a
// non-envelope body. Reading response.data.error.code used to throw a
// TypeError from inside the handler itself. See engineering-pod#5760.
name: '5xx with a non-envelope HTML body',
error: asAxiosError({
message: 'Request failed with status code 503',
response: {
status: 503,
data: '<html><body>503 Service Temporarily Unavailable</body></html>',
} as AxiosError['response'],
}),
expected: {
httpStatusCode: 503,
code: 'UPSTREAM_UNAVAILABLE',
message: 'Request failed with status code 503',
errors: [],
},
},
{
name: '5xx with an empty body',
error: asAxiosError({
message: 'Request failed with status code 502',
response: {
status: 502,
data: undefined,
} as AxiosError['response'],
}),
expected: {
httpStatusCode: 502,
code: 'UPSTREAM_UNAVAILABLE',
message: 'Request failed with status code 502',
errors: [],
},
},
{
name: 'no response received (network error)',
error: asAxiosError({
message: 'Network Error',
code: 'ERR_NETWORK',
name: 'AxiosError',
request: {},
}),
expected: {
httpStatusCode: 500,
code: 'ERR_NETWORK',
message: 'Network Error',
errors: [],
},
},
];
describe('ErrorResponseHandlerV2', () => {
it.each(cases)(
'normalizes $name into a consistent APIError',
({ error, expected }) => {
const apiError = runHandler(error);
expect(apiError).toBeInstanceOf(APIError);
expect(apiError.getHttpStatusCode()).toBe(expected.httpStatusCode);
expect(apiError.getErrorCode()).toBe(expected.code);
expect(apiError.getErrorMessage()).toBe(expected.message);
// The sub-error messages feed several parts of the UI, so assert them.
expect(apiError.getErrorDetails().error.errors).toStrictEqual(
expected.errors,
);
},
);
});

View File

@@ -4298,6 +4298,18 @@ export interface Querybuildertypesv5QueryEnvelopeBuilderDTO {
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType;
}
export enum Querybuildertypesv5QueryEnvelopeBuilderAIDTOType {
builder_ai_query = 'builder_ai_query',
}
export interface Querybuildertypesv5QueryEnvelopeBuilderAIDTO {
spec?: Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregationDTO;
/**
* @type string
* @enum builder_ai_query
*/
type: Querybuildertypesv5QueryEnvelopeBuilderAIDTOType;
}
export interface Querybuildertypesv5QueryBuilderFormulaDTO {
/**
* @type boolean
@@ -4481,6 +4493,7 @@ export interface Querybuildertypesv5QueryEnvelopeClickHouseSQLDTO {
export type Querybuildertypesv5QueryEnvelopeDTO =
| Querybuildertypesv5QueryEnvelopeBuilderDTO
| Querybuildertypesv5QueryEnvelopeBuilderAIDTO
| Querybuildertypesv5QueryEnvelopeFormulaDTO
| Querybuildertypesv5QueryEnvelopeTraceOperatorDTO
| Querybuildertypesv5QueryEnvelopePromQLDTO
@@ -8284,6 +8297,7 @@ export interface Querybuildertypesv5QueryRangeResponseDTO {
export enum Querybuildertypesv5QueryTypeDTO {
builder_query = 'builder_query',
builder_ai_query = 'builder_ai_query',
builder_formula = 'builder_formula',
builder_trace_operator = 'builder_trace_operator',
clickhouse_sql = 'clickhouse_sql',

View File

@@ -8,7 +8,6 @@ import { buildCompositeKey } from 'container/OptionsMenu/utils';
import { useGetQueryKeySuggestions } from 'hooks/querySuggestions/useGetQueryKeySuggestions';
import {
FieldContext,
FieldDataType,
SignalType,
TelemetryFieldKey,
} from 'types/api/v5/queryRange';
@@ -55,7 +54,7 @@ function OtherFields({
key: buildCompositeKey(attr.name, attr.fieldContext as string),
signal: attr.signal as SignalType,
fieldContext: attr.fieldContext as FieldContext,
fieldDataType: attr.fieldDataType as FieldDataType,
fieldDataType: attr.fieldDataType,
}),
);
const addedIds = new Set(

View File

@@ -25,12 +25,12 @@ import CodeMirror, {
} from '@uiw/react-codemirror';
import { Button, Popover, Tooltip } from 'antd';
import { getKeySuggestions } from 'api/querySuggestions/getKeySuggestions';
import { QUERY_BUILDER_KEY_TYPES } from 'constants/antlrQueryConstants';
import { QueryBuilderKeys } from 'constants/queryBuilder';
import { tracesAggregateOperatorOptions } from 'constants/queryBuilderOperators';
import { useIsDarkMode } from 'hooks/useDarkMode';
import { Info, TriangleAlert } from '@signozhq/icons';
import { IBuilderQuery } from 'types/api/queryBuilder/queryBuilderData';
import { FieldDataType } from 'types/api/v5/queryRange';
import { TracesAggregatorOperator } from 'types/common/queryBuilder';
import { useQueryBuilderV2Context } from '../../QueryBuilderV2Context';
@@ -323,16 +323,16 @@ function QueryAggregationSelect({
TracesAggregatorOperator.RATE,
];
const fieldDataType =
const fieldDataType: FieldDataType | undefined =
functionContextForFetch &&
operatorsWithoutDataType.includes(functionContextForFetch)
? undefined
: QUERY_BUILDER_KEY_TYPES.NUMBER;
: 'number';
return getKeySuggestions({
signal: queryData.dataSource,
searchText: '',
fieldDataType: fieldDataType as QUERY_BUILDER_KEY_TYPES,
fieldDataType,
});
},
{

View File

@@ -92,52 +92,76 @@
color: var(--l3-foreground);
}
.only-btn {
cursor: not-allowed;
color: var(--l3-foreground);
}
.only-btn,
.toggle-btn {
cursor: not-allowed;
color: var(--l3-foreground);
}
}
.only-btn {
display: none;
// Stack "Only"/"All" and "Toggle" in a single cell so revealing
// one swaps in place instead of shifting the row layout.
.value-actions {
display: grid;
align-items: center;
justify-items: end;
> * {
grid-area: 1 / 1;
}
}
.only-btn,
.toggle-btn {
display: none;
}
.toggle-btn:hover {
background-color: unset;
}
.only-btn:hover {
background-color: unset;
}
}
.checkbox-value-section:hover {
.toggle-btn {
display: none;
}
.only-btn {
display: flex;
align-items: center;
justify-content: center;
height: 21px;
opacity: 0;
transform: translateX(4px);
// `display` is animated as a discrete property so the button
// stays mounted through its fade-out on exit.
transition:
opacity 0.16s ease,
transform 0.16s ease,
display 0.16s allow-discrete;
&:hover {
background-color: unset;
}
}
}
// Hovering the label/value area reveals the "Only"/"All" action.
.checkbox-value-section:hover .only-btn {
display: flex;
opacity: 1;
transform: translateX(0);
// Entry animation start state (fires on the display switch).
@starting-style {
opacity: 0;
transform: translateX(4px);
}
}
// Hovering the checkbox reveals the "Toggle" action.
.check-box:hover ~ .checkbox-value-section .toggle-btn {
display: flex;
opacity: 1;
transform: translateX(0);
@starting-style {
opacity: 0;
transform: translateX(4px);
}
}
}
.value:hover {
@media (prefers-reduced-motion: reduce) {
.only-btn,
.toggle-btn {
display: flex;
align-items: center;
justify-content: center;
height: 21px;
transition: none;
}
}
}

View File

@@ -53,12 +53,14 @@ function CheckboxValueRow({
</Typography.Text>
</TooltipSimple>
)}
<Button type="text" className="only-btn">
{onlyButtonLabel}
</Button>
<Button type="text" className="toggle-btn">
Toggle
</Button>
<div className="value-actions">
<Button type="text" className="only-btn">
{onlyButtonLabel}
</Button>
<Button type="text" className="toggle-btn">
Toggle
</Button>
</div>
</div>
</div>
);

View File

@@ -1,6 +1,11 @@
.warning-popover-overlay {
--antd-arrow-background-color: var(--l2-background);
}
.warning-content {
display: flex;
flex-direction: column;
background-color: var(--l2-background);
// === SECTION: Summary (Top)
&__summary-section {
@@ -10,16 +15,25 @@
&__summary {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: var(--spacing-8);
padding: 16px;
}
&__summary-left {
display: flex;
align-items: baseline;
align-items: flex-start;
gap: 8px;
}
&__icon-wrapper {
display: flex;
align-items: center;
flex-shrink: 0;
height: var(--spacing-12);
}
&__summary-text {
display: flex;
flex-direction: column;

View File

@@ -2,6 +2,7 @@ import { ReactNode, useMemo } from 'react';
import { Color } from '@signozhq/design-tokens';
import { Button, Popover, PopoverProps } from 'antd';
import ErrorIcon from 'assets/Error';
import cx from 'classnames';
import OverlayScrollbar from 'components/OverlayScrollbar/OverlayScrollbar';
import { BookOpenText, ChevronsDown, TriangleAlert } from '@signozhq/icons';
import KeyValueLabel from 'periscope/components/KeyValueLabel';
@@ -30,16 +31,18 @@ export function WarningContent({ warning }: WarningContentProps): JSX.Element {
{/* Summary Header */}
<section className="warning-content__summary-section">
<header className="warning-content__summary">
<div className="warning-content__summary-left">
<div className="warning-content__icon-wrapper">
<ErrorIcon />
</div>
{(warningCode || warningMessage) && (
<div className="warning-content__summary-left">
<div className="warning-content__icon-wrapper">
<ErrorIcon />
</div>
<div className="warning-content__summary-text">
<h2 className="warning-content__warning-code">{warningCode}</h2>
<p className="warning-content__warning-message">{warningMessage}</p>
<div className="warning-content__summary-text">
<h2 className="warning-content__warning-code">{warningCode}</h2>
<p className="warning-content__warning-message">{warningMessage}</p>
</div>
</div>
</div>
)}
{warningUrl && (
<div className="warning-content__summary-right">
@@ -154,6 +157,10 @@ function WarningPopover({
overlayInnerStyle={{ padding: 0 }}
autoAdjustOverflow
{...popoverProps}
overlayClassName={cx(
'warning-popover-overlay',
popoverProps.overlayClassName,
)}
>
{children || (
<TriangleAlert

View File

@@ -1,8 +1,5 @@
// ** Helpers
import {
MetrictypesTemporalityDTO,
MetrictypesTypeDTO,
} from 'api/generated/services/sigNoz.schemas';
import { MetrictypesTypeDTO } from 'api/generated/services/sigNoz.schemas';
import { defaultTraceSelectedColumns } from 'container/OptionsMenu/constants';
import { createIdFromObjectFields } from 'lib/createIdFromObjectFields';
import { createNewBuilderItemName } from 'lib/newQueryBuilder/createNewBuilderItemName';
@@ -392,19 +389,11 @@ const METRIC_TYPE_TO_ATTRIBUTE_TYPE: Record<
export function toAttributeType(
metricType: MetrictypesTypeDTO | undefined,
isMonotonic?: boolean,
temporality?: MetrictypesTemporalityDTO,
): ATTRIBUTE_TYPES | '' {
if (!metricType) {
return '';
}
// Monotonicity carries meaning only for cumulative sums: a non-monotonic
// cumulative sum is a gauge for all practical purposes. Delta sums are
// counters regardless of monotonicity.
if (
metricType === MetrictypesTypeDTO.sum &&
isMonotonic === false &&
temporality === MetrictypesTemporalityDTO.cumulative
) {
if (metricType === MetrictypesTypeDTO.sum && isMonotonic === false) {
return ATTRIBUTE_TYPES.GAUGE;
}
return METRIC_TYPE_TO_ATTRIBUTE_TYPE[metricType] || '';

View File

@@ -33,7 +33,6 @@ function AllAttributes({
metricName,
metricType,
isMonotonic,
temporality,
minTime,
maxTime,
}: AllAttributesProps): JSX.Element {
@@ -72,7 +71,6 @@ function AllAttributes({
groupBy,
limit,
isMonotonic,
temporality,
);
handleExplorerTabChange(
PANEL_TYPES.TIME_SERIES,
@@ -91,7 +89,7 @@ function AllAttributes({
[MetricsExplorerEventKeys.AttributeKey]: groupBy,
});
},
[metricName, metricType, isMonotonic, temporality, handleExplorerTabChange],
[metricName, metricType, isMonotonic, handleExplorerTabChange],
);
const goToMetricsExploreWithAppliedAttribute = useCallback(
@@ -103,7 +101,6 @@ function AllAttributes({
undefined,
undefined,
isMonotonic,
temporality,
);
handleExplorerTabChange(
PANEL_TYPES.TIME_SERIES,
@@ -123,7 +120,7 @@ function AllAttributes({
[MetricsExplorerEventKeys.AttributeValue]: value,
});
},
[metricName, metricType, isMonotonic, temporality, handleExplorerTabChange],
[metricName, metricType, isMonotonic, handleExplorerTabChange],
);
const handleKeyMenuItemClick = useCallback(

View File

@@ -6,7 +6,7 @@ import { Skeleton } from 'antd';
import { Typography } from '@signozhq/ui/typography';
import {
useGetMetricAlerts,
useGetMetricDashboards,
useGetMetricDashboardsV2,
} from 'api/generated/services/metrics';
import { QueryParams } from 'constants/query';
import ROUTES from 'constants/routes';
@@ -38,7 +38,7 @@ function DashboardsAndAlertsPopover({
data: dashboardsData,
isLoading: isLoadingDashboards,
isError: isErrorDashboards,
} = useGetMetricDashboards(
} = useGetMetricDashboardsV2(
{
metricName,
},
@@ -55,7 +55,8 @@ function DashboardsAndAlertsPopover({
const dashboards = useMemo(() => {
const currentDashboards = dashboardsData?.data.dashboards ?? [];
// Remove duplicate dashboards
// The API returns one entry per referencing panel, so a dashboard repeats
// once per panel that uses the metric.
return currentDashboards.filter(
(dashboard, index, self) =>
index === self.findIndex((t) => t.dashboardId === dashboard.dashboardId),

View File

@@ -86,7 +86,6 @@ function MetricDetails({
undefined,
undefined,
metadata?.isMonotonic,
metadata?.temporality,
);
handleExplorerTabChange(
PANEL_TYPES.TIME_SERIES,
@@ -109,7 +108,6 @@ function MetricDetails({
handleExplorerTabChange,
metadata?.type,
metadata?.isMonotonic,
metadata?.temporality,
]);
useEffect(() => {
@@ -198,7 +196,6 @@ function MetricDetails({
metricName={metricName}
metricType={metadata?.type}
isMonotonic={metadata?.isMonotonic}
temporality={metadata?.temporality}
minTime={minTime}
maxTime={maxTime}
/>

View File

@@ -23,7 +23,7 @@ const useGetMetricAlertsMock = jest.spyOn(
);
const useGetMetricDashboardsMock = jest.spyOn(
metricsExplorerHooks,
'useGetMetricDashboards',
'useGetMetricDashboardsV2',
);
describe('DashboardsAndAlertsPopover', () => {
@@ -153,11 +153,15 @@ describe('DashboardsAndAlertsPopover', () => {
);
});
it('renders unique dashboards even when there are duplicates', async () => {
it('collapses multiple panels of the same dashboard into one entry', async () => {
useGetMetricDashboardsMock.mockReturnValue(
getMockDashboardsData({
data: {
dashboards: [MOCK_DASHBOARD_1, MOCK_DASHBOARD_2, MOCK_DASHBOARD_1],
dashboards: [
MOCK_DASHBOARD_1,
MOCK_DASHBOARD_2,
{ ...MOCK_DASHBOARD_1, panelId: '3', panelName: 'Panel 3' },
],
},
}),
);

View File

@@ -2,7 +2,7 @@ import * as metricsExplorerHooks from 'api/generated/services/metrics';
import {
GetMetricAlerts200,
GetMetricAttributes200,
GetMetricDashboards200,
GetMetricDashboardsV2200,
GetMetricHighlights200,
GetMetricMetadata200,
MetrictypesTemporalityDTO,
@@ -40,14 +40,14 @@ export function getMockMetricHighlightsData(
export const MOCK_DASHBOARD_1 = {
dashboardName: 'Dashboard 1',
dashboardId: '1',
widgetId: '1',
widgetName: 'Widget 1',
panelId: '1',
panelName: 'Panel 1',
};
export const MOCK_DASHBOARD_2 = {
dashboardName: 'Dashboard 2',
dashboardId: '2',
widgetId: '2',
widgetName: 'Widget 2',
panelId: '2',
panelName: 'Panel 2',
};
export const MOCK_ALERT_1 = {
alertName: 'Alert 1',
@@ -59,7 +59,7 @@ export const MOCK_ALERT_2 = {
};
export function getMockDashboardsData(
overrides?: Partial<GetMetricDashboards200>,
overrides?: Partial<GetMetricDashboardsV2200>,
{
isLoading = false,
isError = false,
@@ -67,7 +67,7 @@ export function getMockDashboardsData(
isLoading?: boolean;
isError?: boolean;
} = {},
): ReturnType<typeof metricsExplorerHooks.useGetMetricDashboards> {
): ReturnType<typeof metricsExplorerHooks.useGetMetricDashboardsV2> {
return {
data: {
data: {
@@ -79,7 +79,7 @@ export function getMockDashboardsData(
isLoading,
isError,
} as ReturnType<typeof metricsExplorerHooks.useGetMetricDashboards>;
} as ReturnType<typeof metricsExplorerHooks.useGetMetricDashboardsV2>;
}
export function getMockAlertsData(

View File

@@ -147,44 +147,6 @@ describe('MetricDetails utils', () => {
expect(query.builder.queryData[0]?.spaceAggregation).toBe('sum');
});
it('should treat a non-monotonic cumulative SUM as a gauge', () => {
const query = getMetricDetailsQuery(
TEST_METRIC_NAME,
MetrictypesTypeDTO.sum,
undefined,
undefined,
undefined,
false,
MetrictypesTemporalityDTO.cumulative,
);
expect(query.builder.queryData[0]?.aggregateAttribute?.type).toBe(
ATTRIBUTE_TYPES.GAUGE,
);
expect(query.builder.queryData[0]?.aggregateOperator).toBe('avg');
expect(query.builder.queryData[0]?.timeAggregation).toBe('avg');
expect(query.builder.queryData[0]?.spaceAggregation).toBe('avg');
});
it('should treat a non-monotonic delta SUM as a counter', () => {
const query = getMetricDetailsQuery(
TEST_METRIC_NAME,
MetrictypesTypeDTO.sum,
undefined,
undefined,
undefined,
false,
MetrictypesTemporalityDTO.delta,
);
expect(query.builder.queryData[0]?.aggregateAttribute?.type).toBe(
ATTRIBUTE_TYPES.SUM,
);
expect(query.builder.queryData[0]?.aggregateOperator).toBe('rate');
expect(query.builder.queryData[0]?.timeAggregation).toBe('rate');
expect(query.builder.queryData[0]?.spaceAggregation).toBe('sum');
});
it('should create correct query for GAUGE metric type', () => {
const query = getMetricDetailsQuery(
TEST_METRIC_NAME,

View File

@@ -35,7 +35,6 @@ export interface AllAttributesProps {
metricName: string;
metricType: MetrictypesTypeDTO | undefined;
isMonotonic?: boolean;
temporality?: MetrictypesTemporalityDTO;
minTime?: number;
maxTime?: number;
}

View File

@@ -89,21 +89,16 @@ export function getMetricDetailsQuery(
groupBy?: string,
limit?: number,
isMonotonic?: boolean,
temporality?: MetrictypesTemporalityDTO,
): Query {
let timeAggregation;
let spaceAggregation;
let aggregateOperator;
// Monotonicity carries meaning only for cumulative sums; delta sums are
// counters regardless of monotonicity.
const isNonMonotonicCumulativeSum =
metricType === MetrictypesTypeDTO.sum &&
isMonotonic === false &&
temporality === MetrictypesTemporalityDTO.cumulative;
const isNonMonotonicSum =
metricType === MetrictypesTypeDTO.sum && isMonotonic === false;
switch (metricType) {
case MetrictypesTypeDTO.sum:
if (isNonMonotonicCumulativeSum) {
if (isNonMonotonicSum) {
timeAggregation = 'avg';
spaceAggregation = 'avg';
aggregateOperator = 'avg';
@@ -136,7 +131,7 @@ export function getMetricDetailsQuery(
break;
}
const attributeType = toAttributeType(metricType, isMonotonic, temporality);
const attributeType = toAttributeType(metricType, isMonotonic);
return {
...initialQueriesMap[DataSource.METRICS],

View File

@@ -113,7 +113,7 @@ const useOptionsMenu = ({
(suggestion) => ({
name: suggestion.name,
signal: suggestion.signal as SignalType,
fieldDataType: suggestion.fieldDataType as FieldDataType,
fieldDataType: suggestion.fieldDataType,
fieldContext: suggestion.fieldContext as FieldContext,
}),
);
@@ -192,7 +192,7 @@ const useOptionsMenu = ({
name: e.name,
signal: e.signal as SignalType,
fieldContext: e.fieldContext as FieldContext,
fieldDataType: e.fieldDataType as FieldDataType,
fieldDataType: e.fieldDataType,
}));
}
if (dataSource === DataSource.TRACES) {

View File

@@ -8,7 +8,6 @@ import {
} from '@testing-library/react';
import {
MetricsexplorertypesListMetricDTO,
MetrictypesTemporalityDTO,
MetrictypesTypeDTO,
} from 'api/generated/services/sigNoz.schemas';
import { ENTITY_VERSION_V5 } from 'constants/app';
@@ -71,7 +70,7 @@ function makeMetric(
type: MetrictypesTypeDTO.sum,
isMonotonic: true,
description: '',
temporality: MetrictypesTemporalityDTO.cumulative,
temporality: 'cumulative' as never,
unit: '',
...overrides,
};
@@ -394,13 +393,12 @@ describe('selecting a metric type updates the aggregation options', () => {
]);
});
it('non-monotonic cumulative Sum metric is treated as Gauge', () => {
it('non-monotonic Sum metric is treated as Gauge', () => {
returnMetrics([
makeMetric({
metricName: 'active_connections',
type: MetrictypesTypeDTO.sum,
isMonotonic: false,
temporality: MetrictypesTemporalityDTO.cumulative,
}),
]);
@@ -429,36 +427,6 @@ describe('selecting a metric type updates the aggregation options', () => {
]);
});
it('non-monotonic delta Sum metric remains a counter with Rate/Increase options', () => {
returnMetrics([
makeMetric({
metricName: 'gcp_reject_connections_count',
type: MetrictypesTypeDTO.sum,
isMonotonic: false,
temporality: MetrictypesTemporalityDTO.delta,
}),
]);
render(<MetricQueryHarness query={makeQuery()} />);
const input = screen.getByRole('combobox');
fireEvent.change(input, {
target: { value: 'gcp_reject_connections_count' },
});
fireEvent.blur(input);
expect(getOptionLabels('time-agg-options')).toStrictEqual([
'Rate',
'Increase',
]);
expect(getOptionLabels('space-agg-options')).toStrictEqual([
'Sum',
'Avg',
'Min',
'Max',
]);
});
it('Histogram metric shows no time options and P50P99 space options', () => {
returnMetrics([
makeMetric({

View File

@@ -34,7 +34,7 @@ export type MetricNameSelectorProps = {
function getAttributeType(
metric: MetricsexplorertypesListMetricDTO,
): ATTRIBUTE_TYPES | '' {
return toAttributeType(metric.type, metric.isMonotonic, metric.temporality);
return toAttributeType(metric.type, metric.isMonotonic);
}
function createAutocompleteData(

View File

@@ -4,11 +4,11 @@ import { Input } from '@signozhq/ui/input';
import { Skeleton } from 'antd';
import { getKeySuggestions } from 'api/querySuggestions/getKeySuggestions';
import OverlayScrollbar from 'components/OverlayScrollbar/OverlayScrollbar';
import { QUERY_BUILDER_KEY_TYPES } from 'constants/antlrQueryConstants';
import useDebounce from 'hooks/useDebounce';
import { ContextMenu } from 'periscope/components/ContextMenu';
import { BaseAutocompleteData } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { MetricAggregation } from 'types/api/v5/queryRange';
import { fieldDataTypeToDataType } from 'utils/fieldDataType';
import { BreakoutOptionsProps } from './contextConfig';
import { BreakoutAttributeType } from './types';
@@ -80,7 +80,7 @@ function BreakoutOptions({
keyArray.forEach((keyData) => {
transformedOptions.push({
key: keyData.name,
dataType: keyData.fieldDataType as QUERY_BUILDER_KEY_TYPES,
dataType: fieldDataTypeToDataType(keyData.fieldDataType),
type: '',
});
});

View File

@@ -238,6 +238,10 @@ describe('TableDrilldown Breakout Functionality', () => {
expect(aggregateQueryData.groupBy).toHaveLength(1);
expect(aggregateQueryData.groupBy[0].key).toBe('deployment.environment');
// The picked field's type travels with it — dropping it leaves the breakout query
// untyped, so a drilldown on its result can't tell a number from a string.
expect(aggregateQueryData.groupBy[0].dataType).toBe('string');
// Verify that orderBy has been cleared (as per getBreakoutQuery logic)
expect(aggregateQueryData.orderBy).toStrictEqual([]);

View File

@@ -1,6 +1,5 @@
import { OPERATORS, PANEL_TYPES } from 'constants/queryBuilder';
import cloneDeep from 'lodash-es/cloneDeep';
import { BaseAutocompleteData } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { IBuilderQuery, Query } from 'types/api/queryBuilder/queryBuilderData';
import { addFilterToSelectedQuery, FilterData } from './drilldownUtils';
@@ -89,7 +88,11 @@ export const getBreakoutQuery = (
.filter((item: IBuilderQuery) => item.queryName === aggregateData.queryName)
.map((item: IBuilderQuery) => ({
...item,
groupBy: [{ key: groupBy.key, type: groupBy.type } as BaseAutocompleteData],
// The picked field's type travels with it: dropped, the breakout query goes out
// untyped and a drilldown on its result can't tell a number from a string.
groupBy: [
{ key: groupBy.key, dataType: groupBy.dataType, type: groupBy.type },
],
orderBy: [],
legend: item.legend && groupBy.key ? `{{${groupBy.key}}}` : '',
}));

View File

@@ -1,6 +1,8 @@
import { ReactNode } from 'react';
import { QUERY_BUILDER_KEY_TYPES } from 'constants/antlrQueryConstants';
import { BaseAutocompleteData } from 'types/api/queryBuilder/queryAutocompleteResponse';
import {
BaseAutocompleteData,
DataTypes,
} from 'types/api/queryBuilder/queryAutocompleteResponse';
import { IBuilderQuery } from 'types/api/queryBuilder/queryBuilderData';
export type ContextMenuItem = ReactNode;
@@ -31,7 +33,8 @@ export interface AggregateContextMenuConfig {
export interface BreakoutAttributeType {
key: string;
dataType: QUERY_BUILDER_KEY_TYPES;
/** The picked field's type, in the vocabulary the group-by it becomes is read with. */
dataType: DataTypes;
type: string;
}

View File

@@ -11,3 +11,9 @@
border: 1px solid var(--l3-border) !important;
box-shadow: none !important;
}
/* Highlights the dashboard name inside the delete-confirm title. */
.deleteName {
color: var(--danger-background);
font-weight: var(--font-weight-medium);
}

View File

@@ -27,16 +27,13 @@ import logEvent from 'api/common/logEvent';
import { cloneDashboardV2 } from 'api/generated/services/dashboard';
import type { DashboardtypesGettableDashboardV2DTO } from 'api/generated/services/sigNoz.schemas';
import ROUTES from 'constants/routes';
import { useDeleteDashboard } from 'hooks/dashboard/useDeleteDashboard';
import { useSafeNavigate } from 'hooks/useSafeNavigate';
import history from 'lib/history';
import { DashboardDetailEvents } from 'pages/DashboardPageV2/constants/events';
import { useAppContext } from 'providers/App/App';
import { useErrorModal } from 'providers/ErrorModalProvider';
import APIError from 'types/api/error';
import { USER_ROLES } from 'types/roles';
import ConfirmDeleteDialog from '../../components/ConfirmDeleteDialog/ConfirmDeleteDialog';
import DisabledControlTooltip from '../../components/DisabledControlTooltip/DisabledControlTooltip';
import DisabledMenuItemLabel from '../../components/DisabledMenuItemLabel/DisabledMenuItemLabel';
import DashboardSettings from '../../DashboardSettings';
@@ -45,6 +42,7 @@ import SectionTitleModal from '../../PanelsAndSectionsLayout/Section/SectionTitl
import JsonEditorDrawer from '../JsonEditorDrawer/JsonEditorDrawer';
import SettingsDrawer from '../SettingsDrawer';
import styles from './DashboardActions.module.scss';
import { useDeleteDashboardAction } from './useDeleteDashboardAction';
import { DASHBOARD_LOCKED_REASON } from '../../hooks/useDashboardEditGuard';
import { useDashboardStore } from '../../store/useDashboardStore';
@@ -84,8 +82,12 @@ function DashboardActions({
const [isCloning, setIsCloning] = useState<boolean>(false);
const [isNewSectionOpen, setIsNewSectionOpen] = useState<boolean>(false);
const [isDeleteOpen, setIsDeleteOpen] = useState<boolean>(false);
const deleteDashboardMutation = useDeleteDashboard(dashboard.id);
const { contextHolder: deleteConfirmHolder, confirmDeleteDashboard } =
useDeleteDashboardAction({
dashboardId: dashboard.id,
dashboardName: title,
panelCount: Object.keys(dashboard.spec.panels).length,
});
// Open the settings drawer when something in the tree requests it (e.g. the
// variables bar's "Add variable" button).
@@ -132,19 +134,6 @@ function DashboardActions({
}
}, [dashboard.id, title, safeNavigate, showErrorModal]);
const handleConfirmDelete = useCallback((): void => {
deleteDashboardMutation.mutate(undefined, {
onSuccess: () => {
void logEvent(DashboardDetailEvents.Deleted, {
dashboardId: dashboard.id,
panelCount: Object.keys(dashboard.spec.panels).length,
});
setIsDeleteOpen(false);
history.replace(ROUTES.ALL_DASHBOARD);
},
});
}, [deleteDashboardMutation, dashboard.id, dashboard.spec.panels]);
const handleOpenSettings = useCallback((): void => {
void logEvent(DashboardDetailEvents.SettingsOpened, {
dashboardId: dashboard.id,
@@ -248,7 +237,7 @@ function DashboardActions({
icon: <Trash2 size={14} />,
danger: true,
disabled: isLocked,
onClick: (): void => setIsDeleteOpen(true),
onClick: confirmDeleteDashboard,
},
);
}
@@ -266,6 +255,7 @@ function DashboardActions({
handleClone,
onLockToggle,
handleEnterFullScreen,
confirmDeleteDashboard,
]);
return (
@@ -348,14 +338,7 @@ function DashboardActions({
isOpen={isJsonEditorOpen}
onClose={(): void => setIsJsonEditorOpen(false)}
/>
<ConfirmDeleteDialog
open={isDeleteOpen}
title={`Delete dashboard"?`}
description={`Are you sure you want to delete this dashboard - "${title}"? This action cannot be undone.`}
isLoading={deleteDashboardMutation.isLoading}
onConfirm={handleConfirmDelete}
onClose={(): void => setIsDeleteOpen(false)}
/>
{deleteConfirmHolder}
<SectionTitleModal
open={isNewSectionOpen}
heading="New section"

View File

@@ -0,0 +1,89 @@
import { type ReactNode, useCallback } from 'react';
import { useQueryClient } from 'react-query';
import { toast } from '@signozhq/ui/sonner';
import { Typography } from '@signozhq/ui/typography';
import logEvent from 'api/common/logEvent';
import {
invalidateListDashboardsForUserV2,
useDeleteDashboardV2,
} from 'api/generated/services/dashboard';
import { useDeleteConfirm } from 'components/DeleteConfirmModal/useDeleteConfirm';
import ROUTES from 'constants/routes';
import { useDashboardPreferencesStore } from 'hooks/dashboard/useDashboardPreference';
import history from 'lib/history';
import { DashboardDetailEvents } from 'pages/DashboardPageV2/constants/events';
import { useErrorModal } from 'providers/ErrorModalProvider';
import APIError from 'types/api/error';
import styles from './DashboardActions.module.scss';
interface UseDeleteDashboardActionArgs {
dashboardId: string;
dashboardName: string;
panelCount: number;
}
interface UseDeleteDashboardAction {
/** Must be rendered in the calling component for the modal to appear. */
contextHolder: ReactNode;
confirmDeleteDashboard: () => void;
}
/**
* Deletes the open dashboard through the v2 endpoint behind the shared
* destructive confirmation, then drops its local preferences, refreshes the
* dashboard list cache and sends the user back to the list.
*/
export function useDeleteDashboardAction({
dashboardId,
dashboardName,
panelCount,
}: UseDeleteDashboardActionArgs): UseDeleteDashboardAction {
const queryClient = useQueryClient();
const { showErrorModal } = useErrorModal();
const { contextHolder, confirmDelete } = useDeleteConfirm();
const removePreferences = useDashboardPreferencesStore(
(state) => state.removePreferences,
);
const { mutate: deleteDashboard } = useDeleteDashboardV2({
mutation: {
onSuccess: async (): Promise<void> => {
void logEvent(DashboardDetailEvents.Deleted, { dashboardId, panelCount });
removePreferences(dashboardId);
await invalidateListDashboardsForUserV2(queryClient);
toast.success('Dashboard deleted successfully');
history.replace(ROUTES.ALL_DASHBOARD);
},
onError: (error: unknown): void => {
showErrorModal(error as APIError);
},
},
});
const confirmDeleteDashboard = useCallback((): void => {
confirmDelete({
title: (
<Typography.Title level={5}>
Are you sure you want to delete the
<Typography.Text className={styles.deleteName}>
{' '}
{dashboardName}{' '}
</Typography.Text>
dashboard?
</Typography.Title>
),
content: 'This action cannot be undone.',
// Keeps the Delete button loading until the mutation settles, then closes.
onConfirm: () =>
new Promise<void>((resolve) => {
deleteDashboard(
{ pathParams: { id: dashboardId } },
{ onSettled: () => resolve() },
);
}),
});
}, [confirmDelete, dashboardName, deleteDashboard, dashboardId]);
return { contextHolder, confirmDeleteDashboard };
}

View File

@@ -1,5 +1,6 @@
import { useCallback } from 'react';
import { SolidAlertTriangle, X } from '@signozhq/icons';
import { Badge } from '@signozhq/ui/badge';
import { Button } from '@signozhq/ui/button';
import { DialogWrapper } from '@signozhq/ui/dialog';
import { Divider } from '@signozhq/ui/divider';
@@ -13,6 +14,7 @@ import DisabledControlTooltip from '../../components/DisabledControlTooltip/Disa
import styles from './Header.module.scss';
interface HeaderProps {
/** Unsaved edits exist — shows the "Unsaved Changes" badge and gates the discard confirmation on close (not the Save button). */
isDirty: boolean;
isSaving: boolean;
showSwitchToView?: boolean;
@@ -66,6 +68,11 @@ function Header({
/>
<Divider type="vertical" />
<Typography.Text>Configure panel</Typography.Text>
{isDirty && (
<Badge color="warning" data-testid="panel-editor-v2-unsaved-badge">
Unsaved Changes
</Badge>
)}
</div>
<div className={styles.actions}>
<HeaderRightSection
@@ -88,7 +95,7 @@ function Header({
variant="solid"
color="primary"
data-testid="panel-editor-v2-save"
disabled={readOnly || !isDirty || isSaving}
disabled={readOnly || isSaving}
loading={!readOnly && isSaving}
onClick={readOnly ? undefined : onSave}
>

View File

@@ -1,3 +1,4 @@
import type { ComponentProps } from 'react';
import { MemoryRouter } from 'react-router-dom';
import { render, screen } from '@testing-library/react';
import { TooltipProvider } from '@signozhq/ui/tooltip';
@@ -23,7 +24,9 @@ jest.mock('api/common/logEvent', () => ({
const mockUseIsAIAssistantEnabled = useIsAIAssistantEnabled as jest.Mock;
function renderHeader(): void {
function renderHeader(
props: Partial<ComponentProps<typeof Header>> = {},
): void {
// AppLayout supplies the TooltipProvider in the app; the header is rendered bare here.
render(
<MemoryRouter>
@@ -33,6 +36,7 @@ function renderHeader(): void {
isSaving={false}
onSave={jest.fn()}
onClose={jest.fn()}
{...props}
/>
</TooltipProvider>
</MemoryRouter>,
@@ -66,4 +70,34 @@ describe('PanelEditor Header', () => {
).not.toBeInTheDocument();
expect(screen.getByTestId('panel-editor-v2-save')).toBeInTheDocument();
});
it('keeps Save enabled even when there are no unsaved edits', () => {
mockUseIsAIAssistantEnabled.mockReturnValue(false);
renderHeader({ isDirty: false });
expect(screen.getByTestId('panel-editor-v2-save')).toBeEnabled();
});
it('disables Save only while read-only or saving', () => {
mockUseIsAIAssistantEnabled.mockReturnValue(false);
renderHeader({ isDirty: true, readOnly: true, readOnlyReason: 'Locked' });
expect(screen.getByTestId('panel-editor-v2-save')).toBeDisabled();
});
it('shows the Unsaved Changes badge only when there are unsaved edits', () => {
mockUseIsAIAssistantEnabled.mockReturnValue(false);
renderHeader({ isDirty: false });
expect(
screen.queryByTestId('panel-editor-v2-unsaved-badge'),
).not.toBeInTheDocument();
renderHeader({ isDirty: true });
expect(
screen.getByTestId('panel-editor-v2-unsaved-badge'),
).toBeInTheDocument();
});
});

View File

@@ -1,5 +1,6 @@
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { toast } from '@signozhq/ui/sonner';
import type { DashboardtypesPanelDTO } from 'api/generated/services/sigNoz.schemas';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { getPanelDefinition } from 'pages/DashboardPageV2/DashboardContainer/Panels/registry';
@@ -100,6 +101,12 @@ jest.mock('@signozhq/ui/resizable', () => ({
jest.mock('@signozhq/ui/sonner', () => ({
toast: { success: jest.fn(), error: jest.fn() },
}));
const mockShowErrorModal = jest.fn();
jest.mock('providers/ErrorModalProvider', () => ({
useErrorModal: (): { showErrorModal: jest.Mock } => ({
showErrorModal: mockShowErrorModal,
}),
}));
// Children mocked to capture props (and expose a Save trigger / footer slot).
const mockHeaderProps = jest.fn();
@@ -261,13 +268,13 @@ describe('PanelEditorContainer composition', () => {
);
});
it('keeps a query-less new panel unsaveable but still serializes its seed query', () => {
it('keeps a query-less new panel clean but still serializes its seed query', () => {
setup(makePanel('signoz/TimeSeriesPanel'), { isNew: true });
expect(mockUseQuerySync).toHaveBeenCalledWith(
expect.objectContaining({ alwaysSerializeQuery: true }),
);
// No query and no edits yet → nothing to save, so Save stays disabled.
// No query and no edits yet → not dirty, so closing won't prompt to discard.
expect(mockHeaderProps).toHaveBeenCalledWith(
expect.objectContaining({ isDirty: false }),
);
@@ -290,7 +297,7 @@ describe('PanelEditorContainer composition', () => {
);
});
it('marks a new panel that already has a query saveable (e.g. list auto-runs one)', () => {
it('marks a new panel that already has a query dirty (e.g. list auto-runs one)', () => {
const seededQuery = {
spec: { plugin: { kind: 'signoz/BuilderQuery', spec: { signal: 'logs' } } },
};
@@ -327,6 +334,24 @@ describe('PanelEditorContainer composition', () => {
expect(mockSave).toHaveBeenCalledWith(panel.spec);
});
it('surfaces a save failure through the error modal', async () => {
// The raw thrown value flows straight to the modal, which normalizes it to an
// APIError itself (that normalization is covered by ErrorModalProvider's tests).
const failure = {
response: {
status: 400,
data: { error: { code: 'INVALID', message: 'Panel name already exists' } },
},
};
mockSave.mockRejectedValueOnce(failure);
setup(makePanel('signoz/TimeSeriesPanel'));
await userEvent.click(screen.getByTestId('editor-save'));
await waitFor(() => expect(mockShowErrorModal).toHaveBeenCalledWith(failure));
expect(toast.error).not.toHaveBeenCalled();
});
it('marks the saved panel to be scrolled into view on the dashboard', async () => {
setup(makePanel('signoz/TimeSeriesPanel'));

View File

@@ -117,6 +117,49 @@ describe('usePanelEditorQuerySync (real query builder)', () => {
},
);
it('a NEW panel stays query-dirty after Stage & Run commits the edited query into the draft', async () => {
// Repro of the P0: new panel → edit → Run commits the query into the draft; the
// dirty baseline must not drift onto it, or Save re-disables.
const editedInUrl: Query = {
...initialQueriesMap[DataSource.METRICS],
id: 'edited-new-panel',
builder: {
...initialQueriesMap[DataSource.METRICS].builder,
queryData: [
{
...initialQueriesMap[DataSource.METRICS].builder.queryData[0],
legend: 'edited-legend',
},
],
},
};
const committed = toPerses(editedInUrl, panelType);
const { result, rerender } = renderHook(
({ draftQueries }: { draftQueries: DashboardtypesQueryDTO[] }) =>
usePanelEditorQuerySync({
draft: makePanel(draftQueries),
panelType,
setSpec: jest.fn(),
refetch: jest.fn(),
alwaysSerializeQuery: true,
signal: DataSource.METRICS as unknown as TelemetrytypesSignalDTO,
// savedQueries omitted — new panel.
}),
{
wrapper: makeUrlWrapper(editedInUrl),
initialProps: { draftQueries: [] as DashboardtypesQueryDTO[] },
},
);
// The edited query (from the URL) diverges from the seeded default → dirty.
await waitFor(() => expect(result.current.isQueryDirty).toBe(true));
// Stage & Run commits the edited query into the draft → must stay dirty.
rerender({ draftQueries: committed });
expect(result.current.isQueryDirty).toBe(true);
});
it('an untouched panel with a minimal/older stored query is NOT dirty (drift fix)', async () => {
// An older saved query carries only a few fields; the builder re-emits many more
// (source, stepInterval, filter, spaceAggregation, …). Comparing raw would read

View File

@@ -79,6 +79,11 @@ export function usePanelEditorQuerySync({
// in-editor edit in the URL survives a refresh / browser Back-Forward.
useShareBuilderUrl({ defaultValue: seedQuery });
// Frozen seed for the new-panel dirty baseline: Stage & Run commits the live query
// into `draft.spec.queries`, so re-reading `seedQuery` would drift the baseline onto
// the run query and Save would re-disable right after a run.
const initialSeedRef = useRef(seedQuery);
// Commit the live query into the draft (what the preview fetches).
const commitQuery = useCallback(
(query: Query): boolean => {
@@ -152,16 +157,19 @@ export function usePanelEditorQuerySync({
// the V5 envelope level. Anchoring to `savedQueries` (not the builder-seed) keeps a
// handed-off / URL-restored edit reading as dirty; routing both sides through the same
// `fromPerses → toPerses` round-trip stops builder-added defaults (absent from an older
// stored query) reading an untouched panel as modified. New panel: fall back to seed.
// stored query) reading an untouched panel as modified. New panel: fall back to the
// frozen initial seed (see `initialSeedRef`).
const baselineEnvelopes = useMemo(
() =>
toQueryEnvelopes(
toPerses(
savedQueries ? fromPerses(savedQueries, panelType) : seedQuery,
savedQueries
? fromPerses(savedQueries, panelType)
: initialSeedRef.current,
panelType,
),
),
[savedQueries, seedQuery, panelType],
[savedQueries, panelType],
);
const isQueryDirty = useMemo(
() =>

View File

@@ -19,6 +19,7 @@ import {
SectionKind,
} from 'pages/DashboardPageV2/DashboardContainer/Panels/types/sections';
import { getBuilderQueries } from 'pages/DashboardPageV2/DashboardContainer/Panels/utils/getBuilderQueries';
import { useErrorModal } from 'providers/ErrorModalProvider';
import { getExecStats } from '../queryV5/v5ResponseData';
import { usePanelInteractions } from '../PanelsAndSectionsLayout/Panel/hooks/usePanelInteractions';
@@ -159,9 +160,10 @@ function PanelEditorContainer({
return section?.controls;
}, [panelDefinition]);
// New panels are savable once seeded with a query (List auto-seeds one). Read the
// seed `panel`, not the live `draft` — the staged-query sync commits the seed into
// the draft on open, which would falsely dirty an untouched query-less new panel.
// Unsaved-edits flag driving the discard confirmation on close (Save is always
// enabled). Read the seed `panel`, not the live `draft` — the staged-query sync
// commits the seed into the draft on open, which would falsely dirty an untouched
// query-less new panel.
const isDirty = useMemo(
() => isSpecDirty || isQueryDirty || (isNew && panel.spec.queries.length > 0),
[isSpecDirty, isQueryDirty, isNew, panel.spec.queries.length],
@@ -226,6 +228,7 @@ function PanelEditorContainer({
});
const setScrollTargetId = useScrollIntoViewStore((s) => s.setScrollTargetId);
const { showErrorModal } = useErrorModal();
const onSave = useCallback(async (): Promise<void> => {
if (!isEditable) {
@@ -236,12 +239,22 @@ function PanelEditorContainer({
const savedPanelId = await save(buildSaveSpec(draft.spec));
// Reveal the saved panel once the dashboard re-renders.
setScrollTargetId(savedPanelId);
toast.success('Panel saved');
toast.success('Panel saved', {
position: 'top-center',
});
onSaved();
} catch {
toast.error('Failed to save panel');
} catch (err) {
showErrorModal(err);
}
}, [isEditable, save, buildSaveSpec, draft.spec, setScrollTargetId, onSaved]);
}, [
isEditable,
save,
buildSaveSpec,
draft.spec,
setScrollTargetId,
onSaved,
showErrorModal,
]);
// Leaving an existing panel's editor (without saving) still returns to it, so
// the dashboard lands on that panel rather than scrolled to the top. A new,

View File

@@ -51,7 +51,7 @@ describe('panelStatusFromError', () => {
it('falls back to the error message when there is no structured body', () => {
expect(panelStatusFromError(new Error('boom'))).toStrictEqual({
code: 'unknown_error',
code: 'UPSTREAM_UNAVAILABLE',
message: 'boom',
docsUrl: undefined,
messages: [],

View File

@@ -12,13 +12,13 @@ import {
deleteDashboardV2,
invalidateListDashboardsForUserV2,
} from 'api/generated/services/dashboard';
import { useDeleteConfirm } from 'components/DeleteConfirmModal/useDeleteConfirm';
import { DashboardListEvents } from 'pages/DashboardsListPageV2/constants/events';
import { useAppContext } from 'providers/App/App';
import { useErrorModal } from 'providers/ErrorModalProvider';
import APIError from 'types/api/error';
import { USER_ROLES } from 'types/roles';
import { useDeleteConfirm } from '../DeleteConfirmModal/useDeleteConfirm';
import styles from './ActionsPopover.module.scss';
interface Props {

View File

@@ -6,11 +6,11 @@ import { Typography } from '@signozhq/ui/typography';
import { Bookmark, PenLine, Plus, Search, Trash2 } from '@signozhq/icons';
import cx from 'classnames';
import { useDeleteConfirm } from 'components/DeleteConfirmModal/useDeleteConfirm';
import { DashboardListEvents } from 'pages/DashboardsListPageV2/constants/events';
import type { SavedView } from '../../types';
import { type BuiltinView } from '../../utils/views';
import { useDeleteConfirm } from '../DeleteConfirmModal/useDeleteConfirm';
import ViewNamePopover from './ViewNamePopover';
import styles from './ViewsRail.module.scss';

View File

@@ -1,4 +1,4 @@
import { QUERY_BUILDER_KEY_TYPES } from 'constants/antlrQueryConstants';
import { FieldDataType } from 'types/api/v5/queryRange';
export interface QueryKeyDataSuggestionsProps {
label: string;
@@ -7,7 +7,12 @@ export interface QueryKeyDataSuggestionsProps {
apply?: string;
detail?: string;
fieldContext?: 'resource' | 'scope' | 'attribute' | 'span';
fieldDataType?: QUERY_BUILDER_KEY_TYPES;
/**
* The field's type as the API reports it. Was declared as the antlr key-type enum
* (`string | number | boolean`), which the endpoint never returns — every consumer cast it
* back to `FieldDataType`.
*/
fieldDataType?: FieldDataType;
name: string;
signal: 'traces' | 'logs' | 'metrics';
}
@@ -26,7 +31,7 @@ export interface QueryKeyRequestProps {
signal: 'traces' | 'logs' | 'metrics';
searchText: string;
fieldContext?: 'resource' | 'scope' | 'attribute' | 'span';
fieldDataType?: QUERY_BUILDER_KEY_TYPES;
fieldDataType?: FieldDataType;
metricName?: string;
metricNamespace?: string;
signalSource?: 'meter' | '';

View File

@@ -0,0 +1,34 @@
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import type { FieldDataType } from 'types/api/v5/queryRange';
import { fieldDataTypeToDataType } from '../fieldDataType';
describe('fieldDataTypeToDataType', () => {
it('maps the reported numeric spellings onto the query-builder numerics', () => {
expect(fieldDataTypeToDataType('number')).toBe(DataTypes.Float64);
expect(fieldDataTypeToDataType('int64')).toBe(DataTypes.Int64);
expect(fieldDataTypeToDataType('float64')).toBe(DataTypes.Float64);
});
it('maps the list spellings onto the array types', () => {
expect(fieldDataTypeToDataType('[]string')).toBe(DataTypes.ArrayString);
expect(fieldDataTypeToDataType('[]bool')).toBe(DataTypes.ArrayBool);
expect(fieldDataTypeToDataType('[]int64')).toBe(DataTypes.ArrayInt64);
expect(fieldDataTypeToDataType('[]float64')).toBe(DataTypes.ArrayFloat64);
expect(fieldDataTypeToDataType('[]number')).toBe(DataTypes.ArrayFloat64);
});
it('passes through the spellings the two vocabularies share', () => {
expect(fieldDataTypeToDataType('string')).toBe(DataTypes.String);
expect(fieldDataTypeToDataType('bool')).toBe(DataTypes.bool);
});
it('returns EMPTY for empty, missing and not-yet-known types', () => {
expect(fieldDataTypeToDataType('')).toBe(DataTypes.EMPTY);
expect(fieldDataTypeToDataType(undefined)).toBe(DataTypes.EMPTY);
// The backend can store `[]json` / `[]dynamic`, which the API union doesn't list.
expect(fieldDataTypeToDataType('[]json' as FieldDataType)).toBe(
DataTypes.EMPTY,
);
});
});

View File

@@ -42,7 +42,12 @@ export function toAPIError(
try {
ErrorResponseHandlerForGeneratedAPIs(error);
} catch (apiError) {
if (apiError instanceof APIError) {
// UPSTREAM_UNAVAILABLE means the handler couldn't find a backend error code
// (non-envelope body); prefer the caller's context-specific defaultMessage.
if (
apiError instanceof APIError &&
apiError.getErrorCode() !== 'UPSTREAM_UNAVAILABLE'
) {
return apiError;
}
}

View File

@@ -0,0 +1,31 @@
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import type { FieldDataType } from 'types/api/v5/queryRange';
/**
* The field metadata APIs and the query builder spell field types differently: the metadata
* collapses the numerics to `number` and writes lists as `[]string`, while `DataTypes` — which
* keys the operator and attribute-value lookups — uses `float64` and `array(string)`. Convert
* where a reported type becomes a query-builder one, so those lookups can't miss.
*/
const TO_DATA_TYPE: Record<FieldDataType, DataTypes> = {
'': DataTypes.EMPTY,
string: DataTypes.String,
bool: DataTypes.bool,
int64: DataTypes.Int64,
float64: DataTypes.Float64,
number: DataTypes.Float64,
'[]string': DataTypes.ArrayString,
'[]bool': DataTypes.ArrayBool,
'[]int64': DataTypes.ArrayInt64,
'[]float64': DataTypes.ArrayFloat64,
'[]number': DataTypes.ArrayFloat64,
};
/**
* Exhaustive over `FieldDataType`, so a type added to the API union fails to compile here
* rather than going missing at runtime. The fallback is for a spelling the union doesn't know
* yet — the backend can store `[]json` and `[]dynamic` — not for any of the cases above.
*/
export const fieldDataTypeToDataType = (
fieldDataType?: FieldDataType,
): DataTypes => TO_DATA_TYPE[fieldDataType ?? ''] ?? DataTypes.EMPTY;

2
go.mod
View File

@@ -4,7 +4,7 @@ go 1.25.7
require (
dario.cat/mergo v1.0.2
github.com/AfterShip/clickhouse-sql-parser v0.4.16
github.com/AfterShip/clickhouse-sql-parser v0.5.2
github.com/ClickHouse/clickhouse-go/v2 v2.44.0
github.com/DATA-DOG/go-sqlmock v1.5.2
github.com/SigNoz/clickhouse-go-mock v0.14.0

4
go.sum
View File

@@ -66,8 +66,8 @@ dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/AfterShip/clickhouse-sql-parser v0.4.16 h1:gpl+wXclYUKT0p4+gBq22XeRYWwEoZ9f35vogqMvkLQ=
github.com/AfterShip/clickhouse-sql-parser v0.4.16/go.mod h1:W0Z82wJWkJxz2RVun/RMwxue3g7ut47Xxl+SFqdJGus=
github.com/AfterShip/clickhouse-sql-parser v0.5.2 h1:KCxdmvuVawVF4yl0ZS33q7olgmLZwvZG5BjWHp6o414=
github.com/AfterShip/clickhouse-sql-parser v0.5.2/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible h1:fcYLmCpyNYRnvJbPerq7U0hS+6+I79yEDJBqVNcqUzU=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0/go.mod h1:t76Ruy8AHvUAC8GfMWJMa0ElSbuIcO03NLpynfbgsPA=

View File

@@ -5,14 +5,14 @@ import (
"fmt"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (m *module) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
metadataTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName)
metadataTable := fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName)
var (
systemMetricCount uint64
k8sMetricCount uint64

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -281,7 +281,7 @@ func (m *module) getPerGroupContainerStatusCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
// Built once; identical across the two fps CTEs (buildFilterClause hits the
// metadata store + parses the expression). AddWhereClause only reads it.
@@ -310,7 +310,7 @@ func (m *module) getPerGroupContainerStatusCounts(
)
}
stateFps.Select(stateFpsCols...)
stateFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
stateFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
stateFps.Where(
stateFps.E("metric_name", containerStatusStateMetricName),
stateFps.GE("unix_milli", tsAdjustedStart),
@@ -342,7 +342,7 @@ func (m *module) getPerGroupContainerStatusCounts(
containerState.Select(containerStateCols...)
containerState.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN state_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerState.Where(
containerState.E("samples.metric_name", containerStatusStateMetricName),
@@ -361,7 +361,7 @@ func (m *module) getPerGroupContainerStatusCounts(
fmt.Sprintf("JSONExtractString(labels, %s) AS container_name", reasonFps.Var(containerNameAttrKey)),
fmt.Sprintf("JSONExtractString(labels, %s) AS reason", reasonFps.Var(containerStatusReasonAttrKey)),
)
reasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
reasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
reasonFps.Where(
reasonFps.E("metric_name", containerStatusReasonMetricName),
reasonFps.GE("unix_milli", tsAdjustedStart),
@@ -391,7 +391,7 @@ func (m *module) getPerGroupContainerStatusCounts(
)
reasonInner.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
reasonInner.Where(
reasonInner.E("samples.metric_name", containerStatusReasonMetricName),
@@ -546,7 +546,7 @@ func (m *module) getPerGroupContainerRestartCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
var (
filterClause *sqlbuilder.WhereClause
@@ -572,7 +572,7 @@ func (m *module) getPerGroupContainerRestartCounts(
)
}
restartFps.Select(restartFpsCols...)
restartFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
restartFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
restartFps.Where(
restartFps.E("metric_name", containerRestartsMetricName),
restartFps.GE("unix_milli", tsAdjustedStart),
@@ -602,7 +602,7 @@ func (m *module) getPerGroupContainerRestartCounts(
containerRestarts.Select(containerRestartsCols...)
containerRestarts.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN restart_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerRestarts.Where(
containerRestarts.E("samples.metric_name", containerRestartsMetricName),
@@ -690,7 +690,7 @@ func (m *module) getPerGroupContainerReadyCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
var (
filterClause *sqlbuilder.WhereClause
@@ -716,7 +716,7 @@ func (m *module) getPerGroupContainerReadyCounts(
)
}
readyFps.Select(readyFpsCols...)
readyFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
readyFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
readyFps.Where(
readyFps.E("metric_name", containerReadyMetricName),
readyFps.GE("unix_milli", tsAdjustedStart),
@@ -746,7 +746,7 @@ func (m *module) getPerGroupContainerReadyCounts(
containerReady.Select(containerReadyCols...)
containerReady.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN ready_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerReady.Where(
containerReady.E("samples.metric_name", containerReadyMetricName),

View File

@@ -9,7 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
@@ -344,11 +344,11 @@ func alignedMetricWindow(startMs, endMs int64) (
flooredEndMs = flooredEndMs - (flooredEndMs % (adjustStep * 1000))
}
tsAdjustedStartMs, _, distributedTSTable, localTSTable := telemetrymetrics.WhichTSTableToUse(
tsAdjustedStartMs, _, distributedTSTable, localTSTable := metricstelemetryschema.WhichTSTableToUse(
samplesAdjustedStartMs, flooredEndMs, false, nil,
)
distributedSamplesTable, localSamplesTable := telemetrymetrics.WhichSamplesTableToUse(
distributedSamplesTable, localSamplesTable := metricstelemetryschema.WhichSamplesTableToUse(
samplesAdjustedStartMs, flooredEndMs,
metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil,
)
@@ -362,7 +362,7 @@ func alignedMetricWindow(startMs, endMs int64) (
func (m *module) buildSamplesTblFingerprintSubQuery(metricNames []string, samplesTable string, flooredStart, flooredEnd uint64) *sqlbuilder.SelectBuilder {
fpSB := sqlbuilder.NewSelectBuilder()
fpSB.Select("DISTINCT fingerprint")
fpSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, samplesTable))
fpSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, samplesTable))
fpSB.Where(
fpSB.In("metric_name", sqlbuilder.List(metricNames)),
fpSB.GE("unix_milli", flooredStart),
@@ -376,7 +376,7 @@ func (m *module) buildSamplesTblFingerprintSubQuery(metricNames []string, sample
func (m *module) buildReducedSamplesTblFingerprintSubQuery(metricNames []string, flooredStart, flooredEnd uint64) *sqlbuilder.SelectBuilder {
lastSB := sqlbuilder.NewSelectBuilder()
lastSB.Select("reduced_fingerprint")
lastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
lastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
lastSB.Where(
lastSB.In("metric_name", sqlbuilder.List(metricNames)),
lastSB.GE("unix_milli", flooredStart),
@@ -385,7 +385,7 @@ func (m *module) buildReducedSamplesTblFingerprintSubQuery(metricNames []string,
sumSB := sqlbuilder.NewSelectBuilder()
sumSB.Select("reduced_fingerprint")
sumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
sumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
sumSB.Where(
sumSB.In("metric_name", sqlbuilder.List(metricNames)),
sumSB.GE("unix_milli", flooredStart),
@@ -478,7 +478,7 @@ func (m *module) getEarliestMetricTime(ctx context.Context, metricNames []string
sb := sqlbuilder.NewSelectBuilder()
sb.Select("min(first_reported_unix_milli) AS min_first_reported")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -504,7 +504,7 @@ func (m *module) getMetricsExistence(ctx context.Context, metricNames []string)
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", "count(*) AS cnt")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
sb.GroupBy("metric_name")
@@ -549,7 +549,7 @@ func (m *module) getAttributesExistence(ctx context.Context, metricNames, attrNa
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select("attr_name", "count(*) AS cnt")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.In("attr_name", sqlbuilder.List(attrNames)),
@@ -656,7 +656,7 @@ func (m *module) getMetadata(
rawSrc := sqlbuilder.NewSelectBuilder()
rawSrc.Select("labels", "unix_milli")
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTableName))
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTableName))
rawSrc.Where(
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
rawSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -669,7 +669,7 @@ func (m *module) getMetadata(
reducedSrc := sqlbuilder.NewSelectBuilder()
reducedSrc.Select("labels", "unix_milli")
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedSrc.Where(
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -683,7 +683,7 @@ func (m *module) getMetadata(
// Inner query reads over the union of raw + reduced series.
innerSB.From(innerSB.BuilderAs(sqlbuilder.UnionAll(rawSrc, reducedSrc), "series"))
} else {
innerSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTableName))
innerSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTableName))
innerSB.Where(
innerSB.In("metric_name", sqlbuilder.List(metricNames)),
innerSB.GE("unix_milli", tsAdjustedStartMs),
@@ -870,7 +870,7 @@ func (m *module) getPerGroupDistinctCounts(
rawSrc := sqlbuilder.NewSelectBuilder()
rawSrc.Select("labels")
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTbl))
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTbl))
rawSrc.Where(
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
rawSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -883,7 +883,7 @@ func (m *module) getPerGroupDistinctCounts(
reducedSrc := sqlbuilder.NewSelectBuilder()
reducedSrc.Select("labels")
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedSrc.Where(
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -896,7 +896,7 @@ func (m *module) getPerGroupDistinctCounts(
sb.From(sb.BuilderAs(sqlbuilder.UnionAll(rawSrc, reducedSrc), "series"))
} else {
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTbl))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTbl))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.GE("unix_milli", tsAdjustedStartMs),

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -72,7 +72,7 @@ func (m *module) getPerGroupHostStatusCounts(
rawSrc := sqlbuilder.NewSelectBuilder()
rawSrc.Select("labels")
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTableName))
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTableName))
rawSrc.Where(
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
rawSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -85,7 +85,7 @@ func (m *module) getPerGroupHostStatusCounts(
reducedSrc := sqlbuilder.NewSelectBuilder()
reducedSrc.Select("labels")
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedSrc.Where(
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -101,7 +101,7 @@ func (m *module) getPerGroupHostStatusCounts(
fpSB := m.buildSamplesTblFingerprintSubQuery(metricNames, localSamplesTable, samplesStartMs, flooredEndMs)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTableName))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.GE("unix_milli", tsAdjustedStartMs),
@@ -357,7 +357,7 @@ func (m *module) getActiveHostsQuery(metricNames []string, hostNameAttr string,
sb := sqlbuilder.NewSelectBuilder()
sb.Distinct()
sb.Select("attr_string_value")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.E("attr_name", hostNameAttr),

View File

@@ -9,7 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/modules/inframonitoring"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
@@ -40,8 +40,8 @@ func NewModule(
providerSettings factory.ProviderSettings,
cfg inframonitoring.Config,
) inframonitoring.Module {
fieldMapper := telemetrymetrics.NewFieldMapper()
condBuilder := telemetrymetrics.NewConditionBuilder(fieldMapper)
fieldMapper := metricstelemetryschema.NewFieldMapper()
condBuilder := metricstelemetryschema.NewConditionBuilder(fieldMapper)
return &module{
telemetryStore: telemetryStore,
telemetryMetadataStore: telemetryMetadataStore,

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -186,7 +186,7 @@ func (m *module) getPerGroupNodeConditionCounts(
// Step-floor bounds + resolve tables in one shot to match QB v5 querier.
samplesStartMs, flooredEndMs, tsAdjustedStartMs, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
// ----- timeSeriesFPs -----
timeSeriesFPs := sqlbuilder.NewSelectBuilder()
@@ -200,7 +200,7 @@ func (m *module) getPerGroupNodeConditionCounts(
)
}
timeSeriesFPs.Select(timeSeriesFPsSelectCols...)
timeSeriesFPs.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
timeSeriesFPs.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
timeSeriesFPs.Where(
timeSeriesFPs.E("metric_name", nodeConditionMetricName),
timeSeriesFPs.GE("unix_milli", tsAdjustedStartMs),
@@ -237,7 +237,7 @@ func (m *module) getPerGroupNodeConditionCounts(
latestConditionPerNode.Select(latestConditionPerNodeSelectCols...)
latestConditionPerNode.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN time_series_fps AS tsfp ON samples.fingerprint = tsfp.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
latestConditionPerNode.Where(
latestConditionPerNode.E("samples.metric_name", nodeConditionMetricName),

View File

@@ -8,7 +8,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -284,7 +284,7 @@ func (m *module) getPerGroupPodStatusCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
// Build the merged filter clause once; it's identical across the three fps
// CTEs, and buildFilterClause hits the metadata store + parses the
@@ -313,7 +313,7 @@ func (m *module) getPerGroupPodStatusCounts(
)
}
phaseFps.Select(phaseFpsCols...)
phaseFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
phaseFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
phaseFps.Where(
phaseFps.E("metric_name", podPhaseMetricName),
phaseFps.GE("unix_milli", tsAdjustedStart),
@@ -340,7 +340,7 @@ func (m *module) getPerGroupPodStatusCounts(
phasePerPod.Select(phasePerPodCols...)
phasePerPod.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN phase_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
phasePerPod.Where(
phasePerPod.E("samples.metric_name", podPhaseMetricName),
@@ -357,7 +357,7 @@ func (m *module) getPerGroupPodStatusCounts(
"fingerprint",
fmt.Sprintf("JSONExtractString(labels, %s) AS pod_uid", podReasonFps.Var(podUIDAttrKey)),
)
podReasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
podReasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
podReasonFps.Where(
podReasonFps.E("metric_name", podStatusReasonMetricName),
podReasonFps.GE("unix_milli", tsAdjustedStart),
@@ -377,7 +377,7 @@ func (m *module) getPerGroupPodStatusCounts(
)
podReasonPerPod.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN pod_reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
podReasonPerPod.Where(
podReasonPerPod.E("samples.metric_name", podStatusReasonMetricName),
@@ -396,7 +396,7 @@ func (m *module) getPerGroupPodStatusCounts(
fmt.Sprintf("JSONExtractString(labels, %s) AS container_name", containerReasonFps.Var(containerNameAttrKey)),
fmt.Sprintf("JSONExtractString(labels, %s) AS reason", containerReasonFps.Var(containerStatusReasonAttrKey)),
)
containerReasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
containerReasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
containerReasonFps.Where(
containerReasonFps.E("metric_name", containerStatusReasonMetricName),
containerReasonFps.GE("unix_milli", tsAdjustedStart),
@@ -432,7 +432,7 @@ func (m *module) getPerGroupPodStatusCounts(
)
containerInner.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN container_reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerInner.Where(
containerInner.E("samples.metric_name", containerStatusReasonMetricName),
@@ -611,7 +611,7 @@ func (m *module) getPerGroupPodRestartCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
var (
filterClause *sqlbuilder.WhereClause
@@ -637,7 +637,7 @@ func (m *module) getPerGroupPodRestartCounts(
)
}
restartFps.Select(restartFpsCols...)
restartFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
restartFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
restartFps.Where(
restartFps.E("metric_name", containerRestartsMetricName),
restartFps.GE("unix_milli", tsAdjustedStart),
@@ -667,7 +667,7 @@ func (m *module) getPerGroupPodRestartCounts(
containerRestarts.Select(containerRestartsCols...)
containerRestarts.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN restart_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerRestarts.Where(
containerRestarts.E("samples.metric_name", containerRestartsMetricName),

View File

@@ -213,18 +213,18 @@ func (module *module) discoverModels(ctx context.Context, orgID valuer.UUID) ([]
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: fmt.Sprintf("%s EXISTS", llmpricingruletypes.GenAIRequestModel)},
Filter: &qbtypes.Filter{Expression: fmt.Sprintf("%s EXISTS", telemetrytypes.GenAIRequestModel)},
Aggregations: []qbtypes.TraceAggregation{
{Expression: "count()", Alias: "spanCount"},
},
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: llmpricingruletypes.GenAIRequestModel,
Name: telemetrytypes.GenAIRequestModel,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: llmpricingruletypes.GenAIProviderName,
Name: telemetrytypes.GenAIProviderName,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
@@ -254,9 +254,9 @@ func (module *module) discoverModels(ctx context.Context, orgID valuer.UUID) ([]
switch c.Type {
case qbtypes.ColumnTypeGroup:
switch c.Name {
case llmpricingruletypes.GenAIRequestModel:
case telemetrytypes.GenAIRequestModel:
modelIdx = i
case llmpricingruletypes.GenAIProviderName:
case telemetrytypes.GenAIProviderName:
providerIdx = i
}
case qbtypes.ColumnTypeAggregation:

View File

@@ -19,8 +19,8 @@ import (
"github.com/SigNoz/signoz/pkg/modules/dashboard"
"github.com/SigNoz/signoz/pkg/modules/metricsexplorer"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
@@ -49,8 +49,8 @@ type module struct {
// NewModule constructs the metrics module with the provided dependencies.
func NewModule(ts telemetrystore.TelemetryStore, telemetryMetadataStore telemetrytypes.MetadataStore, cache cache.Cache, ruleStore ruletypes.RuleStore, dashboardModule dashboard.Module, fl flagger.Flagger, providerSettings factory.ProviderSettings, cfg metricsexplorer.Config) metricsexplorer.Module {
fieldMapper := telemetrymetrics.NewFieldMapper()
condBuilder := telemetrymetrics.NewConditionBuilder(fieldMapper)
fieldMapper := metricstelemetryschema.NewFieldMapper()
condBuilder := metricstelemetryschema.NewConditionBuilder(fieldMapper)
return &module{
telemetryStore: ts,
fieldMapper: fieldMapper,
@@ -89,7 +89,7 @@ func (m *module) listMeterMetrics(ctx context.Context, params *metricsexplorerty
"argMax(temporality, unix_milli) AS temporality",
"argMax(is_monotonic, unix_milli) AS is_monotonic",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymeter.DBName, telemetrymeter.SamplesTableName))
sb.From(fmt.Sprintf("%s.%s", metertelemetryschema.DBName, metertelemetryschema.SamplesTableName))
if params.Start != nil && params.End != nil {
sb.Where(sb.Between("unix_milli", *params.Start, *params.End))
@@ -160,11 +160,11 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
hasRange := params.Start != nil && params.End != nil
if hasRange {
var distributedTsTable string
rangeStart, rangeEnd, distributedTsTable, _ = telemetrymetrics.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
rawSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
rangeStart, rangeEnd, distributedTsTable, _ = metricstelemetryschema.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
rawSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
rawSB.Where(rawSB.Between("unix_milli", rangeStart, rangeEnd))
} else {
rawSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
rawSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
}
applyListFilters(rawSB)
@@ -174,7 +174,7 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
if reductionEnabled {
reducedSB := sqlbuilder.NewSelectBuilder()
reducedSB.Select("DISTINCT metric_name")
reducedSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
if hasRange {
reducedSB.Where(reducedSB.Between("unix_milli", rangeStart, rangeEnd))
}
@@ -512,7 +512,7 @@ func (m *module) CheckMetricExists(ctx context.Context, orgID valuer.UUID, metri
sb := sqlbuilder.NewSelectBuilder()
sb.Select("count(*) > 0 as metricExists")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -534,7 +534,7 @@ func (m *module) HasNonSigNozMetrics(ctx context.Context) (bool, error) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("count(*) > 0")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
sb.Where("metric_name NOT LIKE 'signoz_%'")
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -570,10 +570,10 @@ func (m *module) InspectMetrics(
return nil, err
}
tsStart, _, tsTable, _ := telemetrymetrics.WhichTSTableToUse(start, end, false, nil)
tsStart, _, tsTable, _ := metricstelemetryschema.WhichTSTableToUse(start, end, false, nil)
tsSb := sqlbuilder.NewSelectBuilder()
tsSb.Select("fingerprint", "labels")
tsSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, tsTable))
tsSb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, tsTable))
tsSb.Where(
tsSb.E("metric_name", req.MetricName),
tsSb.GE("unix_milli", tsStart),
@@ -625,7 +625,7 @@ func (m *module) InspectMetrics(
samplesSb := sqlbuilder.NewSelectBuilder()
samplesSb.Select("fingerprint", "unix_milli", "value")
samplesSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName))
samplesSb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName))
samplesSb.Where(
samplesSb.In("fingerprint", fingerprints...),
samplesSb.E("metric_name", req.MetricName),
@@ -717,7 +717,7 @@ func (m *module) fetchUpdatedMetadata(ctx context.Context, orgID valuer.UUID, me
"argMax(temporality, created_at) AS temporality",
"argMax(is_monotonic, created_at) AS is_monotonic",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.UpdatedMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.UpdatedMetadataTableName))
sb.Where(sb.In("metric_name", args...))
sb.GroupBy("metric_name")
@@ -777,7 +777,7 @@ func (m *module) fetchTimeseriesMetadata(ctx context.Context, orgID valuer.UUID,
"anyLast(temporality) AS temporality",
"argMax(is_monotonic, unix_milli) AS is_monotonic",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4TableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4TableName))
sb.Where(sb.In("metric_name", args...))
sb.GroupBy("metric_name")
@@ -891,7 +891,7 @@ func (m *module) checkForLabelInMetric(ctx context.Context, metricName string, l
sb := sqlbuilder.NewSelectBuilder()
sb.Select("count(*) > 0 AS has_label")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
sb.Where(sb.E("attr_name", label))
sb.Limit(1)
@@ -914,7 +914,7 @@ func (m *module) insertMetricsMetadata(ctx context.Context, orgID valuer.UUID, r
createdAt := time.Now().UnixMilli()
ib := sqlbuilder.NewInsertBuilder()
ib.InsertInto(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.UpdatedMetadataTableName))
ib.InsertInto(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.UpdatedMetadataTableName))
ib.Cols("metric_name", "temporality", "is_monotonic", "type", "description", "unit", "created_at")
ib.Values(
req.MetricName,
@@ -1016,9 +1016,9 @@ func (m *module) fetchMetricsStatsWithSamples(
}
}
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
start, end, distributedTsTable, localTsTable := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := metricstelemetryschema.CountExpressionForSamplesTable(distributedSamplesTable)
// Timeseries counts per metric
tsSB := sqlbuilder.NewSelectBuilder()
@@ -1026,7 +1026,7 @@ func (m *module) fetchMetricsStatsWithSamples(
"metric_name",
"uniq(fingerprint) AS timeseries",
)
tsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
tsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
tsSB.Where(tsSB.Between("unix_milli", start, end))
tsSB.Where("NOT startsWith(metric_name, 'signoz')")
if filterWhereClause != nil {
@@ -1040,7 +1040,7 @@ func (m *module) fetchMetricsStatsWithSamples(
"metric_name",
fmt.Sprintf("%s AS samples", countExp),
)
samplesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
samplesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
samplesSB.Where(samplesSB.Between("unix_milli", req.Start, req.End))
samplesSB.Where("NOT startsWith(metric_name, 'signoz')")
@@ -1053,7 +1053,7 @@ func (m *module) fetchMetricsStatsWithSamples(
if filterWhereClause != nil {
fingerprintSB := sqlbuilder.NewSelectBuilder()
fingerprintSB.Select("fingerprint")
fingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
fingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
fingerprintSB.Where(fingerprintSB.Between("unix_milli", start, end))
fingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
fingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1064,7 +1064,7 @@ func (m *module) fetchMetricsStatsWithSamples(
} else {
metricNamesSB := sqlbuilder.NewSelectBuilder()
metricNamesSB.Select("DISTINCT metric_name")
metricNamesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
metricNamesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
metricNamesSB.Where(metricNamesSB.Between("unix_milli", start, end))
metricNamesSB.Where("NOT startsWith(metric_name, 'signoz')")
@@ -1090,18 +1090,18 @@ func (m *module) fetchMetricsStatsWithSamples(
// the data lands either of the _reduced_last/_reduced_sum tables
reducedLastSB := sqlbuilder.NewSelectBuilder()
reducedLastSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
reducedLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
reducedLastSB.Where(reducedLastSB.Between("unix_milli", req.Start, req.End))
reducedLastSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedSumSB := sqlbuilder.NewSelectBuilder()
reducedSumSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
reducedSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
reducedSumSB.Where(reducedSumSB.Between("unix_milli", req.Start, req.End))
reducedSumSB.Where("NOT startsWith(metric_name, 'signoz')")
// separate query for reduced series counts
reducedTsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedTsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedTsSB.Where(reducedTsSB.Between("unix_milli", start, end))
reducedTsSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedTsSB.GroupBy("metric_name")
@@ -1112,7 +1112,7 @@ func (m *module) fetchMetricsStatsWithSamples(
// samples uses a separate cte with local table
reducedFpSB := sqlbuilder.NewSelectBuilder()
reducedFpSB.Select("fingerprint")
reducedFpSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedLocalTableName))
reducedFpSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
reducedFpSB.Where(reducedFpSB.Between("unix_milli", start, end))
reducedFpSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedFpSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1220,11 +1220,11 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
}
}
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
start, end, distributedTsTable, _ := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
totalTSBuilder := sqlbuilder.NewSelectBuilder()
totalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
totalTSBuilder.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
totalTSBuilder.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
totalTSBuilder.Where(totalTSBuilder.Between("unix_milli", start, end))
metricsSB := sqlbuilder.NewSelectBuilder()
@@ -1232,7 +1232,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
"metric_name",
"uniq(fingerprint) AS total_value",
)
metricsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
metricsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
metricsSB.Where(metricsSB.Between("unix_milli", start, end))
metricsSB.Where("NOT startsWith(metric_name, 'signoz')")
if filterWhereClause != nil {
@@ -1244,7 +1244,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
if reductionEnabled {
reducedTotalTSBuilder := sqlbuilder.NewSelectBuilder()
reducedTotalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
reducedTotalTSBuilder.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedTotalTSBuilder.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedTotalTSBuilder.Where(reducedTotalTSBuilder.Between("unix_milli", start, end))
reducedMetricsSB := sqlbuilder.NewSelectBuilder()
@@ -1252,7 +1252,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
"metric_name",
"uniq(fingerprint) AS total_value",
)
reducedMetricsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedMetricsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedMetricsSB.Where(reducedMetricsSB.Between("unix_milli", start, end))
reducedMetricsSB.Where("NOT startsWith(metric_name, 'signoz')")
if filterWhereClause != nil {
@@ -1338,15 +1338,15 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
}
}
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
start, end, distributedTsTable, localTsTable := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := metricstelemetryschema.CountExpressionForSamplesTable(distributedSamplesTable)
candidateLimit := req.Limit + 50
metricCandidatesSB := sqlbuilder.NewSelectBuilder()
metricCandidatesSB.Select("metric_name")
metricCandidatesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
metricCandidatesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
metricCandidatesSB.Where("NOT startsWith(metric_name, 'signoz')")
metricCandidatesSB.Where(metricCandidatesSB.Between("unix_milli", start, end))
if filterWhereClause != nil {
@@ -1364,7 +1364,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if reductionEnabled {
reducedCandidatesSB = sqlbuilder.NewSelectBuilder()
reducedCandidatesSB.Select("metric_name")
reducedCandidatesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedCandidatesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedCandidatesSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedCandidatesSB.Where(reducedCandidatesSB.Between("unix_milli", start, end))
if filterWhereClause != nil {
@@ -1379,7 +1379,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
totalSamplesSB := sqlbuilder.NewSelectBuilder()
totalSamplesSB.Select(fmt.Sprintf("%s AS total_samples", countExp))
totalSamplesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
totalSamplesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
totalSamplesSB.Where(totalSamplesSB.Between("unix_milli", req.Start, req.End))
sampleCountsSB := sqlbuilder.NewSelectBuilder()
@@ -1387,7 +1387,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
"metric_name",
fmt.Sprintf("%s AS samples", countExp),
)
sampleCountsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
sampleCountsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
sampleCountsSB.Where(sampleCountsSB.Between("unix_milli", req.Start, req.End))
sampleCountsSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __metric_candidates)")
@@ -1397,13 +1397,13 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if reductionEnabled {
reducedLastSB = sqlbuilder.NewSelectBuilder()
reducedLastSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
reducedLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
reducedLastSB.Where(reducedLastSB.Between("unix_milli", req.Start, req.End))
reducedLastSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __reduced_metric_candidates)")
reducedSumSB = sqlbuilder.NewSelectBuilder()
reducedSumSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
reducedSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
reducedSumSB.Where(reducedSumSB.Between("unix_milli", req.Start, req.End))
reducedSumSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __reduced_metric_candidates)")
}
@@ -1411,7 +1411,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if filterWhereClause != nil {
fingerprintSB := sqlbuilder.NewSelectBuilder()
fingerprintSB.Select("fingerprint")
fingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
fingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
fingerprintSB.Where(fingerprintSB.Between("unix_milli", start, end))
fingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
fingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1423,7 +1423,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if reductionEnabled {
reducedFingerprintSB := sqlbuilder.NewSelectBuilder()
reducedFingerprintSB.Select("fingerprint")
reducedFingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedLocalTableName))
reducedFingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
reducedFingerprintSB.Where(reducedFingerprintSB.Between("unix_milli", start, end))
reducedFingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedFingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1457,12 +1457,12 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
// Grand total includes reduced sample volume so percentages stay consistent.
reducedTotalLastSB := sqlbuilder.NewSelectBuilder()
reducedTotalLastSB.Select("uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedTotalLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
reducedTotalLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
reducedTotalLastSB.Where(reducedTotalLastSB.Between("unix_milli", req.Start, req.End))
reducedTotalSumSB := sqlbuilder.NewSelectBuilder()
reducedTotalSumSB.Select("uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedTotalSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
reducedTotalSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
reducedTotalSumSB.Where(reducedTotalSumSB.Between("unix_milli", req.Start, req.End))
reducedTotalSamplesSB := sqlbuilder.NewSelectBuilder()
@@ -1552,7 +1552,7 @@ func (m *module) getMetricDataPoints(ctx context.Context, metricName string) (ui
sb := sqlbuilder.NewSelectBuilder()
sb.Select("sum(count) AS data_points")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4Agg30mTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4Agg30mTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -1574,7 +1574,7 @@ func (m *module) getMetricLastReceived(ctx context.Context, metricName string) (
sb := sqlbuilder.NewSelectBuilder()
sb.Select("MAX(last_reported_unix_milli) AS last_received_time")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -1599,7 +1599,7 @@ func (m *module) getTotalTimeSeriesForMetricName(ctx context.Context, metricName
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint) AS time_series_count")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -1623,7 +1623,7 @@ func (m *module) getActiveTimeSeriesForMetricName(ctx context.Context, metricNam
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint) AS active_time_series")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4TableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4TableName))
sb.Where(sb.E("metric_name", metricName))
sb.Where(sb.GTE("unix_milli", milli))
@@ -1649,7 +1649,7 @@ func (m *module) fetchMetricAttributes(ctx context.Context, metricName string, s
"groupUniqArray(1000)(attr_string_value) AS values",
"uniq(attr_string_value) AS valueCount",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
sb.Where("NOT startsWith(attr_name, '__')")

View File

@@ -9,7 +9,7 @@ import (
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
@@ -52,7 +52,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
response := []promotetypes.PromotePath{}
for _, path := range promotedPaths {
fullPath := telemetrylogs.BodyPromotedColumnPrefix + path
fullPath := logstelemetryschema.BodyPromotedColumnPrefix + path
path = telemetrytypes.BodyJSONStringSearchPrefix + path
item := promotetypes.PromotePath{
Path: path,
@@ -68,7 +68,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
// add the paths that are not promoted but have indexes
for path, indexes := range aggr {
path := strings.TrimPrefix(path, telemetrylogs.BodyV2ColumnPrefix)
path := strings.TrimPrefix(path, logstelemetryschema.BodyV2ColumnPrefix)
path = telemetrytypes.BodyJSONStringSearchPrefix + path
response = append(response, promotetypes.PromotePath{
Path: path,
@@ -108,8 +108,8 @@ func (m *module) createIndexes(ctx context.Context, indexes []schemamigrator.Ind
for _, index := range indexes {
alterStmt := schemamigrator.AlterTableAddIndex{
Database: telemetrylogs.DBName,
Table: telemetrylogs.LogsV2LocalTableName,
Database: logstelemetryschema.DBName,
Table: logstelemetryschema.LogsV2LocalTableName,
Index: index,
}
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
@@ -153,10 +153,10 @@ func (m *module) PromoteAndIndexPaths(
}
}
if len(it.Indexes) > 0 {
parentColumn := telemetrylogs.LogsV2BodyV2Column
parentColumn := logstelemetryschema.LogsV2BodyV2Column
// if the path is already promoted or is being promoted, add it to the promoted column
if _, promoted := existingPromotedPaths[it.Path]; promoted || it.Promote {
parentColumn = telemetrylogs.LogsV2BodyPromotedColumn
parentColumn = logstelemetryschema.LogsV2BodyPromotedColumn
}
for _, index := range it.Indexes {

View File

@@ -11,8 +11,8 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/services"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -39,7 +39,7 @@ func (m *module) FetchTopLevelOperations(ctx context.Context, start time.Time, s
ctx = m.withServicesContext(ctx, "FetchTopLevelOperations")
db := m.TelemetryStore.ClickhouseDB()
query := fmt.Sprintf("SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start", telemetrytraces.DBName, telemetrytraces.TopLevelOperationsTableName)
query := fmt.Sprintf("SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start", tracestelemetryschema.DBName, tracestelemetryschema.TopLevelOperationsTableName)
args := []any{clickhouse.Named("start", start)}
if len(services) > 0 {
query += " AND serviceName IN @services"

View File

@@ -22,6 +22,13 @@ import (
"github.com/SigNoz/signoz/pkg/variables"
)
type Handler interface {
QueryRange(rw http.ResponseWriter, req *http.Request)
QueryRangePreview(rw http.ResponseWriter, req *http.Request)
QueryRawStream(rw http.ResponseWriter, req *http.Request)
ReplaceVariables(rw http.ResponseWriter, req *http.Request)
}
type handler struct {
set factory.ProviderSettings
analytics analytics.Analytics
@@ -242,7 +249,7 @@ func (handler *handler) ReplaceVariables(rw http.ResponseWriter, req *http.Reque
errs := []error{}
for idx, item := range queryRangeRequest.CompositeQuery.Queries {
if item.Type == qbtypes.QueryTypeBuilder {
if item.Type == qbtypes.QueryTypeBuilder || item.Type == qbtypes.QueryTypeBuilderAI {
switch spec := item.Spec.(type) {
case qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]:
if spec.Filter != nil && spec.Filter.Expression != "" {

View File

@@ -15,6 +15,14 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
)
// BucketCache is the interface for bucket-based caching.
type BucketCache interface {
// cached portion + list of gaps to fetch
GetMissRanges(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step) (cached *qbtypes.Result, missing []*qbtypes.TimeRange)
// store fresh buckets for future hits
Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step, fresh *qbtypes.Result)
}
// bucketCache implements the BucketCache interface.
type bucketCache struct {
cache cache.Cache

View File

@@ -11,8 +11,8 @@ import (
"github.com/ClickHouse/clickhouse-go/v2"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -277,12 +277,12 @@ func (q *builderQuery[T]) narrowWindowByTraceID(ctx context.Context, fromMS, toM
return fromMS, toMS, true, ""
}
traceIDs, found := telemetrytraces.ExtractTraceIDsFromFilter(q.spec.Filter.Expression)
traceIDs, found := tracestelemetryschema.ExtractTraceIDsFromFilter(q.spec.Filter.Expression)
if !found || len(traceIDs) == 0 {
return fromMS, toMS, true, ""
}
finder := telemetrytraces.NewTraceTimeRangeFinder(q.telemetryStore)
finder := tracestelemetryschema.NewTraceTimeRangeFinder(q.telemetryStore)
traceStart, traceEnd, exists, err := finder.GetTraceTimeRangeMulti(ctx, traceIDs)
if err != nil {
return fromMS, toMS, true, ""

View File

@@ -100,9 +100,20 @@ func (q *chSQLQuery) renderVars(query string, vars map[string]qbtypes.VariableIt
return newQuery.String(), nil
}
// Statement renders the SQL without executing it, for the preview path.
func (q *chSQLQuery) Statement(_ context.Context) (*qbtypes.Statement, error) {
func (q *chSQLQuery) render(ctx context.Context) (string, error) {
rendered, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
if err != nil {
return "", err
}
querybuilder.LogIfStatementIsNotValid(ctx, q.logger, rendered)
return rendered, nil
}
// Statement renders the SQL without executing it, for the preview path.
func (q *chSQLQuery) Statement(ctx context.Context) (*qbtypes.Statement, error) {
rendered, err := q.render(ctx)
if err != nil {
return nil, err
}
@@ -124,7 +135,7 @@ func (q *chSQLQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
elapsed += p.Elapsed
}))
query, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
query, err := q.render(ctx)
if err != nil {
return nil, err
}
@@ -135,11 +146,11 @@ func (q *chSQLQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
}
defer rows.Close()
// TODO: map the errors from ClickHouse to our error types
payload, err := consume(rows, q.kind, nil, qbtypes.Step{}, q.query.Name)
if err != nil {
return nil, err
}
return &qbtypes.Result{
Type: q.kind,
Value: payload,

View File

@@ -6,18 +6,18 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
tracesTable := fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.SpanIndexV3TableName)
logsTable := fmt.Sprintf("%s.%s", telemetrylogs.DBName, telemetrylogs.LogsV2TableName)
metricsTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName)
tracesTable := fmt.Sprintf("%s.%s", tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName)
logsTable := fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName)
metricsTable := fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName)
var (
traces uint64

View File

@@ -9,12 +9,6 @@ import (
const DefaultMaxConcurrentQueries = 8
type SkipResourceFingerprint struct {
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
// If count of fingerprint is above threshold, skip the fingerprint subquery and filter on main table instead.
Threshold uint64 `yaml:"threshold" mapstructure:"threshold"`
}
// Config represents the configuration for the querier.
type Config struct {
// CacheTTL is the TTL for cached query results
@@ -23,8 +17,6 @@ type Config struct {
FluxInterval time.Duration `yaml:"flux_interval" mapstructure:"flux_interval"`
// MaxConcurrentQueries is the maximum number of concurrent queries for missing ranges
MaxConcurrentQueries int `yaml:"max_concurrent_queries" mapstructure:"max_concurrent_queries"`
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
SkipResourceFingerprint SkipResourceFingerprint `yaml:"skip_resource_fingerprint" mapstructure:"skip_resource_fingerprint"`
// LogTraceIDWindowPadding is the padding added to narrowed down timerange from trace summary to logs with trace_id filter.
LogTraceIDWindowPadding time.Duration `yaml:"log_trace_id_window_padding" mapstructure:"log_trace_id_window_padding"`
}
@@ -39,11 +31,7 @@ func newConfig() factory.Config {
// Default values
CacheTTL: 168 * time.Hour,
FluxInterval: 5 * time.Minute,
MaxConcurrentQueries: DefaultMaxConcurrentQueries,
SkipResourceFingerprint: SkipResourceFingerprint{
Enabled: false,
Threshold: 100000,
},
MaxConcurrentQueries: DefaultMaxConcurrentQueries,
LogTraceIDWindowPadding: 5 * time.Minute,
}
}
@@ -59,9 +47,6 @@ func (c Config) Validate() error {
if c.MaxConcurrentQueries <= 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "max_concurrent_queries must be positive, got %v", c.MaxConcurrentQueries)
}
if c.SkipResourceFingerprint.Enabled && c.SkipResourceFingerprint.Threshold == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "skip_resource_fingerprint.threshold must be > 0 when enabled")
}
if c.LogTraceIDWindowPadding < 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "log_trace_id_window_padding must not be negative, got %v", c.LogTraceIDWindowPadding)
}

View File

@@ -1,35 +0,0 @@
package querier
import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/statsreporter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
// Querier interface defines the contract for querying data.
type Querier interface {
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
statsreporter.StatsCollector
// QueryRangePreview validates and renders the queries without executing them.
QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
}
// BucketCache is the interface for bucket-based caching.
type BucketCache interface {
// cached portion + list of gaps to fetch
GetMissRanges(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step) (cached *qbtypes.Result, missing []*qbtypes.TimeRange)
// store fresh buckets for future hits
Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step, fresh *qbtypes.Result)
}
type Handler interface {
QueryRange(rw http.ResponseWriter, req *http.Request)
// QueryRangePreview is the dry-run endpoint: validate and render without executing.
QueryRangePreview(rw http.ResponseWriter, req *http.Request)
QueryRawStream(rw http.ResponseWriter, req *http.Request)
ReplaceVariables(rw http.ResponseWriter, req *http.Request)
}

View File

@@ -249,6 +249,7 @@ func (q *querier) buildPreviewProviders(
func rendersStandaloneStatement(t qbtypes.QueryType) bool {
switch t {
case qbtypes.QueryTypeBuilder,
qbtypes.QueryTypeBuilderAI,
qbtypes.QueryTypePromQL,
qbtypes.QueryTypeClickHouseSQL,
qbtypes.QueryTypeTraceOperator:

View File

@@ -331,11 +331,14 @@ func (q *promqlQuery) Execute(ctx context.Context) (*qbv5.Result, error) {
return nil, errors.WrapInternalf(promErr, errors.CodeInternal, "error getting matrix from promql query %q", query)
}
// Hide only known SigNoz storage keys: label names are user data and may
// legitimately start with "__" (e.g. __address__), so a blanket dunder
// strip mangles user labelsets. The __scope./__resource. prefixes cover
// every exporter version's keys.
excludeLabel := func(labelName string) bool {
if labelName == "__name__" {
return false
}
return strings.HasPrefix(labelName, "__") || labelName == "fingerprint"
return labelName == "__temporality__" ||
strings.HasPrefix(labelName, "__scope.") ||
strings.HasPrefix(labelName, "__resource.")
}
var series []*qbv5.TimeSeries

View File

@@ -21,6 +21,7 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/query-service/utils"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
@@ -35,6 +36,15 @@ var (
intervalWarn = "Query %s is requesting aggregation interval %v seconds, which is smaller than the minimum allowed interval of %v seconds for selected time range. Using the minimum instead"
)
// Querier interface defines the contract for querying data.
type Querier interface {
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
statsreporter.StatsCollector
// QueryRangePreview validates and renders the queries without executing them.
QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
}
type querier struct {
logger *slog.Logger
fl flagger.Flagger
@@ -42,6 +52,7 @@ type querier struct {
metadataStore telemetrytypes.MetadataStore
promEngine prometheus.Prometheus
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation]
@@ -61,6 +72,7 @@ func New(
metadataStore telemetrytypes.MetadataStore,
promEngine prometheus.Prometheus,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
@@ -82,6 +94,7 @@ func New(
metadataStore: metadataStore,
promEngine: promEngine,
traceStmtBuilder: traceStmtBuilder,
aiTraceStmtBuilder: aiTraceStmtBuilder,
logStmtBuilder: logStmtBuilder,
auditStmtBuilder: auditStmtBuilder,
metricStmtBuilder: metricStmtBuilder,
@@ -232,6 +245,16 @@ func (q *querier) buildQueries(
}
queries[traceOpQuery.Name] = toq
steps[traceOpQuery.Name] = traceOpQuery.StepInterval
case qbtypes.QueryTypeBuilderAI:
spec, ok := query.Spec.(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation])
if !ok {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid AI builder query spec %T", query.Spec)
}
spec.ShiftBy = extractShiftFromBuilderQuery(spec)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, req.RequestType)
bq := newBuilderQuery(q.logger, q.telemetryStore, orgID, q.aiTraceStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
queries[spec.Name] = bq
steps[spec.Name] = spec.StepInterval
case qbtypes.QueryTypeBuilder:
switch spec := query.Spec.(type) {
case qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]:
@@ -298,6 +321,11 @@ func (q *querier) populateQBEvent(event *qbtypes.QBEvent, queries []qbtypes.Quer
case qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]:
event.MetricsUsed = true
}
case qbtypes.QueryTypeBuilderAI:
filter := query.GetFilter()
event.FilterApplied = event.FilterApplied || (filter != nil && filter.Expression != "")
event.GroupByApplied = event.GroupByApplied || len(query.GetGroupBy()) > 0
event.TracesUsed = true
case qbtypes.QueryTypePromQL:
event.MetricsUsed = true
case qbtypes.QueryTypeTraceOperator:
@@ -860,7 +888,9 @@ func (q *querier) createRangedQuery(_ valuer.UUID, originalQuery qbtypes.Query,
specCopy := qt.spec.Copy()
specCopy.ShiftBy = extractShiftFromBuilderQuery(specCopy)
adjustedTimeRange := adjustTimeRangeForShift(specCopy, timeRange, qt.kind)
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, q.traceStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
// Reuse the statement builder the original query was created with, so an AI
// query keeps its AI builder without re-deriving it from the spec.
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, qt.stmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
case *builderQuery[qbtypes.LogAggregation]:
specCopy := qt.spec.Copy()
@@ -1217,6 +1247,8 @@ func (q *querier) adjustStepInterval(queries []qbtypes.QueryEnvelope, start, end
if qe.GetStepInterval().Seconds() == 0 {
qe.SetStepInterval(secondsStep(metricRecommended))
}
case qbtypes.QueryTypeBuilderAI:
clampStep(qe, traceLogRecommended, traceLogMin, &warnings)
case qbtypes.QueryTypeTraceOperator:
clampStep(qe, traceLogRecommended, traceLogMin, &warnings)
}

View File

@@ -49,6 +49,7 @@ func TestQueryRange_MetricTypeMissing(t *testing.T) {
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
@@ -119,17 +120,18 @@ func TestQueryRange_MetricTypeFromStore(t *testing.T) {
providerSettings,
telemetryStore,
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
&mockMetricStmtBuilder{}, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
flaggertest.New(t), // flagger
0, // logTraceIDWindowPadding
0, // maxConcurrentQueries
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
&mockMetricStmtBuilder{},
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
flaggertest.New(t), // flagger
0, // logTraceIDWindowPadding
0, // maxConcurrentQueries
)
req := &qbtypes.QueryRangeRequest{

View File

@@ -0,0 +1,52 @@
package queriertest
import (
"context"
"github.com/SigNoz/signoz/pkg/querier"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
// MockQuerier implements querier.Querier for testing. Each behavior can be
// overridden via a function field; unset fields return empty, non-error values.
type MockQuerier struct {
QueryRangeFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStreamFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
QueryRangePreviewFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
CollectFunc func(ctx context.Context, orgID valuer.UUID) (map[string]any, error)
}
var _ querier.Querier = (*MockQuerier)(nil)
// NewMockQuerier creates a new MockQuerier.
func NewMockQuerier() *MockQuerier {
return &MockQuerier{}
}
func (m *MockQuerier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error) {
if m.QueryRangeFunc != nil {
return m.QueryRangeFunc(ctx, orgID, req)
}
return &qbtypes.QueryRangeResponse{}, nil
}
func (m *MockQuerier) QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream) {
if m.QueryRawStreamFunc != nil {
m.QueryRawStreamFunc(ctx, orgID, req, client)
}
}
func (m *MockQuerier) QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error) {
if m.QueryRangePreviewFunc != nil {
return m.QueryRangePreviewFunc(ctx, orgID, req, opts)
}
return &qbtypes.QueryRangePreviewResponse{}, nil
}
func (m *MockQuerier) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
if m.CollectFunc != nil {
return m.CollectFunc(ctx, orgID)
}
return map[string]any{}, nil
}

View File

@@ -3,192 +3,56 @@ package signozquerier
import (
"context"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// NewFactory creates a new factory for the signoz querier provider.
// NewFactory creates a new factory for the signoz querier provider. The query
// stack (metadata store, statement builders, bucket cache) is assembled once in
// signoz.go and injected here.
func NewFactory(
telemetryStore telemetrystore.TelemetryStore,
prometheus prometheus.Prometheus,
cache cache.Cache,
metadataStore telemetrytypes.MetadataStore,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder,
bucketCache querier.BucketCache,
flagger flagger.Flagger,
) factory.ProviderFactory[querier.Querier, querier.Config] {
return factory.NewProviderFactory(
factory.MustNewName("signoz"),
func(
ctx context.Context,
_ context.Context,
settings factory.ProviderSettings,
cfg querier.Config,
) (querier.Querier, error) {
return newProvider(ctx, settings, cfg, telemetryStore, prometheus, cache, flagger)
return querier.New(
settings,
telemetryStore,
metadataStore,
prometheus,
traceStmtBuilder,
aiTraceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,
meterStmtBuilder,
traceOperatorStmtBuilder,
bucketCache,
flagger,
cfg.LogTraceIDWindowPadding,
cfg.MaxConcurrentQueries,
), nil
},
)
}
func newProvider(
_ context.Context,
settings factory.ProviderSettings,
cfg querier.Config,
telemetryStore telemetrystore.TelemetryStore,
prometheus prometheus.Prometheus,
cache cache.Cache,
flagger flagger.Flagger,
) (querier.Querier, error) {
// Create telemetry metadata store
telemetryMetadataStore := telemetrymetadata.NewTelemetryMetaStore(
settings,
telemetryStore,
telemetrytraces.DBName,
telemetrytraces.TagAttributesV2TableName,
telemetrytraces.SpanAttributesKeysTblName,
telemetrytraces.SpanIndexV3TableName,
telemetrymetrics.DBName,
telemetrymetrics.AttributesMetadataTableName,
telemetrymeter.DBName,
telemetrymeter.SamplesAgg1dTableName,
telemetrylogs.DBName,
telemetrylogs.LogsV2TableName,
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.AuditLogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
flagger,
)
// Create trace statement builder
traceFieldMapper := telemetrytraces.NewFieldMapper()
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, traceFieldMapper, traceConditionBuilder, flagger)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
settings,
telemetryMetadataStore,
traceFieldMapper,
traceConditionBuilder,
traceAggExprRewriter,
telemetryStore,
flagger,
cfg.SkipResourceFingerprint.Enabled,
cfg.SkipResourceFingerprint.Threshold,
)
// Create trace operator statement builder
traceOperatorStmtBuilder := telemetrytraces.NewTraceOperatorStatementBuilder(
settings,
telemetryMetadataStore,
traceFieldMapper,
traceConditionBuilder,
traceStmtBuilder,
traceAggExprRewriter,
flagger,
)
// Create log statement builder
logFieldMapper := telemetrylogs.NewFieldMapper(flagger)
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, flagger)
logAggExprRewriter := querybuilder.NewAggExprRewriter(
settings,
telemetrylogs.DefaultFullTextColumn,
logFieldMapper,
logConditionBuilder,
flagger,
)
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
settings,
telemetryMetadataStore,
logFieldMapper,
logConditionBuilder,
logAggExprRewriter,
telemetrylogs.DefaultFullTextColumn,
flagger,
telemetryStore,
cfg.SkipResourceFingerprint.Enabled,
cfg.SkipResourceFingerprint.Threshold,
)
// Create audit statement builder
auditFieldMapper := telemetryaudit.NewFieldMapper()
auditConditionBuilder := telemetryaudit.NewConditionBuilder(auditFieldMapper)
auditAggExprRewriter := querybuilder.NewAggExprRewriter(
settings,
telemetryaudit.DefaultFullTextColumn,
auditFieldMapper,
auditConditionBuilder,
flagger,
)
auditStmtBuilder := telemetryaudit.NewAuditQueryStatementBuilder(
settings,
telemetryMetadataStore,
auditFieldMapper,
auditConditionBuilder,
auditAggExprRewriter,
telemetryaudit.DefaultFullTextColumn,
flagger,
)
// Create metric statement builder
metricFieldMapper := telemetrymetrics.NewFieldMapper()
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(
settings,
telemetryMetadataStore,
metricFieldMapper,
metricConditionBuilder,
flagger,
)
// Create meter statement builder
meterStmtBuilder := telemetrymeter.NewMeterQueryStatementBuilder(
settings,
telemetryMetadataStore,
metricFieldMapper,
metricConditionBuilder,
metricStmtBuilder,
)
// Create bucket cache
bucketCache := querier.NewBucketCache(
settings,
cache,
cfg.CacheTTL,
cfg.FluxInterval,
)
// Create and return the querier
return querier.New(
settings,
telemetryStore,
telemetryMetadataStore,
prometheus,
traceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,
meterStmtBuilder,
traceOperatorStmtBuilder,
bucketCache,
flagger,
cfg.LogTraceIDWindowPadding,
cfg.MaxConcurrentQueries,
), nil
}

View File

@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/modules/thirdpartyapi"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/queryparser"
"log/slog"
@@ -1092,6 +1093,8 @@ func (aH *APIHandler) queryDashboardVarsV2(w http.ResponseWriter, r *http.Reques
return
}
querybuilder.LogIfStatementIsNotValid(r.Context(), aH.logger, query)
dashboardVars, err := aH.reader.QueryDashboardVars(r.Context(), query)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorBadData, Err: err}, nil)

View File

@@ -18,6 +18,14 @@ import (
"github.com/SigNoz/signoz/pkg/querier/signozquerier"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstoretest"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/aistatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/auditstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/meterstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/stretchr/testify/require"
@@ -105,7 +113,25 @@ func NewTestManager(t *testing.T, testOpts *TestManagerOptions) *Manager {
}
// Create querier with test values
providerFactory := signozquerier.NewFactory(telemetryStore, prometheus, cache, flagger)
metadataStore := telemetrymetadata.NewTelemetryMetaStore(providerSettings, telemetryStore, flagger)
cfg := statementbuilder.Config{}
ctx := context.Background()
traceStmtBuilder, err := tracesstatementbuilder.NewFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
aiTraceStmtBuilder, err := aistatementbuilder.NewFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
traceOperatorStmtBuilder, err := tracesstatementbuilder.NewOperatorFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
logStmtBuilder, err := logsstatementbuilder.NewFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
auditStmtBuilder, err := auditstatementbuilder.NewFactory(metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
metricStmtBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
meterStmtBuilder, err := meterstatementbuilder.NewFactory(metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
bucketCache := querier.NewBucketCache(providerSettings, cache, 0, 0)
providerFactory := signozquerier.NewFactory(telemetryStore, prometheus, metadataStore, traceStmtBuilder, aiTraceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, flagger)
mockQuerier, err := providerFactory.New(context.Background(), providerSettings, querier.Config{})
require.NoError(t, err)

View File

@@ -8,11 +8,11 @@ import (
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/require"
@@ -24,7 +24,7 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
providerSettings := instrumentationtest.New().ToProviderSettings()
metadataStore := telemetrytypestest.NewMockMetadataStore()
flagger, err := flagger.New(
fl, err := flagger.New(
context.Background(),
instrumentationtest.New().ToProviderSettings(),
flagger.Config{},
@@ -32,15 +32,8 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
)
require.NoError(t, err)
metricFieldMapper := telemetrymetrics.NewFieldMapper()
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(
providerSettings,
metadataStore,
metricFieldMapper,
metricConditionBuilder,
flagger,
)
metricStmtBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, fl).New(context.Background(), providerSettings, statementbuilder.Config{})
require.NoError(t, err)
return querier.New(
providerSettings,
@@ -48,13 +41,14 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
metricStmtBuilder,
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
flagger,
fl,
0,
0, // maxConcurrentQueries (0 means default)
), metadataStore
@@ -73,40 +67,22 @@ func prepareQuerierForLogs(t *testing.T, telemetryStore telemetrystore.Telemetry
metadataStore.KeysMap = keysMap
fl := flaggertest.New(t)
logFieldMapper := telemetrylogs.NewFieldMapper(fl)
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, fl)
logAggExprRewriter := querybuilder.NewAggExprRewriter(
providerSettings,
telemetrylogs.DefaultFullTextColumn,
logFieldMapper,
logConditionBuilder,
fl,
)
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
providerSettings,
metadataStore,
logFieldMapper,
logConditionBuilder,
logAggExprRewriter,
telemetrylogs.DefaultFullTextColumn,
fl,
nil,
false,
100000,
)
logStmtBuilder, err := logsstatementbuilder.NewFactory(nil, metadataStore, fl).New(context.Background(), providerSettings, statementbuilder.Config{})
require.NoError(t, err)
return querier.New(
providerSettings,
telemetryStore,
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
logStmtBuilder, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
nil, // prometheus
nil, // traceStmtBuilder
nil, // aiTraceStmtBuilder
logStmtBuilder,
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
fl,
5*time.Minute, // logTraceIDWindowPadding
0, // maxConcurrentQueries (0 means default)
@@ -126,36 +102,23 @@ func prepareQuerierForTraces(t *testing.T, telemetryStore telemetrystore.Telemet
}
metadataStore.KeysMap = keysMap
// Create trace statement builder
traceFieldMapper := telemetrytraces.NewFieldMapper()
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
fl := flaggertest.New(t)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(providerSettings, nil, traceFieldMapper, traceConditionBuilder, fl)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
providerSettings,
metadataStore,
traceFieldMapper,
traceConditionBuilder,
traceAggExprRewriter,
telemetryStore,
fl,
false,
100000,
)
traceStmtBuilder, err := tracesstatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(context.Background(), providerSettings, statementbuilder.Config{})
require.NoError(t, err)
return querier.New(
providerSettings,
telemetryStore,
metadataStore,
nil, // prometheus
traceStmtBuilder, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
nil, // prometheus
traceStmtBuilder,
nil, // aiTraceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
fl,
0,
0, // maxConcurrentQueries (0 means default)

View File

@@ -96,9 +96,9 @@ func (r *aggExprRewriter) Rewrite(
}
if visitor.isRate {
return fmt.Sprintf("%s/%d", sel.SelectItems[0].String(), rateInterval), visitor.chArgs, nil
return fmt.Sprintf("%s/%d", chparser.Format(sel.SelectItems[0]), rateInterval), visitor.chArgs, nil
}
return sel.SelectItems[0].String(), visitor.chArgs, nil
return chparser.Format(sel.SelectItems[0]), visitor.chArgs, nil
}
// RewriteMulti rewrites a slice of expressions.
@@ -207,7 +207,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
// Handle *If functions with predicate + values
if aggFunc.FuncCombinator {
// Map the predicate (last argument)
origPred := args[len(args)-1].String()
origPred := chparser.Format(args[len(args)-1])
whereClause, err := PrepareWhereClause(
origPred,
FilterExprVisitorOpts{
@@ -242,7 +242,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
// Map each value column argument
for i := 0; i < len(args)-1; i++ {
origVal := args[i].String()
origVal := chparser.Format(args[i])
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(origVal)
expr, err := v.fieldMapper.ColumnExpressionFor(v.ctx, v.orgID, v.startNs, v.endNs, &fieldKey, dataType, v.fieldKeys)
if err != nil {
@@ -259,7 +259,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
} else {
// Non-If functions: map every argument as a column/value
for i, arg := range args {
orig := arg.String()
orig := chparser.Format(arg)
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(orig)
expr, err := v.fieldMapper.ColumnExpressionFor(v.ctx, v.orgID, v.startNs, v.endNs, &fieldKey, dataType, v.fieldKeys)
if err != nil {

View File

@@ -0,0 +1,76 @@
package querybuilder
import (
"context"
"log/slog"
"strings"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/SigNoz/signoz/pkg/errors"
)
// internalDatabases hold server metadata, credentials and grants rather than telemetry.
var internalDatabases = map[string]struct{}{
"system": {},
"information_schema": {},
}
// The parser's grammar has gaps against SQL that ClickHouse itself accepts. See TestErrIfStatementIsNotValid_ShouldPassButFails.
func ErrIfStatementIsNotValid(query string) (err error) {
defer func() {
// The parser has a history of panicking on malformed input rather than returning an error.
if recovered := recover(); recovered != nil {
err = errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid ClickHouse SQL (recovered): %v", recovered)
}
}()
stmts, parseErr := chparser.NewParser(query).ParseStmts()
if parseErr != nil {
// Wrapped rather than formatted in, so that callers can recover the parser's *ParseError and read the position off it.
return errors.WrapInvalidInputf(parseErr, errors.CodeInvalidInput, "invalid ClickHouse SQL: %s", parseErr.Error())
}
if len(stmts) != 1 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse SQL must contain exactly one statement, found %d statements", len(stmts))
}
selectQuery, ok := stmts[0].(*chparser.SelectQuery)
if !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "only SELECT statements are allowed in ClickHouse SQL queries")
}
visitor := &chparser.DefaultASTVisitor{Visit: func(node chparser.Expr) error {
switch expr := node.(type) {
case *chparser.TableFunctionExpr:
// Source table functions remain usable in ClickHouse read-only mode.
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse table functions are not allowed in SQL queries: %s", chparser.Format(expr.Name))
case *chparser.TableIdentifier:
// Reading these is unaffected by ClickHouse read-only mode.
if expr.Database == nil {
return nil
}
if _, ok := internalDatabases[strings.ToLower(expr.Database.Name)]; ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse %s database is not allowed in SQL queries", expr.Database.Name)
}
case *chparser.SettingExpr:
// A query-level setting takes precedence over the context setting.
if strings.EqualFold(expr.Name.Name, "readonly") {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse readonly setting cannot be overridden")
}
}
return nil
}}
return selectQuery.Accept(visitor)
}
// TODO(@therealpandey): remove this and move to ErrIfStatementIsNotValid.
func LogIfStatementIsNotValid(ctx context.Context, logger *slog.Logger, query string) {
if err := ErrIfStatementIsNotValid(query); err != nil {
logger.WarnContext(ctx, "clickhouse sql is not valid", errors.Attr(err), slog.String("query", query))
}
}

View File

@@ -0,0 +1,154 @@
package querybuilder
import (
"testing"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
testCases := []struct {
name string
query string
}{
// Shapes a telemetry read is allowed to take.
{"Select", "SELECT region AS r, zone FROM metrics WHERE metric_name = 'cpu' GROUP BY region, zone"},
{"TrailingSemicolon", "SELECT count() FROM signoz_logs.distributed_logs_v2;"},
{"CommonTableExpression", "WITH t AS (SELECT fingerprint FROM signoz_metrics.time_series_v4) SELECT * FROM t"},
{"Join", "SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.b"},
{"GlobalIn", "SELECT a FROM t WHERE a GLOBAL IN (SELECT b FROM t2)"},
{"Union", "SELECT * FROM t UNION ALL SELECT * FROM t2"},
{"Intersect", "SELECT * FROM t INTERSECT SELECT * FROM t2"},
{"WindowFunction", "SELECT sum(v) OVER (PARTITION BY a ORDER BY t) FROM t"},
{"UnrelatedSetting", "SELECT * FROM t SETTINGS max_threads = 4"},
{"TerminatedBlockComment", "SELECT /* keep me */ count() FROM t"},
{"BlockCommentMarkerInsideStringLiteral", "SELECT count() FROM t WHERE body = '/* not a comment'"},
// The parser used to loop forever on this; it now reads the comment to the end of
// the input, so this doubles as a canary for that regression.
{"TrailingUnterminatedBlockComment", "SELECT count() FROM t /* unterminated"},
// The rule keys on the database, not on the table name.
{"TableNamedSystemInTelemetryDatabase", "SELECT * FROM signoz_logs.system"},
{"SignedLiteralAfterClosingParen", "SELECT (toUnixTimestamp(now()) - 3600)*1000000000"},
// order by interval
{"OrderByInterval", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval ORDER BY interval"},
{"OrderByIntervalAndDirection", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS `interval` ORDER BY `interval` ASC"},
{"TrimFunction", "SELECT trimBoth('/api/endpoint/', '/');"},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
assert.NoError(t, err)
})
}
}
func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
testCases := []struct {
name string
query string
}{
// Not a single statement, or not a statement at all.
{"Empty", ""},
{"UnterminatedBlockCommentOnly", "/* x"},
{"Unparseable", "SELECT FROM WHERE"},
{"MultipleStatements", "SELECT 1; DROP TABLE signoz_logs.logs_v2"},
// Parses, but is not a SELECT.
{"Drop", "DROP TABLE signoz_logs.logs_v2"},
{"Insert", "INSERT INTO signoz_logs.logs_v2 SELECT * FROM signoz_logs.logs_v2"},
{"AlterDelete", "ALTER TABLE signoz_logs.logs_v2 DELETE WHERE 1 = 1"},
{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory"},
{"Grant", "GRANT ALL ON *.* TO admin"},
{"Set", "SET readonly = 0"},
// These the parser rejects outright rather than classifying.
{"ShowGrants", "SHOW GRANTS"},
{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'"},
// Table functions, which read through something other than a telemetry table.
{"UrlTableFunction", "SELECT * FROM url('http://attacker.example/x', CSV, 'a String')"},
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')"},
{"ExecutableTableFunction", "SELECT * FROM executable('script.sh', CSV, 'a String')"},
{"TableFunctionInJoin", "SELECT * FROM t1 JOIN url('http://x', CSV, 'a String') u ON 1 = 1"},
{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c"},
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))"},
{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')"},
// Internal databases, which hold grants and server metadata rather than telemetry.
{"SystemUsers", "SELECT * FROM system.users"},
{"SystemUppercase", "SELECT * FROM SYSTEM.USERS"},
{"SystemQuoted", "SELECT count() FROM `system`.`tables`"},
{"SystemInSubquery", "SELECT * FROM (SELECT name FROM system.parts)"},
{"SystemInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l JOIN system.users AS u ON 1 = 1"},
{"SystemInIntersect", "SELECT * FROM t INTERSECT SELECT * FROM system.users"},
{"InformationSchema", "SELECT * FROM information_schema.tables"},
// A query-level setting takes precedence over the one the caller applies.
{"ReadonlySettingOverride", "SELECT * FROM t SETTINGS readonly = 0"},
{"ReadonlySettingOverrideAmongOthers", "SELECT * FROM t SETTINGS max_threads = 4, readonly = 0"},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
assert.Error(t, err)
})
}
}
// Queries the parser cannot read. ClickHouse runs all of them.
func TestErrIfStatementIsNotValid_ShouldPassButFails(t *testing.T) {
testCases := []struct {
name string
query string
// The construct the parser stops after, which is the one it cannot read.
expectedStopsAfter string
// The same construct written so the parser accepts it.
fix string
}{
{
name: "IntervalAliasInOrderBy",
query: "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, resource_string_service$$name AS `service.name`, attributes_string['http.route'] AS `http.route`, quantile(0.95)(duration_nano) / 1000000000 AS value FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_string_service$$name = 'svc-a' AND resources_string['deployment.environment'] = 'dev' AND attributes_string['http.route'] = '/v1' AND http_method = 'POST' AND timestamp BETWEEN toDateTime(1784601720) AND toDateTime(1784602620) AND ts_bucket_start BETWEEN 1784601720 - 1800 AND 1784602620 GROUP BY `service.name`, `http.route`, interval ORDER BY interval ASC",
expectedStopsAfter: "ORDER BY interval ASC",
fix: "SELECT count() AS interval FROM t ORDER BY `interval` ASC",
},
{
name: "IntervalAliasInOrderByDesc",
query: "SELECT count() AS value, toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, serviceName, resourceTagsMap['deployment.environment'] AS environment, exceptionStacktrace FROM signoz_traces.distributed_signoz_error_index_v2 WHERE exceptionType != 'OSError' AND resourceTagsMap['deployment.environment'] = 'staging' AND timestamp BETWEEN toDateTime(1785186300) AND toDateTime(1785186600) GROUP BY serviceName, interval, environment, exceptionStacktrace ORDER BY interval DESC",
expectedStopsAfter: "ORDER BY interval DESC",
fix: "SELECT count() AS interval FROM t ORDER BY `interval` DESC",
},
{
name: "StandardTrimSyntax",
query: "SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 5 MINUTE) AS interval, resources_string['host.name'] as host_name, toFloat64(countIf( lower(trim(BOTH ' ' FROM replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ))) IN ('unhealthy','starting','failing') )) as value FROM signoz_logs.distributed_logs_v2 WHERE timestamp BETWEEN 1784602320000000000 AND 1784602620000000000 AND ts_bucket_start BETWEEN 1784602320 - 300 AND 1784602620 AND JSONExtractString(body, 'Type') = 'container' AND JSONExtractString(body, 'Actor', 'Attributes', 'name') IS NOT NULL AND resources_string['host.name'] IS NOT NULL AND resources_string['host.name'] = 'aihub-nightly' GROUP BY interval, host_name ORDER BY interval, host_name",
expectedStopsAfter: "trim(BOTH '",
fix: "SELECT trimBoth(replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ), ' ')",
},
{
name: "SignedLiteralAfterClosingParen",
query: "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)",
expectedStopsAfter: "(toUnixTimestamp(now())",
fix: "SELECT (toUnixTimestamp(now()) - 3600) * 1000000000",
},
{
name: "SignedLiteralAfterClosingParenMinimal",
query: "SELECT (1)-1",
expectedStopsAfter: "(1)",
fix: "SELECT (1) - 1",
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
var parseErr *chparser.ParseError
require.ErrorAs(t, err, &parseErr, "expected a parser failure rather than a rule violation")
// The parser reports the offset it stopped at, which sits just past the construct
// it choked on, so the text leading up to it is what needs looking at.
consumed := testCase.query[:parseErr.Pos]
assert.Equal(t, testCase.expectedStopsAfter, consumed[max(0, len(consumed)-len(testCase.expectedStopsAfter)):])
assert.NoError(t, ErrIfStatementIsNotValid(testCase.fix))
})
}
}

View File

@@ -0,0 +1,176 @@
package querybuilder
import (
"strings"
"github.com/SigNoz/signoz/pkg/errors"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/antlr4-go/antlr/v4"
)
// SplitFilterForAggregates partitions a single filter expression into a span-level
// part (a WHERE over spans) and a trace-level part (a HAVING over per-trace
// aggregates), splitting on the top-level AND.
//
// A key is trace-level when it carries the trace field context (`trace.completion_tokens`)
// or, with no context, its bare name is in aggregateNames. Any other explicit context
// (`span.`, `resource.`, …) is span-level. Trace-level and span-level keys may be
// AND-combined (they run at different query stages) but not OR-combined; an OR that
// mixes the two is an error.
func SplitFilterForAggregates(query string, aggregateNames map[string]struct{}) (spanExpr string, havingExpr string, err error) {
if strings.TrimSpace(query) == "" {
return "", "", nil
}
tree, syntaxErrors := parseFilterQuery(query)
if len(syntaxErrors) > 0 {
combinedErrors := errors.Newf(
errors.TypeInvalidInput,
errors.CodeInvalidInput,
"Found %d syntax errors while parsing the filter expression.",
len(syntaxErrors),
)
additionals := make([]string, 0, len(syntaxErrors))
for _, syntaxError := range syntaxErrors {
if syntaxError.Error() != "" {
additionals = append(additionals, syntaxError.Error())
}
}
// TODO: add troubleshooting link to the filter query syntax guide once it's published.
return "", "", combinedErrors.WithAdditional(additionals...)
}
s := filterSplitter{query: []rune(query), aggregateNames: aggregateNames}
s.visit(tree)
if s.mixed {
return "", "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level and span-level filters cannot be combined within an OR/NOT group; separate them with a top-level AND")
}
return strings.Join(s.span, " AND "), strings.Join(s.having, " AND "), nil
}
func parseFilterQuery(query string) (antlr.Tree, []*SyntaxErr) {
lexerErrorListener := NewErrorListener()
lexer := grammar.NewFilterQueryLexer(antlr.NewInputStream(query))
lexer.RemoveErrorListeners()
lexer.AddErrorListener(lexerErrorListener)
parserErrorListener := NewErrorListener()
parser := grammar.NewFilterQueryParser(antlr.NewCommonTokenStream(lexer, 0))
parser.RemoveErrorListeners()
parser.AddErrorListener(parserErrorListener)
tree := parser.Query()
return tree, append(lexerErrorListener.SyntaxErrors, parserErrorListener.SyntaxErrors...)
}
// filterSplitter walks the parse tree once, flattening the top-level AND chain and
// routing each atom (a comparison, a NOT expression, or a whole multi-branch OR group)
// to the span or having bucket by the class of the keys it references.
type filterSplitter struct {
query []rune
aggregateNames map[string]struct{}
span []string
having []string
mixed bool
}
func (s *filterSplitter) visit(node antlr.Tree) {
switch n := node.(type) {
case *grammar.QueryContext:
if n.Expression() != nil {
s.visit(n.Expression())
}
case *grammar.ExpressionContext:
if n.OrExpression() != nil {
s.visit(n.OrExpression())
}
case *grammar.OrExpressionContext:
// a single branch is just an AND chain; multiple branches are a real OR, kept
// whole so a class-mixing OR can be rejected.
if ands := n.AllAndExpression(); len(ands) == 1 {
s.visit(ands[0])
} else {
s.route(n)
}
case *grammar.AndExpressionContext:
for _, u := range n.AllUnaryExpression() {
s.visit(u)
}
case *grammar.UnaryExpressionContext:
if n.NOT() != nil {
s.route(n)
} else if n.Primary() != nil {
s.visit(n.Primary())
}
case *grammar.PrimaryContext:
if n.OrExpression() != nil { // parenthesized sub-expression
s.visit(n.OrExpression())
} else {
s.route(n)
}
}
}
// route classifies an atom and appends its original source text to the right bucket.
func (s *filterSplitter) route(atom antlr.ParserRuleContext) {
isTrace, isSpan := classifyKeys(atom, s.aggregateNames)
if isTrace && isSpan {
s.mixed = true
return
}
text := atomSourceText(s.query, atom)
// A multi-branch OR group's source slice excludes its enclosing parens (they belong
// to the parent Primary). Re-wrap it so rejoining a bucket with " AND " cannot invert
// OR/AND precedence, e.g. `a AND (b OR c)` must not flatten to `a AND b OR c`.
if or, ok := atom.(*grammar.OrExpressionContext); ok && len(or.AllAndExpression()) > 1 {
text = "(" + text + ")"
}
if isTrace {
s.having = append(s.having, text)
} else {
s.span = append(s.span, text)
}
}
// classifyKeys reports whether a subtree references trace-level and/or span-level keys.
// A key is trace-level when it carries the trace field context or, with no context,
// its name is a known aggregate; an unknown name under the trace context stays
// trace-level so the aggregate validation rejects it with a targeted error. Any other
// explicit context (`span.`, `resource.`, …) is span-level.
func classifyKeys(node antlr.Tree, aggregateNames map[string]struct{}) (isTrace, isSpan bool) {
kc, ok := node.(*grammar.KeyContext)
if ok {
key := telemetrytypes.GetFieldKeyFromKeyText(kc.GetText())
switch key.FieldContext {
case telemetrytypes.FieldContextTrace:
isTrace = true
case telemetrytypes.FieldContextUnspecified:
_, isTrace = aggregateNames[key.Name]
isSpan = !isTrace
default:
isSpan = true
}
return
}
for i := 0; i < node.GetChildCount(); i++ {
t, s := classifyKeys(node.GetChild(i), aggregateNames)
isTrace = isTrace || t
isSpan = isSpan || s
}
return
}
// atomSourceText returns the original source substring for an atom, preserving
// whitespace. The token stream drops skipped whitespace, which would glue word
// operators (OR/AND/NOT) to their operands, so slice the input by token offsets.
// ANTLR offsets are rune indices (InputStream holds []rune), hence the rune slice.
func atomSourceText(query []rune, atom antlr.ParserRuleContext) string {
start, stop := atom.GetStart(), atom.GetStop()
if start == nil || stop == nil || start.GetStart() < 0 || stop.GetStop() >= len(query) || stop.GetStop() < start.GetStart() {
return atom.GetText()
}
return string(query[start.GetStart() : stop.GetStop()+1])
}

View File

@@ -0,0 +1,222 @@
package querybuilder
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestSplitFilterForAggregates(t *testing.T) {
agg := map[string]struct{}{"completion_tokens": {}, "span_count": {}, "prompt_tokens": {}}
type tc struct {
name string
query string
span string // expected span-level (WHERE) part; "" => empty
having string // expected trace-level (HAVING) part; "" => empty
wantErr bool
}
cases := []tc{
// --- empty input ---------------------------------------------------------
{
name: "empty",
},
{
name: "whitespace only",
query: " ",
},
// --- single class --------------------------------------------------------
{
name: "span only",
query: "service.name = 'x'",
span: "service.name = 'x'",
},
{
name: "agg only bare",
query: "completion_tokens > 1000",
having: "completion_tokens > 1000",
},
{
// the user-facing `trace.` prefix marks a trace-level aggregate.
name: "agg only trace prefix",
query: "trace.completion_tokens > 1000",
having: "trace.completion_tokens > 1000",
},
{
// an unknown name under the trace context still routes trace-level, so the
// aggregate validation rejects it with a targeted error instead of the span
// path failing on an unknown field.
name: "unknown aggregate under trace context stays trace-level",
query: "trace.not_an_aggregate > 1000",
having: "trace.not_an_aggregate > 1000",
},
{
// ANTLR token offsets are rune indices; slicing must not shift after a
// multi-byte char (this used to truncate 1000 → 100).
name: "unicode value before the split",
query: "service.name = 'héllo' AND completion_tokens > 1000",
span: "service.name = 'héllo'",
having: "completion_tokens > 1000",
},
// --- top-level AND splits across the two buckets -------------------------
{
name: "span AND agg",
query: "service.name = 'x' AND completion_tokens > 1000",
span: "service.name = 'x'",
having: "completion_tokens > 1000",
},
{
// order within a bucket is preserved; the two span atoms join with AND.
name: "span AND span AND agg",
query: "service.name = 'x' AND kind_string = 'Internal' AND completion_tokens > 1000",
span: "service.name = 'x' AND kind_string = 'Internal'",
having: "completion_tokens > 1000",
},
{
// a parenthesized top-level AND still splits across the two buckets.
name: "parenthesized span AND agg",
query: "(service.name = 'x' AND completion_tokens > 1000)",
span: "service.name = 'x'",
having: "completion_tokens > 1000",
},
// --- OR groups are re-wrapped in parens so a later AND-join can't invert
// precedence (`a AND (b OR c)` must not flatten to `a AND b OR c`) ------
{
name: "agg OR agg",
query: "completion_tokens > 1000 OR span_count > 3",
having: "(completion_tokens > 1000 OR span_count > 3)",
},
{
name: "span OR span",
query: "service.name = 'x' OR kind_string = 'Internal'",
span: "(service.name = 'x' OR kind_string = 'Internal')",
},
{
name: "span AND (span OR span)",
query: "service.name = 'x' AND (kind_string = 'Internal' OR kind_string = 'Client')",
span: "service.name = 'x' AND (kind_string = 'Internal' OR kind_string = 'Client')",
},
{
name: "agg AND (agg OR agg)",
query: "prompt_tokens > 5 AND (completion_tokens > 1000 OR span_count > 3)",
having: "prompt_tokens > 5 AND (completion_tokens > 1000 OR span_count > 3)",
},
{
// the OR group routes to span, the trailing aggregate to having.
name: "span AND (span OR span) AND agg",
query: "a.b = 'x' AND (c.d = 'y' OR e.f = 'z') AND completion_tokens > 1000",
span: "a.b = 'x' AND (c.d = 'y' OR e.f = 'z')",
having: "completion_tokens > 1000",
},
// --- a nested AND group flattens across the buckets (no spurious parens) --
{
name: "(span AND agg) AND agg",
query: "(service.name = 'x' AND completion_tokens > 1000) AND prompt_tokens > 5",
span: "service.name = 'x'",
having: "completion_tokens > 1000 AND prompt_tokens > 5",
},
// --- NOT wrapping a single-class group is routed whole to that class ------
{
name: "not agg",
query: "NOT (completion_tokens > 1000)",
having: "NOT (completion_tokens > 1000)",
},
{
name: "not span",
query: "NOT (service.name = 'x')",
span: "NOT (service.name = 'x')",
},
// --- an explicit non-trace context escapes the aggregate-alias shadow -----
{
// a span attribute named like an aggregate stays reachable via `attribute.`.
name: "attribute prefix on aggregate name routes span-level",
query: "attribute.completion_tokens > 5",
span: "attribute.completion_tokens > 5",
},
{
name: "span prefix on aggregate name routes span-level",
query: "span.completion_tokens > 5",
span: "span.completion_tokens > 5",
},
{
name: "prefixed attribute AND bare aggregate split across buckets",
query: "attribute.completion_tokens > 5 AND completion_tokens > 1000",
span: "attribute.completion_tokens > 5",
having: "completion_tokens > 1000",
},
// --- class-mixing is rejected in an OR group, a NOT group, or a nested OR -
{
name: "agg OR span rejected",
query: "completion_tokens > 1000 OR service.name = 'x'",
wantErr: true,
},
{
name: "not mixed rejected",
query: "NOT (completion_tokens > 1000 AND service.name = 'x')",
wantErr: true,
},
{
name: "span AND (agg OR span) rejected",
query: "service.name = 'x' AND (completion_tokens > 1000 OR kind_string = 'Client')",
wantErr: true,
},
// --- syntax errors are rejected, not silently dropped by error recovery ---
{
// recovery would yield an empty tree → both buckets empty → filter ignored.
name: "lone paren rejected",
query: ")",
wantErr: true,
},
{
name: "unbalanced parens rejected",
query: "((",
wantErr: true,
},
{
name: "bare operator rejected",
query: "AND",
wantErr: true,
},
{
// lexer-level error: recovery drops the whole expression.
name: "unterminated quote rejected",
query: "'unterminated",
wantErr: true,
},
{
// recovery would drop only the malformed atom and keep the rest — a
// partially applied filter with no error.
name: "garbage atom alongside valid agg rejected",
query: ") AND completion_tokens > 5",
wantErr: true,
},
{
name: "missing value rejected",
query: "completion_tokens >",
wantErr: true,
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
span, having, err := SplitFilterForAggregates(c.query, agg)
if c.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
require.Equal(t, c.span, span, "span part")
require.Equal(t, c.having, having, "having part")
})
}
}

View File

@@ -18,6 +18,19 @@ func NewHavingExpressionRewriter() *HavingExpressionRewriter {
}
}
// Rewrite rewrites and validates a HAVING expression against a caller-supplied
// column map (user-facing name -> SQL identifier/expression). Values are inlined, so
// the result is a bare SQL boolean expression with no bound args. Used by callers
// that project their own aggregate columns (e.g. the AI trace list) rather than the
// query's Aggregations.
func (r *HavingExpressionRewriter) Rewrite(expression string, columnMap map[string]string) (string, error) {
if len(strings.TrimSpace(expression)) == 0 {
return "", nil
}
r.columnMap = columnMap
return r.rewriteAndValidate(expression)
}
// RewriteForTraces rewrites and validates the HAVING expression for a traces query.
func (r *HavingExpressionRewriter) RewriteForTraces(expression string, aggregations []qbtypes.TraceAggregation) (string, error) {
if len(strings.TrimSpace(expression)) == 0 {

View File

@@ -1 +0,0 @@
package querybuilder

View File

@@ -82,6 +82,10 @@ func resourcesForQuery(query gjson.Result, variables map[string]qbtypes.Variable
switch queryType {
case qbtypes.QueryTypeBuilder.StringValue(), qbtypes.QueryTypeSubQuery.StringValue():
return resourcesForBuilderQuery(queryType, query.Get("spec"), variables)
case qbtypes.QueryTypeBuilderAI.StringValue():
// An AI builder query is always a traces query; the signal is implied by the
// type (and may be absent from the payload), so pin the resource directly.
return builderQueryResourceRefs(queryType, coretypes.ResourceTelemetryResourceTraces, query.Get("spec"), variables)
case qbtypes.QueryTypePromQL.StringValue():
return []coretypes.ResourceWithID{{Resource: coretypes.ResourceTelemetryResourceMetrics, ID: typeWildcard}}, nil
case qbtypes.QueryTypeClickHouseSQL.StringValue():
@@ -103,7 +107,10 @@ func resourcesForBuilderQuery(queryType string, spec gjson.Result, variables map
if err != nil {
return nil, err
}
return builderQueryResourceRefs(queryType, resource, spec, variables)
}
func builderQueryResourceRefs(queryType string, resource coretypes.Resource, spec gjson.Result, variables map[string]qbtypes.VariableItem) ([]coretypes.ResourceWithID, error) {
ids, err := builderQuerySelectors(queryType, spec.Get("filter.expression").String(), variables)
if err != nil {
return nil, err

View File

@@ -92,6 +92,20 @@ func TestQueryRangeResources(t *testing.T) {
{Resource: coretypes.ResourceTelemetryResourceAuditLogs, ID: "builder_query/signoz.workspace.key.id/a"},
},
},
{
name: "ai builder query maps to traces resource without a signal",
body: `{"compositeQuery":{"queries":[{"type":"builder_ai_query","spec":{"filter":{"expression":"signoz.workspace.key.id = 'checkout'"}}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_ai_query/signoz.workspace.key.id/checkout"},
},
},
{
name: "ai builder query without filter is wildcard",
body: `{"compositeQuery":{"queries":[{"type":"builder_ai_query","spec":{}}]}}`,
expected: []coretypes.ResourceWithID{
{Resource: coretypes.ResourceTelemetryResourceTraces, ID: "builder_ai_query/*"},
},
},
{
name: "promql is wildcard only",
body: `{"compositeQuery":{"queries":[{"type":"promql","spec":{"query":"up"}}]}}`,

View File

@@ -316,17 +316,17 @@ func (e *ClickHouseFilterExtractor) extractColumnStrByExpr(expr clickhouse.Expr)
// FunctionExpr is a function call like "toDate(timestamp)"
case *clickhouse.FunctionExpr:
// For function expressions, return the complete function call string
return ex.String()
return clickhouse.Format(ex)
// ColumnExpr is a column expression like "m.region", "toDate(timestamp)"
case *clickhouse.ColumnExpr:
// ColumnExpr wraps another expression - extract the underlying expression
if ex.Expr != nil {
return e.extractColumnStrByExpr(ex.Expr)
}
return ex.String()
return clickhouse.Format(ex)
default:
// For other expression types, return the string representation
return expr.String()
return clickhouse.Format(expr)
}
}
@@ -524,7 +524,7 @@ func (e *ClickHouseFilterExtractor) extractFullExpression(expr clickhouse.Expr)
if expr == nil {
return ""
}
return expr.String()
return clickhouse.Format(expr)
}
// isSimpleColumnReference checks if an expression is just a simple column reference
@@ -657,7 +657,7 @@ func (e *ClickHouseFilterExtractor) extractCTEName(cte *clickhouse.CTEStmt) stri
case *clickhouse.Ident:
return name.Name
default:
return cte.Expr.String()
return clickhouse.Format(cte.Expr)
}
}

View File

@@ -39,6 +39,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlmigrator"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/tokenizer"
@@ -97,6 +98,9 @@ type Config struct {
// Querier config
Querier querier.Config `mapstructure:"querier"`
// StatementBuilder config
StatementBuilder statementbuilder.Config `mapstructure:"statementbuilder"`
// Ruler config
Ruler ruler.Config `mapstructure:"ruler"`
@@ -166,6 +170,7 @@ func NewConfig(ctx context.Context, logger *slog.Logger, resolverConfig config.R
prometheus.NewConfigFactory(),
alertmanager.NewConfigFactory(),
querier.NewConfigFactory(),
statementbuilder.NewConfigFactory(),
ruler.NewConfigFactory(),
emailing.NewConfigFactory(),
sharder.NewConfigFactory(),

View File

@@ -69,6 +69,8 @@ import (
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/web"
"github.com/SigNoz/signoz/pkg/web/noopweb"
@@ -227,6 +229,7 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddTelemetryTuplesFactory(sqlstore),
sqlmigration.NewAddTagRelationRankFactory(sqlstore, sqlschema),
sqlmigration.NewMigrateDashboardsV1ToV2Factory(sqlstore, sqlschema, dashboardStore, tagModule),
sqlmigration.NewFillDashboardMeterSourceFactory(sqlstore, dashboardStore),
)
}
@@ -285,9 +288,9 @@ func NewStatsReporterProviderFactories(aggregator statsreporter.Aggregator, orgG
)
}
func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, prometheus prometheus.Prometheus, cache cache.Cache, flagger flagger.Flagger) factory.NamedMap[factory.ProviderFactory[querier.Querier, querier.Config]] {
func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, prometheus prometheus.Prometheus, metadataStore telemetrytypes.MetadataStore, traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation], aiTraceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation], logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation], auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation], metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation], meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation], traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder, bucketCache querier.BucketCache, flagger flagger.Flagger) factory.NamedMap[factory.ProviderFactory[querier.Querier, querier.Config]] {
return factory.MustNewNamedMap(
signozquerier.NewFactory(telemetryStore, prometheus, cache, flagger),
signozquerier.NewFactory(telemetryStore, prometheus, metadataStore, traceStmtBuilder, aiTraceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, flagger),
)
}

View File

@@ -48,16 +48,18 @@ import (
"github.com/SigNoz/signoz/pkg/sqlmigrator"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/statementbuilder/aistatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/auditstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/meterstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
pkgtokenizer "github.com/SigNoz/signoz/pkg/tokenizer"
"github.com/SigNoz/signoz/pkg/types/authtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
@@ -96,6 +98,67 @@ type SigNoz struct {
MeterReporter meterreporter.Reporter
}
// newQueryStack assembles the query stack once and returns, in order: the shared
// telemetry metadata store (reused elsewhere in signoz.New), the per-signal
// statement builders (trace, ai-trace, log, audit, metric, meter, trace-operator),
// and the bucket cache. It is the only place that imports the concrete
// statement-builder sub-packages.
func newQueryStack(
ctx context.Context,
settings factory.ProviderSettings,
config Config,
telemetryStore telemetrystore.TelemetryStore,
cache cache.Cache,
fl flagger.Flagger,
) (
telemetrytypes.MetadataStore,
qbtypes.StatementBuilder[qbtypes.TraceAggregation],
qbtypes.StatementBuilder[qbtypes.TraceAggregation],
qbtypes.StatementBuilder[qbtypes.LogAggregation],
qbtypes.StatementBuilder[qbtypes.LogAggregation],
qbtypes.StatementBuilder[qbtypes.MetricAggregation],
qbtypes.StatementBuilder[qbtypes.MetricAggregation],
qbtypes.TraceOperatorStatementBuilder,
querier.BucketCache,
error,
) {
metadataStore := telemetrymetadata.NewTelemetryMetaStore(settings, telemetryStore, fl)
cfg := config.StatementBuilder
traceStmtBuilder, err := tracesstatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
aiTraceStmtBuilder, err := aistatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
traceOperatorStmtBuilder, err := tracesstatementbuilder.NewOperatorFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
logStmtBuilder, err := logsstatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
auditStmtBuilder, err := auditstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
metricStmtBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
meterStmtBuilder, err := meterstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, nil, err
}
bucketCache := querier.NewBucketCache(settings, cache, config.Querier.CacheTTL, config.Querier.FluxInterval)
return metadataStore, traceStmtBuilder, aiTraceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, nil
}
func New(
ctx context.Context,
config Config,
@@ -254,12 +317,19 @@ func New(
return nil, err
}
// Assemble the query stack (metadata store, statement builders, bucket cache) once,
// and reuse the single metadata store everywhere downstream.
telemetryMetadataStore, traceStmtBuilder, aiTraceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, err := newQueryStack(ctx, providerSettings, config, telemetrystore, cache, flagger)
if err != nil {
return nil, err
}
// Initialize querier from the available querier provider factories
querier, err := factory.NewProviderFromNamedMap(
ctx,
providerSettings,
config.Querier,
NewQuerierProviderFactories(telemetrystore, prometheus, cache, flagger),
NewQuerierProviderFactories(telemetrystore, prometheus, telemetryMetadataStore, traceStmtBuilder, aiTraceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, flagger),
config.Querier.Provider(),
)
if err != nil {
@@ -427,35 +497,6 @@ func New(
return nil, err
}
// Initialize telemetry metadata store
// TODO: consolidate other telemetrymetadata.NewTelemetryMetaStore initializations to reuse this instance instead.
telemetryMetadataStore := telemetrymetadata.NewTelemetryMetaStore(
providerSettings,
telemetrystore,
telemetrytraces.DBName,
telemetrytraces.TagAttributesV2TableName,
telemetrytraces.SpanAttributesKeysTblName,
telemetrytraces.SpanIndexV3TableName,
telemetrymetrics.DBName,
telemetrymetrics.AttributesMetadataTableName,
telemetrymeter.DBName,
telemetrymeter.SamplesAgg1dTableName,
telemetrylogs.DBName,
telemetrylogs.LogsV2TableName,
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.AuditLogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
flagger,
)
global, err := factory.NewProviderFromNamedMap(
ctx,
providerSettings,

View File

@@ -0,0 +1,159 @@
package sqlmigration
import (
"context"
"log/slog"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
qb "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
)
// meterMetricNames are the SigNoz meter metrics; a metrics query aggregating one is a
// meter query.
var meterMetricNames = map[string]bool{
"signoz.meter.log.count": true,
"signoz.meter.log.size": true,
"signoz.meter.span.count": true,
"signoz.meter.span.size": true,
"signoz.meter.metric.datapoint.count": true,
"signoz.meter.platform.active": true,
}
type fillDashboardMeterSource struct {
sqlstore sqlstore.SQLStore
dashboardStore dashboardtypes.Store
settings factory.ProviderSettings
}
func NewFillDashboardMeterSourceFactory(sqlstore sqlstore.SQLStore, dashboardStore dashboardtypes.Store) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(
factory.MustNewName("fill_dashboard_meter_source"),
func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &fillDashboardMeterSource{sqlstore: sqlstore, dashboardStore: dashboardStore, settings: ps}, nil
},
)
}
func (migration *fillDashboardMeterSource) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
// Up resets the meter source that 103 dropped when converting v1 meter queries to v2:
// every metrics query aggregating a meter metric is set back to source "meter". One
// transaction; v1 (unparseable-as-v2) dashboards are skipped.
func (migration *fillDashboardMeterSource) Up(ctx context.Context, _ *bun.DB) error {
return migration.sqlstore.RunInTxCtx(ctx, nil, func(ctx context.Context) error {
var orgIDs []string
if err := migration.sqlstore.BunDBCtx(ctx).NewSelect().Model((*types.Organization)(nil)).Column("id").Scan(ctx, &orgIDs); err != nil {
return err
}
for _, id := range orgIDs {
orgID, err := valuer.NewUUID(id)
if err != nil {
return err
}
if err := migration.fillOrg(ctx, orgID); err != nil {
return err
}
}
return nil
})
}
// fillOrg fills the meter source on every v2 dashboard in the org that needs it. Runs
// inside the caller's transaction.
func (migration *fillDashboardMeterSource) fillOrg(ctx context.Context, orgID valuer.UUID) error {
// List, not ListV2: ListV2 paginates and excludes system dashboards; a migration needs every row.
storables, err := migration.dashboardStore.List(ctx, orgID)
if err != nil {
return err
}
logger := migration.settings.Logger
var stillInV1, skippedNoMeter, wouldMigrateButErrored, migrated int
for _, storable := range storables {
// ToDashboardV2 rejects non-v2 data, so v1 dashboards fall through untouched.
dashboard, err := storable.ToDashboardV2(nil)
if err != nil {
stillInV1++
continue
}
if !fillMeterSource(&dashboard.Spec) {
skippedNoMeter++
continue
}
restorable, err := dashboard.ToStorableDashboard()
if err != nil {
wouldMigrateButErrored++
logger.WarnContext(ctx, "failed to re-serialize dashboard after filling meter source", slog.String("org_id", orgID.String()), slog.String("dashboard_id", storable.ID.String()), slog.Any("error", err))
continue
}
if err := migration.dashboardStore.Update(ctx, orgID, restorable); err != nil {
return err
}
migrated++
}
logger.InfoContext(ctx, "filled meter source on v2 dashboards",
slog.String("org_id", orgID.String()),
slog.Int("total", len(storables)),
slog.Int("still_in_v1", stillInV1),
slog.Int("skipped_no_meter", skippedNoMeter),
slog.Int("would_migrate_but_errored", wouldMigrateButErrored),
slog.Int("migrated", migrated),
)
return nil
}
// fillMeterSource sets Source=meter on every metrics builder query in the spec whose
// aggregation targets a meter metric, reporting whether anything changed.
func fillMeterSource(spec *dashboardtypes.DashboardSpec) bool {
changed := false
for _, panel := range spec.Panels {
if panel == nil {
continue
}
for _, query := range panel.Spec.Queries {
switch plugin := query.Spec.Plugin.Spec.(type) {
case *qb.CompositeQuery:
for i := range plugin.Queries {
changed = setMeterSource(&plugin.Queries[i].Spec) || changed
}
case *dashboardtypes.BuilderQuerySpec:
changed = setMeterSource(&plugin.Spec) || changed
}
}
}
return changed
}
// setMeterSource sets source "meter" on a metrics query aggregating a meter metric. The
// type assertion doubles as the metrics-signal check; the already-meter guard keeps it
// idempotent. Written back through the pointer, as an interface value isn't addressable.
func setMeterSource(spec *any) bool {
query, ok := (*spec).(qb.QueryBuilderQuery[qb.MetricAggregation])
if !ok || query.Source == telemetrytypes.SourceMeter {
return false
}
for _, aggregation := range query.Aggregations {
if meterMetricNames[aggregation.MetricName] {
query.Source = telemetrytypes.SourceMeter
*spec = query
return true
}
}
return false
}
func (migration *fillDashboardMeterSource) Down(context.Context, *bun.DB) error {
return nil
}

View File

@@ -0,0 +1,84 @@
package aistatementbuilder
import (
"strings"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/statementbuilder"
scopedtraces "github.com/SigNoz/signoz/pkg/statementbuilder/scopedtracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetrystore"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// NewFactory returns a provider factory for the AI trace statement builder
// (builder_ai_query): the gen_ai Scope paired with the domain-neutral scoped-trace
// topology, which owns the query construction.
//
// The gen_ai gate/column keys are surfaced by the metadata store itself
// (enrichWithGenAIKeys), so queries work before any gen_ai metadata is ingested.
func NewFactory(
telemetryStore telemetrystore.TelemetryStore,
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[qbtypes.StatementBuilder[qbtypes.TraceAggregation], statementbuilder.Config] {
return scopedtraces.NewFactory(factory.MustNewName("ai"), Scope(), telemetryStore, metadataStore, fl)
}
// Scope describes gen_ai for the scoped trace builder: an AI trace has >=1 gen_ai
// LLM, tool, or agent span, and its list adds AI/LLM per-trace metrics on top of the
// common columns. This package holds only gen_ai domain knowledge; the query
// topology lives in scopedtracesstatementbuilder.
func Scope() scopedtraces.TraceScope {
gateKeyNames := []string{telemetrytypes.GenAIRequestModel, telemetrytypes.GenAIToolName, telemetrytypes.GenAIAgentName}
gateExprs := make([]string, 0, len(gateKeyNames))
gateKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(gateKeyNames))
for _, name := range gateKeyNames {
gateExprs = append(gateExprs, name+" EXISTS")
gateKeys = append(gateKeys, &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextAttribute,
})
}
defs := telemetrytypes.GenAIFieldDefinitions
reqModel := defs[telemetrytypes.GenAIRequestModel]
toolName := defs[telemetrytypes.GenAIToolName]
inTok := defs[telemetrytypes.GenAIUsageInputTokens]
outTok := defs[telemetrytypes.GenAIUsageOutputTokens]
cost := defs[telemetrytypes.SignozGenAITotalCost]
inMsg := defs[telemetrytypes.GenAIInputMessages]
outMsg := defs[telemetrytypes.GenAIOutputMessages]
str := telemetrytypes.FieldDataTypeString
columns := append(scopedtraces.CommonTraceColumns(),
// LLM calls only (request model present), not the full gate.
scopedtraces.TraceColumn{Alias: "llm_call_count", Orderable: true, Expr: scopedtraces.CountExists(&reqModel)},
scopedtraces.TraceColumn{Alias: "tool_call_count", Orderable: true, Expr: scopedtraces.CountExists(&toolName)},
scopedtraces.TraceColumn{Alias: "distinct_tool_count", Orderable: true, Expr: scopedtraces.UniqCount(&toolName, str)},
// tokens live only on LLM spans, so a plain sum needs no gate scoping.
scopedtraces.TraceColumn{Alias: "input_tokens", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &inTok)},
scopedtraces.TraceColumn{Alias: "output_tokens", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &outTok)},
scopedtraces.TraceColumn{Alias: "total_tokens", Orderable: true, Expr: scopedtraces.SumOfKeys(telemetrytypes.FieldDataTypeFloat64, &inTok, &outTok)},
// per-span cost attached by the SigNoz LLM pricing processor.
scopedtraces.TraceColumn{Alias: "estimated_total_cost", Orderable: true, Expr: scopedtraces.Reduce(scopedtraces.AggSum, &cost)},
// slowest single LLM call in the trace.
scopedtraces.TraceColumn{Alias: "max_llm_duration_nano", Orderable: true, Expr: scopedtraces.ScopedToKeyColumn(scopedtraces.AggMax, scopedtraces.IntrinsicSpanKey("duration_nano"), &reqModel)},
// errors across the whole trace (any span), so display-only.
scopedtraces.TraceColumn{Alias: "error_count", Expr: scopedtraces.CondCount(scopedtraces.IntrinsicSpanKey("has_error"), qbtypes.FilterOperatorEqual, true)},
// timestamp of the last gen_ai span (LLM/tool/agent), hence gate-scoped.
scopedtraces.TraceColumn{Alias: "last_activity_time", Orderable: true, Expr: scopedtraces.ScopedReduce(scopedtraces.AggMax, scopedtraces.IntrinsicSpanKey("timestamp"))},
// previews: first call's input (the prompt), last call's output (the answer).
scopedtraces.TraceColumn{Alias: "input", SpanLevel: true, Expr: scopedtraces.PickBy(&inMsg, str, scopedtraces.IntrinsicSpanKey("timestamp"), scopedtraces.PickEarliest)},
scopedtraces.TraceColumn{Alias: "output", SpanLevel: true, Expr: scopedtraces.PickBy(&outMsg, str, scopedtraces.IntrinsicSpanKey("timestamp"), scopedtraces.PickLatest)},
)
return scopedtraces.TraceScope{
FilterExpression: strings.Join(gateExprs, " OR "),
FieldKeys: gateKeys,
Columns: columns,
DefaultOrderAlias: "last_activity_time",
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,4 +1,4 @@
package telemetryaudit
package auditstatementbuilder
import (
"context"
@@ -10,7 +10,9 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -29,6 +31,25 @@ type auditQueryStatementBuilder struct {
var _ qbtypes.StatementBuilder[qbtypes.LogAggregation] = (*auditQueryStatementBuilder)(nil)
// NewFactory returns a provider factory for the audit statement builder. Its New
// internalizes the FieldMapper, ConditionBuilder, and AggExprRewriter.
func NewFactory(
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[qbtypes.StatementBuilder[qbtypes.LogAggregation], statementbuilder.Config] {
return factory.NewProviderFactory(
factory.MustNewName("audit"),
func(_ context.Context, settings factory.ProviderSettings, _ statementbuilder.Config) (qbtypes.StatementBuilder[qbtypes.LogAggregation], error) {
fm := audittelemetryschema.NewFieldMapper()
cb := audittelemetryschema.NewConditionBuilder(fm)
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, audittelemetryschema.DefaultFullTextColumn, fm, cb, fl)
return NewAuditQueryStatementBuilder(
settings, metadataStore, fm, cb, aggExprRewriter, audittelemetryschema.DefaultFullTextColumn, fl,
), nil
},
)
}
func NewAuditQueryStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
@@ -38,12 +59,12 @@ func NewAuditQueryStatementBuilder(
fullTextColumn *telemetrytypes.TelemetryFieldKey,
flagger flagger.Flagger,
) *auditQueryStatementBuilder {
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryaudit")
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema")
resourceFilterStmtBuilder := telemetryresourcefilter.New[qbtypes.LogAggregation](
resourceFilterStmtBuilder := resourcefilter.New[qbtypes.LogAggregation](
settings,
DBName,
LogsResourceTableName,
audittelemetryschema.DBName,
audittelemetryschema.LogsResourceTableName,
telemetrytypes.SignalLogs,
telemetrytypes.SourceAudit,
metadataStore,
@@ -191,8 +212,8 @@ func (b *auditQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[st
}
func (b *auditQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
if _, ok := IntrinsicFields[key.Name]; ok {
intrinsicField := IntrinsicFields[key.Name]
if _, ok := audittelemetryschema.IntrinsicFields[key.Name]; ok {
intrinsicField := audittelemetryschema.IntrinsicFields[key.Name]
return querybuilder.AdjustKey(key, keys, &intrinsicField)
}
return querybuilder.AdjustKey(key, keys, nil)
@@ -219,26 +240,26 @@ func (b *auditQueryStatementBuilder) buildListQuery(
cteArgs = append(cteArgs, args)
}
sb.Select(TimestampColumn)
sb.SelectMore(IDColumn)
sb.Select(audittelemetryschema.TimestampColumn)
sb.SelectMore(audittelemetryschema.IDColumn)
if len(query.SelectFields) == 0 {
sb.SelectMore(TraceIDColumn)
sb.SelectMore(SpanIDColumn)
sb.SelectMore(TraceFlagsColumn)
sb.SelectMore(SeverityTextColumn)
sb.SelectMore(SeverityNumberColumn)
sb.SelectMore(ScopeNameColumn)
sb.SelectMore(ScopeVersionColumn)
sb.SelectMore(BodyColumn)
sb.SelectMore(EventNameColumn)
sb.SelectMore(AttributesStringColumn)
sb.SelectMore(AttributesNumberColumn)
sb.SelectMore(AttributesBoolColumn)
sb.SelectMore(ResourceColumn)
sb.SelectMore(ScopeStringColumn)
sb.SelectMore(audittelemetryschema.TraceIDColumn)
sb.SelectMore(audittelemetryschema.SpanIDColumn)
sb.SelectMore(audittelemetryschema.TraceFlagsColumn)
sb.SelectMore(audittelemetryschema.SeverityTextColumn)
sb.SelectMore(audittelemetryschema.SeverityNumberColumn)
sb.SelectMore(audittelemetryschema.ScopeNameColumn)
sb.SelectMore(audittelemetryschema.ScopeVersionColumn)
sb.SelectMore(audittelemetryschema.BodyColumn)
sb.SelectMore(audittelemetryschema.EventNameColumn)
sb.SelectMore(audittelemetryschema.AttributesStringColumn)
sb.SelectMore(audittelemetryschema.AttributesNumberColumn)
sb.SelectMore(audittelemetryschema.AttributesBoolColumn)
sb.SelectMore(audittelemetryschema.ResourceColumn)
sb.SelectMore(audittelemetryschema.ScopeStringColumn)
} else {
for index := range query.SelectFields {
if query.SelectFields[index].Name == TimestampColumn || query.SelectFields[index].Name == IDColumn {
if query.SelectFields[index].Name == audittelemetryschema.TimestampColumn || query.SelectFields[index].Name == audittelemetryschema.IDColumn {
continue
}
@@ -250,7 +271,7 @@ func (b *auditQueryStatementBuilder) buildListQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
@@ -340,7 +361,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
sb.SelectMore(fmt.Sprintf("%s AS __result_%d", rewritten, i))
}
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
@@ -478,7 +499,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {

View File

@@ -1,4 +1,4 @@
package telemetryaudit
package auditstatementbuilder
import (
"context"
@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
@@ -54,8 +55,8 @@ func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = auditFieldKeyMap()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := audittelemetryschema.NewFieldMapper()
cb := audittelemetryschema.NewConditionBuilder(fm)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
return NewAuditQueryStatementBuilder(
@@ -64,7 +65,7 @@ func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
audittelemetryschema.DefaultFullTextColumn,
fl,
)
}

View File

@@ -0,0 +1,41 @@
package statementbuilder
import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
)
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
type SkipResourceFingerprint struct {
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
// If count of fingerprint is above threshold, skip the fingerprint subquery and filter on main table instead.
Threshold uint64 `yaml:"threshold" mapstructure:"threshold"`
}
// Config represents the configuration for the statement builders.
type Config struct {
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
SkipResourceFingerprint SkipResourceFingerprint `yaml:"skip_resource_fingerprint" mapstructure:"skip_resource_fingerprint"`
}
// NewConfigFactory creates a new config factory for the statement builders.
func NewConfigFactory() factory.ConfigFactory {
return factory.NewConfigFactory(factory.MustNewName("statementbuilder"), newConfig)
}
func newConfig() factory.Config {
return Config{
SkipResourceFingerprint: SkipResourceFingerprint{
Enabled: false,
Threshold: 100000,
},
}
}
// Validate validates the configuration.
func (c Config) Validate() error {
if c.SkipResourceFingerprint.Enabled && c.SkipResourceFingerprint.Threshold == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "skip_resource_fingerprint.threshold must be > 0 when enabled")
}
return nil
}

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
@@ -43,14 +44,14 @@ func TestStatementBuilderGroupByUnknownKey(t *testing.T) {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.WithUseJSONBody(t, c.useJSONBody)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore, fm, cb, aggExprRewriter,
DefaultFullTextColumn, fl, nil, false, 100000,
logstelemetryschema.DefaultFullTextColumn, fl, nil, false, 100000,
)
query := qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
@@ -1225,7 +1226,7 @@ func TestJSONStmtBuilder_OrderBy(t *testing.T) {
func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
statementBuilder, metadataStore := buildJSONTestStatementBuilder(t, false)
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
keysMap := buildCompleteFieldKeyMap(releaseTime)
keysMap := logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
for _, keys := range keysMap {
for _, key := range keys {
metadataStore.SetKey(key)
@@ -1287,7 +1288,7 @@ func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetrytypestest.MockMetadataStore {
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.SetStaticFields(IntrinsicFields)
mockMetadataStore.SetStaticFields(logstelemetryschema.IntrinsicFields)
types, _ := telemetrytypes.TestJSONTypeSet()
for path, fieldDataTypes := range types {
for _, fdt := range fieldDataTypes {
@@ -1307,15 +1308,15 @@ func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetryty
Name: path,
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: fdt,
BaseColumn: LogsV2BodyV2Column,
IndexExpression: schemamigrator.JSONSubColumnIndexExpr(LogsV2BodyV2Column, path, jsonType.StringValue()),
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
IndexExpression: schemamigrator.JSONSubColumnIndexExpr(logstelemetryschema.LogsV2BodyV2Column, path, jsonType.StringValue()),
})
}
}
err := key.SetJSONAccessPlan(telemetrytypes.JSONColumnMetadata{
BaseColumn: LogsV2BodyV2Column,
PromotedColumn: LogsV2BodyPromotedColumn,
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
PromotedColumn: logstelemetryschema.LogsV2BodyPromotedColumn,
}, types)
require.NoError(t, err)
@@ -1331,8 +1332,8 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) (*logQueryStat
mockMetadataStore := buildTestTelemetryMetadataStore(t, addIndexes)
fl := flaggertest.WithUseJSONBody(t, true)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -1342,7 +1343,7 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) (*logQueryStat
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -10,7 +10,9 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -19,12 +21,22 @@ import (
"github.com/huandu/go-sqlbuilder"
)
const bodySearchDefaultWarning = "body searches default to `body.message:string`. Use `body.<key>` to search a different field inside body"
func bodyAliasExpression(bodyJSONEnabled bool) string {
if !bodyJSONEnabled {
return logstelemetryschema.LogsV2BodyColumn
}
return fmt.Sprintf("%s as body", logstelemetryschema.LogsV2BodyV2Column)
}
type logQueryStatementBuilder struct {
logger *slog.Logger
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
resourceFilterResolver *telemetryresourcefilter.ResourceFingerprintResolver[qbtypes.LogAggregation]
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.LogAggregation]
aggExprRewriter qbtypes.AggExprRewriter
fl flagger.Flagger
skipResourceFingerprintEnabled bool
@@ -34,6 +46,28 @@ type logQueryStatementBuilder struct {
var _ qbtypes.StatementBuilder[qbtypes.LogAggregation] = (*logQueryStatementBuilder)(nil)
// NewFactory returns a provider factory for the logs statement builder. Its New
// internalizes the FieldMapper, ConditionBuilder, and AggExprRewriter, and reads
// SkipResourceFingerprint from the config.
func NewFactory(
telemetryStore telemetrystore.TelemetryStore,
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[qbtypes.StatementBuilder[qbtypes.LogAggregation], statementbuilder.Config] {
return factory.NewProviderFactory(
factory.MustNewName("logs"),
func(_ context.Context, settings factory.ProviderSettings, cfg statementbuilder.Config) (qbtypes.StatementBuilder[qbtypes.LogAggregation], error) {
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, logstelemetryschema.DefaultFullTextColumn, fm, cb, fl)
return NewLogQueryStatementBuilder(
settings, metadataStore, fm, cb, aggExprRewriter, logstelemetryschema.DefaultFullTextColumn,
fl, telemetryStore, cfg.SkipResourceFingerprint.Enabled, cfg.SkipResourceFingerprint.Threshold,
), nil
},
)
}
func NewLogQueryStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
@@ -46,12 +80,12 @@ func NewLogQueryStatementBuilder(
skipResourceFingerprintEnable bool,
skipResourceFingerprintThreshold uint64,
) *logQueryStatementBuilder {
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrylogs")
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema")
resourceFilterResolver := telemetryresourcefilter.NewResolver[qbtypes.LogAggregation](
resourceFilterResolver := resourcefilter.NewResolver[qbtypes.LogAggregation](
settings,
DBName,
LogsResourceV2TableName,
logstelemetryschema.DBName,
logstelemetryschema.LogsResourceV2TableName,
telemetrytypes.SignalLogs,
telemetrytypes.SourceUnspecified,
metadataStore,
@@ -174,7 +208,7 @@ func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation], bo
// TODO(Piyush): Setup better for coming FTS support.
if bodyJSONEnabled {
for _, sel := range keySelectors {
if sel.Name == LogsV2BodyColumn {
if sel.Name == logstelemetryschema.LogsV2BodyColumn {
warnings = append(warnings, bodySearchDefaultWarning)
break
}
@@ -250,8 +284,8 @@ func (b *logQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[stri
func (b *logQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
// First check if it matches with any intrinsic fields
var intrinsicOrCalculatedField telemetrytypes.TelemetryFieldKey
if _, ok := IntrinsicFields[key.Name]; ok {
intrinsicOrCalculatedField = IntrinsicFields[key.Name]
if _, ok := logstelemetryschema.IntrinsicFields[key.Name]; ok {
intrinsicOrCalculatedField = logstelemetryschema.IntrinsicFields[key.Name]
return querybuilder.AdjustKey(key, keys, &intrinsicOrCalculatedField)
}
@@ -284,28 +318,28 @@ func (b *logQueryStatementBuilder) buildListQuery(
}
// Select timestamp and id by default
sb.Select(LogsV2TimestampColumn)
sb.SelectMore(LogsV2IDColumn)
sb.Select(logstelemetryschema.LogsV2TimestampColumn)
sb.SelectMore(logstelemetryschema.LogsV2IDColumn)
if len(query.SelectFields) == 0 {
// Select all default columns
sb.SelectMore(LogsV2TraceIDColumn)
sb.SelectMore(LogsV2SpanIDColumn)
sb.SelectMore(LogsV2TraceFlagsColumn)
sb.SelectMore(LogsV2SeverityTextColumn)
sb.SelectMore(LogsV2SeverityNumberColumn)
sb.SelectMore(LogsV2ScopeNameColumn)
sb.SelectMore(LogsV2ScopeVersionColumn)
sb.SelectMore(logstelemetryschema.LogsV2TraceIDColumn)
sb.SelectMore(logstelemetryschema.LogsV2SpanIDColumn)
sb.SelectMore(logstelemetryschema.LogsV2TraceFlagsColumn)
sb.SelectMore(logstelemetryschema.LogsV2SeverityTextColumn)
sb.SelectMore(logstelemetryschema.LogsV2SeverityNumberColumn)
sb.SelectMore(logstelemetryschema.LogsV2ScopeNameColumn)
sb.SelectMore(logstelemetryschema.LogsV2ScopeVersionColumn)
sb.SelectMore(bodyAliasExpression(b.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))))
sb.SelectMore(LogsV2AttributesStringColumn)
sb.SelectMore(LogsV2AttributesNumberColumn)
sb.SelectMore(LogsV2AttributesBoolColumn)
sb.SelectMore(LogsV2ResourcesStringColumn)
sb.SelectMore(LogsV2ScopeStringColumn)
sb.SelectMore(logstelemetryschema.LogsV2AttributesStringColumn)
sb.SelectMore(logstelemetryschema.LogsV2AttributesNumberColumn)
sb.SelectMore(logstelemetryschema.LogsV2AttributesBoolColumn)
sb.SelectMore(logstelemetryschema.LogsV2ResourcesStringColumn)
sb.SelectMore(logstelemetryschema.LogsV2ScopeStringColumn)
} else {
// Select specified columns
for index := range query.SelectFields {
if query.SelectFields[index].Name == LogsV2TimestampColumn || query.SelectFields[index].Name == LogsV2IDColumn {
if query.SelectFields[index].Name == logstelemetryschema.LogsV2TimestampColumn || query.SelectFields[index].Name == logstelemetryschema.LogsV2IDColumn {
continue
}
@@ -318,7 +352,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
@@ -424,7 +458,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
}
// Add FROM clause
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
@@ -592,7 +626,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)

Some files were not shown because too many files have changed in this diff Show More