Compare commits

...

41 Commits

Author SHA1 Message Date
Vinícius Lourenço
83f18ccedc refactor(alerts): clean and re-organize the tests 2026-07-30 10:21:20 -03:00
Vinícius Lourenço
9293a3cf8a feat(alerts): add initial e2e 2026-07-29 17:33:27 -03:00
Vinícius Lourenço
db3e2c59a5 chore(alert): add test ids 2026-07-29 17:28:40 -03:00
Vinícius Lourenço
0f488261df refactor(graph): improve a little bit the state colors for recovering/pending 2026-07-29 12:39:40 -03:00
Vinícius Lourenço
f9a31e05b0 fix(store): be extract safe in case fieldKeys is nil 2026-07-29 12:38:23 -03:00
Vinícius Lourenço
ee5713c9c8 fix(alert-history-query-search): a simple fix for the issue of the visual artifacts 2026-07-29 12:36:53 -03:00
Vinícius Lourenço
7c72e6123e Revert "fix(query-search): backdrop-filter leaking visual artifacts for some reason"
This reverts commit c8b28c81ef.
2026-07-29 12:13:15 -03:00
Vinícius Lourenço
3fedb4f6a1 fix(alert-history-filter-suggestions): broken types after openapi update 2026-07-29 11:44:28 -03:00
Vinícius Lourenço
728fda9964 Merge branch 'main' into feat/alert-history-api-v2 2026-07-29 11:38:06 -03:00
Vinícius Lourenço
c8b28c81ef fix(query-search): backdrop-filter leaking visual artifacts for some reason
I was only able to reproduce this on alert-history,
not sure why other places don't have the same issue
2026-07-29 11:30:58 -03:00
Vinícius Lourenço
9f16b6c37b fix(alert-history-suggestions): render empty keys when api call fails
And also shows good loading feedback
2026-07-29 11:04:20 -03:00
Ashwin Bhatkal
e51c464417 fix(frontend): alias lodash to lodash-es to survive a page-level AMD loader (#12332)
@grafana/data's ESM build imports bare CJS `lodash` in ~28 modules.
lodash's UMD footer checks for an AMD loader before assigning
module.exports, so when a third-party script has already defined
window.define.amd, lodash takes that branch and never exports. rolldown
can't statically detect lodash's dynamic named exports, so those imports
compile to runtime namespace reads against an empty object and every
method is undefined.

lodash-es is the same version, real ESM, and tree-shakes.
2026-07-29 14:03:32 +00:00
Vinícius Lourenço
396790f2ba fix(table): remove type hack with new support for links from backend 2026-07-29 10:06:06 -03:00
Tushar Vats
04637bd960 test(querier): assert rendered SQL in skip_resource_fingerprint tests (#12334)
Identical rows can't distinguish the optimized instance from the
fingerprint baseline, so the tests now also diff the rendered ClickHouse
statement via /query_range/preview: the baseline must join the fingerprint
CTE and the optimized instance must not.

The assertion is pinned to the "resource_fingerprint GLOBAL IN (SELECT
fingerprint FROM __resource_filter)" clause, a literal in the statement
builder. The resource condition the fallback path renders on the main
table is deliberately not asserted: its column form follows the resource
JSON evolution boundary, rendering as a multiIf over resources_string
while the query window straddles release_time and collapsing to
resource.`key`::String once the window sits past it.

Also fixes the fixture env prefix. skip_resource_fingerprint moved out of
the querier config namespace into its own, so
SIGNOZ_QUERIER_SKIP__RESOURCE__FINGERPRINT_* was being ignored and both
instances ran with the optimization disabled.
2026-07-29 12:51:03 +00:00
Swapnil Nakade
0703fbf961 feat: adding compute engine service in GCP integration (#12166)
* feat: adding gcp memorystore redis service

* refactor: updating dashboard title

* refactor: extending width of uptime gauge panel

* refactor: updating cpu utilization panel

* refactor: updating dashboard panel to use rate function instead of hack

* feat: adding compute engine service

* refactor: updating dashboard panels to use rate aggregation

* fix: correct typo and unit in compute engine dashboard

* refactor: migrating dashboard to v6
2026-07-29 11:28:58 +00:00
Tushar Vats
4e45620b72 refactor(statementbuilder): drop Builders bundle; fold factories into statement_builder.go (#12330)
Remove the statementbuilder.Builders aggregate. The querier chain (querier.New,
signozquerier.NewFactory, NewQuerierProviderFactories) now takes the six per-signal
statement builders individually, and signoz.go::newQueryStack returns them directly
via a plain multi-return. The statementbuilder package is now config-only.

Remove each <signal>statementbuilder/new.go and fold its NewFactory into that
package's statement_builder.go (traces' NewOperatorFactory into
trace_operator_statement_builder.go), giving the layout: struct -> NewFactory ->
New<X>QueryStatementBuilder.
2026-07-29 10:46:55 +00:00
Naman Verma
a9506f1354 fix: add meter source where needed during dashboard migration (#12322)
* fix: add meter source during migration

* chore: move source field handling to wrapinv5envelope

* fix: add migration to backfill source for migrated dashboards

* chore: shorten comments
2026-07-29 10:39:41 +00:00
Tushar Vats
e2e7caf1ca refactor(querier): 3-layer per-signal query architecture (#12304)
Split the five telemetry<signal> packages, which mixed three concerns, into
three per-signal layers with a cycle-free dependency direction:

- telemetryschema/<signal>telemetryschema — primitives (const + table
  selection, field_mapper, condition_builder, trace helpers); leaf layer.
- statementbuilder/<signal>statementbuilder — SQL generation. The parent
  statementbuilder package is contract-only (the Builders bundle + Config);
  each sub-package exposes a factory.ProviderFactory[..., statementbuilder.Config]
  whose New internalizes FieldMapper/ConditionBuilder/AggExprRewriter and reads
  SkipResourceFingerprint. Traces exposes two factories (query + operator).
  telemetryresourcefilter moves here as statementbuilder/resourcefilter.
- telemetrymetadata — key/value resolution; NewTelemetryMetaStore collapses
  from 24 args to (settings, telemetrystore, flagger), sourcing table names
  from the schema constants.

Centralize query-stack assembly in signoz.go via newQueryStack: build the
single metadata store, run each per-signal statement-builder factory, assemble
the statementbuilder.Builders bundle, and build the bucket cache — once. This
is the only place that imports the concrete sub-packages (so the edge runs
subs -> parent, cycle-free), and it removes the duplicate metadata store that
signozquerier used to build, leaving signozquerier a thin
querier.New(*statementbuilder.Builders) adapter.

Also:
- statementbuilder.Config owns SkipResourceFingerprint (moved off
  querier.Config). YAML key moves querier.skip_resource_fingerprint ->
  statementbuilder.skip_resource_fingerprint.
- Querier interface moves into querier.go (interfaces.go removed); BucketCache
  -> bucket_cache.go, Handler -> api.go.
- Add pkg/querier/queriertest.MockQuerier.
2026-07-29 09:24:08 +00:00
Srikanth Chekuri
bca2370862 test(promql): add upstream promqltest conformance corpus and suite (#12156)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
Freezes an absolute-truth oracle from prometheus@v0.311.3's own
promql/promqltest testdata: a generator command (scripts/promqltestcorpus, go run .)
evaluates upstream's load scripts with the vendored reference engine and
writes 755 cases to a committed corpus; a new integration suite replays
the ingestion and asserts /api/v5/query_range responses against it. The
known-divergences ledger is enforced exactly in both directions and is
empty: the serving path matches the reference engine on every case.

Also folds in the last serving-path fix the corpus surfaced: the v5
output filter dropped every __-prefixed label, mangling labelsets that
legitimately carry one (e.g. __address__); it now strips only known
storage keys (__temporality__, __scope./__resource. prefixes).

The metrics fixture writes registration rows per (series, hour bucket)
with hour-floored timestamps, matching the exporter; the queriermetrics
dormant-metric warning test now uses genuinely dormant data (production-
shaped registration shares an hour bucket with recently-stale data).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 03:54:22 +00:00
Gaurav Tewari
591837152e fix(warning-popover): background color to l2 & message alignment (#12302)
Some checks failed
build-staging / prepare (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
* fix: update css styles

* feat: add warning popover

* chore: revert bg color change

* feat: update styles

* chore: css conventios

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-07-28 17:16:05 +00:00
Pandey
2c5b4184c4 feat(querier): log user-authored clickhouse sql after running them through the parser (#12301)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* fix(querier): restrict user-authored clickhouse sql to read-only selects

Validate every user-authored ClickHouse statement before it runs: exactly one
statement, SELECT only, no table functions and no readonly override. Validation
runs on the rendered statement, since substituted variable values are user input
too.

Apply it to both entry points that reach the telemetry store with user SQL, the
clickhouse_sql query type in query_range and the dashboard variables query, and
replace the substring blacklist in the latter, which ran before substitution.

Also run these statements with readonly = 2 as a backstop. The connection is
shared with write paths, so it is opt-in per query and writers never set it.

* fix(querier): address review on clickhouse sql validation

Rename the validator to ValidateReadOnlySelect and collapse the multi-line error
constructions onto single lines.

Carry the parse failure in the message rather than as a wrapped cause. Neither
renderer surfaces the cause: render.Error reads only the message off the base
error, and RespondError reads Error(), which returns the cause alone and drops
the message. Both now show the same text with a 400.

Add unit tests covering statement kinds, table functions nested in joins, CTEs,
subqueries and unions, and readonly overrides.

* fix(querier): reject internal databases in user-authored clickhouse sql

Reading system or information_schema exposes grants, users and server metadata,
and ClickHouse read-only mode does not prevent it, so reject any table reference
into them.

Update the dashboard variables test for the new rejection message and cover both
a statement smuggled through a variable value and a system table read.

* fix(querier): reject unterminated block comments before parsing

The parser loops forever on an unterminated block comment, so validating a query
containing one would spin a request goroutine at full CPU instead of rejecting
it. Detect it up front, skipping comment markers that sit inside string literals.

Cover the parser panicking on a settings clause with no value in its own test,
which asserts the input still panics the parser so the case cannot quietly stop
exercising the recover.

* fix(querier): drop the block comment guard now the parser handles it

The parser no longer loops on an unterminated block comment, so the hand-rolled
scan that worked around it is redundant, and it was the riskier of the two: it
duplicated the lexer's handling of string literals and could have rejected a
legitimate query. Leave the parser as the single source of truth.

The panic case likewise no longer panics, so its test can no longer cover the
recover and is removed. The recover stays as insurance, since this reaches
user-authored SQL and the parser has regressed this way before.

Cover INTERSECT and EXCEPT, which the parser only started accepting in this
version and which reach a second query through the same rules.

* fix(querier): log clickhouse sql validation failures instead of rejecting

The parser's grammar has gaps against SQL that ClickHouse accepts, so rejecting
whatever it cannot read would break working dashboards and alerts. Sampling a
week of production clickhouse_sql found roughly one query in eighteen tripping
one of three gaps, none of them reading anything a telemetry query should not.

Log the failure with the rendered query instead, so the gaps can be told apart
from statements that genuinely break the rules before anything is enforced.

Wrap the parse error rather than formatting it in, so a caller can recover the
parser's *ParseError and read the position off it. The tests use that to pin the
construct each gap stops at, alongside the rewrite that the parser does accept.

* chore(querier): tighten the clickhouse sql validation comments

Condense the comments to single lines, move the recover note inside the defer it
explains, separate the visitor cases, and shorten the log message. Report how
many statements were found when rejecting a multi-statement query.

* fix(querier): log invalid clickhouse sql on the dashboard variables path too

The two callers disagreed: query_range logged and carried on, while dashboard
variables rejected. Given the parser trips on roughly one real query in eighteen,
that path could refuse a legitimate variable query, and being a rejection it left
nothing behind to show it had happened.

Both now go through LogIfStatementIsNotValid. prepareQuery is back to rendering
and nothing else, so the cases that expected it to police the statement move to
where the rules actually live.

* fix(querier): drop the read-only clickhouse session

Setting readonly on the session was defence in depth for a validator that now
only logs, so it guarded nothing while adding a context key and a settings branch
to a connection that is shared with every write path.

Restore the dashboard variables handler to what it was and call the validation
alongside, rather than reworking the rendering to accommodate it.

* chore(querybuilder): drop the redundant suffix from the failing case names

The test they sit in is already named _Fail.
2026-07-28 14:54:42 +00:00
Gaurav Tewari
0b69f5f677 fix: smooth Only/Toggle hover reveal in quick-filters checkbox (#12248)
The Only and Toggle buttons were revealed by overlapping hover triggers
(`.value` for Toggle, `.checkbox-value-section` for Only), so hovering the
label matched both rules and showed the two buttons side by side, shifting
the row layout — the visible hover "jerk".

- Scope the Toggle trigger to the checkbox (`.check-box:hover ~ ...`) so it
  no longer overlaps the label region; Only and Toggle are now mutually
  exclusive by cursor position.
- Stack both buttons in a single grid cell (`.value-actions`) so revealing
  one swaps in place instead of shifting the row.
- Add a fade + slide entry/exit animation (opacity + translateX with
  `@starting-style` and `display` `allow-discrete`), disabled under
  prefers-reduced-motion.

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-07-28 12:21:39 +00:00
Srikanth Chekuri
d6ef1c5f63 Merge branch 'main' into feat/alert-history-api-v2 2026-07-27 12:20:22 +05:30
Srikanth Chekuri
81b6188455 Merge branch 'main' into feat/alert-history-api-v2 2026-07-27 00:21:04 +05:30
Vinicius Lourenço
1eb322a7a7 Merge branch 'main' into feat/alert-history-api-v2 2026-07-21 11:08:45 -03:00
Vinícius Lourenço
518fdbfcc1 feat(links): add back the links 2026-07-20 19:06:15 -03:00
Vinícius Lourenço
af5944cf1e fix(filter-suggestions): remove limit 2026-07-20 18:49:56 -03:00
Vinícius Lourenço
2ca03140e4 fix(query-search): clean the code a little bit 2026-07-20 18:23:18 -03:00
Vinícius Lourenço
405dbad2ec fix(query-search): pass complete prop 2026-07-20 18:13:01 -03:00
Vinícius Lourenço
8c5b110bd4 fix(pr): reset page on change time 2026-07-20 18:05:26 -03:00
Vinícius Lourenço
da699858de fix(pr): fallback to known existing keys on error for keys 2026-07-20 18:05:09 -03:00
Vinícius Lourenço
06cecef628 fix(pr): address comments 2026-07-17 10:17:00 -03:00
Vinícius Lourenço
01de1fa17a fix(alert-history-table): add more safe checks 2026-07-16 17:00:55 -03:00
Vinícius Lourenço
77455c7e04 refactor(alert-history): removed unused code 2026-07-16 16:57:51 -03:00
Vinícius Lourenço
4490bde90d fix(hooks): reset page after filter change 2026-07-16 16:56:35 -03:00
Vinícius Lourenço
a954d4e548 fix(timeline-table): keep as asc
No idea why it was preferred this way
2026-07-16 14:19:12 -03:00
Vinícius Lourenço
e352012e53 fix(cursor-pagination): simplify pagination 2026-07-16 14:19:12 -03:00
Vinícius Lourenço
85d18c980f fix(rule-history): not accepting severity/threshold.name as filter 2026-07-16 14:19:12 -03:00
Vinícius Lourenço
a23aa7eaad refactor(api): remove types and old apis 2026-07-16 14:19:11 -03:00
Vinícius Lourenço
45d52d1730 feat(alert-history): use v2 apis 2026-07-16 14:19:11 -03:00
Vinícius Lourenço
501cb1588e feat(query-search): add support for custom value fetcher 2026-07-16 14:19:11 -03:00
198 changed files with 140542 additions and 1768 deletions

View File

@@ -54,6 +54,7 @@ jobs:
- querierscalar
- queriercommon
- rawexportdata
- promqlconformance
- querierauthz
- role
- rootuser

View File

@@ -2,9 +2,9 @@ package main
import (
"github.com/SigNoz/signoz/pkg/metercollector"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
"github.com/SigNoz/signoz/pkg/types/zeustypes"
)
@@ -25,8 +25,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterLogSize,
Unit: zeustypes.MeterUnitBytes,
Aggregation: zeustypes.MeterAggregationSum,
DBName: telemetrylogs.DBName,
TableName: telemetrylogs.LogsV2LocalTableName,
DBName: logstelemetryschema.DBName,
TableName: logstelemetryschema.LogsV2LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultLogsRetentionDays,
},
},
@@ -36,8 +36,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterSpanSize,
Unit: zeustypes.MeterUnitBytes,
Aggregation: zeustypes.MeterAggregationSum,
DBName: telemetrytraces.DBName,
TableName: telemetrytraces.SpanIndexV3LocalTableName,
DBName: tracestelemetryschema.DBName,
TableName: tracestelemetryschema.SpanIndexV3LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultTracesRetentionDays,
},
},
@@ -47,8 +47,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterDatapointCount,
Unit: zeustypes.MeterUnitCount,
Aggregation: zeustypes.MeterAggregationSum,
DBName: telemetrymetrics.DBName,
TableName: telemetrymetrics.SamplesV4LocalTableName,
DBName: metricstelemetryschema.DBName,
TableName: metricstelemetryschema.SamplesV4LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultMetricsRetentionDays,
},
},

View File

@@ -1496,6 +1496,7 @@ components:
- redis
- cloudsql_postgres
- memorystore_redis
- computeengine
type: string
CloudintegrationtypesServiceMetadata:
properties:

View File

@@ -16,7 +16,7 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/metercollector"
"github.com/SigNoz/signoz/pkg/modules/retention"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/licensetypes"
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
@@ -153,7 +153,7 @@ func (provider *Provider) Collect(
func buildOriginQuery(meterName string) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("toInt64(ifNull(min(unix_milli), 0))")
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
sb.Where(sb.Equal("metric_name", meterName))
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
@@ -171,7 +171,7 @@ func buildQuery(meterName string, segment *retentiontypes.RetentionPolicySegment
sb := sqlbuilder.NewSelectBuilder()
sb.Select(selects...)
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
sb.Where(
sb.Equal("metric_name", meterName),
sb.GTE("unix_milli", segment.StartMs),

View File

@@ -8,16 +8,16 @@ import (
sqlbuilder "github.com/huandu/go-sqlbuilder"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/metricreductionruletypes"
)
var (
reductionRulesTable = telemetrymetrics.DBName + "." + telemetrymetrics.ReductionRulesTableName
metadataTable = telemetrymetrics.DBName + "." + telemetrymetrics.AttributesMetadataTableName
bufferSeriesTable = telemetrymetrics.DBName + "." + telemetrymetrics.TimeseriesV4BufferTableName
reductionRulesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.ReductionRulesTableName
metadataTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.AttributesMetadataTableName
bufferSeriesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.TimeseriesV4BufferTableName
)
const timeSeriesBucketMilli = int64(time.Hour / time.Millisecond)
@@ -192,7 +192,7 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", "uniq(fingerprint)")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
conds := []string{
sb.In("metric_name", names...),
sb.GE("unix_milli", startMs),
@@ -229,8 +229,8 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
// reduced sample tables.
func (c *clickhouse) reducedSeriesCount(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[string]uint64, error) {
out := make(map[string]uint64, len(metricNames))
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
counts, err := c.reducedSeriesCountForTable(ctx, telemetrymetrics.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
counts, err := c.reducedSeriesCountForTable(ctx, metricstelemetryschema.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
@@ -300,9 +300,9 @@ func (c *clickhouse) RankByVolume(ctx context.Context, metricNames []string, eff
direction = "DESC"
}
ingestedTable := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName
reducedLast := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedLastTableName
reducedSum := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedSumTableName
ingestedTable := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName
reducedLast := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedLastTableName
reducedSum := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedSumTableName
sb := sqlbuilder.NewSelectBuilder()
sb.Select("base.metric_name AS metric_name", "ifNull(i.cnt, 0) AS ingested", "ifNull(d.cnt, 0) AS reduced")
@@ -352,16 +352,16 @@ func (c *clickhouse) SampleVolumeByMetric(ctx context.Context, metricNames []str
}
ctx = c.withThreads(ctx)
ingested, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
ingested, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
last, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
last, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
sum, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
sum, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
@@ -419,7 +419,7 @@ func (c *clickhouse) TotalVolume(ctx context.Context, startMs, endMs int64) (uin
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint)", "count()")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -470,7 +470,7 @@ func (c *clickhouse) SampleTimeseries(ctx context.Context, ruledMetrics []string
func (c *clickhouse) totalSamplesByBucket(ctx context.Context, startMs, endMs int64) (map[int64]uint64, error) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "count()")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
sb.GroupBy("bucket")
@@ -485,7 +485,7 @@ func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNam
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "count()")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
@@ -499,7 +499,7 @@ func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNam
// reduced 60s rows are versioned by computed_at, so count distinct buckets.
func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
out := make(map[int64]uint64)
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
@@ -507,7 +507,7 @@ func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNam
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "uniq(reduced_fingerprint, unix_milli)")
sb.From(telemetrymetrics.DBName + "." + table)
sb.From(metricstelemetryschema.DBName + "." + table)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))

View File

@@ -1,28 +0,0 @@
import axios from 'api';
import { ErrorResponseHandler } from 'api/ErrorResponseHandler';
import { AxiosError } from 'axios';
import { ErrorResponse, SuccessResponse } from 'types/api';
import { AlertRuleStatsPayload } from 'types/api/alerts/def';
import { RuleStatsProps } from 'types/api/alerts/ruleStats';
const ruleStats = async (
props: RuleStatsProps,
): Promise<SuccessResponse<AlertRuleStatsPayload> | ErrorResponse> => {
try {
const response = await axios.post(`/rules/${props.id}/history/stats`, {
start: props.start,
end: props.end,
});
return {
statusCode: 200,
error: null,
message: response.data.status,
payload: response.data,
};
} catch (error) {
return ErrorResponseHandler(error as AxiosError);
}
};
export default ruleStats;

View File

@@ -1,33 +0,0 @@
import axios from 'api';
import { ErrorResponseHandler } from 'api/ErrorResponseHandler';
import { AxiosError } from 'axios';
import { ErrorResponse, SuccessResponse } from 'types/api';
import { AlertRuleTimelineGraphResponsePayload } from 'types/api/alerts/def';
import { GetTimelineGraphRequestProps } from 'types/api/alerts/timelineGraph';
const timelineGraph = async (
props: GetTimelineGraphRequestProps,
): Promise<
SuccessResponse<AlertRuleTimelineGraphResponsePayload> | ErrorResponse
> => {
try {
const response = await axios.post(
`/rules/${props.id}/history/overall_status`,
{
start: props.start,
end: props.end,
},
);
return {
statusCode: 200,
error: null,
message: response.data.status,
payload: response.data,
};
} catch (error) {
return ErrorResponseHandler(error as AxiosError);
}
};
export default timelineGraph;

View File

@@ -1,36 +0,0 @@
import axios from 'api';
import { ErrorResponseHandler } from 'api/ErrorResponseHandler';
import { AxiosError } from 'axios';
import { ErrorResponse, SuccessResponse } from 'types/api';
import { AlertRuleTimelineTableResponsePayload } from 'types/api/alerts/def';
import { GetTimelineTableRequestProps } from 'types/api/alerts/timelineTable';
const timelineTable = async (
props: GetTimelineTableRequestProps,
): Promise<
SuccessResponse<AlertRuleTimelineTableResponsePayload> | ErrorResponse
> => {
try {
const response = await axios.post(`/rules/${props.id}/history/timeline`, {
start: props.start,
end: props.end,
offset: props.offset,
limit: props.limit,
order: props.order,
state: props.state,
// TODO(shaheer): implement filters
filters: props.filters,
});
return {
statusCode: 200,
error: null,
message: response.data.status,
payload: response.data,
};
} catch (error) {
return ErrorResponseHandler(error as AxiosError);
}
};
export default timelineTable;

View File

@@ -1,33 +0,0 @@
import axios from 'api';
import { ErrorResponseHandler } from 'api/ErrorResponseHandler';
import { AxiosError } from 'axios';
import { ErrorResponse, SuccessResponse } from 'types/api';
import { AlertRuleTopContributorsPayload } from 'types/api/alerts/def';
import { TopContributorsProps } from 'types/api/alerts/topContributors';
const topContributors = async (
props: TopContributorsProps,
): Promise<
SuccessResponse<AlertRuleTopContributorsPayload> | ErrorResponse
> => {
try {
const response = await axios.post(
`/rules/${props.id}/history/top_contributors`,
{
start: props.start,
end: props.end,
},
);
return {
statusCode: 200,
error: null,
message: response.data.status,
payload: response.data,
};
} catch (error) {
return ErrorResponseHandler(error as AxiosError);
}
};
export default topContributors;

View File

@@ -2815,6 +2815,7 @@ export enum CloudintegrationtypesServiceIDDTO {
redis = 'redis',
cloudsql_postgres = 'cloudsql_postgres',
memorystore_redis = 'memorystore_redis',
computeengine = 'computeengine',
}
export type CloudintegrationtypesCloudIntegrationServiceDTOAnyOf = {
/**

View File

@@ -98,6 +98,15 @@ interface QuerySearchProps {
showFilterSuggestionsWithoutMetric?: boolean;
/** When set, the editor shows only the user expression; API/filter uses `initial AND (user)`. */
initialExpression?: string;
/** When set, replaces the generic value-suggestion API with a custom fetcher. */
valueSuggestionsOverride?: (
key: string,
searchText: string,
) => Promise<{
stringValues: string[];
numberValues: number[];
complete: boolean;
}>;
}
function QuerySearch({
@@ -111,6 +120,7 @@ function QuerySearch({
showFilterSuggestionsWithoutMetric,
initialExpression,
metricNamespace,
valueSuggestionsOverride,
}: QuerySearchProps): JSX.Element {
const isDarkMode = useIsDarkMode();
const [valueSuggestions, setValueSuggestions] = useState<any[]>([]);
@@ -480,13 +490,24 @@ function QuerySearch({
const sanitizedSearchText = searchText ? searchText?.trim() : '';
try {
const response = await getValueSuggestions({
key,
searchText: sanitizedSearchText,
signal: dataSource,
signalSource: signalSource as 'meter' | '',
metricName: debouncedMetricName ?? undefined,
});
const values = valueSuggestionsOverride
? await valueSuggestionsOverride(key, sanitizedSearchText)
: await getValueSuggestions({
key,
searchText: sanitizedSearchText,
signal: dataSource,
signalSource: signalSource as 'meter' | '',
metricName: debouncedMetricName ?? undefined,
}).then((response) => {
const responseData = response.data as any;
const data = responseData.data || {};
const values = data.values || {};
return {
stringValues: values.stringValues || [],
numberValues: values.numberValues || [],
complete: data.complete ?? false,
};
});
// Skip updates if component unmounted or key changed
if (
@@ -498,8 +519,6 @@ function QuerySearch({
}
// Process the response data
const responseData = response.data as any;
const values = responseData.data?.values || {};
const stringValues = values.stringValues || [];
const numberValues = values.numberValues || [];
@@ -580,6 +599,7 @@ function QuerySearch({
debouncedMetricName,
signalSource,
toggleSuggestions,
valueSuggestionsOverride,
],
);

View File

@@ -92,52 +92,76 @@
color: var(--l3-foreground);
}
.only-btn {
cursor: not-allowed;
color: var(--l3-foreground);
}
.only-btn,
.toggle-btn {
cursor: not-allowed;
color: var(--l3-foreground);
}
}
.only-btn {
display: none;
// Stack "Only"/"All" and "Toggle" in a single cell so revealing
// one swaps in place instead of shifting the row layout.
.value-actions {
display: grid;
align-items: center;
justify-items: end;
> * {
grid-area: 1 / 1;
}
}
.only-btn,
.toggle-btn {
display: none;
}
.toggle-btn:hover {
background-color: unset;
}
.only-btn:hover {
background-color: unset;
}
}
.checkbox-value-section:hover {
.toggle-btn {
display: none;
}
.only-btn {
display: flex;
align-items: center;
justify-content: center;
height: 21px;
opacity: 0;
transform: translateX(4px);
// `display` is animated as a discrete property so the button
// stays mounted through its fade-out on exit.
transition:
opacity 0.16s ease,
transform 0.16s ease,
display 0.16s allow-discrete;
&:hover {
background-color: unset;
}
}
}
// Hovering the label/value area reveals the "Only"/"All" action.
.checkbox-value-section:hover .only-btn {
display: flex;
opacity: 1;
transform: translateX(0);
// Entry animation start state (fires on the display switch).
@starting-style {
opacity: 0;
transform: translateX(4px);
}
}
// Hovering the checkbox reveals the "Toggle" action.
.check-box:hover ~ .checkbox-value-section .toggle-btn {
display: flex;
opacity: 1;
transform: translateX(0);
@starting-style {
opacity: 0;
transform: translateX(4px);
}
}
}
.value:hover {
@media (prefers-reduced-motion: reduce) {
.only-btn,
.toggle-btn {
display: flex;
align-items: center;
justify-content: center;
height: 21px;
transition: none;
}
}
}

View File

@@ -53,12 +53,14 @@ function CheckboxValueRow({
</Typography.Text>
</TooltipSimple>
)}
<Button type="text" className="only-btn">
{onlyButtonLabel}
</Button>
<Button type="text" className="toggle-btn">
Toggle
</Button>
<div className="value-actions">
<Button type="text" className="only-btn">
{onlyButtonLabel}
</Button>
<Button type="text" className="toggle-btn">
Toggle
</Button>
</div>
</div>
</div>
);

View File

@@ -1,6 +1,11 @@
.warning-popover-overlay {
--antd-arrow-background-color: var(--l2-background);
}
.warning-content {
display: flex;
flex-direction: column;
background-color: var(--l2-background);
// === SECTION: Summary (Top)
&__summary-section {
@@ -10,16 +15,25 @@
&__summary {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: var(--spacing-8);
padding: 16px;
}
&__summary-left {
display: flex;
align-items: baseline;
align-items: flex-start;
gap: 8px;
}
&__icon-wrapper {
display: flex;
align-items: center;
flex-shrink: 0;
height: var(--spacing-12);
}
&__summary-text {
display: flex;
flex-direction: column;

View File

@@ -2,6 +2,7 @@ import { ReactNode, useMemo } from 'react';
import { Color } from '@signozhq/design-tokens';
import { Button, Popover, PopoverProps } from 'antd';
import ErrorIcon from 'assets/Error';
import cx from 'classnames';
import OverlayScrollbar from 'components/OverlayScrollbar/OverlayScrollbar';
import { BookOpenText, ChevronsDown, TriangleAlert } from '@signozhq/icons';
import KeyValueLabel from 'periscope/components/KeyValueLabel';
@@ -30,16 +31,18 @@ export function WarningContent({ warning }: WarningContentProps): JSX.Element {
{/* Summary Header */}
<section className="warning-content__summary-section">
<header className="warning-content__summary">
<div className="warning-content__summary-left">
<div className="warning-content__icon-wrapper">
<ErrorIcon />
</div>
{(warningCode || warningMessage) && (
<div className="warning-content__summary-left">
<div className="warning-content__icon-wrapper">
<ErrorIcon />
</div>
<div className="warning-content__summary-text">
<h2 className="warning-content__warning-code">{warningCode}</h2>
<p className="warning-content__warning-message">{warningMessage}</p>
<div className="warning-content__summary-text">
<h2 className="warning-content__warning-code">{warningCode}</h2>
<p className="warning-content__warning-message">{warningMessage}</p>
</div>
</div>
</div>
)}
{warningUrl && (
<div className="warning-content__summary-right">
@@ -154,6 +157,10 @@ function WarningPopover({
overlayInnerStyle={{ padding: 0 }}
autoAdjustOverflow
{...popoverProps}
overlayClassName={cx(
'warning-popover-overlay',
popoverProps.overlayClassName,
)}
>
{children || (
<TriangleAlert

View File

@@ -20,10 +20,6 @@ export const REACT_QUERY_KEY = {
DELETE_DASHBOARD: 'DELETE_DASHBOARD',
LOGS_PIPELINE_PREVIEW: 'LOGS_PIPELINE_PREVIEW',
ALERT_RULE_DETAILS: 'ALERT_RULE_DETAILS',
ALERT_RULE_STATS: 'ALERT_RULE_STATS',
ALERT_RULE_TOP_CONTRIBUTORS: 'ALERT_RULE_TOP_CONTRIBUTORS',
ALERT_RULE_TIMELINE_TABLE: 'ALERT_RULE_TIMELINE_TABLE',
ALERT_RULE_TIMELINE_GRAPH: 'ALERT_RULE_TIMELINE_GRAPH',
GET_CONSUMER_LAG_DETAILS: 'GET_CONSUMER_LAG_DETAILS',
TOGGLE_ALERT_STATE: 'TOGGLE_ALERT_STATE',
GET_ALL_ALERTS: 'GET_ALL_ALERTS',

View File

@@ -29,6 +29,7 @@ function PopoverContent({
<Link
to={`${ROUTES.LOGS_EXPLORER}?${relatedLogsLink}`}
className="contributor-row-popover-buttons__button"
data-testid="alert-popover-view-logs"
>
<div className="icon">
<LogsIcon />
@@ -40,6 +41,7 @@ function PopoverContent({
<Link
to={`${ROUTES.TRACES_EXPLORER}?${relatedTracesLink}`}
className="contributor-row-popover-buttons__button"
data-testid="alert-popover-view-traces"
>
<div className="icon">
<DraftingCompass

View File

@@ -26,7 +26,10 @@ function ChangePercentage({
}: ChangePercentageProps): JSX.Element {
if (direction > 0) {
return (
<div className="change-percentage change-percentage--success">
<div
className="change-percentage change-percentage--success"
data-testid="stats-card-change"
>
<div className="change-percentage__icon">
<ArrowDownLeft size={14} color={Color.BG_FOREST_500} />
</div>
@@ -38,7 +41,10 @@ function ChangePercentage({
}
if (direction < 0) {
return (
<div className="change-percentage change-percentage--error">
<div
className="change-percentage change-percentage--error"
data-testid="stats-card-change"
>
<div className="change-percentage__icon">
<ArrowUpRight size={14} color={Color.BG_CHERRY_500} />
</div>
@@ -50,7 +56,10 @@ function ChangePercentage({
}
return (
<div className="change-percentage change-percentage--no-previous-data">
<div
className="change-percentage change-percentage--no-previous-data"
data-testid="stats-card-change"
>
<div className="change-percentage__label">no previous data</div>
</div>
);
@@ -103,7 +112,12 @@ function StatsCard({
const formattedEndTimeForTooltip = convertTimestampToLocaleDateString(endTime);
return (
<div className={`stats-card ${isEmpty ? 'stats-card--empty' : ''}`}>
<div
className={`stats-card ${isEmpty ? 'stats-card--empty' : ''}`}
data-testid="stats-card"
data-stats-title={title}
data-empty={isEmpty ? 'true' : 'false'}
>
<div className="stats-card__title-wrapper">
<div className="title">{title}</div>
<div className="duration-indicator">
@@ -123,7 +137,7 @@ function StatsCard({
</div>
<div className="stats-card__stats">
<div className="count-label">
<div className="count-label" data-testid="stats-card-value">
{isEmpty ? emptyMessage : displayValue || totalCurrentCount}
</div>

View File

@@ -81,7 +81,11 @@ function StatsGraph({ timeSeries, changeDirection }: Props): JSX.Element {
);
return (
<div style={{ height: '100%', width: '100%' }} ref={graphRef}>
<div
style={{ height: '100%', width: '100%' }}
ref={graphRef}
data-testid="stats-card-sparkline"
>
<Uplot data={[xData, yData]} options={options} />
</div>
);

View File

@@ -1,6 +1,7 @@
import { useEffect } from 'react';
import { useEffect, useMemo } from 'react';
import { useGetAlertRuleDetailsStats } from 'pages/AlertDetails/hooks';
import DataStateRenderer from 'periscope/components/DataStateRenderer/DataStateRenderer';
import { StatsTimeSeriesItem } from 'types/api/alerts/def';
import AverageResolutionCard from '../AverageResolutionCard/AverageResolutionCard';
import StatsCard from '../StatsCard/StatsCard';
@@ -25,24 +26,59 @@ type StatsCardsRendererProps = {
};
// TODO(shaheer): render the DataStateRenderer inside the TotalTriggeredCard/AverageResolutionCard, it should display the title
type AdaptedStatsData = {
totalCurrentTriggers: number;
totalPastTriggers: number;
currentAvgResolutionTime: string;
pastAvgResolutionTime: string;
currentTriggersSeries: StatsTimeSeriesItem[];
currentAvgResolutionTimeSeries: StatsTimeSeriesItem[];
};
function StatsCardsRenderer({
setTotalCurrentTriggers,
}: StatsCardsRendererProps): JSX.Element {
const { isLoading, isRefetching, isError, data, isValidRuleId, ruleId } =
useGetAlertRuleDetailsStats();
useEffect(() => {
if (data?.payload?.data?.totalCurrentTriggers !== undefined) {
setTotalCurrentTriggers(data.payload.data.totalCurrentTriggers);
const adaptedData = useMemo((): AdaptedStatsData | null => {
if (!data?.data) {
return null;
}
}, [data, setTotalCurrentTriggers]);
const statsData = data.data;
const adaptTimeSeries = (
series: typeof statsData.currentTriggersSeries,
): StatsTimeSeriesItem[] =>
series?.values?.map((item) => ({
timestamp: item.timestamp ?? 0,
value: String(item.value ?? 0),
})) ?? [];
return {
totalCurrentTriggers: statsData.totalCurrentTriggers,
totalPastTriggers: statsData.totalPastTriggers,
currentAvgResolutionTime: String(statsData.currentAvgResolutionTime),
pastAvgResolutionTime: String(statsData.pastAvgResolutionTime),
currentTriggersSeries: adaptTimeSeries(statsData.currentTriggersSeries),
currentAvgResolutionTimeSeries: adaptTimeSeries(
statsData.currentAvgResolutionTimeSeries,
),
};
}, [data?.data]);
useEffect(() => {
if (adaptedData?.totalCurrentTriggers !== undefined) {
setTotalCurrentTriggers(adaptedData.totalCurrentTriggers);
}
}, [adaptedData, setTotalCurrentTriggers]);
return (
<DataStateRenderer
isLoading={isLoading}
isRefetching={isRefetching}
isError={isError || !isValidRuleId || !ruleId}
data={data?.payload?.data || null}
data={adaptedData}
>
{(data): JSX.Element => {
const {
@@ -60,7 +96,7 @@ function StatsCardsRenderer({
<TotalTriggeredCard
totalCurrentTriggers={totalCurrentTriggers}
totalPastTriggers={totalPastTriggers}
timeSeries={currentTriggersSeries?.values}
timeSeries={currentTriggersSeries}
/>
) : (
<StatsCard
@@ -77,7 +113,7 @@ function StatsCardsRenderer({
<AverageResolutionCard
currentAvgResolutionTime={currentAvgResolutionTime}
pastAvgResolutionTime={pastAvgResolutionTime}
timeSeries={currentAvgResolutionTimeSeries?.values}
timeSeries={currentAvgResolutionTimeSeries}
/>
) : (
<StatsCard

View File

@@ -48,11 +48,16 @@ function TopContributorsCard({
return (
<>
<div className="top-contributors-card">
<div className="top-contributors-card" data-testid="top-contributors-card">
<div className="top-contributors-card__header">
<div className="title">top contributors</div>
{topContributorsData.length > 3 && (
<Button type="text" className="view-all" onClick={toggleViewAllDrawer}>
<Button
type="text"
className="view-all"
onClick={toggleViewAllDrawer}
data-testid="top-contributors-view-all"
>
<div className="label">View all</div>
<div className="icon">
<ArrowRight

View File

@@ -68,7 +68,10 @@ function TopContributorsRows({
relatedTracesLink={record.relatedTracesLink}
relatedLogsLink={record.relatedLogsLink}
>
<div className="total-contribution">
<div
className="total-contribution"
data-testid="top-contributors-row-count"
>
{count}/{totalCurrentTriggers}
</div>
</ConditionalAlertPopover>
@@ -78,7 +81,10 @@ function TopContributorsRows({
const handleRowClick = (
record: AlertRuleTopContributors,
): HTMLAttributes<AlertRuleTimelineTableResponse> => ({
): HTMLAttributes<AlertRuleTimelineTableResponse> & {
'data-testid': string;
} => ({
'data-testid': 'top-contributors-row',
onClick: (): void => {
logEvent('Alert history: Top contributors row: Clicked', {
labels: record.labels,

View File

@@ -31,7 +31,10 @@ function ViewAllDrawer({
}}
title="Viewing All Contributors"
>
<div className="top-contributors-card--view-all">
<div
className="top-contributors-card--view-all"
data-testid="top-contributors-drawer"
>
<div className="top-contributors-card__content">
<TopContributorsRows
topContributors={topContributorsData}

View File

@@ -1,6 +1,8 @@
import { useMemo } from 'react';
import { useGetAlertRuleDetailsTopContributors } from 'pages/AlertDetails/hooks';
import { labelsArrayToObject } from 'container/AlertHistory/utils/labelAdapters';
import DataStateRenderer from 'periscope/components/DataStateRenderer/DataStateRenderer';
import { AlertRuleStats } from 'types/api/alerts/def';
import { AlertRuleStats, AlertRuleTopContributors } from 'types/api/alerts/def';
import TopContributorsCard from '../TopContributorsCard/TopContributorsCard';
@@ -13,15 +15,27 @@ function TopContributorsRenderer({
}: TopContributorsRendererProps): JSX.Element {
const { isLoading, isRefetching, isError, data, isValidRuleId, ruleId } =
useGetAlertRuleDetailsTopContributors();
const response = data?.payload?.data;
// TODO(shaheer): render the DataStateRenderer inside the TopContributorsCard, it should display the title and view all
const adaptedData = useMemo((): AlertRuleTopContributors[] | null => {
if (!data?.data) {
return null;
}
return data.data.map((contributor) => ({
fingerprint: contributor.fingerprint,
count: contributor.count,
labels: labelsArrayToObject(contributor.labels),
relatedLogsLink: contributor.relatedLogsLink ?? '',
relatedTracesLink: contributor.relatedTracesLink ?? '',
}));
}, [data?.data]);
return (
<DataStateRenderer
isLoading={isLoading}
isRefetching={isRefetching}
isError={isError || !isValidRuleId || !ruleId}
data={response || null}
data={adaptedData}
>
{(topContributorsData): JSX.Element => (
<TopContributorsCard

View File

@@ -1,12 +1,18 @@
import { Color } from '@signozhq/design-tokens';
import { RuletypesAlertStateDTO } from 'api/generated/services/sigNoz.schemas';
export const ALERT_STATUS: { [key: string]: number } = {
firing: 0,
inactive: 1,
export const ALERT_STATUS: Record<RuletypesAlertStateDTO, number> & {
[key: string]: number;
} = {
[RuletypesAlertStateDTO.firing]: 0,
[RuletypesAlertStateDTO.inactive]: 1,
normal: 1,
'no-data': 2,
disabled: 3,
muted: 4,
[RuletypesAlertStateDTO.pending]: 2,
[RuletypesAlertStateDTO.recovering]: 2,
'no-data': 3,
[RuletypesAlertStateDTO.nodata]: 3,
[RuletypesAlertStateDTO.disabled]: 4,
muted: 5,
};
export const STATE_VS_COLOR: {
@@ -16,9 +22,10 @@ export const STATE_VS_COLOR: {
{
0: { stroke: Color.BG_CHERRY_500, fill: Color.BG_CHERRY_500 },
1: { stroke: Color.BG_FOREST_500, fill: Color.BG_FOREST_500 },
2: { stroke: Color.BG_SIENNA_400, fill: Color.BG_SIENNA_400 },
3: { stroke: Color.BG_VANILLA_400, fill: Color.BG_VANILLA_400 },
4: { stroke: Color.BG_INK_100, fill: Color.BG_INK_100 },
2: { stroke: Color.BG_AMBER_500, fill: Color.BG_AMBER_500 },
3: { stroke: Color.BG_SIENNA_400, fill: Color.BG_SIENNA_400 },
4: { stroke: Color.BG_VANILLA_400, fill: Color.BG_VANILLA_400 },
5: { stroke: Color.BG_INK_100, fill: Color.BG_INK_100 },
},
];

View File

@@ -1,6 +1,8 @@
import { useMemo } from 'react';
import useUrlQuery from 'hooks/useUrlQuery';
import { useGetAlertRuleDetailsTimelineGraphData } from 'pages/AlertDetails/hooks';
import DataStateRenderer from 'periscope/components/DataStateRenderer/DataStateRenderer';
import { AlertRuleTimelineGraphResponse } from 'types/api/alerts/def';
import Graph from '../Graph/Graph';
@@ -18,30 +20,20 @@ function GraphWrapper({
const { isLoading, isRefetching, isError, data, isValidRuleId, ruleId } =
useGetAlertRuleDetailsTimelineGraphData();
// TODO(shaheer): uncomment when the API is ready for
// const { startTime } = useAlertHistoryQueryParams();
// const [isVerticalGraph, setIsVerticalGraph] = useState(false);
// useEffect(() => {
// const checkVerticalGraph = (): void => {
// if (startTime) {
// const startTimeDate = dayjs(Number(startTime));
// const twentyFourHoursAgo = dayjs().subtract(
// HORIZONTAL_GRAPH_HOURS_THRESHOLD,
// DAYJS_MANIPULATE_TYPES.HOUR,
// );
// setIsVerticalGraph(startTimeDate.isBefore(twentyFourHoursAgo));
// }
// };
// checkVerticalGraph();
// }, [startTime]);
const adaptedData = useMemo((): AlertRuleTimelineGraphResponse[] | null => {
if (!data?.data) {
return null;
}
return data.data.map((item) => ({
start: item.start,
end: item.end,
state: item.state as AlertRuleTimelineGraphResponse['state'],
}));
}, [data?.data]);
return (
<div className="timeline-graph">
<div className="timeline-graph__title">
<div className="timeline-graph" data-testid="timeline-graph">
<div className="timeline-graph__title" data-testid="timeline-graph-title">
{totalCurrentTriggers} triggers in {relativeTime}
</div>
<div className="timeline-graph__chart">
@@ -49,7 +41,7 @@ function GraphWrapper({
isLoading={isLoading}
isError={isError || !isValidRuleId || !ruleId}
isRefetching={isRefetching}
data={data?.payload?.data || null}
data={adaptedData}
>
{(data): JSX.Element => <Graph type="horizontal" data={data} />}
</DataStateRenderer>

View File

@@ -1,11 +1,35 @@
.timeline-table {
border-top: 1px solid var(--l1-border);
border-radius: 6px;
overflow: hidden;
margin-top: 4px;
min-height: 600px;
&__filter {
padding: 12px 16px;
background: var(--l1-background);
border-bottom: 1px solid var(--l1-border);
}
&__filter-row {
display: flex;
align-items: center;
gap: 8px;
}
&__filter-search {
flex: 1;
}
&__filter--loading,
&__filter--loading-skeleton {
width: 100% !important;
}
.ant-table {
background: var(--l1-background);
&-placeholder {
background: var(--l1-background) !important;
}
&-cell {
padding: 12px 16px !important;
vertical-align: baseline;
@@ -23,6 +47,9 @@
&-tbody > tr > td {
border: none;
}
&-footer {
background-color: var(--l1-background);
}
}
.label-filter {
@@ -86,4 +113,38 @@
background: var(--l2-background);
}
}
&__error {
display: flex;
align-self: flex-start;
justify-content: flex-start;
text-align: left;
}
&__pagination {
display: flex;
justify-content: flex-end;
align-items: center;
gap: var(--spacing-4);
padding: var(--spacing-6) var(--spacing-8);
background: var(--l1-background);
.pagination-controls {
display: flex;
gap: 4px;
.ant-btn {
display: flex;
align-items: center;
justify-content: center;
padding: 4px;
min-width: 24px;
height: 24px;
&:disabled {
opacity: 0.4;
}
}
}
}
}

View File

@@ -1,52 +1,129 @@
import { HTMLAttributes, useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Table } from 'antd';
import { HTMLAttributes, useCallback, useMemo } from 'react';
import { Button, Skeleton, Table } from 'antd';
import { ChevronLeft, ChevronRight } from '@signozhq/icons';
import logEvent from 'api/common/logEvent';
import { initialFilters } from 'constants/queryBuilder';
import { convertToApiError } from 'api/ErrorResponseHandlerForGeneratedAPIs';
import ErrorContent from 'components/ErrorModal/components/ErrorContent';
import {
QuerySearchV2Provider,
useExpression,
useInputExpression,
useQuerySearchOnChange,
useQuerySearchOnRun,
} from 'components/QueryBuilderV2';
import QuerySearch from 'components/QueryBuilderV2/QueryV2/QuerySearch/QuerySearch';
import { initialQueryBuilderFormValuesMap } from 'constants/queryBuilder';
import RunQueryBtn from 'container/QueryBuilder/components/RunQueryBtn/RunQueryBtn';
import {
useGetAlertRuleDetailsTimelineTable,
useTimelineTable,
} from 'pages/AlertDetails/hooks';
import { labelsArrayToObject } from 'container/AlertHistory/utils/labelAdapters';
import { useTimezone } from 'providers/Timezone';
import { AlertRuleTimelineTableResponse } from 'types/api/alerts/def';
import { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import { useAlertHistoryFilterSuggestions } from './useAlertHistoryFilterSuggestions';
import { timelineTableColumns } from './useTimelineTable';
import './Table.styles.scss';
function TimelineTable(): JSX.Element {
const [filters, setFilters] = useState<TagFilter>(initialFilters);
export const ALERT_HISTORY_EXPRESSION_KEY = 'alertHistoryExpression';
const { isLoading, isRefetching, isError, data, isValidRuleId, ruleId } =
useGetAlertRuleDetailsTimelineTable({ filters });
function TimelineTableContent(): JSX.Element {
const expression = useExpression();
const inputExpression = useInputExpression();
const querySearchOnChange = useQuerySearchOnChange();
const querySearchOnRun = useQuerySearchOnRun();
const {
isLoading,
isRefetching,
isError,
data,
error,
ruleId,
refetch,
cancel,
} = useGetAlertRuleDetailsTimelineTable({ filterExpression: expression });
const apiError = useMemo(() => convertToApiError(error), [error]);
const { hardcodedAttributeKeys, valueSuggestionsOverride, isLoadingKeys } =
useAlertHistoryFilterSuggestions(ruleId ?? null);
const { timelineData, totalItems, nextCursor } = useMemo(() => {
const response = data?.data;
const items: AlertRuleTimelineTableResponse[] | undefined =
response?.items?.map((item) => {
return {
ruleID: item.ruleId,
ruleName: item.ruleName,
overallState: item.overallState as string,
overallStateChanged: item.overallStateChanged,
state: item.state as string,
stateChanged: item.stateChanged,
unixMilli: item.unixMilli,
fingerprint: item.fingerprint,
value: item.value,
labels: labelsArrayToObject(item.labels),
relatedLogsLink: item.relatedLogsLink,
relatedTracesLink: item.relatedTracesLink,
};
});
const { timelineData, totalItems, labels } = useMemo(() => {
const response = data?.payload?.data;
return {
timelineData: response?.items,
totalItems: response?.total,
labels: response?.labels,
timelineData: items,
totalItems: response?.total ?? 0,
nextCursor: response?.nextCursor,
};
}, [data?.payload?.data]);
}, [data?.data]);
const { paginationConfig, onChangeHandler } = useTimelineTable({
const {
paginationConfig,
onChangeHandler,
handleNextPage,
handlePrevPage,
hasNextPage,
hasPrevPage,
} = useTimelineTable({
totalItems: totalItems ?? 0,
nextCursor,
});
const { t } = useTranslation('common');
const { formatTimezoneAdjustedTimestamp } = useTimezone();
if (isError || !isValidRuleId || !ruleId) {
return <div>{t('something_went_wrong')}</div>;
}
const handleRunQuery = useCallback(
(updatedExpression?: string): void => {
const nextExpression = updatedExpression ?? inputExpression;
querySearchOnRun(nextExpression);
if (nextExpression === expression) {
refetch();
}
},
[querySearchOnRun, refetch, inputExpression, expression],
);
const queryData = useMemo(
() => ({
...initialQueryBuilderFormValuesMap.logs,
queryName: 'A',
dataSource: DataSource.LOGS,
filter: { expression },
expression,
}),
[expression],
);
const handleRowClick = (
record: AlertRuleTimelineTableResponse,
): HTMLAttributes<AlertRuleTimelineTableResponse> => ({
): HTMLAttributes<AlertRuleTimelineTableResponse> & {
'data-testid': string;
} => ({
'data-testid': 'timeline-row',
onClick: (): void => {
logEvent('Alert history: Timeline table row: Clicked', {
void logEvent('Alert history: Timeline table row: Clicked', {
ruleId: record.ruleID,
labels: record.labels,
});
@@ -54,24 +131,114 @@ function TimelineTable(): JSX.Element {
});
return (
<div className="timeline-table">
<div className="timeline-table" data-testid="timeline-table">
{/* If we don't wait to have the keys, the QuerySearch will not render them at first usage */}
{!isLoadingKeys && hardcodedAttributeKeys ? (
<div className="timeline-table__filter">
<div className="timeline-table__filter-row">
<div
className="timeline-table__filter-search"
data-testid="timeline-filter-search"
>
<QuerySearch
onChange={querySearchOnChange}
queryData={queryData}
dataSource={DataSource.LOGS}
onRun={handleRunQuery}
hardcodedAttributeKeys={hardcodedAttributeKeys}
valueSuggestionsOverride={valueSuggestionsOverride}
/>
</div>
<RunQueryBtn
isLoadingQueries={isLoading || isRefetching}
onStageRunQuery={(): void => handleRunQuery()}
handleCancelQuery={cancel}
/>
</div>
</div>
) : (
<div className="timeline-table__filter timeline-table__filter--loading">
<Skeleton.Input
className="timeline-table__filter--loading-skeleton"
active
data-testid="timeline-filter-skeleton"
/>
</div>
)}
<Table
rowKey={(row): string => `${row.fingerprint}-${row.value}-${row.unixMilli}`}
columns={timelineTableColumns({
filters,
labels: labels ?? {},
setFilters,
formatTimezoneAdjustedTimestamp,
})}
onRow={handleRowClick}
dataSource={timelineData}
pagination={paginationConfig}
pagination={false}
size="middle"
onChange={onChangeHandler}
loading={isLoading || isRefetching}
locale={{
emptyText:
isError && apiError ? (
<div className="timeline-table__error" data-testid="timeline-error">
<ErrorContent error={apiError} />
</div>
) : undefined,
}}
footer={(): JSX.Element => (
<div className="timeline-table__pagination">
<div
className="timeline-table__pagination-info"
data-testid="timeline-footer-range"
>
{paginationConfig.showTotal?.(totalItems, [
totalItems === 0
? 0
: ((paginationConfig.current ?? 1) - 1) *
(paginationConfig.pageSize ?? 10) +
1,
Math.min(
(paginationConfig.current ?? 1) * (paginationConfig.pageSize ?? 10),
totalItems,
),
])}
</div>
<div className="pagination-controls">
<Button
type="text"
size="small"
disabled={!hasPrevPage}
onClick={handlePrevPage}
data-testid="timeline-prev-page"
>
<ChevronLeft size={14} />
</Button>
<Button
type="text"
size="small"
disabled={!hasNextPage}
onClick={handleNextPage}
data-testid="timeline-next-page"
>
<ChevronRight size={14} />
</Button>
</div>
</div>
)}
/>
</div>
);
}
function TimelineTable(): JSX.Element {
return (
<QuerySearchV2Provider
queryParamKey={ALERT_HISTORY_EXPRESSION_KEY}
initialExpression=""
persistOnUnmount
>
<TimelineTableContent />
</QuerySearchV2Provider>
);
}
export default TimelineTable;

View File

@@ -0,0 +1,100 @@
import { useCallback, useMemo } from 'react';
import {
getRuleHistoryFilterValues,
useGetRuleHistoryFilterKeys,
} from 'api/generated/services/rules';
import { useAlertHistoryQueryParams } from 'pages/AlertDetails/hooks';
import { QueryKeyDataSuggestionsProps } from 'types/api/querySuggestions/types';
import { fieldContextToSuggestionContext } from 'container/AlertHistory/Timeline/Table/utils';
export interface AlertHistoryFilterSuggestions {
hardcodedAttributeKeys: QueryKeyDataSuggestionsProps[];
valueSuggestionsOverride: (
key: string,
searchText: string,
) => Promise<{
stringValues: string[];
numberValues: number[];
complete: boolean;
}>;
isLoadingKeys: boolean;
}
export function useAlertHistoryFilterSuggestions(
ruleId: string | null,
): AlertHistoryFilterSuggestions {
const { startTime, endTime } = useAlertHistoryQueryParams();
const { data: filterKeysData, isLoading: isLoadingKeys } =
useGetRuleHistoryFilterKeys(
{ id: ruleId ?? '' },
{ startUnixMilli: startTime, endUnixMilli: endTime },
{
query: {
enabled: !!ruleId,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
},
);
const hardcodedAttributeKeys = useMemo((): QueryKeyDataSuggestionsProps[] => {
const keys = filterKeysData?.data?.keys;
if (!keys) {
// by default, when QuerySearch keys fails, we don't render fallback keys
// we just return empty to let user write whatever they want with no
// key suggestion
return [];
}
return Object.values(keys).flatMap((items) =>
items.map(
(item) =>
({
label: item.name,
name: item.name,
type: item.fieldDataType || 'string',
signal: 'logs' as const,
fieldDataType: item.fieldDataType,
fieldContext: fieldContextToSuggestionContext(item.fieldContext),
}) satisfies QueryKeyDataSuggestionsProps,
),
);
}, [filterKeysData]);
const valueSuggestionsOverride = useCallback(
async (
key: string,
searchText: string,
): Promise<{
stringValues: string[];
numberValues: number[];
complete: boolean;
}> => {
if (!ruleId) {
return {
stringValues: [],
numberValues: [],
complete: true,
};
}
const response = await getRuleHistoryFilterValues(
{ id: ruleId },
{
name: key,
searchText,
startUnixMilli: startTime,
endUnixMilli: endTime,
},
);
const values = response.data?.values;
return {
stringValues: values?.stringValues ?? [],
numberValues: values?.numberValues ?? [],
complete: response.data?.complete ?? false,
};
},
[ruleId, startTime, endTime],
);
return { hardcodedAttributeKeys, valueSuggestionsOverride, isLoadingKeys };
}

View File

@@ -1,83 +1,15 @@
import { useMemo } from 'react';
import { Ellipsis, Search } from '@signozhq/icons';
import { Color } from '@signozhq/design-tokens';
import { Button, TableColumnsType as ColumnsType } from 'antd';
import ClientSideQBSearch, {
AttributeKey,
} from 'components/ClientSideQBSearch/ClientSideQBSearch';
import { Ellipsis } from '@signozhq/icons';
import { Button, TableColumnsType as ColumnsType, Tooltip } from 'antd';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import { ConditionalAlertPopover } from 'container/AlertHistory/AlertPopover/AlertPopover';
import { transformKeyValuesToAttributeValuesMap } from 'container/QueryBuilder/filters/utils';
import { useIsDarkMode } from 'hooks/useDarkMode';
import { TimestampInput } from 'hooks/useTimezoneFormatter/useTimezoneFormatter';
import AlertLabels, {
AlertLabelsProps,
} from 'pages/AlertDetails/AlertHeader/AlertLabels/AlertLabels';
import AlertLabels from 'pages/AlertDetails/AlertHeader/AlertLabels/AlertLabels';
import AlertState from 'pages/AlertDetails/AlertHeader/AlertState/AlertState';
import { AlertRuleTimelineTableResponse } from 'types/api/alerts/def';
import { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
const transformLabelsToQbKeys = (
labels: AlertRuleTimelineTableResponse['labels'],
): AttributeKey[] => Object.keys(labels).flatMap((key) => [{ key }]);
function LabelFilter({
filters,
setFilters,
labels,
}: {
setFilters: (filters: TagFilter) => void;
filters: TagFilter;
labels: AlertLabelsProps['labels'];
}): JSX.Element | null {
const isDarkMode = useIsDarkMode();
const { transformedKeys, attributesMap } = useMemo(
() => ({
transformedKeys: transformLabelsToQbKeys(labels || {}),
attributesMap: transformKeyValuesToAttributeValuesMap(labels),
}),
[labels],
);
const handleSearch = (tagFilters: TagFilter): void => {
const tagFiltersLength = tagFilters.items.length;
if (
(!tagFiltersLength && (!filters || !filters.items.length)) ||
tagFiltersLength === filters?.items.length
) {
return;
}
setFilters(tagFilters);
};
return (
<ClientSideQBSearch
onChange={handleSearch}
filters={filters}
className="alert-history-label-search"
attributeKeys={transformedKeys}
attributeValuesMap={attributesMap}
suffixIcon={
<Search
size={14}
color={isDarkMode ? Color.TEXT_VANILLA_100 : Color.TEXT_INK_100}
/>
}
/>
);
}
export const timelineTableColumns = ({
filters,
labels,
setFilters,
formatTimezoneAdjustedTimestamp,
}: {
filters: TagFilter;
labels: AlertLabelsProps['labels'];
setFilters: (filters: TagFilter) => void;
formatTimezoneAdjustedTimestamp: (
input: TimestampInput,
format?: string,
@@ -89,18 +21,16 @@ export const timelineTableColumns = ({
sorter: true,
width: 140,
render: (value): JSX.Element => (
<div className="alert-rule-state">
<div className="alert-rule-state" data-testid="timeline-row-state">
<AlertState state={value} showLabel />
</div>
),
},
{
title: (
<LabelFilter setFilters={setFilters} filters={filters} labels={labels} />
),
title: 'LABELS',
dataIndex: 'labels',
render: (labels): JSX.Element => (
<div className="alert-rule-labels">
<div className="alert-rule-labels" data-testid="timeline-row-labels">
<AlertLabels labels={labels} />
</div>
),
@@ -110,7 +40,10 @@ export const timelineTableColumns = ({
dataIndex: 'unixMilli',
width: 200,
render: (value): JSX.Element => (
<div className="alert-rule__created-at">
<div
className="alert-rule__created-at"
data-testid="timeline-row-created-at"
>
{formatTimezoneAdjustedTimestamp(value, DATE_TIME_FORMATS.DASH_DATETIME)}
</div>
),
@@ -119,15 +52,27 @@ export const timelineTableColumns = ({
title: 'ACTIONS',
width: 140,
align: 'right',
render: (record): JSX.Element => (
<ConditionalAlertPopover
relatedTracesLink={record.relatedTracesLink}
relatedLogsLink={record.relatedLogsLink}
>
<Button type="text" ghost>
<Ellipsis className="dropdown-icon" size="md" />
</Button>
</ConditionalAlertPopover>
),
render: (_, record): JSX.Element => {
if (!record.relatedTracesLink && !record.relatedLogsLink) {
return (
<Tooltip title="No links available for this item">
<Button type="text" ghost disabled data-testid="timeline-row-actions">
<Ellipsis className="dropdown-icon" size="md" />
</Button>
</Tooltip>
);
}
return (
<ConditionalAlertPopover
relatedTracesLink={record.relatedTracesLink ?? ''}
relatedLogsLink={record.relatedLogsLink ?? ''}
>
<Button type="text" ghost data-testid="timeline-row-actions">
<Ellipsis className="dropdown-icon" size="md" />
</Button>
</ConditionalAlertPopover>
);
},
},
];

View File

@@ -0,0 +1,53 @@
import {
Options,
parseAsInteger,
parseAsStringLiteral,
useQueryState,
UseQueryStateReturn,
} from 'nuqs';
import { TIMELINE_TABLE_PAGE_SIZE } from 'container/AlertHistory/constants';
const defaultNuqsOptions: Options = {
history: 'push',
};
export const TIMELINE_TABLE_PARAMS = {
PAGE: 'page',
ORDER: 'order',
} as const;
const ORDER_VALUES = ['asc', 'desc'] as const;
export type OrderDirection = (typeof ORDER_VALUES)[number];
export const useTimelineTablePage = (): UseQueryStateReturn<number, number> =>
useQueryState(
TIMELINE_TABLE_PARAMS.PAGE,
parseAsInteger.withDefault(1).withOptions(defaultNuqsOptions),
);
export const useTimelineTableOrder = (): UseQueryStateReturn<
OrderDirection,
OrderDirection
> =>
useQueryState(
TIMELINE_TABLE_PARAMS.ORDER,
parseAsStringLiteral(ORDER_VALUES)
.withDefault('asc')
.withOptions(defaultNuqsOptions),
);
export function encodeCursor(page: number, limit: number): string | undefined {
if (page <= 1) {
return undefined;
}
const offset = (page - 1) * limit;
// Backend uses base64.RawURLEncoding (URL-safe, no padding)
return btoa(JSON.stringify({ offset, limit }))
.replace(/\+/g, '-')
.replace(/\//g, '_')
.replace(/=+$/, '');
}
export function computeCursorForPage(page: number): string | undefined {
return encodeCursor(page, TIMELINE_TABLE_PAGE_SIZE);
}

View File

@@ -0,0 +1,26 @@
import { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import { QueryKeyDataSuggestionsProps } from 'types/api/querySuggestions/types';
const fieldContextToSuggestionMap: Record<
TelemetrytypesFieldContextDTO,
QueryKeyDataSuggestionsProps['fieldContext']
> = {
[TelemetrytypesFieldContextDTO.resource]: 'resource',
[TelemetrytypesFieldContextDTO.span]: 'span',
[TelemetrytypesFieldContextDTO.attribute]: 'attribute',
// no maps for the following values on suggestion context
[TelemetrytypesFieldContextDTO.body]: undefined,
[TelemetrytypesFieldContextDTO.metric]: undefined,
[TelemetrytypesFieldContextDTO.log]: undefined,
[TelemetrytypesFieldContextDTO['']]: undefined,
};
export function fieldContextToSuggestionContext(
fc: TelemetrytypesFieldContextDTO | undefined,
): QueryKeyDataSuggestionsProps['fieldContext'] {
if (fc === undefined) {
return undefined;
}
return fieldContextToSuggestionMap[fc];
}

View File

@@ -23,6 +23,7 @@ function TimelineTabs(): JSX.Element {
{
value: TimelineTab.OVERALL_STATUS,
label: 'Overall Status',
testId: 'timeline-tab-overall-status',
},
{
value: TimelineTab.TOP_5_CONTRIBUTORS,
@@ -33,6 +34,7 @@ function TimelineTabs(): JSX.Element {
</div>
),
disabled: true,
testId: 'timeline-tab-top-contributors',
},
];
@@ -57,14 +59,17 @@ function TimelineFilters(): JSX.Element {
{
value: TimelineFilter.ALL,
label: 'All',
testId: 'timeline-filter-all',
},
{
value: TimelineFilter.FIRED,
label: 'Fired',
testId: 'timeline-filter-fired',
},
{
value: TimelineFilter.RESOLVED,
label: 'Resolved',
testId: 'timeline-filter-resolved',
},
];

View File

@@ -0,0 +1,19 @@
import type { Querybuildertypesv5LabelDTO } from 'api/generated/services/sigNoz.schemas';
import type { Labels } from 'types/api/alerts/def';
export function labelsArrayToObject(
labels: Querybuildertypesv5LabelDTO[] | null | undefined,
): Labels {
if (!labels) {
return {};
}
return labels.reduce<Labels>((acc, label) => {
const key = label.key?.name ?? '';
const value = String(label.value ?? '');
if (key) {
acc[key] = value;
}
return acc;
}, {});
}

View File

@@ -94,6 +94,8 @@ function AlertDetails(): JSX.Element {
>
<div
className={classNames('alert-details', { 'alert-details-v2': isV2Alert })}
data-testid="alert-details-root"
data-schema-version={isV2Alert ? NEW_ALERT_SCHEMA_VERSION : 'v1'}
>
<AlertBreadcrumb
className="alert-details__breadcrumb"

View File

@@ -117,7 +117,11 @@ function AlertActionButtons({
<div className="alert-action-buttons">
<Tooltip title={isAlertRuleDisabled ? 'Enable alert' : 'Disable alert'}>
{isAlertRuleDisabled !== undefined && (
<Switch onChange={toggleAlertRule} value={!isAlertRuleDisabled} />
<Switch
onChange={toggleAlertRule}
value={!isAlertRuleDisabled}
testId="alert-actions-toggle"
/>
)}
</Tooltip>
<CopyToClipboard textToCopy={window.location.href} />
@@ -129,6 +133,7 @@ function AlertActionButtons({
<Tooltip title="More options">
<Button
type="text"
data-testid="alert-actions-menu"
icon={
<Ellipsis
size={16}

View File

@@ -47,21 +47,29 @@ function AlertHeader({ alertDetails }: AlertHeaderProps): JSX.Element {
<div className="alert-info__info-wrapper">
<div className="top-section">
<div className="alert-title-wrapper">
<AlertState state={alertRuleState ?? state ?? ''} />
<div className="alert-title">
<div data-testid="alert-header-state">
<AlertState state={alertRuleState ?? state ?? ''} />
</div>
<div className="alert-title" data-testid="alert-header-title">
<LineClampedText text={displayName || ''} />
</div>
</div>
</div>
<div className="bottom-section">
{labels?.severity && <AlertSeverity severity={labels.severity} />}
{labels?.severity && (
<div data-testid="alert-header-severity">
<AlertSeverity severity={labels.severity} />
</div>
)}
{/* // TODO(shaheer): Get actual data when we are able to get alert firing from state from API */}
{/* <AlertStatus
status="firing"
timestamp={dayjs().subtract(1, 'd').valueOf()}
/> */}
<AlertLabels labels={labelsWithoutSeverity} />
<div data-testid="alert-header-labels">
<AlertLabels labels={labelsWithoutSeverity} />
</div>
</div>
</div>
);

View File

@@ -1,11 +1,12 @@
import { Color } from '@signozhq/design-tokens';
import { useIsDarkMode } from 'hooks/useDarkMode';
import { BellOff, CircleCheck, CircleOff, Flame } from '@signozhq/icons';
import { RuletypesAlertStateDTO } from 'api/generated/services/sigNoz.schemas';
import './AlertState.styles.scss';
type AlertStateProps = {
state: string;
state: RuletypesAlertStateDTO | string;
showLabel?: boolean;
};
@@ -17,7 +18,7 @@ export default function AlertState({
let label;
const isDarkMode = useIsDarkMode();
switch (state) {
case 'nodata':
case RuletypesAlertStateDTO.nodata:
icon = (
<CircleOff
size={18}
@@ -28,7 +29,7 @@ export default function AlertState({
label = <span style={{ color: Color.BG_SIENNA_400 }}>No Data</span>;
break;
case 'disabled':
case RuletypesAlertStateDTO.disabled:
icon = (
<BellOff
size={18}
@@ -38,15 +39,16 @@ export default function AlertState({
);
label = <span style={{ color: Color.BG_VANILLA_400 }}>Muted</span>;
break;
case 'firing':
case RuletypesAlertStateDTO.firing:
icon = (
<Flame size={18} fill={Color.BG_CHERRY_500} color={Color.BG_CHERRY_500} />
);
label = <span style={{ color: Color.BG_CHERRY_500 }}>Firing</span>;
break;
case 'normal':
case 'inactive':
case RuletypesAlertStateDTO.inactive:
case 'normal': // legacy
icon = (
<CircleCheck
size={18}

View File

@@ -1,28 +1,34 @@
import { useCallback, useMemo } from 'react';
import { useMutation, useQuery, useQueryClient } from 'react-query';
import { generatePath, useLocation } from 'react-router-dom';
import { useCallback, useEffect, useMemo, useRef } from 'react';
import { useMutation, useQueryClient, useQuery } from 'react-query';
import { generatePath } from 'react-router-dom';
import { TablePaginationConfig, TableProps } from 'antd';
import type { FilterValue, SorterResult } from 'antd/es/table/interface';
import { patchRulePartial } from 'api/alerts/patchRulePartial';
import ruleStats from 'api/alerts/ruleStats';
import timelineGraph from 'api/alerts/timelineGraph';
import timelineTable from 'api/alerts/timelineTable';
import topContributors from 'api/alerts/topContributors';
import { convertToApiError } from 'api/ErrorResponseHandlerForGeneratedAPIs';
import {
createRule,
deleteRuleByID,
getGetRuleByIDQueryKey,
getGetRuleHistoryTimelineQueryOptions,
invalidateGetRuleByID,
invalidateListRules,
updateRuleByID,
useGetRuleByID,
useGetRuleHistoryOverallStatus,
useGetRuleHistoryStats,
useGetRuleHistoryTopContributors,
useListRules,
} from 'api/generated/services/rules';
import type {
GetRuleByID200,
RenderErrorResponseDTO,
RuletypesPostableRuleDTO,
import {
Querybuildertypesv5OrderDirectionDTO,
RuletypesAlertStateDTO,
type GetRuleByID200,
type GetRuleHistoryOverallStatus200,
type GetRuleHistoryStats200,
type GetRuleHistoryTimeline200,
type GetRuleHistoryTopContributors200,
type RenderErrorResponseDTO,
type RuletypesPostableRuleDTO,
} from 'api/generated/services/sigNoz.schemas';
import { AxiosError } from 'axios';
import { TabRoutes } from 'components/RouteTab/types';
@@ -31,35 +37,27 @@ import { REACT_QUERY_KEY } from 'constants/reactQueryKeys';
import ROUTES from 'constants/routes';
import AlertHistory from 'container/AlertHistory';
import { TIMELINE_TABLE_PAGE_SIZE } from 'container/AlertHistory/constants';
import {
computeCursorForPage,
useTimelineTableOrder,
useTimelineTablePage,
} from 'container/AlertHistory/Timeline/Table/useTimelineTableCursor';
import { AlertDetailsTab, TimelineFilter } from 'container/AlertHistory/types';
import { urlKey } from 'container/AllError/utils';
import { DEFAULT_TIME_RANGE } from 'container/TopNav/DateTimeSelectionV2/constants';
import { useNotifications } from 'hooks/useNotifications';
import { useSafeNavigate } from 'hooks/useSafeNavigate';
import useUrlQuery from 'hooks/useUrlQuery';
import createQueryParams from 'lib/createQueryParams';
import GetMinMax from 'lib/getMinMax';
import history from 'lib/history';
import { History, Table } from '@signozhq/icons';
import EditRules from 'pages/EditRules';
import { OrderPreferenceItems } from 'pages/Logs/config';
import BetaTag from 'periscope/components/BetaTag/BetaTag';
import PaginationInfoText from 'periscope/components/PaginationInfoText/PaginationInfoText';
import { useAlertRule } from 'providers/Alert';
import { useErrorModal } from 'providers/ErrorModalProvider';
import { ErrorResponse, SuccessResponse } from 'types/api';
import { toPostableRuleDTOFromAlertDef } from 'types/api/alerts/convert';
import {
AlertDef,
AlertRuleStatsPayload,
AlertRuleTimelineGraphResponsePayload,
AlertRuleTimelineTableResponse,
AlertRuleTimelineTableResponsePayload,
AlertRuleTopContributorsPayload,
} from 'types/api/alerts/def';
import { AlertDef, AlertRuleTimelineTableResponse } from 'types/api/alerts/def';
import APIError from 'types/api/error';
import { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
import { nanoToMilli } from 'utils/timeUtils';
import { Typography } from '@signozhq/ui/typography';
export const useAlertHistoryQueryParams = (): {
ruleId: string | null;
@@ -129,7 +127,7 @@ export const useRouteTabUtils = (): { routes: TabRoutes[] } => {
{
Component: EditRules,
name: (
<div className="tab-item">
<div className="tab-item" data-testid="alert-details-tab-overview">
<Table size={14} />
Overview
</div>
@@ -140,7 +138,7 @@ export const useRouteTabUtils = (): { routes: TabRoutes[] } => {
{
Component: AlertHistory,
name: (
<div className="tab-item">
<div className="tab-item" data-testid="alert-details-tab-history">
<History size={14} />
History
<BetaTag />
@@ -201,10 +199,7 @@ type GetAlertRuleDetailsApiProps = {
};
type GetAlertRuleDetailsStatsProps = GetAlertRuleDetailsApiProps & {
data:
| SuccessResponse<AlertRuleStatsPayload, unknown>
| ErrorResponse
| undefined;
data: GetRuleHistoryStats200 | undefined;
};
export const useGetAlertRuleDetailsStats =
@@ -213,18 +208,15 @@ export const useGetAlertRuleDetailsStats =
const isValidRuleId = ruleId !== null && String(ruleId).length !== 0;
const { isLoading, isRefetching, isError, data } = useQuery(
[REACT_QUERY_KEY.ALERT_RULE_STATS, ruleId, startTime, endTime],
const { isLoading, isRefetching, isError, data } = useGetRuleHistoryStats(
{ id: ruleId || '' },
{ start: startTime, end: endTime },
{
queryFn: () =>
ruleStats({
id: ruleId || '',
start: startTime,
end: endTime,
}),
enabled: isValidRuleId && !!startTime && !!endTime,
refetchOnMount: false,
refetchOnWindowFocus: false,
query: {
enabled: isValidRuleId && !!startTime && !!endTime,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
},
);
@@ -232,10 +224,7 @@ export const useGetAlertRuleDetailsStats =
};
type GetAlertRuleDetailsTopContributorsProps = GetAlertRuleDetailsApiProps & {
data:
| SuccessResponse<AlertRuleTopContributorsPayload, unknown>
| ErrorResponse
| undefined;
data: GetRuleHistoryTopContributors200 | undefined;
};
export const useGetAlertRuleDetailsTopContributors =
@@ -244,90 +233,128 @@ export const useGetAlertRuleDetailsTopContributors =
const isValidRuleId = ruleId !== null && String(ruleId).length !== 0;
const { isLoading, isRefetching, isError, data } = useQuery(
[REACT_QUERY_KEY.ALERT_RULE_TOP_CONTRIBUTORS, ruleId, startTime, endTime],
{
queryFn: () =>
topContributors({
id: ruleId || '',
start: startTime,
end: endTime,
}),
enabled: isValidRuleId,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
);
const { isLoading, isRefetching, isError, data } =
useGetRuleHistoryTopContributors(
{ id: ruleId || '' },
{ start: startTime, end: endTime },
{
query: {
enabled: isValidRuleId && !!startTime && !!endTime,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
},
);
return { isLoading, isRefetching, isError, data, isValidRuleId, ruleId };
};
type GetAlertRuleDetailsTimelineTableProps = GetAlertRuleDetailsApiProps & {
data:
| SuccessResponse<AlertRuleTimelineTableResponsePayload, unknown>
| ErrorResponse
| undefined;
data: GetRuleHistoryTimeline200 | undefined;
error: AxiosError<RenderErrorResponseDTO> | null;
refetch: () => void;
cancel: () => void;
};
export const useGetAlertRuleDetailsTimelineTable = ({
filters,
filterExpression,
}: {
filters: TagFilter;
filterExpression: string;
}): GetAlertRuleDetailsTimelineTableProps => {
const queryClient = useQueryClient();
const { ruleId, startTime, endTime, params } = useAlertHistoryQueryParams();
const { updatedOrder, offset } = useMemo(
() => ({
updatedOrder: params.get(urlKey.order) ?? OrderPreferenceItems.ASC,
offset: parseInt(params.get(urlKey.offset) ?? '0', 10),
}),
[params],
const [page, setPage] = useTimelineTablePage();
const [order] = useTimelineTableOrder();
const updatedOrder = useMemo(
() =>
order === 'asc'
? Querybuildertypesv5OrderDirectionDTO.asc
: Querybuildertypesv5OrderDirectionDTO.desc,
[order],
);
const timelineFilter = params.get('timelineFilter');
const isValidRuleId = ruleId !== null && String(ruleId).length !== 0;
const hasStartAndEnd = startTime !== null && endTime !== null;
const { isLoading, isRefetching, isError, data } = useQuery(
[
REACT_QUERY_KEY.ALERT_RULE_TIMELINE_TABLE,
ruleId,
startTime,
endTime,
timelineFilter,
updatedOrder,
offset,
JSON.stringify(filters.items),
],
const stateFilter = useMemo(() => {
if (!timelineFilter || timelineFilter === TimelineFilter.ALL) {
return undefined;
}
return timelineFilter === TimelineFilter.FIRED
? RuletypesAlertStateDTO.firing
: RuletypesAlertStateDTO.inactive;
}, [timelineFilter]);
const filtersKey = `${filterExpression}|${stateFilter ?? ''}|${startTime}|${endTime}`;
const prevFiltersKeyRef = useRef(filtersKey);
const filtersChanged = prevFiltersKeyRef.current !== filtersKey;
const cursor = computeCursorForPage(filtersChanged ? 1 : page);
useEffect(() => {
if (prevFiltersKeyRef.current !== filtersKey) {
prevFiltersKeyRef.current = filtersKey;
if (page > 1) {
void setPage(1);
}
}
}, [filtersKey, page, setPage]);
const queryParams = useMemo(
() => ({
start: startTime,
end: endTime,
limit: TIMELINE_TABLE_PAGE_SIZE,
order: updatedOrder,
cursor,
filterExpression: filterExpression || undefined,
state: stateFilter,
}),
[startTime, endTime, updatedOrder, cursor, filterExpression, stateFilter],
);
const queryOptions = getGetRuleHistoryTimelineQueryOptions(
{ id: ruleId || '' },
queryParams,
{
queryFn: () =>
timelineTable({
id: ruleId || '',
start: startTime,
end: endTime,
limit: TIMELINE_TABLE_PAGE_SIZE,
order: updatedOrder,
offset,
filters,
...(timelineFilter && timelineFilter !== TimelineFilter.ALL
? {
state: timelineFilter === TimelineFilter.FIRED ? 'firing' : 'normal',
}
: {}),
}),
enabled: isValidRuleId && hasStartAndEnd,
refetchOnMount: false,
refetchOnWindowFocus: false,
query: {
enabled: isValidRuleId,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
},
);
return { isLoading, isRefetching, isError, data, isValidRuleId, ruleId };
const { isLoading, isRefetching, isError, data, error, refetch } =
useQuery(queryOptions);
const queryKeyRef = useRef(queryOptions.queryKey);
queryKeyRef.current = queryOptions.queryKey;
const cancel = useCallback(() => {
void queryClient.cancelQueries({ queryKey: queryKeyRef.current });
}, [queryClient]);
return {
isLoading,
isRefetching,
isError,
data,
error: error as AxiosError<RenderErrorResponseDTO> | null,
isValidRuleId,
ruleId,
refetch,
cancel,
};
};
export const useTimelineTable = ({
totalItems,
nextCursor,
}: {
totalItems: number;
nextCursor?: string;
}): {
paginationConfig: TablePaginationConfig;
onChangeHandler: (
@@ -336,16 +363,13 @@ export const useTimelineTable = ({
filters: any,
extra: any,
) => void;
handleNextPage: () => void;
handlePrevPage: () => void;
hasNextPage: boolean;
hasPrevPage: boolean;
} => {
const { safeNavigate } = useSafeNavigate();
const { pathname } = useLocation();
const { search } = useLocation();
const params = useMemo(() => new URLSearchParams(search), [search]);
const offset = params.get('offset') ?? '0';
const [page, setPage] = useTimelineTablePage();
const [, setOrder] = useTimelineTableOrder();
const onChangeHandler: TableProps<AlertRuleTimelineTableResponse>['onChange'] =
useCallback(
@@ -357,38 +381,52 @@ export const useTimelineTable = ({
| SorterResult<AlertRuleTimelineTableResponse>,
) => {
if (!Array.isArray(sorter)) {
const { pageSize = 0, current = 0 } = pagination;
const { order } = sorter;
const updatedOrder = order === 'ascend' ? 'asc' : 'desc';
const params = new URLSearchParams(window.location.search);
safeNavigate(
`${pathname}?${createQueryParams({
...Object.fromEntries(params),
order: updatedOrder,
offset: current * TIMELINE_TABLE_PAGE_SIZE - TIMELINE_TABLE_PAGE_SIZE,
pageSize,
})}`,
);
void Promise.all([setOrder(updatedOrder), setPage(1)]);
}
},
[pathname, safeNavigate],
[setOrder, setPage],
);
const offsetInt = parseInt(offset, 10);
const pageSize = params.get('pageSize') ?? String(TIMELINE_TABLE_PAGE_SIZE);
const pageSizeInt = parseInt(pageSize, 10);
const handleNextPage = useCallback(() => {
if (!nextCursor) {
return;
}
void setPage(page + 1);
}, [nextCursor, page, setPage]);
const handlePrevPage = useCallback(() => {
if (page <= 1) {
return;
}
void setPage(page - 1);
}, [page, setPage]);
const paginationConfig: TablePaginationConfig = {
pageSize: pageSizeInt,
showTotal: PaginationInfoText,
current: offsetInt / TIMELINE_TABLE_PAGE_SIZE + 1,
pageSize: TIMELINE_TABLE_PAGE_SIZE,
showTotal: (total, [start, end]) => (
<span>
<Typography.Text size="small">
{start} &#8212; {end}
</Typography.Text>
<Typography.Text size="small"> of {total}</Typography.Text>
</span>
),
current: page,
showSizeChanger: false,
hideOnSinglePage: true,
total: totalItems,
};
return { paginationConfig, onChangeHandler };
return {
paginationConfig,
onChangeHandler,
handleNextPage,
handlePrevPage,
hasNextPage: !!nextCursor,
hasPrevPage: page > 1,
};
};
export const useAlertRuleStatusToggle = ({
@@ -581,10 +619,7 @@ export const useAlertRuleDelete = ({
};
type GetAlertRuleDetailsTimelineGraphProps = GetAlertRuleDetailsApiProps & {
data:
| SuccessResponse<AlertRuleTimelineGraphResponsePayload, unknown>
| ErrorResponse
| undefined;
data: GetRuleHistoryOverallStatus200 | undefined;
};
export const useGetAlertRuleDetailsTimelineGraphData =
@@ -594,20 +629,18 @@ export const useGetAlertRuleDetailsTimelineGraphData =
const isValidRuleId = ruleId !== null && String(ruleId).length !== 0;
const hasStartAndEnd = startTime !== null && endTime !== null;
const { isLoading, isRefetching, isError, data } = useQuery(
[REACT_QUERY_KEY.ALERT_RULE_TIMELINE_GRAPH, ruleId, startTime, endTime],
{
queryFn: () =>
timelineGraph({
id: ruleId || '',
start: startTime,
end: endTime,
}),
enabled: isValidRuleId && hasStartAndEnd,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
);
const { isLoading, isRefetching, isError, data } =
useGetRuleHistoryOverallStatus(
{ id: ruleId || '' },
{ start: startTime, end: endTime },
{
query: {
enabled: isValidRuleId && hasStartAndEnd,
refetchOnMount: false,
refetchOnWindowFocus: false,
},
},
);
return { isLoading, isRefetching, isError, data, isValidRuleId, ruleId };
};

View File

@@ -13,6 +13,8 @@ interface Tab {
disabled?: boolean;
icon?: string | JSX.Element;
isBeta?: boolean;
/** Optional `data-testid` for the tab button. */
testId?: string;
}
interface TimelineTabsProps {
@@ -63,6 +65,7 @@ function Tabs2({
disabled={tab.disabled}
icon={tab.icon}
style={{ minWidth: buttonMinWidth }}
data-testid={tab.testId}
>
{tab.label}

View File

@@ -1,4 +1,4 @@
import { AlertLabelsProps } from 'pages/AlertDetails/AlertHeader/AlertLabels/AlertLabels';
import { RuletypesAlertStateDTO } from 'api/generated/services/sigNoz.schemas';
import { ICompositeMetricQuery } from 'types/api/alerts/compositeQuery';
// default match type for threshold
@@ -73,10 +73,6 @@ export interface StatsTimeSeriesItem {
value: string;
}
export type AlertRuleStatsPayload = {
data: AlertRuleStats;
};
export interface AlertRuleTopContributors {
fingerprint: number;
labels: Labels;
@@ -84,9 +80,6 @@ export interface AlertRuleTopContributors {
relatedLogsLink: string;
relatedTracesLink: string;
}
export type AlertRuleTopContributorsPayload = {
data: AlertRuleTopContributors[];
};
export interface AlertRuleTimelineTableResponse {
ruleID: string;
@@ -99,24 +92,12 @@ export interface AlertRuleTimelineTableResponse {
labels: Labels;
fingerprint: number;
value: number;
relatedTracesLink: string;
relatedLogsLink: string;
relatedLogsLink?: string;
relatedTracesLink?: string;
}
export type AlertRuleTimelineTableResponsePayload = {
data: {
items: AlertRuleTimelineTableResponse[];
total: number;
labels: AlertLabelsProps['labels'];
};
};
type AlertState = 'firing' | 'normal' | 'nodata' | 'muted';
export interface AlertRuleTimelineGraphResponse {
start: number;
end: number;
state: AlertState;
state: RuletypesAlertStateDTO;
}
export type AlertRuleTimelineGraphResponsePayload = {
data: AlertRuleTimelineGraphResponse[];
};

View File

@@ -1,7 +0,0 @@
import { AlertDef } from './def';
export interface RuleStatsProps {
id: AlertDef['id'];
start: number;
end: number;
}

View File

@@ -1,7 +0,0 @@
import { AlertDef } from './def';
export interface GetTimelineGraphRequestProps {
id: AlertDef['id'];
start: number;
end: number;
}

View File

@@ -1,13 +0,0 @@
import { TagFilter } from '../queryBuilder/queryBuilderData';
import { AlertDef } from './def';
export interface GetTimelineTableRequestProps {
id: AlertDef['id'];
start: number;
end: number;
offset: number;
limit: number;
order: string;
filters?: TagFilter;
state?: string;
}

View File

@@ -1,7 +0,0 @@
import { AlertDef } from './def';
export interface TopContributorsProps {
id: AlertDef['id'];
start: number;
end: number;
}

View File

@@ -143,6 +143,12 @@ export default defineConfig(({ mode }): UserConfig => {
plugins,
resolve: {
alias: {
// @grafana/data imports bare CJS `lodash`, whose UMD footer checks for an
// AMD loader before assigning module.exports. Any third-party script that
// defines window.define.amd first leaves the bundled namespace empty, so
// every lodash method reached through it is undefined at runtime. lodash-es
// is the same version, real ESM, and tree-shakes.
lodash: 'lodash-es',
'@': resolve(__dirname, './src'),
utils: resolve(__dirname, './src/utils'),
types: resolve(__dirname, './src/types'),

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24px" height="24px" viewBox="0 0 24 24"><defs><style>.cls-1{fill:#aecbfa;}.cls-2{fill:#669df6;}.cls-3{fill:#4285f4;}</style></defs><title>Icon_24px_ComputeEngine_Color</title><g data-name="Product Icons"><rect class="cls-1" x="9" y="9" width="6" height="6"/><rect class="cls-2" x="11" y="2" width="2" height="4"/><rect class="cls-2" x="7" y="2" width="2" height="4"/><rect class="cls-2" x="15" y="2" width="2" height="4"/><rect class="cls-3" x="11" y="18" width="2" height="4"/><rect class="cls-3" x="7" y="18" width="2" height="4"/><rect class="cls-3" x="15" y="18" width="2" height="4"/><rect class="cls-3" x="19" y="10" width="2" height="4" transform="translate(8 32) rotate(-90)"/><rect class="cls-3" x="19" y="14" width="2" height="4" transform="translate(4 36) rotate(-90)"/><rect class="cls-3" x="19" y="6" width="2" height="4" transform="translate(12 28) rotate(-90)"/><rect class="cls-2" x="3" y="10" width="2" height="4" transform="translate(-8 16) rotate(-90)"/><rect class="cls-2" x="3" y="14" width="2" height="4" transform="translate(-12 20) rotate(-90)"/><rect class="cls-2" x="3" y="6" width="2" height="4" transform="translate(-4 12) rotate(-90)"/><path class="cls-1" d="M5,5V19H19V5ZM17,17H7V7H17Z"/><polygon class="cls-2" points="9 15 15 15 12 12 9 15"/><polygon class="cls-3" points="12 12 15 15 15 9 12 12"/></g></svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

View File

@@ -0,0 +1,124 @@
{
"id": "computeengine",
"title": "GCP Compute Engine",
"icon": "file://icon.svg",
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": true
},
"dataCollected": {
"metrics": [
{
"name": "compute.googleapis.com/instance/uptime_total",
"unit": "Seconds",
"type": "Gauge",
"description": ""
},
{
"name": "compute.googleapis.com/instance/cpu/utilization",
"unit": "Percent",
"type": "Gauge",
"description": ""
},
{
"name": "compute.googleapis.com/guest/memory/bytes_used",
"unit": "Bytes",
"type": "Gauge",
"description": ""
},
{
"name": "compute.googleapis.com/guest/memory/percent_used",
"unit": "Percent",
"type": "Gauge",
"description": ""
},
{
"name": "agent.googleapis.com/memory/percent_used",
"unit": "Percent",
"type": "Gauge",
"description": ""
},
{
"name": "compute.googleapis.com/instance/disk/average_io_latency",
"unit": "Microseconds",
"type": "Gauge",
"description": ""
},
{
"name": "compute.googleapis.com/instance/disk/read_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/disk/write_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/disk/read_ops_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/disk/write_ops_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/disk/performance_status",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "compute.googleapis.com/instance/network/received_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/network/sent_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/network/received_packets_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/instance/network/sent_packets_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "compute.googleapis.com/firewall/dropped_packets_count",
"unit": "Count",
"type": "Sum",
"description": ""
}
],
"logs": []
},
"telemetryCollectionStrategy": {
"gcp": {}
},
"assets": {
"dashboards": [
{
"id": "overview",
"title": "GCP Compute Engine Overview",
"description": "Overview of GCP Compute Engine metrics",
"definition": "file://assets/dashboards/overview.json"
}
]
}
}

View File

@@ -0,0 +1,3 @@
### Monitor GCP Compute Engine with SigNoz
Collect key GCP Compute Engine metrics and view them with an out of the box dashboard.

View File

@@ -5,14 +5,14 @@ import (
"fmt"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (m *module) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
metadataTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName)
metadataTable := fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName)
var (
systemMetricCount uint64
k8sMetricCount uint64

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -281,7 +281,7 @@ func (m *module) getPerGroupContainerStatusCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
// Built once; identical across the two fps CTEs (buildFilterClause hits the
// metadata store + parses the expression). AddWhereClause only reads it.
@@ -310,7 +310,7 @@ func (m *module) getPerGroupContainerStatusCounts(
)
}
stateFps.Select(stateFpsCols...)
stateFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
stateFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
stateFps.Where(
stateFps.E("metric_name", containerStatusStateMetricName),
stateFps.GE("unix_milli", tsAdjustedStart),
@@ -342,7 +342,7 @@ func (m *module) getPerGroupContainerStatusCounts(
containerState.Select(containerStateCols...)
containerState.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN state_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerState.Where(
containerState.E("samples.metric_name", containerStatusStateMetricName),
@@ -361,7 +361,7 @@ func (m *module) getPerGroupContainerStatusCounts(
fmt.Sprintf("JSONExtractString(labels, %s) AS container_name", reasonFps.Var(containerNameAttrKey)),
fmt.Sprintf("JSONExtractString(labels, %s) AS reason", reasonFps.Var(containerStatusReasonAttrKey)),
)
reasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
reasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
reasonFps.Where(
reasonFps.E("metric_name", containerStatusReasonMetricName),
reasonFps.GE("unix_milli", tsAdjustedStart),
@@ -391,7 +391,7 @@ func (m *module) getPerGroupContainerStatusCounts(
)
reasonInner.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
reasonInner.Where(
reasonInner.E("samples.metric_name", containerStatusReasonMetricName),
@@ -546,7 +546,7 @@ func (m *module) getPerGroupContainerRestartCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
var (
filterClause *sqlbuilder.WhereClause
@@ -572,7 +572,7 @@ func (m *module) getPerGroupContainerRestartCounts(
)
}
restartFps.Select(restartFpsCols...)
restartFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
restartFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
restartFps.Where(
restartFps.E("metric_name", containerRestartsMetricName),
restartFps.GE("unix_milli", tsAdjustedStart),
@@ -602,7 +602,7 @@ func (m *module) getPerGroupContainerRestartCounts(
containerRestarts.Select(containerRestartsCols...)
containerRestarts.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN restart_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerRestarts.Where(
containerRestarts.E("samples.metric_name", containerRestartsMetricName),
@@ -690,7 +690,7 @@ func (m *module) getPerGroupContainerReadyCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
var (
filterClause *sqlbuilder.WhereClause
@@ -716,7 +716,7 @@ func (m *module) getPerGroupContainerReadyCounts(
)
}
readyFps.Select(readyFpsCols...)
readyFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
readyFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
readyFps.Where(
readyFps.E("metric_name", containerReadyMetricName),
readyFps.GE("unix_milli", tsAdjustedStart),
@@ -746,7 +746,7 @@ func (m *module) getPerGroupContainerReadyCounts(
containerReady.Select(containerReadyCols...)
containerReady.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN ready_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerReady.Where(
containerReady.E("samples.metric_name", containerReadyMetricName),

View File

@@ -9,7 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
@@ -344,11 +344,11 @@ func alignedMetricWindow(startMs, endMs int64) (
flooredEndMs = flooredEndMs - (flooredEndMs % (adjustStep * 1000))
}
tsAdjustedStartMs, _, distributedTSTable, localTSTable := telemetrymetrics.WhichTSTableToUse(
tsAdjustedStartMs, _, distributedTSTable, localTSTable := metricstelemetryschema.WhichTSTableToUse(
samplesAdjustedStartMs, flooredEndMs, false, nil,
)
distributedSamplesTable, localSamplesTable := telemetrymetrics.WhichSamplesTableToUse(
distributedSamplesTable, localSamplesTable := metricstelemetryschema.WhichSamplesTableToUse(
samplesAdjustedStartMs, flooredEndMs,
metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil,
)
@@ -362,7 +362,7 @@ func alignedMetricWindow(startMs, endMs int64) (
func (m *module) buildSamplesTblFingerprintSubQuery(metricNames []string, samplesTable string, flooredStart, flooredEnd uint64) *sqlbuilder.SelectBuilder {
fpSB := sqlbuilder.NewSelectBuilder()
fpSB.Select("DISTINCT fingerprint")
fpSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, samplesTable))
fpSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, samplesTable))
fpSB.Where(
fpSB.In("metric_name", sqlbuilder.List(metricNames)),
fpSB.GE("unix_milli", flooredStart),
@@ -376,7 +376,7 @@ func (m *module) buildSamplesTblFingerprintSubQuery(metricNames []string, sample
func (m *module) buildReducedSamplesTblFingerprintSubQuery(metricNames []string, flooredStart, flooredEnd uint64) *sqlbuilder.SelectBuilder {
lastSB := sqlbuilder.NewSelectBuilder()
lastSB.Select("reduced_fingerprint")
lastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
lastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
lastSB.Where(
lastSB.In("metric_name", sqlbuilder.List(metricNames)),
lastSB.GE("unix_milli", flooredStart),
@@ -385,7 +385,7 @@ func (m *module) buildReducedSamplesTblFingerprintSubQuery(metricNames []string,
sumSB := sqlbuilder.NewSelectBuilder()
sumSB.Select("reduced_fingerprint")
sumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
sumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
sumSB.Where(
sumSB.In("metric_name", sqlbuilder.List(metricNames)),
sumSB.GE("unix_milli", flooredStart),
@@ -478,7 +478,7 @@ func (m *module) getEarliestMetricTime(ctx context.Context, metricNames []string
sb := sqlbuilder.NewSelectBuilder()
sb.Select("min(first_reported_unix_milli) AS min_first_reported")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -504,7 +504,7 @@ func (m *module) getMetricsExistence(ctx context.Context, metricNames []string)
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", "count(*) AS cnt")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
sb.GroupBy("metric_name")
@@ -549,7 +549,7 @@ func (m *module) getAttributesExistence(ctx context.Context, metricNames, attrNa
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select("attr_name", "count(*) AS cnt")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.In("attr_name", sqlbuilder.List(attrNames)),
@@ -656,7 +656,7 @@ func (m *module) getMetadata(
rawSrc := sqlbuilder.NewSelectBuilder()
rawSrc.Select("labels", "unix_milli")
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTableName))
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTableName))
rawSrc.Where(
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
rawSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -669,7 +669,7 @@ func (m *module) getMetadata(
reducedSrc := sqlbuilder.NewSelectBuilder()
reducedSrc.Select("labels", "unix_milli")
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedSrc.Where(
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -683,7 +683,7 @@ func (m *module) getMetadata(
// Inner query reads over the union of raw + reduced series.
innerSB.From(innerSB.BuilderAs(sqlbuilder.UnionAll(rawSrc, reducedSrc), "series"))
} else {
innerSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTableName))
innerSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTableName))
innerSB.Where(
innerSB.In("metric_name", sqlbuilder.List(metricNames)),
innerSB.GE("unix_milli", tsAdjustedStartMs),
@@ -870,7 +870,7 @@ func (m *module) getPerGroupDistinctCounts(
rawSrc := sqlbuilder.NewSelectBuilder()
rawSrc.Select("labels")
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTbl))
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTbl))
rawSrc.Where(
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
rawSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -883,7 +883,7 @@ func (m *module) getPerGroupDistinctCounts(
reducedSrc := sqlbuilder.NewSelectBuilder()
reducedSrc.Select("labels")
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedSrc.Where(
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -896,7 +896,7 @@ func (m *module) getPerGroupDistinctCounts(
sb.From(sb.BuilderAs(sqlbuilder.UnionAll(rawSrc, reducedSrc), "series"))
} else {
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTbl))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTbl))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.GE("unix_milli", tsAdjustedStartMs),

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -72,7 +72,7 @@ func (m *module) getPerGroupHostStatusCounts(
rawSrc := sqlbuilder.NewSelectBuilder()
rawSrc.Select("labels")
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTableName))
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTableName))
rawSrc.Where(
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
rawSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -85,7 +85,7 @@ func (m *module) getPerGroupHostStatusCounts(
reducedSrc := sqlbuilder.NewSelectBuilder()
reducedSrc.Select("labels")
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedSrc.Where(
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
@@ -101,7 +101,7 @@ func (m *module) getPerGroupHostStatusCounts(
fpSB := m.buildSamplesTblFingerprintSubQuery(metricNames, localSamplesTable, samplesStartMs, flooredEndMs)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTableName))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.GE("unix_milli", tsAdjustedStartMs),
@@ -357,7 +357,7 @@ func (m *module) getActiveHostsQuery(metricNames []string, hostNameAttr string,
sb := sqlbuilder.NewSelectBuilder()
sb.Distinct()
sb.Select("attr_string_value")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(
sb.In("metric_name", sqlbuilder.List(metricNames)),
sb.E("attr_name", hostNameAttr),

View File

@@ -9,7 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/modules/inframonitoring"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
@@ -40,8 +40,8 @@ func NewModule(
providerSettings factory.ProviderSettings,
cfg inframonitoring.Config,
) inframonitoring.Module {
fieldMapper := telemetrymetrics.NewFieldMapper()
condBuilder := telemetrymetrics.NewConditionBuilder(fieldMapper)
fieldMapper := metricstelemetryschema.NewFieldMapper()
condBuilder := metricstelemetryschema.NewConditionBuilder(fieldMapper)
return &module{
telemetryStore: telemetryStore,
telemetryMetadataStore: telemetryMetadataStore,

View File

@@ -7,7 +7,7 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -186,7 +186,7 @@ func (m *module) getPerGroupNodeConditionCounts(
// Step-floor bounds + resolve tables in one shot to match QB v5 querier.
samplesStartMs, flooredEndMs, tsAdjustedStartMs, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
// ----- timeSeriesFPs -----
timeSeriesFPs := sqlbuilder.NewSelectBuilder()
@@ -200,7 +200,7 @@ func (m *module) getPerGroupNodeConditionCounts(
)
}
timeSeriesFPs.Select(timeSeriesFPsSelectCols...)
timeSeriesFPs.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
timeSeriesFPs.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
timeSeriesFPs.Where(
timeSeriesFPs.E("metric_name", nodeConditionMetricName),
timeSeriesFPs.GE("unix_milli", tsAdjustedStartMs),
@@ -237,7 +237,7 @@ func (m *module) getPerGroupNodeConditionCounts(
latestConditionPerNode.Select(latestConditionPerNodeSelectCols...)
latestConditionPerNode.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN time_series_fps AS tsfp ON samples.fingerprint = tsfp.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
latestConditionPerNode.Where(
latestConditionPerNode.E("samples.metric_name", nodeConditionMetricName),

View File

@@ -8,7 +8,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -284,7 +284,7 @@ func (m *module) getPerGroupPodStatusCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
// Build the merged filter clause once; it's identical across the three fps
// CTEs, and buildFilterClause hits the metadata store + parses the
@@ -313,7 +313,7 @@ func (m *module) getPerGroupPodStatusCounts(
)
}
phaseFps.Select(phaseFpsCols...)
phaseFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
phaseFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
phaseFps.Where(
phaseFps.E("metric_name", podPhaseMetricName),
phaseFps.GE("unix_milli", tsAdjustedStart),
@@ -340,7 +340,7 @@ func (m *module) getPerGroupPodStatusCounts(
phasePerPod.Select(phasePerPodCols...)
phasePerPod.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN phase_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
phasePerPod.Where(
phasePerPod.E("samples.metric_name", podPhaseMetricName),
@@ -357,7 +357,7 @@ func (m *module) getPerGroupPodStatusCounts(
"fingerprint",
fmt.Sprintf("JSONExtractString(labels, %s) AS pod_uid", podReasonFps.Var(podUIDAttrKey)),
)
podReasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
podReasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
podReasonFps.Where(
podReasonFps.E("metric_name", podStatusReasonMetricName),
podReasonFps.GE("unix_milli", tsAdjustedStart),
@@ -377,7 +377,7 @@ func (m *module) getPerGroupPodStatusCounts(
)
podReasonPerPod.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN pod_reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
podReasonPerPod.Where(
podReasonPerPod.E("samples.metric_name", podStatusReasonMetricName),
@@ -396,7 +396,7 @@ func (m *module) getPerGroupPodStatusCounts(
fmt.Sprintf("JSONExtractString(labels, %s) AS container_name", containerReasonFps.Var(containerNameAttrKey)),
fmt.Sprintf("JSONExtractString(labels, %s) AS reason", containerReasonFps.Var(containerStatusReasonAttrKey)),
)
containerReasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
containerReasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
containerReasonFps.Where(
containerReasonFps.E("metric_name", containerStatusReasonMetricName),
containerReasonFps.GE("unix_milli", tsAdjustedStart),
@@ -432,7 +432,7 @@ func (m *module) getPerGroupPodStatusCounts(
)
containerInner.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN container_reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerInner.Where(
containerInner.E("samples.metric_name", containerStatusReasonMetricName),
@@ -611,7 +611,7 @@ func (m *module) getPerGroupPodRestartCounts(
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
var (
filterClause *sqlbuilder.WhereClause
@@ -637,7 +637,7 @@ func (m *module) getPerGroupPodRestartCounts(
)
}
restartFps.Select(restartFpsCols...)
restartFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
restartFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
restartFps.Where(
restartFps.E("metric_name", containerRestartsMetricName),
restartFps.GE("unix_milli", tsAdjustedStart),
@@ -667,7 +667,7 @@ func (m *module) getPerGroupPodRestartCounts(
containerRestarts.Select(containerRestartsCols...)
containerRestarts.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN restart_fps AS fps ON samples.fingerprint = fps.fingerprint",
telemetrymetrics.DBName, distributedSamplesTable,
metricstelemetryschema.DBName, distributedSamplesTable,
))
containerRestarts.Where(
containerRestarts.E("samples.metric_name", containerRestartsMetricName),

View File

@@ -19,8 +19,8 @@ import (
"github.com/SigNoz/signoz/pkg/modules/dashboard"
"github.com/SigNoz/signoz/pkg/modules/metricsexplorer"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
@@ -49,8 +49,8 @@ type module struct {
// NewModule constructs the metrics module with the provided dependencies.
func NewModule(ts telemetrystore.TelemetryStore, telemetryMetadataStore telemetrytypes.MetadataStore, cache cache.Cache, ruleStore ruletypes.RuleStore, dashboardModule dashboard.Module, fl flagger.Flagger, providerSettings factory.ProviderSettings, cfg metricsexplorer.Config) metricsexplorer.Module {
fieldMapper := telemetrymetrics.NewFieldMapper()
condBuilder := telemetrymetrics.NewConditionBuilder(fieldMapper)
fieldMapper := metricstelemetryschema.NewFieldMapper()
condBuilder := metricstelemetryschema.NewConditionBuilder(fieldMapper)
return &module{
telemetryStore: ts,
fieldMapper: fieldMapper,
@@ -89,7 +89,7 @@ func (m *module) listMeterMetrics(ctx context.Context, params *metricsexplorerty
"argMax(temporality, unix_milli) AS temporality",
"argMax(is_monotonic, unix_milli) AS is_monotonic",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymeter.DBName, telemetrymeter.SamplesTableName))
sb.From(fmt.Sprintf("%s.%s", metertelemetryschema.DBName, metertelemetryschema.SamplesTableName))
if params.Start != nil && params.End != nil {
sb.Where(sb.Between("unix_milli", *params.Start, *params.End))
@@ -160,11 +160,11 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
hasRange := params.Start != nil && params.End != nil
if hasRange {
var distributedTsTable string
rangeStart, rangeEnd, distributedTsTable, _ = telemetrymetrics.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
rawSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
rangeStart, rangeEnd, distributedTsTable, _ = metricstelemetryschema.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
rawSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
rawSB.Where(rawSB.Between("unix_milli", rangeStart, rangeEnd))
} else {
rawSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
rawSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
}
applyListFilters(rawSB)
@@ -174,7 +174,7 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
if reductionEnabled {
reducedSB := sqlbuilder.NewSelectBuilder()
reducedSB.Select("DISTINCT metric_name")
reducedSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
if hasRange {
reducedSB.Where(reducedSB.Between("unix_milli", rangeStart, rangeEnd))
}
@@ -512,7 +512,7 @@ func (m *module) CheckMetricExists(ctx context.Context, orgID valuer.UUID, metri
sb := sqlbuilder.NewSelectBuilder()
sb.Select("count(*) > 0 as metricExists")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -534,7 +534,7 @@ func (m *module) HasNonSigNozMetrics(ctx context.Context) (bool, error) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("count(*) > 0")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
sb.Where("metric_name NOT LIKE 'signoz_%'")
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -570,10 +570,10 @@ func (m *module) InspectMetrics(
return nil, err
}
tsStart, _, tsTable, _ := telemetrymetrics.WhichTSTableToUse(start, end, false, nil)
tsStart, _, tsTable, _ := metricstelemetryschema.WhichTSTableToUse(start, end, false, nil)
tsSb := sqlbuilder.NewSelectBuilder()
tsSb.Select("fingerprint", "labels")
tsSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, tsTable))
tsSb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, tsTable))
tsSb.Where(
tsSb.E("metric_name", req.MetricName),
tsSb.GE("unix_milli", tsStart),
@@ -625,7 +625,7 @@ func (m *module) InspectMetrics(
samplesSb := sqlbuilder.NewSelectBuilder()
samplesSb.Select("fingerprint", "unix_milli", "value")
samplesSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName))
samplesSb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName))
samplesSb.Where(
samplesSb.In("fingerprint", fingerprints...),
samplesSb.E("metric_name", req.MetricName),
@@ -717,7 +717,7 @@ func (m *module) fetchUpdatedMetadata(ctx context.Context, orgID valuer.UUID, me
"argMax(temporality, created_at) AS temporality",
"argMax(is_monotonic, created_at) AS is_monotonic",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.UpdatedMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.UpdatedMetadataTableName))
sb.Where(sb.In("metric_name", args...))
sb.GroupBy("metric_name")
@@ -777,7 +777,7 @@ func (m *module) fetchTimeseriesMetadata(ctx context.Context, orgID valuer.UUID,
"anyLast(temporality) AS temporality",
"argMax(is_monotonic, unix_milli) AS is_monotonic",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4TableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4TableName))
sb.Where(sb.In("metric_name", args...))
sb.GroupBy("metric_name")
@@ -891,7 +891,7 @@ func (m *module) checkForLabelInMetric(ctx context.Context, metricName string, l
sb := sqlbuilder.NewSelectBuilder()
sb.Select("count(*) > 0 AS has_label")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
sb.Where(sb.E("attr_name", label))
sb.Limit(1)
@@ -914,7 +914,7 @@ func (m *module) insertMetricsMetadata(ctx context.Context, orgID valuer.UUID, r
createdAt := time.Now().UnixMilli()
ib := sqlbuilder.NewInsertBuilder()
ib.InsertInto(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.UpdatedMetadataTableName))
ib.InsertInto(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.UpdatedMetadataTableName))
ib.Cols("metric_name", "temporality", "is_monotonic", "type", "description", "unit", "created_at")
ib.Values(
req.MetricName,
@@ -1016,9 +1016,9 @@ func (m *module) fetchMetricsStatsWithSamples(
}
}
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
start, end, distributedTsTable, localTsTable := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := metricstelemetryschema.CountExpressionForSamplesTable(distributedSamplesTable)
// Timeseries counts per metric
tsSB := sqlbuilder.NewSelectBuilder()
@@ -1026,7 +1026,7 @@ func (m *module) fetchMetricsStatsWithSamples(
"metric_name",
"uniq(fingerprint) AS timeseries",
)
tsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
tsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
tsSB.Where(tsSB.Between("unix_milli", start, end))
tsSB.Where("NOT startsWith(metric_name, 'signoz')")
if filterWhereClause != nil {
@@ -1040,7 +1040,7 @@ func (m *module) fetchMetricsStatsWithSamples(
"metric_name",
fmt.Sprintf("%s AS samples", countExp),
)
samplesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
samplesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
samplesSB.Where(samplesSB.Between("unix_milli", req.Start, req.End))
samplesSB.Where("NOT startsWith(metric_name, 'signoz')")
@@ -1053,7 +1053,7 @@ func (m *module) fetchMetricsStatsWithSamples(
if filterWhereClause != nil {
fingerprintSB := sqlbuilder.NewSelectBuilder()
fingerprintSB.Select("fingerprint")
fingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
fingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
fingerprintSB.Where(fingerprintSB.Between("unix_milli", start, end))
fingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
fingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1064,7 +1064,7 @@ func (m *module) fetchMetricsStatsWithSamples(
} else {
metricNamesSB := sqlbuilder.NewSelectBuilder()
metricNamesSB.Select("DISTINCT metric_name")
metricNamesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
metricNamesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
metricNamesSB.Where(metricNamesSB.Between("unix_milli", start, end))
metricNamesSB.Where("NOT startsWith(metric_name, 'signoz')")
@@ -1090,18 +1090,18 @@ func (m *module) fetchMetricsStatsWithSamples(
// the data lands either of the _reduced_last/_reduced_sum tables
reducedLastSB := sqlbuilder.NewSelectBuilder()
reducedLastSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
reducedLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
reducedLastSB.Where(reducedLastSB.Between("unix_milli", req.Start, req.End))
reducedLastSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedSumSB := sqlbuilder.NewSelectBuilder()
reducedSumSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
reducedSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
reducedSumSB.Where(reducedSumSB.Between("unix_milli", req.Start, req.End))
reducedSumSB.Where("NOT startsWith(metric_name, 'signoz')")
// separate query for reduced series counts
reducedTsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedTsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedTsSB.Where(reducedTsSB.Between("unix_milli", start, end))
reducedTsSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedTsSB.GroupBy("metric_name")
@@ -1112,7 +1112,7 @@ func (m *module) fetchMetricsStatsWithSamples(
// samples uses a separate cte with local table
reducedFpSB := sqlbuilder.NewSelectBuilder()
reducedFpSB.Select("fingerprint")
reducedFpSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedLocalTableName))
reducedFpSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
reducedFpSB.Where(reducedFpSB.Between("unix_milli", start, end))
reducedFpSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedFpSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1220,11 +1220,11 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
}
}
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
start, end, distributedTsTable, _ := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
totalTSBuilder := sqlbuilder.NewSelectBuilder()
totalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
totalTSBuilder.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
totalTSBuilder.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
totalTSBuilder.Where(totalTSBuilder.Between("unix_milli", start, end))
metricsSB := sqlbuilder.NewSelectBuilder()
@@ -1232,7 +1232,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
"metric_name",
"uniq(fingerprint) AS total_value",
)
metricsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
metricsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
metricsSB.Where(metricsSB.Between("unix_milli", start, end))
metricsSB.Where("NOT startsWith(metric_name, 'signoz')")
if filterWhereClause != nil {
@@ -1244,7 +1244,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
if reductionEnabled {
reducedTotalTSBuilder := sqlbuilder.NewSelectBuilder()
reducedTotalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
reducedTotalTSBuilder.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedTotalTSBuilder.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedTotalTSBuilder.Where(reducedTotalTSBuilder.Between("unix_milli", start, end))
reducedMetricsSB := sqlbuilder.NewSelectBuilder()
@@ -1252,7 +1252,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
"metric_name",
"uniq(fingerprint) AS total_value",
)
reducedMetricsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedMetricsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedMetricsSB.Where(reducedMetricsSB.Between("unix_milli", start, end))
reducedMetricsSB.Where("NOT startsWith(metric_name, 'signoz')")
if filterWhereClause != nil {
@@ -1338,15 +1338,15 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
}
}
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
start, end, distributedTsTable, localTsTable := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
countExp := metricstelemetryschema.CountExpressionForSamplesTable(distributedSamplesTable)
candidateLimit := req.Limit + 50
metricCandidatesSB := sqlbuilder.NewSelectBuilder()
metricCandidatesSB.Select("metric_name")
metricCandidatesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
metricCandidatesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
metricCandidatesSB.Where("NOT startsWith(metric_name, 'signoz')")
metricCandidatesSB.Where(metricCandidatesSB.Between("unix_milli", start, end))
if filterWhereClause != nil {
@@ -1364,7 +1364,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if reductionEnabled {
reducedCandidatesSB = sqlbuilder.NewSelectBuilder()
reducedCandidatesSB.Select("metric_name")
reducedCandidatesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
reducedCandidatesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
reducedCandidatesSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedCandidatesSB.Where(reducedCandidatesSB.Between("unix_milli", start, end))
if filterWhereClause != nil {
@@ -1379,7 +1379,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
totalSamplesSB := sqlbuilder.NewSelectBuilder()
totalSamplesSB.Select(fmt.Sprintf("%s AS total_samples", countExp))
totalSamplesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
totalSamplesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
totalSamplesSB.Where(totalSamplesSB.Between("unix_milli", req.Start, req.End))
sampleCountsSB := sqlbuilder.NewSelectBuilder()
@@ -1387,7 +1387,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
"metric_name",
fmt.Sprintf("%s AS samples", countExp),
)
sampleCountsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
sampleCountsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
sampleCountsSB.Where(sampleCountsSB.Between("unix_milli", req.Start, req.End))
sampleCountsSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __metric_candidates)")
@@ -1397,13 +1397,13 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if reductionEnabled {
reducedLastSB = sqlbuilder.NewSelectBuilder()
reducedLastSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
reducedLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
reducedLastSB.Where(reducedLastSB.Between("unix_milli", req.Start, req.End))
reducedLastSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __reduced_metric_candidates)")
reducedSumSB = sqlbuilder.NewSelectBuilder()
reducedSumSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
reducedSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
reducedSumSB.Where(reducedSumSB.Between("unix_milli", req.Start, req.End))
reducedSumSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __reduced_metric_candidates)")
}
@@ -1411,7 +1411,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if filterWhereClause != nil {
fingerprintSB := sqlbuilder.NewSelectBuilder()
fingerprintSB.Select("fingerprint")
fingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
fingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
fingerprintSB.Where(fingerprintSB.Between("unix_milli", start, end))
fingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
fingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1423,7 +1423,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
if reductionEnabled {
reducedFingerprintSB := sqlbuilder.NewSelectBuilder()
reducedFingerprintSB.Select("fingerprint")
reducedFingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedLocalTableName))
reducedFingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
reducedFingerprintSB.Where(reducedFingerprintSB.Between("unix_milli", start, end))
reducedFingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
reducedFingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
@@ -1457,12 +1457,12 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
// Grand total includes reduced sample volume so percentages stay consistent.
reducedTotalLastSB := sqlbuilder.NewSelectBuilder()
reducedTotalLastSB.Select("uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedTotalLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
reducedTotalLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
reducedTotalLastSB.Where(reducedTotalLastSB.Between("unix_milli", req.Start, req.End))
reducedTotalSumSB := sqlbuilder.NewSelectBuilder()
reducedTotalSumSB.Select("uniq(reduced_fingerprint, unix_milli) AS cnt")
reducedTotalSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
reducedTotalSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
reducedTotalSumSB.Where(reducedTotalSumSB.Between("unix_milli", req.Start, req.End))
reducedTotalSamplesSB := sqlbuilder.NewSelectBuilder()
@@ -1552,7 +1552,7 @@ func (m *module) getMetricDataPoints(ctx context.Context, metricName string) (ui
sb := sqlbuilder.NewSelectBuilder()
sb.Select("sum(count) AS data_points")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4Agg30mTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4Agg30mTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -1574,7 +1574,7 @@ func (m *module) getMetricLastReceived(ctx context.Context, metricName string) (
sb := sqlbuilder.NewSelectBuilder()
sb.Select("MAX(last_reported_unix_milli) AS last_received_time")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -1599,7 +1599,7 @@ func (m *module) getTotalTimeSeriesForMetricName(ctx context.Context, metricName
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint) AS time_series_count")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
sb.Where(sb.E("metric_name", metricName))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -1623,7 +1623,7 @@ func (m *module) getActiveTimeSeriesForMetricName(ctx context.Context, metricNam
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint) AS active_time_series")
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4TableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4TableName))
sb.Where(sb.E("metric_name", metricName))
sb.Where(sb.GTE("unix_milli", milli))
@@ -1649,7 +1649,7 @@ func (m *module) fetchMetricAttributes(ctx context.Context, metricName string, s
"groupUniqArray(1000)(attr_string_value) AS values",
"uniq(attr_string_value) AS valueCount",
)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
sb.Where(sb.E("metric_name", metricName))
sb.Where("NOT startsWith(attr_name, '__')")

View File

@@ -9,7 +9,7 @@ import (
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
@@ -52,7 +52,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
response := []promotetypes.PromotePath{}
for _, path := range promotedPaths {
fullPath := telemetrylogs.BodyPromotedColumnPrefix + path
fullPath := logstelemetryschema.BodyPromotedColumnPrefix + path
path = telemetrytypes.BodyJSONStringSearchPrefix + path
item := promotetypes.PromotePath{
Path: path,
@@ -68,7 +68,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
// add the paths that are not promoted but have indexes
for path, indexes := range aggr {
path := strings.TrimPrefix(path, telemetrylogs.BodyV2ColumnPrefix)
path := strings.TrimPrefix(path, logstelemetryschema.BodyV2ColumnPrefix)
path = telemetrytypes.BodyJSONStringSearchPrefix + path
response = append(response, promotetypes.PromotePath{
Path: path,
@@ -108,8 +108,8 @@ func (m *module) createIndexes(ctx context.Context, indexes []schemamigrator.Ind
for _, index := range indexes {
alterStmt := schemamigrator.AlterTableAddIndex{
Database: telemetrylogs.DBName,
Table: telemetrylogs.LogsV2LocalTableName,
Database: logstelemetryschema.DBName,
Table: logstelemetryschema.LogsV2LocalTableName,
Index: index,
}
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
@@ -153,10 +153,10 @@ func (m *module) PromoteAndIndexPaths(
}
}
if len(it.Indexes) > 0 {
parentColumn := telemetrylogs.LogsV2BodyV2Column
parentColumn := logstelemetryschema.LogsV2BodyV2Column
// if the path is already promoted or is being promoted, add it to the promoted column
if _, promoted := existingPromotedPaths[it.Path]; promoted || it.Promote {
parentColumn = telemetrylogs.LogsV2BodyPromotedColumn
parentColumn = logstelemetryschema.LogsV2BodyPromotedColumn
}
for _, index := range it.Indexes {

View File

@@ -485,9 +485,11 @@ func (s *store) buildFilterClause(ctx context.Context, orgID valuer.UUID, filter
}
fieldKeys, _, err := s.telemetryMetadataStore.GetKeysMulti(ctx, orgID, selectors)
if err != nil || len(fieldKeys) == 0 {
if err != nil || fieldKeys == nil {
fieldKeys = map[string][]*telemetrytypes.TelemetryFieldKey{}
for _, sel := range selectors {
}
for _, sel := range selectors {
if len(fieldKeys[sel.Name]) == 0 {
fieldKeys[sel.Name] = []*telemetrytypes.TelemetryFieldKey{{
Name: sel.Name,
Signal: sel.Signal,

View File

@@ -11,8 +11,8 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/services"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -39,7 +39,7 @@ func (m *module) FetchTopLevelOperations(ctx context.Context, start time.Time, s
ctx = m.withServicesContext(ctx, "FetchTopLevelOperations")
db := m.TelemetryStore.ClickhouseDB()
query := fmt.Sprintf("SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start", telemetrytraces.DBName, telemetrytraces.TopLevelOperationsTableName)
query := fmt.Sprintf("SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start", tracestelemetryschema.DBName, tracestelemetryschema.TopLevelOperationsTableName)
args := []any{clickhouse.Named("start", start)}
if len(services) > 0 {
query += " AND serviceName IN @services"

View File

@@ -22,6 +22,13 @@ import (
"github.com/SigNoz/signoz/pkg/variables"
)
type Handler interface {
QueryRange(rw http.ResponseWriter, req *http.Request)
QueryRangePreview(rw http.ResponseWriter, req *http.Request)
QueryRawStream(rw http.ResponseWriter, req *http.Request)
ReplaceVariables(rw http.ResponseWriter, req *http.Request)
}
type handler struct {
set factory.ProviderSettings
analytics analytics.Analytics

View File

@@ -15,6 +15,14 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
)
// BucketCache is the interface for bucket-based caching.
type BucketCache interface {
// cached portion + list of gaps to fetch
GetMissRanges(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step) (cached *qbtypes.Result, missing []*qbtypes.TimeRange)
// store fresh buckets for future hits
Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step, fresh *qbtypes.Result)
}
// bucketCache implements the BucketCache interface.
type bucketCache struct {
cache cache.Cache

View File

@@ -11,8 +11,8 @@ import (
"github.com/ClickHouse/clickhouse-go/v2"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -277,12 +277,12 @@ func (q *builderQuery[T]) narrowWindowByTraceID(ctx context.Context, fromMS, toM
return fromMS, toMS, true, ""
}
traceIDs, found := telemetrytraces.ExtractTraceIDsFromFilter(q.spec.Filter.Expression)
traceIDs, found := tracestelemetryschema.ExtractTraceIDsFromFilter(q.spec.Filter.Expression)
if !found || len(traceIDs) == 0 {
return fromMS, toMS, true, ""
}
finder := telemetrytraces.NewTraceTimeRangeFinder(q.telemetryStore)
finder := tracestelemetryschema.NewTraceTimeRangeFinder(q.telemetryStore)
traceStart, traceEnd, exists, err := finder.GetTraceTimeRangeMulti(ctx, traceIDs)
if err != nil {
return fromMS, toMS, true, ""

View File

@@ -100,9 +100,20 @@ func (q *chSQLQuery) renderVars(query string, vars map[string]qbtypes.VariableIt
return newQuery.String(), nil
}
// Statement renders the SQL without executing it, for the preview path.
func (q *chSQLQuery) Statement(_ context.Context) (*qbtypes.Statement, error) {
func (q *chSQLQuery) render(ctx context.Context) (string, error) {
rendered, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
if err != nil {
return "", err
}
querybuilder.LogIfStatementIsNotValid(ctx, q.logger, rendered)
return rendered, nil
}
// Statement renders the SQL without executing it, for the preview path.
func (q *chSQLQuery) Statement(ctx context.Context) (*qbtypes.Statement, error) {
rendered, err := q.render(ctx)
if err != nil {
return nil, err
}
@@ -124,7 +135,7 @@ func (q *chSQLQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
elapsed += p.Elapsed
}))
query, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
query, err := q.render(ctx)
if err != nil {
return nil, err
}
@@ -135,11 +146,11 @@ func (q *chSQLQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
}
defer rows.Close()
// TODO: map the errors from ClickHouse to our error types
payload, err := consume(rows, q.kind, nil, qbtypes.Step{}, q.query.Name)
if err != nil {
return nil, err
}
return &qbtypes.Result{
Type: q.kind,
Value: payload,

View File

@@ -6,18 +6,18 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
tracesTable := fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.SpanIndexV3TableName)
logsTable := fmt.Sprintf("%s.%s", telemetrylogs.DBName, telemetrylogs.LogsV2TableName)
metricsTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName)
tracesTable := fmt.Sprintf("%s.%s", tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName)
logsTable := fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName)
metricsTable := fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName)
var (
traces uint64

View File

@@ -9,12 +9,6 @@ import (
const DefaultMaxConcurrentQueries = 8
type SkipResourceFingerprint struct {
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
// If count of fingerprint is above threshold, skip the fingerprint subquery and filter on main table instead.
Threshold uint64 `yaml:"threshold" mapstructure:"threshold"`
}
// Config represents the configuration for the querier.
type Config struct {
// CacheTTL is the TTL for cached query results
@@ -23,8 +17,6 @@ type Config struct {
FluxInterval time.Duration `yaml:"flux_interval" mapstructure:"flux_interval"`
// MaxConcurrentQueries is the maximum number of concurrent queries for missing ranges
MaxConcurrentQueries int `yaml:"max_concurrent_queries" mapstructure:"max_concurrent_queries"`
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
SkipResourceFingerprint SkipResourceFingerprint `yaml:"skip_resource_fingerprint" mapstructure:"skip_resource_fingerprint"`
// LogTraceIDWindowPadding is the padding added to narrowed down timerange from trace summary to logs with trace_id filter.
LogTraceIDWindowPadding time.Duration `yaml:"log_trace_id_window_padding" mapstructure:"log_trace_id_window_padding"`
}
@@ -39,11 +31,7 @@ func newConfig() factory.Config {
// Default values
CacheTTL: 168 * time.Hour,
FluxInterval: 5 * time.Minute,
MaxConcurrentQueries: DefaultMaxConcurrentQueries,
SkipResourceFingerprint: SkipResourceFingerprint{
Enabled: false,
Threshold: 100000,
},
MaxConcurrentQueries: DefaultMaxConcurrentQueries,
LogTraceIDWindowPadding: 5 * time.Minute,
}
}
@@ -59,9 +47,6 @@ func (c Config) Validate() error {
if c.MaxConcurrentQueries <= 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "max_concurrent_queries must be positive, got %v", c.MaxConcurrentQueries)
}
if c.SkipResourceFingerprint.Enabled && c.SkipResourceFingerprint.Threshold == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "skip_resource_fingerprint.threshold must be > 0 when enabled")
}
if c.LogTraceIDWindowPadding < 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "log_trace_id_window_padding must not be negative, got %v", c.LogTraceIDWindowPadding)
}

View File

@@ -1,35 +0,0 @@
package querier
import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/statsreporter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
// Querier interface defines the contract for querying data.
type Querier interface {
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
statsreporter.StatsCollector
// QueryRangePreview validates and renders the queries without executing them.
QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
}
// BucketCache is the interface for bucket-based caching.
type BucketCache interface {
// cached portion + list of gaps to fetch
GetMissRanges(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step) (cached *qbtypes.Result, missing []*qbtypes.TimeRange)
// store fresh buckets for future hits
Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step, fresh *qbtypes.Result)
}
type Handler interface {
QueryRange(rw http.ResponseWriter, req *http.Request)
// QueryRangePreview is the dry-run endpoint: validate and render without executing.
QueryRangePreview(rw http.ResponseWriter, req *http.Request)
QueryRawStream(rw http.ResponseWriter, req *http.Request)
ReplaceVariables(rw http.ResponseWriter, req *http.Request)
}

View File

@@ -331,11 +331,14 @@ func (q *promqlQuery) Execute(ctx context.Context) (*qbv5.Result, error) {
return nil, errors.WrapInternalf(promErr, errors.CodeInternal, "error getting matrix from promql query %q", query)
}
// Hide only known SigNoz storage keys: label names are user data and may
// legitimately start with "__" (e.g. __address__), so a blanket dunder
// strip mangles user labelsets. The __scope./__resource. prefixes cover
// every exporter version's keys.
excludeLabel := func(labelName string) bool {
if labelName == "__name__" {
return false
}
return strings.HasPrefix(labelName, "__") || labelName == "fingerprint"
return labelName == "__temporality__" ||
strings.HasPrefix(labelName, "__scope.") ||
strings.HasPrefix(labelName, "__resource.")
}
var series []*qbv5.TimeSeries

View File

@@ -21,6 +21,7 @@ import (
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/query-service/utils"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
@@ -35,6 +36,15 @@ var (
intervalWarn = "Query %s is requesting aggregation interval %v seconds, which is smaller than the minimum allowed interval of %v seconds for selected time range. Using the minimum instead"
)
// Querier interface defines the contract for querying data.
type Querier interface {
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
statsreporter.StatsCollector
// QueryRangePreview validates and renders the queries without executing them.
QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
}
type querier struct {
logger *slog.Logger
fl flagger.Flagger

View File

@@ -119,17 +119,17 @@ func TestQueryRange_MetricTypeFromStore(t *testing.T) {
providerSettings,
telemetryStore,
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
&mockMetricStmtBuilder{}, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
flaggertest.New(t), // flagger
0, // logTraceIDWindowPadding
0, // maxConcurrentQueries
nil, // prometheus
nil, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
&mockMetricStmtBuilder{},
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
flaggertest.New(t), // flagger
0, // logTraceIDWindowPadding
0, // maxConcurrentQueries
)
req := &qbtypes.QueryRangeRequest{

View File

@@ -0,0 +1,52 @@
package queriertest
import (
"context"
"github.com/SigNoz/signoz/pkg/querier"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
// MockQuerier implements querier.Querier for testing. Each behavior can be
// overridden via a function field; unset fields return empty, non-error values.
type MockQuerier struct {
QueryRangeFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStreamFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
QueryRangePreviewFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
CollectFunc func(ctx context.Context, orgID valuer.UUID) (map[string]any, error)
}
var _ querier.Querier = (*MockQuerier)(nil)
// NewMockQuerier creates a new MockQuerier.
func NewMockQuerier() *MockQuerier {
return &MockQuerier{}
}
func (m *MockQuerier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error) {
if m.QueryRangeFunc != nil {
return m.QueryRangeFunc(ctx, orgID, req)
}
return &qbtypes.QueryRangeResponse{}, nil
}
func (m *MockQuerier) QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream) {
if m.QueryRawStreamFunc != nil {
m.QueryRawStreamFunc(ctx, orgID, req, client)
}
}
func (m *MockQuerier) QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error) {
if m.QueryRangePreviewFunc != nil {
return m.QueryRangePreviewFunc(ctx, orgID, req, opts)
}
return &qbtypes.QueryRangePreviewResponse{}, nil
}
func (m *MockQuerier) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
if m.CollectFunc != nil {
return m.CollectFunc(ctx, orgID)
}
return map[string]any{}, nil
}

View File

@@ -3,192 +3,54 @@ package signozquerier
import (
"context"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// NewFactory creates a new factory for the signoz querier provider.
// NewFactory creates a new factory for the signoz querier provider. The query
// stack (metadata store, statement builders, bucket cache) is assembled once in
// signoz.go and injected here.
func NewFactory(
telemetryStore telemetrystore.TelemetryStore,
prometheus prometheus.Prometheus,
cache cache.Cache,
metadataStore telemetrytypes.MetadataStore,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder,
bucketCache querier.BucketCache,
flagger flagger.Flagger,
) factory.ProviderFactory[querier.Querier, querier.Config] {
return factory.NewProviderFactory(
factory.MustNewName("signoz"),
func(
ctx context.Context,
_ context.Context,
settings factory.ProviderSettings,
cfg querier.Config,
) (querier.Querier, error) {
return newProvider(ctx, settings, cfg, telemetryStore, prometheus, cache, flagger)
return querier.New(
settings,
telemetryStore,
metadataStore,
prometheus,
traceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,
meterStmtBuilder,
traceOperatorStmtBuilder,
bucketCache,
flagger,
cfg.LogTraceIDWindowPadding,
cfg.MaxConcurrentQueries,
), nil
},
)
}
func newProvider(
_ context.Context,
settings factory.ProviderSettings,
cfg querier.Config,
telemetryStore telemetrystore.TelemetryStore,
prometheus prometheus.Prometheus,
cache cache.Cache,
flagger flagger.Flagger,
) (querier.Querier, error) {
// Create telemetry metadata store
telemetryMetadataStore := telemetrymetadata.NewTelemetryMetaStore(
settings,
telemetryStore,
telemetrytraces.DBName,
telemetrytraces.TagAttributesV2TableName,
telemetrytraces.SpanAttributesKeysTblName,
telemetrytraces.SpanIndexV3TableName,
telemetrymetrics.DBName,
telemetrymetrics.AttributesMetadataTableName,
telemetrymeter.DBName,
telemetrymeter.SamplesAgg1dTableName,
telemetrylogs.DBName,
telemetrylogs.LogsV2TableName,
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.AuditLogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
flagger,
)
// Create trace statement builder
traceFieldMapper := telemetrytraces.NewFieldMapper()
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, traceFieldMapper, traceConditionBuilder, flagger)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
settings,
telemetryMetadataStore,
traceFieldMapper,
traceConditionBuilder,
traceAggExprRewriter,
telemetryStore,
flagger,
cfg.SkipResourceFingerprint.Enabled,
cfg.SkipResourceFingerprint.Threshold,
)
// Create trace operator statement builder
traceOperatorStmtBuilder := telemetrytraces.NewTraceOperatorStatementBuilder(
settings,
telemetryMetadataStore,
traceFieldMapper,
traceConditionBuilder,
traceStmtBuilder,
traceAggExprRewriter,
flagger,
)
// Create log statement builder
logFieldMapper := telemetrylogs.NewFieldMapper(flagger)
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, flagger)
logAggExprRewriter := querybuilder.NewAggExprRewriter(
settings,
telemetrylogs.DefaultFullTextColumn,
logFieldMapper,
logConditionBuilder,
flagger,
)
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
settings,
telemetryMetadataStore,
logFieldMapper,
logConditionBuilder,
logAggExprRewriter,
telemetrylogs.DefaultFullTextColumn,
flagger,
telemetryStore,
cfg.SkipResourceFingerprint.Enabled,
cfg.SkipResourceFingerprint.Threshold,
)
// Create audit statement builder
auditFieldMapper := telemetryaudit.NewFieldMapper()
auditConditionBuilder := telemetryaudit.NewConditionBuilder(auditFieldMapper)
auditAggExprRewriter := querybuilder.NewAggExprRewriter(
settings,
telemetryaudit.DefaultFullTextColumn,
auditFieldMapper,
auditConditionBuilder,
flagger,
)
auditStmtBuilder := telemetryaudit.NewAuditQueryStatementBuilder(
settings,
telemetryMetadataStore,
auditFieldMapper,
auditConditionBuilder,
auditAggExprRewriter,
telemetryaudit.DefaultFullTextColumn,
flagger,
)
// Create metric statement builder
metricFieldMapper := telemetrymetrics.NewFieldMapper()
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(
settings,
telemetryMetadataStore,
metricFieldMapper,
metricConditionBuilder,
flagger,
)
// Create meter statement builder
meterStmtBuilder := telemetrymeter.NewMeterQueryStatementBuilder(
settings,
telemetryMetadataStore,
metricFieldMapper,
metricConditionBuilder,
metricStmtBuilder,
)
// Create bucket cache
bucketCache := querier.NewBucketCache(
settings,
cache,
cfg.CacheTTL,
cfg.FluxInterval,
)
// Create and return the querier
return querier.New(
settings,
telemetryStore,
telemetryMetadataStore,
prometheus,
traceStmtBuilder,
logStmtBuilder,
auditStmtBuilder,
metricStmtBuilder,
meterStmtBuilder,
traceOperatorStmtBuilder,
bucketCache,
flagger,
cfg.LogTraceIDWindowPadding,
cfg.MaxConcurrentQueries,
), nil
}

View File

@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/modules/thirdpartyapi"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/queryparser"
"log/slog"
@@ -1092,6 +1093,8 @@ func (aH *APIHandler) queryDashboardVarsV2(w http.ResponseWriter, r *http.Reques
return
}
querybuilder.LogIfStatementIsNotValid(r.Context(), aH.logger, query)
dashboardVars, err := aH.reader.QueryDashboardVars(r.Context(), query)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorBadData, Err: err}, nil)

View File

@@ -18,6 +18,13 @@ import (
"github.com/SigNoz/signoz/pkg/querier/signozquerier"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstoretest"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/auditstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/meterstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/stretchr/testify/require"
@@ -105,7 +112,23 @@ func NewTestManager(t *testing.T, testOpts *TestManagerOptions) *Manager {
}
// Create querier with test values
providerFactory := signozquerier.NewFactory(telemetryStore, prometheus, cache, flagger)
metadataStore := telemetrymetadata.NewTelemetryMetaStore(providerSettings, telemetryStore, flagger)
cfg := statementbuilder.Config{}
ctx := context.Background()
traceStmtBuilder, err := tracesstatementbuilder.NewFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
traceOperatorStmtBuilder, err := tracesstatementbuilder.NewOperatorFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
logStmtBuilder, err := logsstatementbuilder.NewFactory(telemetryStore, metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
auditStmtBuilder, err := auditstatementbuilder.NewFactory(metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
metricStmtBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
meterStmtBuilder, err := meterstatementbuilder.NewFactory(metadataStore, flagger).New(ctx, providerSettings, cfg)
require.NoError(t, err)
bucketCache := querier.NewBucketCache(providerSettings, cache, 0, 0)
providerFactory := signozquerier.NewFactory(telemetryStore, prometheus, metadataStore, traceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, flagger)
mockQuerier, err := providerFactory.New(context.Background(), providerSettings, querier.Config{})
require.NoError(t, err)

View File

@@ -8,11 +8,11 @@ import (
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/require"
@@ -24,7 +24,7 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
providerSettings := instrumentationtest.New().ToProviderSettings()
metadataStore := telemetrytypestest.NewMockMetadataStore()
flagger, err := flagger.New(
fl, err := flagger.New(
context.Background(),
instrumentationtest.New().ToProviderSettings(),
flagger.Config{},
@@ -32,15 +32,8 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
)
require.NoError(t, err)
metricFieldMapper := telemetrymetrics.NewFieldMapper()
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(
providerSettings,
metadataStore,
metricFieldMapper,
metricConditionBuilder,
flagger,
)
metricStmtBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, fl).New(context.Background(), providerSettings, statementbuilder.Config{})
require.NoError(t, err)
return querier.New(
providerSettings,
@@ -54,7 +47,7 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
flagger,
fl,
0,
0, // maxConcurrentQueries (0 means default)
), metadataStore
@@ -73,40 +66,21 @@ func prepareQuerierForLogs(t *testing.T, telemetryStore telemetrystore.Telemetry
metadataStore.KeysMap = keysMap
fl := flaggertest.New(t)
logFieldMapper := telemetrylogs.NewFieldMapper(fl)
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, fl)
logAggExprRewriter := querybuilder.NewAggExprRewriter(
providerSettings,
telemetrylogs.DefaultFullTextColumn,
logFieldMapper,
logConditionBuilder,
fl,
)
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
providerSettings,
metadataStore,
logFieldMapper,
logConditionBuilder,
logAggExprRewriter,
telemetrylogs.DefaultFullTextColumn,
fl,
nil,
false,
100000,
)
logStmtBuilder, err := logsstatementbuilder.NewFactory(nil, metadataStore, fl).New(context.Background(), providerSettings, statementbuilder.Config{})
require.NoError(t, err)
return querier.New(
providerSettings,
telemetryStore,
metadataStore,
nil, // prometheus
nil, // traceStmtBuilder
logStmtBuilder, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
nil, // prometheus
nil, // traceStmtBuilder
logStmtBuilder,
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
fl,
5*time.Minute, // logTraceIDWindowPadding
0, // maxConcurrentQueries (0 means default)
@@ -126,36 +100,22 @@ func prepareQuerierForTraces(t *testing.T, telemetryStore telemetrystore.Telemet
}
metadataStore.KeysMap = keysMap
// Create trace statement builder
traceFieldMapper := telemetrytraces.NewFieldMapper()
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
fl := flaggertest.New(t)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(providerSettings, nil, traceFieldMapper, traceConditionBuilder, fl)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
providerSettings,
metadataStore,
traceFieldMapper,
traceConditionBuilder,
traceAggExprRewriter,
telemetryStore,
fl,
false,
100000,
)
traceStmtBuilder, err := tracesstatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(context.Background(), providerSettings, statementbuilder.Config{})
require.NoError(t, err)
return querier.New(
providerSettings,
telemetryStore,
metadataStore,
nil, // prometheus
traceStmtBuilder, // traceStmtBuilder
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
nil, // prometheus
traceStmtBuilder,
nil, // logStmtBuilder
nil, // auditStmtBuilder
nil, // metricStmtBuilder
nil, // meterStmtBuilder
nil, // traceOperatorStmtBuilder
nil, // bucketCache
fl,
0,
0, // maxConcurrentQueries (0 means default)

View File

@@ -0,0 +1,76 @@
package querybuilder
import (
"context"
"log/slog"
"strings"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/SigNoz/signoz/pkg/errors"
)
// internalDatabases hold server metadata, credentials and grants rather than telemetry.
var internalDatabases = map[string]struct{}{
"system": {},
"information_schema": {},
}
// The parser's grammar has gaps against SQL that ClickHouse itself accepts. See TestErrIfStatementIsNotValid_ShouldPassButFails.
func ErrIfStatementIsNotValid(query string) (err error) {
defer func() {
// The parser has a history of panicking on malformed input rather than returning an error.
if recovered := recover(); recovered != nil {
err = errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid ClickHouse SQL (recovered): %v", recovered)
}
}()
stmts, parseErr := chparser.NewParser(query).ParseStmts()
if parseErr != nil {
// Wrapped rather than formatted in, so that callers can recover the parser's *ParseError and read the position off it.
return errors.WrapInvalidInputf(parseErr, errors.CodeInvalidInput, "invalid ClickHouse SQL: %s", parseErr.Error())
}
if len(stmts) != 1 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse SQL must contain exactly one statement, found %d statements", len(stmts))
}
selectQuery, ok := stmts[0].(*chparser.SelectQuery)
if !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "only SELECT statements are allowed in ClickHouse SQL queries")
}
visitor := &chparser.DefaultASTVisitor{Visit: func(node chparser.Expr) error {
switch expr := node.(type) {
case *chparser.TableFunctionExpr:
// Source table functions remain usable in ClickHouse read-only mode.
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse table functions are not allowed in SQL queries: %s", chparser.Format(expr.Name))
case *chparser.TableIdentifier:
// Reading these is unaffected by ClickHouse read-only mode.
if expr.Database == nil {
return nil
}
if _, ok := internalDatabases[strings.ToLower(expr.Database.Name)]; ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse %s database is not allowed in SQL queries", expr.Database.Name)
}
case *chparser.SettingExpr:
// A query-level setting takes precedence over the context setting.
if strings.EqualFold(expr.Name.Name, "readonly") {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse readonly setting cannot be overridden")
}
}
return nil
}}
return selectQuery.Accept(visitor)
}
// TODO(@therealpandey): remove this and move to ErrIfStatementIsNotValid.
func LogIfStatementIsNotValid(ctx context.Context, logger *slog.Logger, query string) {
if err := ErrIfStatementIsNotValid(query); err != nil {
logger.WarnContext(ctx, "clickhouse sql is not valid", errors.Attr(err), slog.String("query", query))
}
}

View File

@@ -0,0 +1,154 @@
package querybuilder
import (
"testing"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
testCases := []struct {
name string
query string
}{
// Shapes a telemetry read is allowed to take.
{"Select", "SELECT region AS r, zone FROM metrics WHERE metric_name = 'cpu' GROUP BY region, zone"},
{"TrailingSemicolon", "SELECT count() FROM signoz_logs.distributed_logs_v2;"},
{"CommonTableExpression", "WITH t AS (SELECT fingerprint FROM signoz_metrics.time_series_v4) SELECT * FROM t"},
{"Join", "SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.b"},
{"GlobalIn", "SELECT a FROM t WHERE a GLOBAL IN (SELECT b FROM t2)"},
{"Union", "SELECT * FROM t UNION ALL SELECT * FROM t2"},
{"Intersect", "SELECT * FROM t INTERSECT SELECT * FROM t2"},
{"WindowFunction", "SELECT sum(v) OVER (PARTITION BY a ORDER BY t) FROM t"},
{"UnrelatedSetting", "SELECT * FROM t SETTINGS max_threads = 4"},
{"TerminatedBlockComment", "SELECT /* keep me */ count() FROM t"},
{"BlockCommentMarkerInsideStringLiteral", "SELECT count() FROM t WHERE body = '/* not a comment'"},
// The parser used to loop forever on this; it now reads the comment to the end of
// the input, so this doubles as a canary for that regression.
{"TrailingUnterminatedBlockComment", "SELECT count() FROM t /* unterminated"},
// The rule keys on the database, not on the table name.
{"TableNamedSystemInTelemetryDatabase", "SELECT * FROM signoz_logs.system"},
{"SignedLiteralAfterClosingParen", "SELECT (toUnixTimestamp(now()) - 3600)*1000000000"},
// order by interval
{"OrderByInterval", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval ORDER BY interval"},
{"OrderByIntervalAndDirection", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS `interval` ORDER BY `interval` ASC"},
{"TrimFunction", "SELECT trimBoth('/api/endpoint/', '/');"},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
assert.NoError(t, err)
})
}
}
func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
testCases := []struct {
name string
query string
}{
// Not a single statement, or not a statement at all.
{"Empty", ""},
{"UnterminatedBlockCommentOnly", "/* x"},
{"Unparseable", "SELECT FROM WHERE"},
{"MultipleStatements", "SELECT 1; DROP TABLE signoz_logs.logs_v2"},
// Parses, but is not a SELECT.
{"Drop", "DROP TABLE signoz_logs.logs_v2"},
{"Insert", "INSERT INTO signoz_logs.logs_v2 SELECT * FROM signoz_logs.logs_v2"},
{"AlterDelete", "ALTER TABLE signoz_logs.logs_v2 DELETE WHERE 1 = 1"},
{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory"},
{"Grant", "GRANT ALL ON *.* TO admin"},
{"Set", "SET readonly = 0"},
// These the parser rejects outright rather than classifying.
{"ShowGrants", "SHOW GRANTS"},
{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'"},
// Table functions, which read through something other than a telemetry table.
{"UrlTableFunction", "SELECT * FROM url('http://attacker.example/x', CSV, 'a String')"},
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')"},
{"ExecutableTableFunction", "SELECT * FROM executable('script.sh', CSV, 'a String')"},
{"TableFunctionInJoin", "SELECT * FROM t1 JOIN url('http://x', CSV, 'a String') u ON 1 = 1"},
{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c"},
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))"},
{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')"},
// Internal databases, which hold grants and server metadata rather than telemetry.
{"SystemUsers", "SELECT * FROM system.users"},
{"SystemUppercase", "SELECT * FROM SYSTEM.USERS"},
{"SystemQuoted", "SELECT count() FROM `system`.`tables`"},
{"SystemInSubquery", "SELECT * FROM (SELECT name FROM system.parts)"},
{"SystemInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l JOIN system.users AS u ON 1 = 1"},
{"SystemInIntersect", "SELECT * FROM t INTERSECT SELECT * FROM system.users"},
{"InformationSchema", "SELECT * FROM information_schema.tables"},
// A query-level setting takes precedence over the one the caller applies.
{"ReadonlySettingOverride", "SELECT * FROM t SETTINGS readonly = 0"},
{"ReadonlySettingOverrideAmongOthers", "SELECT * FROM t SETTINGS max_threads = 4, readonly = 0"},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
assert.Error(t, err)
})
}
}
// Queries the parser cannot read. ClickHouse runs all of them.
func TestErrIfStatementIsNotValid_ShouldPassButFails(t *testing.T) {
testCases := []struct {
name string
query string
// The construct the parser stops after, which is the one it cannot read.
expectedStopsAfter string
// The same construct written so the parser accepts it.
fix string
}{
{
name: "IntervalAliasInOrderBy",
query: "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, resource_string_service$$name AS `service.name`, attributes_string['http.route'] AS `http.route`, quantile(0.95)(duration_nano) / 1000000000 AS value FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_string_service$$name = 'svc-a' AND resources_string['deployment.environment'] = 'dev' AND attributes_string['http.route'] = '/v1' AND http_method = 'POST' AND timestamp BETWEEN toDateTime(1784601720) AND toDateTime(1784602620) AND ts_bucket_start BETWEEN 1784601720 - 1800 AND 1784602620 GROUP BY `service.name`, `http.route`, interval ORDER BY interval ASC",
expectedStopsAfter: "ORDER BY interval ASC",
fix: "SELECT count() AS interval FROM t ORDER BY `interval` ASC",
},
{
name: "IntervalAliasInOrderByDesc",
query: "SELECT count() AS value, toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, serviceName, resourceTagsMap['deployment.environment'] AS environment, exceptionStacktrace FROM signoz_traces.distributed_signoz_error_index_v2 WHERE exceptionType != 'OSError' AND resourceTagsMap['deployment.environment'] = 'staging' AND timestamp BETWEEN toDateTime(1785186300) AND toDateTime(1785186600) GROUP BY serviceName, interval, environment, exceptionStacktrace ORDER BY interval DESC",
expectedStopsAfter: "ORDER BY interval DESC",
fix: "SELECT count() AS interval FROM t ORDER BY `interval` DESC",
},
{
name: "StandardTrimSyntax",
query: "SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 5 MINUTE) AS interval, resources_string['host.name'] as host_name, toFloat64(countIf( lower(trim(BOTH ' ' FROM replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ))) IN ('unhealthy','starting','failing') )) as value FROM signoz_logs.distributed_logs_v2 WHERE timestamp BETWEEN 1784602320000000000 AND 1784602620000000000 AND ts_bucket_start BETWEEN 1784602320 - 300 AND 1784602620 AND JSONExtractString(body, 'Type') = 'container' AND JSONExtractString(body, 'Actor', 'Attributes', 'name') IS NOT NULL AND resources_string['host.name'] IS NOT NULL AND resources_string['host.name'] = 'aihub-nightly' GROUP BY interval, host_name ORDER BY interval, host_name",
expectedStopsAfter: "trim(BOTH '",
fix: "SELECT trimBoth(replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ), ' ')",
},
{
name: "SignedLiteralAfterClosingParen",
query: "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)",
expectedStopsAfter: "(toUnixTimestamp(now())",
fix: "SELECT (toUnixTimestamp(now()) - 3600) * 1000000000",
},
{
name: "SignedLiteralAfterClosingParenMinimal",
query: "SELECT (1)-1",
expectedStopsAfter: "(1)",
fix: "SELECT (1) - 1",
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
var parseErr *chparser.ParseError
require.ErrorAs(t, err, &parseErr, "expected a parser failure rather than a rule violation")
// The parser reports the offset it stopped at, which sits just past the construct
// it choked on, so the text leading up to it is what needs looking at.
consumed := testCase.query[:parseErr.Pos]
assert.Equal(t, testCase.expectedStopsAfter, consumed[max(0, len(consumed)-len(testCase.expectedStopsAfter)):])
assert.NoError(t, ErrIfStatementIsNotValid(testCase.fix))
})
}
}

View File

@@ -1 +0,0 @@
package querybuilder

View File

@@ -39,6 +39,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlmigrator"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/tokenizer"
@@ -97,6 +98,9 @@ type Config struct {
// Querier config
Querier querier.Config `mapstructure:"querier"`
// StatementBuilder config
StatementBuilder statementbuilder.Config `mapstructure:"statementbuilder"`
// Ruler config
Ruler ruler.Config `mapstructure:"ruler"`
@@ -166,6 +170,7 @@ func NewConfig(ctx context.Context, logger *slog.Logger, resolverConfig config.R
prometheus.NewConfigFactory(),
alertmanager.NewConfigFactory(),
querier.NewConfigFactory(),
statementbuilder.NewConfigFactory(),
ruler.NewConfigFactory(),
emailing.NewConfigFactory(),
sharder.NewConfigFactory(),

View File

@@ -69,6 +69,8 @@ import (
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/web"
"github.com/SigNoz/signoz/pkg/web/noopweb"
@@ -227,6 +229,7 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddTelemetryTuplesFactory(sqlstore),
sqlmigration.NewAddTagRelationRankFactory(sqlstore, sqlschema),
sqlmigration.NewMigrateDashboardsV1ToV2Factory(sqlstore, sqlschema, dashboardStore, tagModule),
sqlmigration.NewFillDashboardMeterSourceFactory(sqlstore, dashboardStore),
)
}
@@ -285,9 +288,9 @@ func NewStatsReporterProviderFactories(aggregator statsreporter.Aggregator, orgG
)
}
func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, prometheus prometheus.Prometheus, cache cache.Cache, flagger flagger.Flagger) factory.NamedMap[factory.ProviderFactory[querier.Querier, querier.Config]] {
func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, prometheus prometheus.Prometheus, metadataStore telemetrytypes.MetadataStore, traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation], logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation], auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation], metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation], meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation], traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder, bucketCache querier.BucketCache, flagger flagger.Flagger) factory.NamedMap[factory.ProviderFactory[querier.Querier, querier.Config]] {
return factory.MustNewNamedMap(
signozquerier.NewFactory(telemetryStore, prometheus, cache, flagger),
signozquerier.NewFactory(telemetryStore, prometheus, metadataStore, traceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, flagger),
)
}

View File

@@ -48,16 +48,17 @@ import (
"github.com/SigNoz/signoz/pkg/sqlmigrator"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/statementbuilder/auditstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/meterstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
pkgtokenizer "github.com/SigNoz/signoz/pkg/tokenizer"
"github.com/SigNoz/signoz/pkg/types/authtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
@@ -96,6 +97,62 @@ type SigNoz struct {
MeterReporter meterreporter.Reporter
}
// newQueryStack assembles the query stack once and returns, in order: the shared
// telemetry metadata store (reused elsewhere in signoz.New), the per-signal
// statement builders (trace, log, audit, metric, meter, trace-operator), and the
// bucket cache. It is the only place that imports the concrete statement-builder
// sub-packages.
func newQueryStack(
ctx context.Context,
settings factory.ProviderSettings,
config Config,
telemetryStore telemetrystore.TelemetryStore,
cache cache.Cache,
fl flagger.Flagger,
) (
telemetrytypes.MetadataStore,
qbtypes.StatementBuilder[qbtypes.TraceAggregation],
qbtypes.StatementBuilder[qbtypes.LogAggregation],
qbtypes.StatementBuilder[qbtypes.LogAggregation],
qbtypes.StatementBuilder[qbtypes.MetricAggregation],
qbtypes.StatementBuilder[qbtypes.MetricAggregation],
qbtypes.TraceOperatorStatementBuilder,
querier.BucketCache,
error,
) {
metadataStore := telemetrymetadata.NewTelemetryMetaStore(settings, telemetryStore, fl)
cfg := config.StatementBuilder
traceStmtBuilder, err := tracesstatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, err
}
traceOperatorStmtBuilder, err := tracesstatementbuilder.NewOperatorFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, err
}
logStmtBuilder, err := logsstatementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, err
}
auditStmtBuilder, err := auditstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, err
}
metricStmtBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, err
}
meterStmtBuilder, err := meterstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, nil, nil, nil, nil, nil, nil, nil, err
}
bucketCache := querier.NewBucketCache(settings, cache, config.Querier.CacheTTL, config.Querier.FluxInterval)
return metadataStore, traceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, nil
}
func New(
ctx context.Context,
config Config,
@@ -254,12 +311,19 @@ func New(
return nil, err
}
// Assemble the query stack (metadata store, statement builders, bucket cache) once,
// and reuse the single metadata store everywhere downstream.
telemetryMetadataStore, traceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, err := newQueryStack(ctx, providerSettings, config, telemetrystore, cache, flagger)
if err != nil {
return nil, err
}
// Initialize querier from the available querier provider factories
querier, err := factory.NewProviderFromNamedMap(
ctx,
providerSettings,
config.Querier,
NewQuerierProviderFactories(telemetrystore, prometheus, cache, flagger),
NewQuerierProviderFactories(telemetrystore, prometheus, telemetryMetadataStore, traceStmtBuilder, logStmtBuilder, auditStmtBuilder, metricStmtBuilder, meterStmtBuilder, traceOperatorStmtBuilder, bucketCache, flagger),
config.Querier.Provider(),
)
if err != nil {
@@ -427,35 +491,6 @@ func New(
return nil, err
}
// Initialize telemetry metadata store
// TODO: consolidate other telemetrymetadata.NewTelemetryMetaStore initializations to reuse this instance instead.
telemetryMetadataStore := telemetrymetadata.NewTelemetryMetaStore(
providerSettings,
telemetrystore,
telemetrytraces.DBName,
telemetrytraces.TagAttributesV2TableName,
telemetrytraces.SpanAttributesKeysTblName,
telemetrytraces.SpanIndexV3TableName,
telemetrymetrics.DBName,
telemetrymetrics.AttributesMetadataTableName,
telemetrymeter.DBName,
telemetrymeter.SamplesAgg1dTableName,
telemetrylogs.DBName,
telemetrylogs.LogsV2TableName,
telemetrylogs.TagAttributesV2TableName,
telemetrylogs.LogAttributeKeysTblName,
telemetrylogs.LogResourceKeysTblName,
telemetryaudit.DBName,
telemetryaudit.AuditLogsTableName,
telemetryaudit.TagAttributesTableName,
telemetryaudit.LogAttributeKeysTblName,
telemetryaudit.LogResourceKeysTblName,
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
flagger,
)
global, err := factory.NewProviderFromNamedMap(
ctx,
providerSettings,

View File

@@ -0,0 +1,159 @@
package sqlmigration
import (
"context"
"log/slog"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
qb "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
)
// meterMetricNames are the SigNoz meter metrics; a metrics query aggregating one is a
// meter query.
var meterMetricNames = map[string]bool{
"signoz.meter.log.count": true,
"signoz.meter.log.size": true,
"signoz.meter.span.count": true,
"signoz.meter.span.size": true,
"signoz.meter.metric.datapoint.count": true,
"signoz.meter.platform.active": true,
}
type fillDashboardMeterSource struct {
sqlstore sqlstore.SQLStore
dashboardStore dashboardtypes.Store
settings factory.ProviderSettings
}
func NewFillDashboardMeterSourceFactory(sqlstore sqlstore.SQLStore, dashboardStore dashboardtypes.Store) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(
factory.MustNewName("fill_dashboard_meter_source"),
func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &fillDashboardMeterSource{sqlstore: sqlstore, dashboardStore: dashboardStore, settings: ps}, nil
},
)
}
func (migration *fillDashboardMeterSource) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
// Up resets the meter source that 103 dropped when converting v1 meter queries to v2:
// every metrics query aggregating a meter metric is set back to source "meter". One
// transaction; v1 (unparseable-as-v2) dashboards are skipped.
func (migration *fillDashboardMeterSource) Up(ctx context.Context, _ *bun.DB) error {
return migration.sqlstore.RunInTxCtx(ctx, nil, func(ctx context.Context) error {
var orgIDs []string
if err := migration.sqlstore.BunDBCtx(ctx).NewSelect().Model((*types.Organization)(nil)).Column("id").Scan(ctx, &orgIDs); err != nil {
return err
}
for _, id := range orgIDs {
orgID, err := valuer.NewUUID(id)
if err != nil {
return err
}
if err := migration.fillOrg(ctx, orgID); err != nil {
return err
}
}
return nil
})
}
// fillOrg fills the meter source on every v2 dashboard in the org that needs it. Runs
// inside the caller's transaction.
func (migration *fillDashboardMeterSource) fillOrg(ctx context.Context, orgID valuer.UUID) error {
// List, not ListV2: ListV2 paginates and excludes system dashboards; a migration needs every row.
storables, err := migration.dashboardStore.List(ctx, orgID)
if err != nil {
return err
}
logger := migration.settings.Logger
var stillInV1, skippedNoMeter, wouldMigrateButErrored, migrated int
for _, storable := range storables {
// ToDashboardV2 rejects non-v2 data, so v1 dashboards fall through untouched.
dashboard, err := storable.ToDashboardV2(nil)
if err != nil {
stillInV1++
continue
}
if !fillMeterSource(&dashboard.Spec) {
skippedNoMeter++
continue
}
restorable, err := dashboard.ToStorableDashboard()
if err != nil {
wouldMigrateButErrored++
logger.WarnContext(ctx, "failed to re-serialize dashboard after filling meter source", slog.String("org_id", orgID.String()), slog.String("dashboard_id", storable.ID.String()), slog.Any("error", err))
continue
}
if err := migration.dashboardStore.Update(ctx, orgID, restorable); err != nil {
return err
}
migrated++
}
logger.InfoContext(ctx, "filled meter source on v2 dashboards",
slog.String("org_id", orgID.String()),
slog.Int("total", len(storables)),
slog.Int("still_in_v1", stillInV1),
slog.Int("skipped_no_meter", skippedNoMeter),
slog.Int("would_migrate_but_errored", wouldMigrateButErrored),
slog.Int("migrated", migrated),
)
return nil
}
// fillMeterSource sets Source=meter on every metrics builder query in the spec whose
// aggregation targets a meter metric, reporting whether anything changed.
func fillMeterSource(spec *dashboardtypes.DashboardSpec) bool {
changed := false
for _, panel := range spec.Panels {
if panel == nil {
continue
}
for _, query := range panel.Spec.Queries {
switch plugin := query.Spec.Plugin.Spec.(type) {
case *qb.CompositeQuery:
for i := range plugin.Queries {
changed = setMeterSource(&plugin.Queries[i].Spec) || changed
}
case *dashboardtypes.BuilderQuerySpec:
changed = setMeterSource(&plugin.Spec) || changed
}
}
}
return changed
}
// setMeterSource sets source "meter" on a metrics query aggregating a meter metric. The
// type assertion doubles as the metrics-signal check; the already-meter guard keeps it
// idempotent. Written back through the pointer, as an interface value isn't addressable.
func setMeterSource(spec *any) bool {
query, ok := (*spec).(qb.QueryBuilderQuery[qb.MetricAggregation])
if !ok || query.Source == telemetrytypes.SourceMeter {
return false
}
for _, aggregation := range query.Aggregations {
if meterMetricNames[aggregation.MetricName] {
query.Source = telemetrytypes.SourceMeter
*spec = query
return true
}
}
return false
}
func (migration *fillDashboardMeterSource) Down(context.Context, *bun.DB) error {
return nil
}

View File

@@ -1,4 +1,4 @@
package telemetryaudit
package auditstatementbuilder
import (
"context"
@@ -10,7 +10,9 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -29,6 +31,25 @@ type auditQueryStatementBuilder struct {
var _ qbtypes.StatementBuilder[qbtypes.LogAggregation] = (*auditQueryStatementBuilder)(nil)
// NewFactory returns a provider factory for the audit statement builder. Its New
// internalizes the FieldMapper, ConditionBuilder, and AggExprRewriter.
func NewFactory(
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[qbtypes.StatementBuilder[qbtypes.LogAggregation], statementbuilder.Config] {
return factory.NewProviderFactory(
factory.MustNewName("audit"),
func(_ context.Context, settings factory.ProviderSettings, _ statementbuilder.Config) (qbtypes.StatementBuilder[qbtypes.LogAggregation], error) {
fm := audittelemetryschema.NewFieldMapper()
cb := audittelemetryschema.NewConditionBuilder(fm)
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, audittelemetryschema.DefaultFullTextColumn, fm, cb, fl)
return NewAuditQueryStatementBuilder(
settings, metadataStore, fm, cb, aggExprRewriter, audittelemetryschema.DefaultFullTextColumn, fl,
), nil
},
)
}
func NewAuditQueryStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
@@ -38,12 +59,12 @@ func NewAuditQueryStatementBuilder(
fullTextColumn *telemetrytypes.TelemetryFieldKey,
flagger flagger.Flagger,
) *auditQueryStatementBuilder {
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryaudit")
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema")
resourceFilterStmtBuilder := telemetryresourcefilter.New[qbtypes.LogAggregation](
resourceFilterStmtBuilder := resourcefilter.New[qbtypes.LogAggregation](
settings,
DBName,
LogsResourceTableName,
audittelemetryschema.DBName,
audittelemetryschema.LogsResourceTableName,
telemetrytypes.SignalLogs,
telemetrytypes.SourceAudit,
metadataStore,
@@ -191,8 +212,8 @@ func (b *auditQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[st
}
func (b *auditQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
if _, ok := IntrinsicFields[key.Name]; ok {
intrinsicField := IntrinsicFields[key.Name]
if _, ok := audittelemetryschema.IntrinsicFields[key.Name]; ok {
intrinsicField := audittelemetryschema.IntrinsicFields[key.Name]
return querybuilder.AdjustKey(key, keys, &intrinsicField)
}
return querybuilder.AdjustKey(key, keys, nil)
@@ -219,26 +240,26 @@ func (b *auditQueryStatementBuilder) buildListQuery(
cteArgs = append(cteArgs, args)
}
sb.Select(TimestampColumn)
sb.SelectMore(IDColumn)
sb.Select(audittelemetryschema.TimestampColumn)
sb.SelectMore(audittelemetryschema.IDColumn)
if len(query.SelectFields) == 0 {
sb.SelectMore(TraceIDColumn)
sb.SelectMore(SpanIDColumn)
sb.SelectMore(TraceFlagsColumn)
sb.SelectMore(SeverityTextColumn)
sb.SelectMore(SeverityNumberColumn)
sb.SelectMore(ScopeNameColumn)
sb.SelectMore(ScopeVersionColumn)
sb.SelectMore(BodyColumn)
sb.SelectMore(EventNameColumn)
sb.SelectMore(AttributesStringColumn)
sb.SelectMore(AttributesNumberColumn)
sb.SelectMore(AttributesBoolColumn)
sb.SelectMore(ResourceColumn)
sb.SelectMore(ScopeStringColumn)
sb.SelectMore(audittelemetryschema.TraceIDColumn)
sb.SelectMore(audittelemetryschema.SpanIDColumn)
sb.SelectMore(audittelemetryschema.TraceFlagsColumn)
sb.SelectMore(audittelemetryschema.SeverityTextColumn)
sb.SelectMore(audittelemetryschema.SeverityNumberColumn)
sb.SelectMore(audittelemetryschema.ScopeNameColumn)
sb.SelectMore(audittelemetryschema.ScopeVersionColumn)
sb.SelectMore(audittelemetryschema.BodyColumn)
sb.SelectMore(audittelemetryschema.EventNameColumn)
sb.SelectMore(audittelemetryschema.AttributesStringColumn)
sb.SelectMore(audittelemetryschema.AttributesNumberColumn)
sb.SelectMore(audittelemetryschema.AttributesBoolColumn)
sb.SelectMore(audittelemetryschema.ResourceColumn)
sb.SelectMore(audittelemetryschema.ScopeStringColumn)
} else {
for index := range query.SelectFields {
if query.SelectFields[index].Name == TimestampColumn || query.SelectFields[index].Name == IDColumn {
if query.SelectFields[index].Name == audittelemetryschema.TimestampColumn || query.SelectFields[index].Name == audittelemetryschema.IDColumn {
continue
}
@@ -250,7 +271,7 @@ func (b *auditQueryStatementBuilder) buildListQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
@@ -340,7 +361,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
sb.SelectMore(fmt.Sprintf("%s AS __result_%d", rewritten, i))
}
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
@@ -478,7 +499,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {

View File

@@ -1,4 +1,4 @@
package telemetryaudit
package auditstatementbuilder
import (
"context"
@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
@@ -54,8 +55,8 @@ func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = auditFieldKeyMap()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := audittelemetryschema.NewFieldMapper()
cb := audittelemetryschema.NewConditionBuilder(fm)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
return NewAuditQueryStatementBuilder(
@@ -64,7 +65,7 @@ func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
audittelemetryschema.DefaultFullTextColumn,
fl,
)
}

View File

@@ -0,0 +1,41 @@
package statementbuilder
import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
)
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
type SkipResourceFingerprint struct {
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
// If count of fingerprint is above threshold, skip the fingerprint subquery and filter on main table instead.
Threshold uint64 `yaml:"threshold" mapstructure:"threshold"`
}
// Config represents the configuration for the statement builders.
type Config struct {
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
SkipResourceFingerprint SkipResourceFingerprint `yaml:"skip_resource_fingerprint" mapstructure:"skip_resource_fingerprint"`
}
// NewConfigFactory creates a new config factory for the statement builders.
func NewConfigFactory() factory.ConfigFactory {
return factory.NewConfigFactory(factory.MustNewName("statementbuilder"), newConfig)
}
func newConfig() factory.Config {
return Config{
SkipResourceFingerprint: SkipResourceFingerprint{
Enabled: false,
Threshold: 100000,
},
}
}
// Validate validates the configuration.
func (c Config) Validate() error {
if c.SkipResourceFingerprint.Enabled && c.SkipResourceFingerprint.Threshold == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "skip_resource_fingerprint.threshold must be > 0 when enabled")
}
return nil
}

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
@@ -43,14 +44,14 @@ func TestStatementBuilderGroupByUnknownKey(t *testing.T) {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.WithUseJSONBody(t, c.useJSONBody)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore, fm, cb, aggExprRewriter,
DefaultFullTextColumn, fl, nil, false, 100000,
logstelemetryschema.DefaultFullTextColumn, fl, nil, false, 100000,
)
query := qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
@@ -1225,7 +1226,7 @@ func TestJSONStmtBuilder_OrderBy(t *testing.T) {
func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
statementBuilder, metadataStore := buildJSONTestStatementBuilder(t, false)
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
keysMap := buildCompleteFieldKeyMap(releaseTime)
keysMap := logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
for _, keys := range keysMap {
for _, key := range keys {
metadataStore.SetKey(key)
@@ -1287,7 +1288,7 @@ func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetrytypestest.MockMetadataStore {
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.SetStaticFields(IntrinsicFields)
mockMetadataStore.SetStaticFields(logstelemetryschema.IntrinsicFields)
types, _ := telemetrytypes.TestJSONTypeSet()
for path, fieldDataTypes := range types {
for _, fdt := range fieldDataTypes {
@@ -1307,15 +1308,15 @@ func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetryty
Name: path,
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: fdt,
BaseColumn: LogsV2BodyV2Column,
IndexExpression: schemamigrator.JSONSubColumnIndexExpr(LogsV2BodyV2Column, path, jsonType.StringValue()),
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
IndexExpression: schemamigrator.JSONSubColumnIndexExpr(logstelemetryschema.LogsV2BodyV2Column, path, jsonType.StringValue()),
})
}
}
err := key.SetJSONAccessPlan(telemetrytypes.JSONColumnMetadata{
BaseColumn: LogsV2BodyV2Column,
PromotedColumn: LogsV2BodyPromotedColumn,
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
PromotedColumn: logstelemetryschema.LogsV2BodyPromotedColumn,
}, types)
require.NoError(t, err)
@@ -1331,8 +1332,8 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) (*logQueryStat
mockMetadataStore := buildTestTelemetryMetadataStore(t, addIndexes)
fl := flaggertest.WithUseJSONBody(t, true)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -1342,7 +1343,7 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) (*logQueryStat
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -10,7 +10,9 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -19,12 +21,22 @@ import (
"github.com/huandu/go-sqlbuilder"
)
const bodySearchDefaultWarning = "body searches default to `body.message:string`. Use `body.<key>` to search a different field inside body"
func bodyAliasExpression(bodyJSONEnabled bool) string {
if !bodyJSONEnabled {
return logstelemetryschema.LogsV2BodyColumn
}
return fmt.Sprintf("%s as body", logstelemetryschema.LogsV2BodyV2Column)
}
type logQueryStatementBuilder struct {
logger *slog.Logger
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
resourceFilterResolver *telemetryresourcefilter.ResourceFingerprintResolver[qbtypes.LogAggregation]
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.LogAggregation]
aggExprRewriter qbtypes.AggExprRewriter
fl flagger.Flagger
skipResourceFingerprintEnabled bool
@@ -34,6 +46,28 @@ type logQueryStatementBuilder struct {
var _ qbtypes.StatementBuilder[qbtypes.LogAggregation] = (*logQueryStatementBuilder)(nil)
// NewFactory returns a provider factory for the logs statement builder. Its New
// internalizes the FieldMapper, ConditionBuilder, and AggExprRewriter, and reads
// SkipResourceFingerprint from the config.
func NewFactory(
telemetryStore telemetrystore.TelemetryStore,
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[qbtypes.StatementBuilder[qbtypes.LogAggregation], statementbuilder.Config] {
return factory.NewProviderFactory(
factory.MustNewName("logs"),
func(_ context.Context, settings factory.ProviderSettings, cfg statementbuilder.Config) (qbtypes.StatementBuilder[qbtypes.LogAggregation], error) {
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, logstelemetryschema.DefaultFullTextColumn, fm, cb, fl)
return NewLogQueryStatementBuilder(
settings, metadataStore, fm, cb, aggExprRewriter, logstelemetryschema.DefaultFullTextColumn,
fl, telemetryStore, cfg.SkipResourceFingerprint.Enabled, cfg.SkipResourceFingerprint.Threshold,
), nil
},
)
}
func NewLogQueryStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
@@ -46,12 +80,12 @@ func NewLogQueryStatementBuilder(
skipResourceFingerprintEnable bool,
skipResourceFingerprintThreshold uint64,
) *logQueryStatementBuilder {
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrylogs")
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema")
resourceFilterResolver := telemetryresourcefilter.NewResolver[qbtypes.LogAggregation](
resourceFilterResolver := resourcefilter.NewResolver[qbtypes.LogAggregation](
settings,
DBName,
LogsResourceV2TableName,
logstelemetryschema.DBName,
logstelemetryschema.LogsResourceV2TableName,
telemetrytypes.SignalLogs,
telemetrytypes.SourceUnspecified,
metadataStore,
@@ -174,7 +208,7 @@ func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation], bo
// TODO(Piyush): Setup better for coming FTS support.
if bodyJSONEnabled {
for _, sel := range keySelectors {
if sel.Name == LogsV2BodyColumn {
if sel.Name == logstelemetryschema.LogsV2BodyColumn {
warnings = append(warnings, bodySearchDefaultWarning)
break
}
@@ -250,8 +284,8 @@ func (b *logQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[stri
func (b *logQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
// First check if it matches with any intrinsic fields
var intrinsicOrCalculatedField telemetrytypes.TelemetryFieldKey
if _, ok := IntrinsicFields[key.Name]; ok {
intrinsicOrCalculatedField = IntrinsicFields[key.Name]
if _, ok := logstelemetryschema.IntrinsicFields[key.Name]; ok {
intrinsicOrCalculatedField = logstelemetryschema.IntrinsicFields[key.Name]
return querybuilder.AdjustKey(key, keys, &intrinsicOrCalculatedField)
}
@@ -284,28 +318,28 @@ func (b *logQueryStatementBuilder) buildListQuery(
}
// Select timestamp and id by default
sb.Select(LogsV2TimestampColumn)
sb.SelectMore(LogsV2IDColumn)
sb.Select(logstelemetryschema.LogsV2TimestampColumn)
sb.SelectMore(logstelemetryschema.LogsV2IDColumn)
if len(query.SelectFields) == 0 {
// Select all default columns
sb.SelectMore(LogsV2TraceIDColumn)
sb.SelectMore(LogsV2SpanIDColumn)
sb.SelectMore(LogsV2TraceFlagsColumn)
sb.SelectMore(LogsV2SeverityTextColumn)
sb.SelectMore(LogsV2SeverityNumberColumn)
sb.SelectMore(LogsV2ScopeNameColumn)
sb.SelectMore(LogsV2ScopeVersionColumn)
sb.SelectMore(logstelemetryschema.LogsV2TraceIDColumn)
sb.SelectMore(logstelemetryschema.LogsV2SpanIDColumn)
sb.SelectMore(logstelemetryschema.LogsV2TraceFlagsColumn)
sb.SelectMore(logstelemetryschema.LogsV2SeverityTextColumn)
sb.SelectMore(logstelemetryschema.LogsV2SeverityNumberColumn)
sb.SelectMore(logstelemetryschema.LogsV2ScopeNameColumn)
sb.SelectMore(logstelemetryschema.LogsV2ScopeVersionColumn)
sb.SelectMore(bodyAliasExpression(b.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))))
sb.SelectMore(LogsV2AttributesStringColumn)
sb.SelectMore(LogsV2AttributesNumberColumn)
sb.SelectMore(LogsV2AttributesBoolColumn)
sb.SelectMore(LogsV2ResourcesStringColumn)
sb.SelectMore(LogsV2ScopeStringColumn)
sb.SelectMore(logstelemetryschema.LogsV2AttributesStringColumn)
sb.SelectMore(logstelemetryschema.LogsV2AttributesNumberColumn)
sb.SelectMore(logstelemetryschema.LogsV2AttributesBoolColumn)
sb.SelectMore(logstelemetryschema.LogsV2ResourcesStringColumn)
sb.SelectMore(logstelemetryschema.LogsV2ScopeStringColumn)
} else {
// Select specified columns
for index := range query.SelectFields {
if query.SelectFields[index].Name == LogsV2TimestampColumn || query.SelectFields[index].Name == LogsV2IDColumn {
if query.SelectFields[index].Name == logstelemetryschema.LogsV2TimestampColumn || query.SelectFields[index].Name == logstelemetryschema.LogsV2IDColumn {
continue
}
@@ -318,7 +352,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
@@ -424,7 +458,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
}
// Add FROM clause
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
@@ -592,7 +626,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)

View File

@@ -1,4 +1,4 @@
package telemetrylogs
package logsstatementbuilder
import (
"context"
@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -212,12 +213,12 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
keysMap := buildCompleteFieldKeyMap(releaseTime)
keysMap := logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
mockMetadataStore.KeysMap = keysMap
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -227,7 +228,7 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -355,12 +356,12 @@ func TestStatementBuilderListQuery(t *testing.T) {
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper(fl)
fm := logstelemetryschema.NewFieldMapper(fl)
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
cb := NewConditionBuilder(fm, fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -370,7 +371,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -507,11 +508,11 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper(fl)
fm := logstelemetryschema.NewFieldMapper(fl)
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
cb := NewConditionBuilder(fm, fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -521,7 +522,7 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -585,11 +586,11 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper(fl)
fm := logstelemetryschema.NewFieldMapper(fl)
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
cb := NewConditionBuilder(fm, fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -599,7 +600,7 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -684,9 +685,9 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
ctx := context.Background()
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMapCollision()
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMapCollision()
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -696,7 +697,7 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -740,8 +741,8 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
expectedKey: IntrinsicFields["severity_text"],
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: logstelemetryschema.IntrinsicFields["severity_text"],
},
{
name: "intrinsic field with other key match - no override",
@@ -777,7 +778,7 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: telemetrytypes.TelemetryFieldKey{
Name: "severity_number",
FieldContext: telemetrytypes.FieldContextBody,
@@ -791,8 +792,8 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["service.name"][0],
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["service.name"][0],
},
{
name: "single matching key with incorrect context specified - no override",
@@ -801,7 +802,7 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: telemetrytypes.TelemetryFieldKey{
Name: "service.name",
FieldContext: telemetrytypes.FieldContextAttribute,
@@ -815,8 +816,8 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["service.name"][0],
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["service.name"][0],
},
{
name: "multiple matching keys - all materialized",
@@ -825,7 +826,7 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: telemetrytypes.TelemetryFieldKey{
Name: "multi.mat.key",
FieldDataType: telemetrytypes.FieldDataTypeString,
@@ -839,7 +840,7 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: telemetrytypes.TelemetryFieldKey{
Name: "mixed.materialization.key",
FieldDataType: telemetrytypes.FieldDataTypeString,
@@ -853,8 +854,8 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["mixed.materialization.key"][0],
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["mixed.materialization.key"][0],
},
{
name: "no matching keys - unknown field",
@@ -863,7 +864,7 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: telemetrytypes.TelemetryFieldKey{
Name: "unknown.field",
FieldContext: telemetrytypes.FieldContextUnspecified,
@@ -878,7 +879,7 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: telemetrytypes.TelemetryFieldKey{
Name: "unknown.field",
FieldContext: telemetrytypes.FieldContextAttribute,
@@ -893,8 +894,8 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["mat.key"][0],
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["mat.key"][0],
},
{
name: "non-materialized field",
@@ -903,16 +904,16 @@ func TestAdjustKey(t *testing.T) {
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
keysMap: buildCompleteFieldKeyMap(releaseTime),
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["user.id"][0],
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["user.id"][0],
},
}
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
fm := logstelemetryschema.NewFieldMapper(fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMapCollision()
cb := NewConditionBuilder(fm, fl)
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMapCollision()
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -922,7 +923,7 @@ func TestAdjustKey(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -1054,11 +1055,11 @@ func TestStmtBuilderBodyField(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.WithUseJSONBody(t, c.enableUseJSONBody)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
// build the key map
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
for _, field := range IntrinsicFields {
for _, field := range logstelemetryschema.IntrinsicFields {
f := field
mockMetadataStore.KeysMap[field.Name] = append(mockMetadataStore.KeysMap[field.Name], &f)
}
@@ -1069,7 +1070,7 @@ func TestStmtBuilderBodyField(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -1155,11 +1156,11 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.WithUseJSONBody(t, c.enableUseJSONBody)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
// build the key map
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
for _, field := range IntrinsicFields {
for _, field := range logstelemetryschema.IntrinsicFields {
f := field
mockMetadataStore.KeysMap[field.Name] = append(mockMetadataStore.KeysMap[field.Name], &f)
}
@@ -1170,7 +1171,7 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
nil,
false,
@@ -1273,14 +1274,14 @@ func newSkipResourceFingerprintLogsBuilder(
t.Helper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
fm := logstelemetryschema.NewFieldMapper(fl)
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC))
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC))
aggExprRewriter := querybuilder.NewAggExprRewriter(
instrumentationtest.New().ToProviderSettings(),
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fm,
cb,
fl,
@@ -1292,7 +1293,7 @@ func newSkipResourceFingerprintLogsBuilder(
fm,
cb,
aggExprRewriter,
DefaultFullTextColumn,
logstelemetryschema.DefaultFullTextColumn,
fl,
telemetryStore,
skipEnable,

View File

@@ -1,4 +1,4 @@
package telemetrymeter
package meterstatementbuilder
import (
"context"
@@ -7,8 +7,12 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -21,19 +25,40 @@ type meterQueryStatementBuilder struct {
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
cb qbtypes.ConditionBuilder
metricsStatementBuilder *telemetrymetrics.MetricQueryStatementBuilder
metricsStatementBuilder *metricsstatementbuilder.StatementBuilder
}
var _ qbtypes.StatementBuilder[qbtypes.MetricAggregation] = (*meterQueryStatementBuilder)(nil)
// NewFactory returns a provider factory for the meter statement builder. Its New
// reuses the metrics FieldMapper/ConditionBuilder and delegates the final SELECT
// to a metrics statement builder built via the metrics factory.
func NewFactory(
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[qbtypes.StatementBuilder[qbtypes.MetricAggregation], statementbuilder.Config] {
return factory.NewProviderFactory(
factory.MustNewName("meter"),
func(ctx context.Context, settings factory.ProviderSettings, cfg statementbuilder.Config) (qbtypes.StatementBuilder[qbtypes.MetricAggregation], error) {
metricsStatementBuilder, err := metricsstatementbuilder.NewFactory(metadataStore, fl).New(ctx, settings, cfg)
if err != nil {
return nil, err
}
fm := metricstelemetryschema.NewFieldMapper()
cb := metricstelemetryschema.NewConditionBuilder(fm)
return NewMeterQueryStatementBuilder(settings, metadataStore, fm, cb, metricsStatementBuilder), nil
},
)
}
func NewMeterQueryStatementBuilder(
settings factory.ProviderSettings,
metadataStore telemetrytypes.MetadataStore,
fieldMapper qbtypes.FieldMapper,
conditionBuilder qbtypes.ConditionBuilder,
metricsStatementBuilder *telemetrymetrics.MetricQueryStatementBuilder,
metricsStatementBuilder *metricsstatementbuilder.StatementBuilder,
) *meterQueryStatementBuilder {
metricsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrymeter")
metricsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema")
return &meterQueryStatementBuilder{
logger: metricsSettings.Logger(),
@@ -53,7 +78,7 @@ func (b *meterQueryStatementBuilder) Build(
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
keySelectors := telemetrymetrics.GetKeySelectors(query)
keySelectors := metricsstatementbuilder.GetKeySelectors(query)
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, keySelectors)
if err != nil {
return nil, err
@@ -133,8 +158,8 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
sb.SelectMore(col)
}
tbl := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
tbl := metertelemetryschema.WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
aggCol, err := metertelemetryschema.AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
}
@@ -143,7 +168,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
}
sb.SelectMore(fmt.Sprintf("%s AS value", aggCol))
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
sb.From(fmt.Sprintf("%s.%s AS points", metertelemetryschema.DBName, tbl))
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
sb.GTE("unix_milli", start),
@@ -222,8 +247,8 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
sb.SelectMore(col)
}
tbl := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality,
tbl := metertelemetryschema.WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
aggCol, err := metertelemetryschema.AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality,
query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
@@ -234,7 +259,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
sb.From(fmt.Sprintf("%s.%s AS points", metertelemetryschema.DBName, tbl))
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
sb.GTE("unix_milli", start),
@@ -300,14 +325,14 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
baseSb.SelectMore(col)
}
tbl := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
tbl := metertelemetryschema.WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
aggCol, err := metertelemetryschema.AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
}
baseSb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
baseSb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
baseSb.From(fmt.Sprintf("%s.%s AS points", metertelemetryschema.DBName, tbl))
baseSb.Where(
baseSb.In("metric_name", query.Aggregations[0].MetricName),
baseSb.GTE("unix_milli", start),
@@ -350,7 +375,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
for _, g := range query.GroupBy {
wrapped.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", telemetrymetrics.RateTmpl))
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", metricsstatementbuilder.RateTmpl))
wrapped.From(fmt.Sprintf("(%s) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)", innerQuery))
q, args := wrapped.BuildWithFlavor(sqlbuilder.ClickHouse, innerArgs...)
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, nil
@@ -361,7 +386,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
for _, g := range query.GroupBy {
wrapped.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", telemetrymetrics.IncreaseTmpl))
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", metricsstatementbuilder.IncreaseTmpl))
wrapped.From(fmt.Sprintf("(%s) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)", innerQuery))
q, args := wrapped.BuildWithFlavor(sqlbuilder.ClickHouse, innerArgs...)
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, nil

View File

@@ -1,4 +1,4 @@
package telemetrymeter
package meterstatementbuilder
import (
"context"
@@ -7,7 +7,8 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -158,8 +159,8 @@ func TestStatementBuilder(t *testing.T) {
},
}
fm := telemetrymetrics.NewFieldMapper()
cb := telemetrymetrics.NewConditionBuilder(fm)
fm := metricstelemetryschema.NewFieldMapper()
cb := metricstelemetryschema.NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
keys, err := telemetrytypestest.LoadFieldKeysFromJSON("testdata/keys_map.json")
if err != nil {
@@ -169,7 +170,7 @@ func TestStatementBuilder(t *testing.T) {
flagger := flaggertest.New(t)
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), mockMetadataStore, fm, cb, flagger)
metricStmtBuilder := metricsstatementbuilder.NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), mockMetadataStore, fm, cb, flagger)
statementBuilder := NewMeterQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),

View File

@@ -1,4 +1,4 @@
package telemetrymetrics
package metricsstatementbuilder
import (
"context"
@@ -7,6 +7,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -118,8 +119,8 @@ func TestReducedStatementBuilder(t *testing.T) {
},
}
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := metricstelemetryschema.NewFieldMapper()
cb := metricstelemetryschema.NewConditionBuilder(fm)
fl, err := flagger.New(context.Background(), instrumentationtest.New().ToProviderSettings(), flagger.Config{}, flagger.MustNewRegistry())
require.NoError(t, err)
sb := NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), telemetrytypestest.NewMockMetadataStore(), fm, cb, fl)

View File

@@ -1,4 +1,4 @@
package telemetrymetrics
package metricsstatementbuilder
import (
"context"
@@ -9,6 +9,8 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/statementbuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -29,7 +31,7 @@ const (
OthersMultiTemporality = `IF(LOWER(temporality) LIKE LOWER('delta'), %s, %s) AS per_series_value`
)
type MetricQueryStatementBuilder struct {
type StatementBuilder struct {
logger *slog.Logger
metadataStore telemetrytypes.MetadataStore
fm qbtypes.FieldMapper
@@ -37,7 +39,24 @@ type MetricQueryStatementBuilder struct {
flagger flagger.Flagger
}
var _ qbtypes.StatementBuilder[qbtypes.MetricAggregation] = (*MetricQueryStatementBuilder)(nil)
var _ qbtypes.StatementBuilder[qbtypes.MetricAggregation] = (*StatementBuilder)(nil)
// NewFactory returns a provider factory for the metrics statement builder. Its
// New internalizes the FieldMapper and ConditionBuilder and yields the concrete
// *StatementBuilder so the meter builder can reuse it.
func NewFactory(
metadataStore telemetrytypes.MetadataStore,
fl flagger.Flagger,
) factory.ProviderFactory[*StatementBuilder, statementbuilder.Config] {
return factory.NewProviderFactory(
factory.MustNewName("metrics"),
func(_ context.Context, settings factory.ProviderSettings, _ statementbuilder.Config) (*StatementBuilder, error) {
fm := metricstelemetryschema.NewFieldMapper()
cb := metricstelemetryschema.NewConditionBuilder(fm)
return NewMetricQueryStatementBuilder(settings, metadataStore, fm, cb, fl), nil
},
)
}
func NewMetricQueryStatementBuilder(
settings factory.ProviderSettings,
@@ -45,9 +64,9 @@ func NewMetricQueryStatementBuilder(
fieldMapper qbtypes.FieldMapper,
conditionBuilder qbtypes.ConditionBuilder,
flagger flagger.Flagger,
) *MetricQueryStatementBuilder {
metricsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrymetrics")
return &MetricQueryStatementBuilder{
) *StatementBuilder {
metricsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema")
return &StatementBuilder{
logger: metricsSettings.Logger(),
metadataStore: metadataStore,
fm: fieldMapper,
@@ -89,7 +108,7 @@ func GetKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation])
return keySelectors
}
func (b *MetricQueryStatementBuilder) Build(
func (b *StatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
@@ -126,7 +145,7 @@ func (b *MetricQueryStatementBuilder) Build(
return b.buildPipelineStatement(ctx, orgID, start, end, query, keys, variables)
}
func (b *MetricQueryStatementBuilder) buildPipelineStatement(
func (b *StatementBuilder) buildPipelineStatement(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
@@ -188,11 +207,11 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
// shaped exactly like samples_v4 / time_series_v4, so once the table names are
// chosen the rest of the pipeline is unchanged.
useBuffer := agg.Reduced &&
end-start < oneDayInMilliseconds &&
start >= uint64(time.Now().UnixMilli())-oneDayInMilliseconds
end-start < metricstelemetryschema.OneDayInMilliseconds &&
start >= uint64(time.Now().UnixMilli())-metricstelemetryschema.OneDayInMilliseconds
samplesTable, _ := WhichSamplesTableToUse(start, end, agg.Type, agg.TimeAggregation, useBuffer, agg.TableHints)
tsStart, tsEnd, _, tsTable := WhichTSTableToUse(start, end, useBuffer, agg.TableHints)
samplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(start, end, agg.Type, agg.TimeAggregation, useBuffer, agg.TableHints)
tsStart, tsEnd, _, tsTable := metricstelemetryschema.WhichTSTableToUse(start, end, useBuffer, agg.TableHints)
var timeSeriesCTE string
var timeSeriesCTEArgs []any
@@ -232,7 +251,7 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
var tsCTE string
var tsArgs []any
// time series rows are written on hour boundaries
tsStart := start - (start % oneHourInMilliseconds)
tsStart := start - (start % metricstelemetryschema.OneHourInMilliseconds)
if tsCTE, tsArgs, err = b.buildReducedTimeSeriesCTE(ctx, orgID, tsStart, end, query, keys, variables); err != nil {
return nil, err
}
@@ -282,7 +301,7 @@ func unionStatements(main, reduced *qbtypes.Statement, query qbtypes.QueryBuilde
return &qbtypes.Statement{Query: q, Args: args, Warnings: warnings}, nil
}
func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
func (b *StatementBuilder) buildReducedTimeSeriesCTE(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
@@ -312,7 +331,7 @@ func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, TimeseriesV4ReducedLocalTableName))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
sb.Select("fingerprint")
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &g.TelemetryFieldKey, telemetrytypes.FieldDataTypeString, keys)
@@ -343,7 +362,7 @@ func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
// spatial_aggregation_cte in one level with no per-series grouping, so shards
// send one state per (step, group) instead of per (series, step, group).
// FINAL still dedups recomputed 60s buckets at scan time.
func (b *MetricQueryStatementBuilder) buildReducedSpatialAggFastPath(
func (b *StatementBuilder) buildReducedSpatialAggFastPath(
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
timeSeriesCTE string,
@@ -352,7 +371,7 @@ func (b *MetricQueryStatementBuilder) buildReducedSpatialAggFastPath(
agg := query.Aggregations[0]
stepSec := int64(query.StepInterval.Seconds())
value, _, ok := ReducedValueColumn(agg.Type, agg.SpaceAggregation)
value, _, ok := metricstelemetryschema.ReducedValueColumn(agg.Type, agg.SpaceAggregation)
if !ok {
return "", nil, false
}
@@ -362,8 +381,8 @@ func (b *MetricQueryStatementBuilder) buildReducedSpatialAggFastPath(
for _, g := range query.GroupBy {
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
sb.SelectMore(fmt.Sprintf("%s AS value", ReducedTimeAggregationColumn(agg.TimeAggregation, stepSec, value)))
sb.From(fmt.Sprintf("%s.%s AS points FINAL", DBName, WhichReducedSamplesTableToUse(agg.Type)))
sb.SelectMore(fmt.Sprintf("%s AS value", metricstelemetryschema.ReducedTimeAggregationColumn(agg.TimeAggregation, stepSec, value)))
sb.From(fmt.Sprintf("%s.%s AS points FINAL", metricstelemetryschema.DBName, metricstelemetryschema.WhichReducedSamplesTableToUse(agg.Type)))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.reduced_fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", agg.MetricName),
@@ -377,7 +396,7 @@ func (b *MetricQueryStatementBuilder) buildReducedSpatialAggFastPath(
return fmt.Sprintf("__spatial_aggregation_cte AS (%s)", q), args, true
}
func (b *MetricQueryStatementBuilder) buildReducedTemporalAggregationCTE(
func (b *StatementBuilder) buildReducedTemporalAggregationCTE(
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
timeSeriesCTE string,
@@ -386,7 +405,7 @@ func (b *MetricQueryStatementBuilder) buildReducedTemporalAggregationCTE(
agg := query.Aggregations[0]
stepSec := int64(query.StepInterval.Seconds())
value, weight, ok := ReducedValueColumn(agg.Type, agg.SpaceAggregation)
value, weight, ok := metricstelemetryschema.ReducedValueColumn(agg.Type, agg.SpaceAggregation)
if !ok {
return "", nil, false
}
@@ -399,13 +418,13 @@ func (b *MetricQueryStatementBuilder) buildReducedTemporalAggregationCTE(
for _, g := range query.GroupBy {
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", ReducedTimeAggregationColumn(agg.TimeAggregation, stepSec, value)))
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", metricstelemetryschema.ReducedTimeAggregationColumn(agg.TimeAggregation, stepSec, value)))
if weight != "" {
// count_series is a series count, not additive over time, so the avg
// denominator is reduced with avg
sb.SelectMore(fmt.Sprintf("avg(%s) AS per_series_weight", weight))
}
sb.From(fmt.Sprintf("%s.%s AS points FINAL", DBName, WhichReducedSamplesTableToUse(agg.Type)))
sb.From(fmt.Sprintf("%s.%s AS points FINAL", metricstelemetryschema.DBName, metricstelemetryschema.WhichReducedSamplesTableToUse(agg.Type)))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.reduced_fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", agg.MetricName),
@@ -419,7 +438,7 @@ func (b *MetricQueryStatementBuilder) buildReducedTemporalAggregationCTE(
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, true
}
func (b *MetricQueryStatementBuilder) buildReducedSpatialAggregationCTE(
func (b *StatementBuilder) buildReducedSpatialAggregationCTE(
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (string, []any) {
spatial := "sum(per_series_value)"
@@ -446,7 +465,7 @@ func (b *MetricQueryStatementBuilder) buildReducedSpatialAggregationCTE(
return fmt.Sprintf("__spatial_aggregation_cte AS (%s)", q), args
}
func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
func (b *StatementBuilder) buildTemporalAggDeltaFastPath(
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
samplesTable string,
@@ -465,7 +484,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggCol, err := AggregationColumnForSamplesTable(
aggCol, err := metricstelemetryschema.AggregationColumnForSamplesTable(
samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation,
)
if err != nil {
@@ -483,7 +502,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
sb.SelectMore(fmt.Sprintf("%s AS value", aggCol))
sb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
sb.From(fmt.Sprintf("%s.%s AS points", metricstelemetryschema.DBName, samplesTable))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
@@ -497,7 +516,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
return fmt.Sprintf("__spatial_aggregation_cte AS (%s)", q), args, nil
}
func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
func (b *StatementBuilder) buildTimeSeriesCTE(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
@@ -529,7 +548,7 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, tsTable))
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, tsTable))
sb.Select("fingerprint")
for _, g := range query.GroupBy {
@@ -552,7 +571,7 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
// the buffer holds both raw rows and the reduced catalog rows; the raw read
// only wants the original series
if tsTable == TimeseriesV4BufferLocalTableName {
if tsTable == metricstelemetryschema.TimeseriesV4BufferLocalTableName {
sb.Where(sb.EQ("is_reduced", false))
}
@@ -567,7 +586,7 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
return fmt.Sprintf("(%s) AS filtered_time_series", q), args, nil
}
func (b *MetricQueryStatementBuilder) buildTemporalAggregationCTE(
func (b *StatementBuilder) buildTemporalAggregationCTE(
ctx context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
@@ -584,7 +603,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggregationCTE(
return b.buildTemporalAggForMultipleTemporalities(ctx, start, end, query, samplesTable, timeSeriesCTE, timeSeriesCTEArgs)
}
func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
func (b *StatementBuilder) buildTemporalAggDelta(
_ context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
@@ -605,7 +624,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggCol, err := AggregationColumnForSamplesTable(samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation)
aggCol, err := metricstelemetryschema.AggregationColumnForSamplesTable(samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation)
if err != nil {
return "", nil, err
}
@@ -616,7 +635,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
sb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
sb.From(fmt.Sprintf("%s.%s AS points", metricstelemetryschema.DBName, samplesTable))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
@@ -631,7 +650,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, nil
}
func (b *MetricQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
func (b *StatementBuilder) buildTemporalAggCumulativeOrUnspecified(
_ context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
@@ -651,13 +670,13 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
baseSb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggCol, err := AggregationColumnForSamplesTable(samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation)
aggCol, err := metricstelemetryschema.AggregationColumnForSamplesTable(samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation)
if err != nil {
return "", nil, err
}
baseSb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
baseSb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
baseSb.From(fmt.Sprintf("%s.%s AS points", metricstelemetryschema.DBName, samplesTable))
baseSb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
baseSb.Where(
baseSb.In("metric_name", query.Aggregations[0].MetricName),
@@ -697,7 +716,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
}
}
func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
func (b *StatementBuilder) buildTemporalAggForMultipleTemporalities(
_ context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
@@ -716,11 +735,11 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggForDeltaTemporality, err := AggregationColumnForSamplesTable(samplesTable, metrictypes.Delta, query.Aggregations[0].TimeAggregation)
aggForDeltaTemporality, err := metricstelemetryschema.AggregationColumnForSamplesTable(samplesTable, metrictypes.Delta, query.Aggregations[0].TimeAggregation)
if err != nil {
return "", nil, err
}
aggForCumulativeTemporality, err := AggregationColumnForSamplesTable(samplesTable, metrictypes.Cumulative, query.Aggregations[0].TimeAggregation)
aggForCumulativeTemporality, err := metricstelemetryschema.AggregationColumnForSamplesTable(samplesTable, metrictypes.Cumulative, query.Aggregations[0].TimeAggregation)
if err != nil {
return "", nil, err
}
@@ -748,7 +767,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
sb.SelectMore(expr)
}
sb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
sb.From(fmt.Sprintf("%s.%s AS points", metricstelemetryschema.DBName, samplesTable))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
@@ -762,7 +781,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", queryWithWindowAndOrder), args, nil
}
func (b *MetricQueryStatementBuilder) buildSpatialAggregationCTE(
func (b *StatementBuilder) buildSpatialAggregationCTE(
_ context.Context,
_ uint64,
_ uint64,
@@ -788,7 +807,7 @@ func (b *MetricQueryStatementBuilder) buildSpatialAggregationCTE(
return fmt.Sprintf("__spatial_aggregation_cte AS (%s)", q), args
}
func (b *MetricQueryStatementBuilder) BuildFinalSelect(
func (b *StatementBuilder) BuildFinalSelect(
cteFragments []string,
cteArgs [][]any,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
@@ -833,7 +852,7 @@ func (b *MetricQueryStatementBuilder) BuildFinalSelect(
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggQuery, err := AggregationQueryForHistogramCountWithParams(query.Aggregations[0].ComparisonSpaceAggregationParam)
aggQuery, err := metricstelemetryschema.AggregationQueryForHistogramCountWithParams(query.Aggregations[0].ComparisonSpaceAggregationParam)
if err != nil {
return nil, err
}

View File

@@ -1,4 +1,4 @@
package telemetrymetrics
package metricsstatementbuilder
import (
"context"
@@ -7,6 +7,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -252,8 +253,8 @@ func TestStatementBuilder(t *testing.T) {
},
}
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := metricstelemetryschema.NewFieldMapper()
cb := metricstelemetryschema.NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
keys, err := telemetrytypestest.LoadFieldKeysFromJSON("testdata/keys_map.json")
if err != nil {

View File

@@ -1,4 +1,4 @@
package telemetryresourcefilter
package resourcefilter
import (
"context"

View File

@@ -1,4 +1,4 @@
package telemetryresourcefilter
package resourcefilter
import (
"context"

View File

@@ -1,4 +1,4 @@
package telemetryresourcefilter
package resourcefilter
import (
"context"

Some files were not shown because too many files have changed in this diff Show More