Commit Graph

269 Commits

Author SHA1 Message Date
Nikhil Soni
e08ef01170 refactor(savedview): restructure api and storage to spec based (#12342)
## Summary

- Saved views now persist a versioned, typed spec (`schemaVersion` +
`spec{compositeQuery, selectedFields, display}`) instead of a bare
composite-query blob plus an opaque, frontend-owned `extraData` string
-- mirroring the pattern dashboards already use for their v2/perses
schema.
- `/api/v1/explorer/views` keeps working exactly as before: a thin
conversion layer translates to/from the legacy wire format, including
folding `extraData`'s ad hoc JSON into the typed spec and back for
backward compatibility.
- A one-time migration rewrites existing rows into the new shape and
drops the now-unused `extra_data`/`category`/`tags` columns.

### Scaffolding decisions 
- Using v2 for new handlers instead of renaming old handlers to
something else for these reasons - keep the diff minimum for easier
reviews, avoiding any git history or last updated at change in old route
registration.
- Keeping the conversion to old saved view type in handler itself rather
than `savedviewtypes` package to keep it un-exported and not let them be
available anywhere else to be used. It also enables `savedviewtypes` to
be independent on query-service models.
- Modified the existing handler and it's interface to include the v2
methods instead of adding another handlerV2 since apiserver already had
handler wired in, so don't want to pass on 2 version simultaneously.

### Breaking change
- Any unknown key in the `ExtraData` will be rejected and dropped
silently in the old APIs and give error in new version.
- If there was any way to add tag or category in saved view earlier,
that data will be lost.
- Old APIs will not support the old QB request payload, only v5 format
is supported.

---

Closes SigNoz/engineering-pod#4651

Alternative discarded https://github.com/SigNoz/signoz/pull/12208

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-07 13:37:56 +00:00
Nikhil Mantri
465e567028 fix(infra-monitoring): use local sample tables in custom queries in metric-reduction-feature path (#12413)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* chore: using local tables in place of distributed tables in infra-monitoring reduced metric path

* test: infra-monitoring list endpoints on the metrics reduction path

* chore: address review comments
2026-08-05 17:18:49 +00:00
Ashwin Bhatkal
a6ac14344e fix(dashboard): count panel stats from the v2 spec (#12396)
* fix(dashboard): count panel stats from the v2 spec

The panel counters walked a top-level `widgets` array and returned early
when the key was missing. A v2 dashboard stores only `metadata` and
`spec`, with panels as a map under `spec.panels`, so every v2 row hit
that early return and all `dashboard.panels.*` stats stayed at zero.
`dashboard.count` was unaffected — it is a row count.

Read the v2 spec instead: count the panels under `spec.panels` and take
each panel's signal from its query envelope, reusing the typed read path
and `QueryEnvelope.GetSignal`. Signal-less queries (promql, clickhouse
sql, formulas) count towards the panel total only. v1 rows are no longer
parsed for panel stats and contribute to `dashboard.count` alone.

* test(dashboard): drop the constant name arg from the stats query helper

statsBuilderQuery only ever received "A", which go-lint flags via unparam.
A panel holds a single query, so the name never mattered to the assertions;
the composite test still names its sub-queries through statsBuilderQuerySpec.

* refactor(dashboard): move v2 panel stats to a perses_ file

All v2 code lives in perses_-prefixed files until the v1 code goes away.
Pure move of the stats block out of dashboard.go, tests alongside it.

* fix(dashboard): count create-v2 stats off the postable spec

CreateV2 already holds the postable dashboard, so decoding the storable
back into a v2 dashboard just to count its panels was a needless type
conversion on the create path.

Split the panel walk into addPanelStats over a DashboardSpec, and add
NewStatsFromPostableDashboardV2 for the create path. The storable variant
keeps its signature for the periodic collectors, which only have rows.
2026-08-05 04:16:02 +00:00
Nityananda Gohain
816ae7760e feat: QB support for llm trace list and span list (#12027)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat: support llm trace list and span list

* fix: take perf into consideration

* fix: more tests

* fix: more cleanup

* fix: cleanup and more tests

* fix: add resource fingerprint cte

* fix: edge cases and correct cost key

* fix: update integration test

* fix: update openapi

* fix: address comments

* fix: address comments

* fix: fix tests

* fix: add back the flag in metadata

* fix: remove source and change to builder ai query

* fix: updated openapi

* fix: minor cleanup

* fix: refactor as requested

* fix: address comments

* fix: address comments

* fix: remove tracefield. explicit rejection

* fix: remove comment

* fix: remove accidentally added file

* fix: add NewFactory

* fix: refactor integration tests

* fix: address comment

* fix: use assert

* fix: use assert and condense comments
2026-08-04 16:44:43 +00:00
Naman Verma
089f9eb4c6 chore: add api to retry migration for a dashboard (#12387)
* chore: add api to retry migration for a dashboard

* feat(dashboard): add retry migration action for legacy dashboards

The legacy-dashboard dialog only offered the dashboard ID and a link to
support. Now that the v1->v2 migration can be re-run on demand, let an
editor trigger it from there and fall back to support only if it still
fails.

Retrying needs edit access (the endpoint is EDITOR-gated), so viewers
keep the ID-and-support dialog unchanged.

---------

Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
2026-08-04 13:16:43 +00:00
Swapnil Nakade
34041be308 fix: adjust aggregation values for GCP metrics (#12391) 2026-08-04 09:41:14 +00:00
Tushar Vats
77c1b601be feat(logs): implement search() across all log fields (#12244)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
- Replace the search() stub: the logs condition builder fans a case-insensitive
  match of the term across every searchable column (log columns, body/body_v2,
  attribute + resource maps).
- Grammar: searchCall takes a valueList (parser regenerated), so the scoped form
  search('term', body, resource) parses and narrows the fan-out to those contexts.
- The visitor emits FilterOperatorSearch and flags the statement scan-heavy; the
  statement builder attaches a CostGuard the querier enforces via EXPLAIN ESTIMATE
  against a per-shard budget, cumulative across buckets on the window-list path.
- body_v2 gets its own, lower budget (search_max_scan_rows_json_body): toString()
  rebuilds every document and no skip index prunes.
- Logs-only: other signals reject search().
- Unit tests for the per-context fan-out SQL and both budgets; integration suites
  run the same matrix over the legacy body and over body_v2.
2026-08-03 19:03:38 +00:00
Nikhil Mantri
16c6fdc600 chore: combined function for goroutines (#12362)
Some checks failed
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
2026-08-01 07:06:09 +00:00
Vinicius Lourenço
ee06d549e6 chore(alert-history): use v2 apis (#12132)
* feat(query-search): add support for custom value fetcher

* feat(alert-history): use v2 apis

* refactor(api): remove types and old apis

* fix(rule-history): not accepting severity/threshold.name as filter

* fix(cursor-pagination): simplify pagination

* fix(timeline-table): keep as asc

No idea why it was preferred this way

* fix(hooks): reset page after filter change

* refactor(alert-history): removed unused code

* fix(alert-history-table): add more safe checks

* fix(pr): address comments

* fix(pr): fallback to known existing keys on error for keys

* fix(pr): reset page on change time

* fix(query-search): pass complete prop

* fix(query-search): clean the code a little bit

* fix(filter-suggestions): remove limit

* feat(links): add back the links

* fix(table): remove type hack with new support for links from backend

* fix(alert-history-suggestions): render empty keys when api call fails

And also shows good loading feedback

* fix(query-search): backdrop-filter leaking visual artifacts for some reason

I was only able to reproduce this on alert-history,
not sure why other places don't have the same issue

* fix(alert-history-filter-suggestions): broken types after openapi update

* Revert "fix(query-search): backdrop-filter leaking visual artifacts for some reason"

This reverts commit c8b28c81ef.

* fix(alert-history-query-search): a simple fix for the issue of the visual artifacts

* fix(store): be extract safe in case fieldKeys is nil

* refactor(graph): improve a little bit the state colors for recovering/pending

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-07-30 17:25:17 +00:00
Swapnil Nakade
bad3850117 feat: adding cloud storage service for GCP integration (#12341)
* feat: adding gcp memorystore redis service

* refactor: updating dashboard title

* refactor: extending width of uptime gauge panel

* refactor: updating cpu utilization panel

* refactor: updating dashboard panel to use rate function instead of hack

* feat: adding compute engine service

* refactor: updating dashboard panels to use rate aggregation

* fix: correct typo and unit in compute engine dashboard

* refactor: migrating dashboard to v6

* feat: adding gcp gke service

* chore: generating openapi spec

* refactor: updating icon in dashboard JSON

* feat: adding gcp cloud storage service

* refactor: updating dashboard and integration config

* refactor: updating cloud storage icon
2026-07-30 10:09:06 +00:00
Swapnil Nakade
5a8ca9573c feat: adding gcp gke service (#12319)
* feat: adding gcp memorystore redis service

* refactor: updating dashboard title

* refactor: extending width of uptime gauge panel

* refactor: updating cpu utilization panel

* refactor: updating dashboard panel to use rate function instead of hack

* feat: adding compute engine service

* refactor: updating dashboard panels to use rate aggregation

* fix: correct typo and unit in compute engine dashboard

* refactor: migrating dashboard to v6

* feat: adding gcp gke service

* chore: generating openapi spec

* refactor: updating icon in dashboard JSON
2026-07-30 08:48:34 +00:00
Swapnil Nakade
0703fbf961 feat: adding compute engine service in GCP integration (#12166)
* feat: adding gcp memorystore redis service

* refactor: updating dashboard title

* refactor: extending width of uptime gauge panel

* refactor: updating cpu utilization panel

* refactor: updating dashboard panel to use rate function instead of hack

* feat: adding compute engine service

* refactor: updating dashboard panels to use rate aggregation

* fix: correct typo and unit in compute engine dashboard

* refactor: migrating dashboard to v6
2026-07-29 11:28:58 +00:00
Tushar Vats
e2e7caf1ca refactor(querier): 3-layer per-signal query architecture (#12304)
Split the five telemetry<signal> packages, which mixed three concerns, into
three per-signal layers with a cycle-free dependency direction:

- telemetryschema/<signal>telemetryschema — primitives (const + table
  selection, field_mapper, condition_builder, trace helpers); leaf layer.
- statementbuilder/<signal>statementbuilder — SQL generation. The parent
  statementbuilder package is contract-only (the Builders bundle + Config);
  each sub-package exposes a factory.ProviderFactory[..., statementbuilder.Config]
  whose New internalizes FieldMapper/ConditionBuilder/AggExprRewriter and reads
  SkipResourceFingerprint. Traces exposes two factories (query + operator).
  telemetryresourcefilter moves here as statementbuilder/resourcefilter.
- telemetrymetadata — key/value resolution; NewTelemetryMetaStore collapses
  from 24 args to (settings, telemetrystore, flagger), sourcing table names
  from the schema constants.

Centralize query-stack assembly in signoz.go via newQueryStack: build the
single metadata store, run each per-signal statement-builder factory, assemble
the statementbuilder.Builders bundle, and build the bucket cache — once. This
is the only place that imports the concrete sub-packages (so the edge runs
subs -> parent, cycle-free), and it removes the duplicate metadata store that
signozquerier used to build, leaving signozquerier a thin
querier.New(*statementbuilder.Builders) adapter.

Also:
- statementbuilder.Config owns SkipResourceFingerprint (moved off
  querier.Config). YAML key moves querier.skip_resource_fingerprint ->
  statementbuilder.skip_resource_fingerprint.
- Querier interface moves into querier.go (interfaces.go removed); BucketCache
  -> bucket_cache.go, Handler -> api.go.
- Add pkg/querier/queriertest.MockQuerier.
2026-07-29 09:24:08 +00:00
Swapnil Nakade
1a6a693466 refactor(cloud-integrations): remove cloud_integration_id query param (#12300) 2026-07-27 17:57:09 +00:00
Srikanth Chekuri
08715a704a feat(rulestatehistory): populate related logs/traces links in v2 history APIs (#12211)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat(rulestatehistory): populate related logs/traces links in v2 history APIs

The v2 rule history timeline dropped the relatedLogsLink/relatedTracesLink
that v1 (getRuleStateHistory) returned per entry, which the alert history
page uses to jump from a state change to the explorer with the rule's
filter and the entry's labels. Load the rule from the rule store in the
module and build the links with contextlinks, scoped to the entry's
evaluation window like v1.

Extract the builder-query filter/group-by selection that the v1 handler
and threshold rule notifications each inlined into
contextlinks.BuilderQueryForSignal and reuse it from both the module and
ThresholdRule.

Also populate the links for top contributors, which both v1 (since #10760)
and v2 returned as always-empty fields even though the UI renders them;
contributor links span the queried range since the counts aggregate it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(contextlinks): remove unused v3 link helpers

PrepareLinksToTraces, PrepareLinksToLogs and PrepareFilters lost their
last callers when the deprecated v3/v4 rule support was removed in #10760;
the v5 equivalents (PrepareParamsFor*V5 and PrepareFilterExpression) are
what all remaining callers use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: end doc comments with a period to satisfy godot

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(alerts): cover v2 rule history related links for logs and traces

Each test fires a rule with a filter and a service.name group-by, then
asserts the recorded firing entry and top contributor carry a related
explorer link for the rule's signal only, with the label-rewritten filter
expression, the evaluation window on timeline entries and the queried
range on contributors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(contextlinks): shrink explorer links to the minimal payload

The explorer pages read only the data source and filter expression from a
shared link and fill in the rest of the query shape with defaults, so stop
shipping the v3 builder-query ceremony (queryName, aggregateOperator,
aggregateAttribute, stepInterval, paging fields) and the timeRange and
options params nothing reads. Links shrink from ~1.2k to ~450 chars and
contextlinks no longer depends on the v3 model.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(rulestatehistory): derive link windows from the evaluation envelope

Rules created through the current UI store the window in the v2alpha1
evaluation envelope with no top-level evalWindow, so the previous 5m
fallback produced wrong link windows for any non-default rolling window
and could not represent cumulative windows at all. Use the envelope's
NextWindowFor like the rule engine does, keeping the top-level
evalWindow (default 5m) as the fallback for rules without an envelope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(contextlinks): simplify double-encoding comment

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(alerts): use literal matchType/op in fixtures and drop link unit tests

Replace the numeric matchType/op codes in all alert scenario fixtures
with their literal forms (at_least_once, above, ...) which the API
normalizes to the same canonical values, and remove the rule history
link unit tests since the integration tests cover the behavior
end-to-end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(alerts): move rule history helpers into the shared alerts fixtures

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 06:38:44 +00:00
Naman Verma
774dfc48ae chore: deprecate v1 dashboard APIs with proper reference to v2 APIs (#12273)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* chore: deprecate v1 dashboard APIs with proper reference to v2 APIs

* fix: use generic doc for all deprecated APIs

* test: update integratoin tests to account for deprecation
2026-07-26 14:43:51 +00:00
Naman Verma
d013eab29d chore: migrate integration dashboards to v2 (#12242)
* chore: add migration script from current to perses dashboard

* fix: move WrapInV5Envelope to types package

* fix: generate internal name from title

* fix: match with lower case signal for variables

* fix: sanitize tags instead of throwing error

* fix: extract row and widget positions to build expanded sections

* chore: add a catch all panic check to log migration error

* chore: catch typecast errors

* fix: remove nil nil return

* test: fix ut to check for v2 internal name

* chore: remove spec md file

* chore: separate error type for migration

* feat: note down all errors in migration

* fix: allow zero value for threshold values

* fix: ignore react placeholders in layout

* fix: note error in query parsing

* fix: read threshold value properly during migration

* fix: adjust to schema changes in variables

* test: fix unit tests

* fix: catch error from decodeTelemetryFields

* fix: make threshold value non-nullable

* fix: handle list of widget IDs in panel maps

* fix: add better error note down for failed conversions

* fix: normalize having array from v4 schema

* fix: remove metric aggregation from non metric queries (malformed json)

* fix: normalize telemetry field keys in query

* chore: add temporary doc for other malformation handlers needed

* fix: add more malformed query handling

* fix: add more malformation handling

* chore: reuse transition package (adds dependency, need to see what to do)

* fix: fix go lint issue

* revert: remove md file

* fix: make none the default fill mode during migration

* fix: add nil check on limit value when normalizing page size

* fix: validate dashboard spec after migration

* fix: remove unused method

* fix: match shape safe version to existing version

* chore: improve variable names

* chore: better comments and names in layout migrator

* chore: add explanation comment for continuing on empty id widget

* fix: ditch dynamic vars with no attr, rearrange overlapping layouts

* fix: add default aggregation count() if none is present in logs/traces

* fix: add empty list tolerance to v1 variables

* fix: add fixes based on backup migration testing

* chore: skip over unreferenced widgets

* chore: remove widgets that have a single query that has no metric name

* chore: assign missing formula names like ui does

* chore: assign query names to queries if missing

* chore: drop legacy filter fields that ui also ignores

* chore: make migration drop widgets that have no aggregations for metrics

* chore: normalize group by

* chore: remove expressions from metric aggregations

* chore: add more error notes

* chore: replace em dash with hyphen in widget id

* chore: add more panel id sanitization

* chore: replace faulty order by keys with aggregation expression

* chore: fill valid span gaps

* chore: handle misformed threshold operators

* chore: remove repeat occurences of widgets in layouts

* chore: remove limits that cross max limit

* chore: restore function args malformed by v4->v5 migration

* chore: remove invalid functions

* chore: skip variables in wrong type

* chore: add wdiget id type check in layout migration

* chore: add a.count to malformed order by keys list

* chore: remove bad aggregations that accompany valid ones

* chore: drop trace operators with no expression

* chore: drop invalid formulas

* chore: skip unknown panel kind

* chore: drop widgets with >1 and all aggregations as malformed

* chore: skip empty type variables

* chore: pass threshold format raw without typecasting

* chore: add handling for trace operators around wrapinv5env

* chore: handle no string value for variable sort

* test: fix span gaps test

* chore: undo schema change in threshold value

* chore: final set of error fixes/hacks

* fix: fix build errors post merge conflicts

* test: remove test for floating pt value

* test: use appendf instead of sprintf

* chore: migrate integration dashboards to v2

* feat: add scaffolding to migrate dashboards

* fix: fill in ref path when building grid layout

* feat: handle v1 payloads in v2 create api

* fix: return empty objects instead of null if no variables/panels/layouts are there

* test: fix unit test for empty layouts

* fix: add empty links array in migration

* chore: rerun migration after links and signal change

* test: add unit test to show v5  takes precedence over v4

* fix: change EXISTS to exists so that migrateCommon handles it properly

* test: add unit test for count with attributes

* fix: run order by handling after default aggregation handling

* fix: restore formula fields that wrapinv5envelope removes

* fix: keep thresholds that do not have an optional label

* fix: force allowAll to be false if allowMultiple is false

* chore: rerun migration

* fix: migrate name column even if data column migration fails

* fix: use slog for logging

* refactor(dashboard): re-host ConvertAllV1ToV2 off the Module interface

Mirror base's removal of ConvertAllV1ToV2 from the shared files so future
base→branch merges stay conflict-free, and re-host the capability in dedicated
branch-only files (V1ToV2Migrator interface + module method + result types).

* refactor(dashboard): drop ConvertAllV1ToV2 module scaffolding from base

Remove the temporary v1→v2 bulk-conversion module method (ConvertAllV1ToV2 +
migrateOneV1ToV2), its result types (V1ToV2MigrationResult / V1ToV2MigrationItem),
the Module interface entry, and the EE delegation. This scaffolding lives on the
migration feature branches, not the base branch that merges to main.

V1Name and store.UpdateName are kept (small helpers; UpdateName is on the shared
Store interface).

* fix: add image validation and base64 to asset conversion

* fix: remove unneeded interface

* chore: rerun migration

* fix: make ReencodeInlineSVGImage so that migration script can call it

* chore: remove temp script

* fix: remove local laptop default value

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-07-26 14:15:54 +00:00
Srikanth Chekuri
18f5baafa0 fix: enforce scalar response contract + reject invalid v2 dashboard bodies (#12276)
* fix: enforce scalar response contract for promql and metric builder queries

* fix: reject invalid v2 dashboard bodies instead of migrating them as v1

* fix(dashboards-v2): post legacy v1 dashboards unchanged from the import flow

* fix: add reduce to for scalar requests if not present

---------

Co-authored-by: Naman Verma <naman.verma@signoz.io>
2026-07-26 11:25:01 +00:00
Naman Verma
51819d6337 chore: add migration script from current to perses dashboard (#11712)
* chore: add migration script from current to perses dashboard

* fix: move WrapInV5Envelope to types package

* fix: generate internal name from title

* fix: match with lower case signal for variables

* fix: sanitize tags instead of throwing error

* fix: extract row and widget positions to build expanded sections

* chore: add a catch all panic check to log migration error

* chore: catch typecast errors

* fix: remove nil nil return

* test: fix ut to check for v2 internal name

* chore: remove spec md file

* chore: separate error type for migration

* feat: note down all errors in migration

* fix: allow zero value for threshold values

* fix: ignore react placeholders in layout

* fix: note error in query parsing

* fix: read threshold value properly during migration

* fix: adjust to schema changes in variables

* test: fix unit tests

* fix: catch error from decodeTelemetryFields

* fix: make threshold value non-nullable

* fix: handle list of widget IDs in panel maps

* fix: add better error note down for failed conversions

* fix: normalize having array from v4 schema

* fix: remove metric aggregation from non metric queries (malformed json)

* fix: normalize telemetry field keys in query

* chore: add temporary doc for other malformation handlers needed

* fix: add more malformed query handling

* fix: add more malformation handling

* chore: reuse transition package (adds dependency, need to see what to do)

* fix: fix go lint issue

* revert: remove md file

* fix: make none the default fill mode during migration

* fix: add nil check on limit value when normalizing page size

* fix: validate dashboard spec after migration

* fix: remove unused method

* fix: match shape safe version to existing version

* chore: improve variable names

* chore: better comments and names in layout migrator

* chore: add explanation comment for continuing on empty id widget

* fix: ditch dynamic vars with no attr, rearrange overlapping layouts

* fix: add default aggregation count() if none is present in logs/traces

* fix: add empty list tolerance to v1 variables

* fix: add fixes based on backup migration testing

* chore: skip over unreferenced widgets

* chore: remove widgets that have a single query that has no metric name

* chore: assign missing formula names like ui does

* chore: assign query names to queries if missing

* chore: drop legacy filter fields that ui also ignores

* chore: make migration drop widgets that have no aggregations for metrics

* chore: normalize group by

* chore: remove expressions from metric aggregations

* chore: add more error notes

* chore: replace em dash with hyphen in widget id

* chore: add more panel id sanitization

* chore: replace faulty order by keys with aggregation expression

* chore: fill valid span gaps

* chore: handle misformed threshold operators

* chore: remove repeat occurences of widgets in layouts

* chore: remove limits that cross max limit

* chore: restore function args malformed by v4->v5 migration

* chore: remove invalid functions

* chore: skip variables in wrong type

* chore: add wdiget id type check in layout migration

* chore: add a.count to malformed order by keys list

* chore: remove bad aggregations that accompany valid ones

* chore: drop trace operators with no expression

* chore: drop invalid formulas

* chore: skip unknown panel kind

* chore: drop widgets with >1 and all aggregations as malformed

* chore: skip empty type variables

* chore: pass threshold format raw without typecasting

* chore: add handling for trace operators around wrapinv5env

* chore: handle no string value for variable sort

* test: fix span gaps test

* chore: undo schema change in threshold value

* chore: final set of error fixes/hacks

* fix: fix build errors post merge conflicts

* test: remove test for floating pt value

* test: use appendf instead of sprintf

* feat: add scaffolding to migrate dashboards

* fix: fill in ref path when building grid layout

* feat: handle v1 payloads in v2 create api

* fix: return empty objects instead of null if no variables/panels/layouts are there

* test: fix unit test for empty layouts

* fix: add empty links array in migration

* test: add unit test to show v5  takes precedence over v4

* fix: change EXISTS to exists so that migrateCommon handles it properly

* test: add unit test for count with attributes

* fix: run order by handling after default aggregation handling

* fix: restore formula fields that wrapinv5envelope removes

* fix: keep thresholds that do not have an optional label

* fix: force allowAll to be false if allowMultiple is false

* fix: migrate name column even if data column migration fails

* fix: use slog for logging

* refactor(dashboard): drop ConvertAllV1ToV2 module scaffolding from base

Remove the temporary v1→v2 bulk-conversion module method (ConvertAllV1ToV2 +
migrateOneV1ToV2), its result types (V1ToV2MigrationResult / V1ToV2MigrationItem),
the Module interface entry, and the EE delegation. This scaffolding lives on the
migration feature branches, not the base branch that merges to main.

V1Name and store.UpdateName are kept (small helpers; UpdateName is on the shared
Store interface).

* fix: add image validation and base64 to asset conversion

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-07-25 08:41:07 +00:00
Nikhil Soni
85abfd626e fix(tracedetail): correct EndTimestampMillis in flamegraph API response (#12251)
* fix(tracedetail): use span-derived bounds for flamegraph EndTimestampMillis

The trace_summary MV stores max(timestamp) as 'end', which is the maximum
span *start* time. For the final span of a trace, this misses the span's
duration entirely, making EndTimestampMillis shorter than the actual trace end.

FlamegraphTrace already computed the correct end via updateTimeRange
(end = max(span_start + duration)) but only used it internally for bucket
sampling. Add TimeRange() to expose these span-derived bounds and use them
in both getFullFlamegraph and getWindowedFlamegraph instead of summary.End.

The waterfall was unaffected — NewWaterfallTraceFromSpans already computes
endTime = max(span.TimeUnix + span.DurationNano) from actual spans.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GxTqz7CqqmoFEPHFqhJUCa

* chore: remove unnecessary comments

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 17:21:29 +00:00
Nikhil Mantri
f0a0b95ecd chore: removed code and tests (#12243)
Co-authored-by: Pandey <vibhupandey28@gmail.com>
2026-07-23 10:48:44 +00:00
Swapnil Nakade
8c6a0de8c3 refactor: gcp integration memorystore redis dashboard update (#12229)
Some checks failed
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
2026-07-22 11:42:12 +00:00
Naman Verma
c7abe0fe7b fix: add rank column to tag relations to keep note of tag ordering in dashboards (#12170)
* fix: always sort dashboard tags in alphabetical order

* fix: add rank column to tag relations to keep note of tag ordering in dashboards

* fix: dont disable FK just for adding a new column

* fix: add back tag relation unique index
2026-07-22 11:34:25 +00:00
Swapnil Nakade
842d44d874 feat: adding gcp memorystore redis service (#12118)
* feat: adding gcp memorystore redis service

* refactor: updating dashboard title

* refactor: extending width of uptime gauge panel

* refactor: updating cpu utilization panel

* refactor: updating dashboard panel to use rate function instead of hack
2026-07-22 07:08:26 +00:00
Naman Verma
fe101a1835 fix: fetch latest monotonicty for metrics instead of any random (#12205)
* fix: fetch latest monotonicty for metrics instead of any random

* test: add integration test for is monotonic flag
2026-07-22 06:01:23 +00:00
Nityananda Gohain
69c2f8d73f feat: span mapper test endpoint (#11795)
* feat: span mapper test endpoint

* feat: integrate with collector

* fix: more cleanup and integration test

* fix: lint

* fix: cleanup go.mod

* fix: update openapi

* fix: more cleanup

* fix: update go.mod

* fix: minor cleanup

* fix: update group_id to groupId

* fix: cleanup integration test

---------

Co-authored-by: Gaurav Tewari <gauravtewari111@gmail.com>
2026-07-22 04:52:13 +00:00
Vikrant Gupta
253ca7dd7e fix(session): validate ref and callback state against global allowed_origins (#12172)
* fix(session): use global external_url instead of ref param for SSO state

The sessions/context endpoint no longer reads the client-controlled ref
query param to build the SSO state and callback URLs. Each callback
authn provider now derives the site URL from the server-configured
global external_url, and siteURL is removed from the CallbackAuthN and
session Module interfaces.

* fix(session): validate ref and callback state against global allowed_origins

Instead of deriving SSO urls from the global external_url, keep the ref
roundtrip and validate its origin against the new optional
global.allowed_origins config. The callback state url is re-validated
before tokens are attached to it, closing the forged RelayState/state
exfiltration path. When allowed_origins is not configured, redirect
targets are not validated, preserving existing installs.

* fix(session): scope ref origin validation to sso auth domains

Move the allowed_origins check from the sessions/context handler to
getOrgSessionContext, right before the SSO login URL is built. A
disallowed ref no longer fails the whole request; only orgs with an
SSO-enabled auth domain get a per-org warning with password fallback.
2026-07-21 10:28:49 +00:00
Tushar Vats
852ca779c9 fix: convert key not found to warnings for logs (#12134)
* fix: convert key not found to warnings

* fix(logs): has-family body-only errors, not-found warnings, and condition-builder cleanup

- has/hasAny/hasAll/hasToken on a non-body key now return "supports only body
  JSON search" (both modes); a not-found body path warns and queries the
  underlying data instead of 400 (JSON mode)
- capture the two has-family 500s (hasToken separator/whitespace needle;
  hasAny/hasAll quoted int >= 2^32) as regression tests
- use telemetrytypes.NewTelemetryFieldKey for derived keys (fresh identity-only
  keys, no stale resolved metadata) instead of struct copies of the field key
- rename ConditionForKeys -> ConditionFor and inline conditionsForKeys into it
  across the builders; rename private conditionFor -> conditionForResolvedKey
- move the trace_noise fixture from queriertraces/conftest.py to fixtures/traces.py

* fix: convert trace_noise fixture to util
2026-07-19 18:41:53 +00:00
Nikhil Mantri
20e13f66e0 fix(infra-monitoring): use proper metric names corresponding to entities to be counted (#12130)
* chore: write proper lists of metricNames to lookup for counts

* chore: updated integration tests
2026-07-16 12:54:21 +00:00
Tushar Vats
62c44f5eac chore: pass org id for flagger (#12126) 2026-07-15 19:47:24 +00:00
Tushar Vats
63cfbe8bfb fix: convert key not found to warnings for traces (#12091) 2026-07-15 12:33:47 +00:00
Naman Verma
707f01bd9c chore: support non-dsl search in dashboards v2 list apis (#12051)
* chore: support non-dsl search in dashboards v2 list apis

* chore: move free text compilation to visitor

* chore: shorten comments

* fix: support both plain text and dsl together

* fix: ensure empty description dashboards are not excluded in NOT checks
2026-07-14 13:02:58 +00:00
Naman Verma
bf35d12398 chore: add functional unique index for tags (#12029)
* feat: add functional unique index

* chore: add tag migration

* chore: add failing scratch test (to discuss)

* chore: fetch expressions as well in getindices call

* fix: remove tag unique index (will be added in a separate PR)

* fix: remove tag unique index (will be added in a separate PR)

* chore: remove temporary tests

* chore: add functional unique index for tags

* fix: go lint fix

* fix: go lint fix

* chore: better comment for unique index

* test: add test for case insensitive expression equality

* test: add equality test for columns with quotes and capital letters

* test: add test for quoted columns

* chore: add separate type for unique indices with expressions

* fix: update migration file to use correct index

* test: add test for postgres provider

* test: add test for postgres provider

* fix: remove round trip tests
2026-07-14 11:08:59 +00:00
Nikhil Mantri
849353b5c0 feat(infra-monitoring): counts query for counting related entities (#12000)
* chore: added types and open api spec changes

* chore: added method to calculate reason

* chore: per group pod status counts with req metric checks method added

* chore: wired up pod status counts

* chore: pod restarts type added

* chore: added restart counts for the group

* chore: bug in query fix

* chore: onboarding API changes

* chore: integration tests added

* chore: added podcountsbyphase in other entities

* chore: added pod status counts for other entities

* chore: added integration tests for other entities

* chore: added checks api changes for other entities

* chore: rearrangement

* chore: removed succeeded status and mark it as completed

* chore: query beautified

* chore: corrected metrics list for metadata lookup

* chore: removed dead constants

* chore: goroutines for ListHosts

* chore: goroutines for ListPods

* chore: goroutines for ListNodes

* chore: goroutines for ListNamespaces

* chore: goroutines for ListClusters

* chore: goroutines for ListDeployments

* chore: goroutines for ListStatefulsets

* chore: added goroutines for ListStatefulsets, ListJobs and ListDaemonsets

* chore: added function

* chore: added struct changes

* chore: added count attr keys

* chore: wired counts to the response fields

* chore: regenerated API spec

* chore: merged main, resolved conflicts

* chore: nodes count surfacing

* chore: integration tests added

* fix: use tuple mapping to uniquely identify attrs:

* chore: integration tests update
2026-07-14 06:38:27 +00:00
Swapnil Nakade
86313ae561 feat: clousql - adding metrics in definition and dashboard JSON (#12072)
* feat: adding metrics in definition and dashboard JSON

* refactor: renaming cloudsql to cloudsql_postgres

* refactor: generating openapi specs
2026-07-13 09:57:49 +00:00
Shivam Gupta
03abb3ca90 fix: update stale docs links in backend and remaining frontend (#12096)
Repairs 10 broken signoz.io/docs links (5 hard 404s + 5 dead anchors)
that survived the frontend-only sweep in #11319 because they live in
the Go backend and two frontend files it did not cover.

- infra-monitoring readiness checks: drop the removed `user-guides/`
  path segment and remap to the current hostmetrics/k8s-metrics anchors
- querybuilder / telemetrylogs search-troubleshooting errors: point to
  the reworded Q&A anchors (update matching test assertion)
- alert generatorURL fallback: `alerts-management/#generator-url` ->
  `alerts/` (anchor removed in docs restructure)
- missing-spans banner: -> traces-management troubleshooting FAQ anchor
- agent-skills install link: `#installation` -> `#install-the-plugin`

Every changed URL verified live (200 + anchor present).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 08:49:31 +00:00
Naman Verma
66f4a63978 fix: don't return error in v2 list dashboard api if there is a v1 dashboard (#12020)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* fix: don't return error in v2 list dashboard api if there is a v1 dashboard

* chore: generate api specs

* feat(dashboard-v2): surface legacy dashboards in the v2 list (#12024)

* feat(dashboard-v2): add legacy dashboard dialog

Surfaces a copyable dashboard ID and a Contact Support action for a
pre-v2 (legacy) dashboard that has no v2 spec to render.

* feat(dashboard-v2): hide v2-only actions for legacy dashboards

A legacy dashboard has no v2 spec, so view/open/copy-link/rename/
duplicate/lock don't apply. Gate them behind !isLegacy, leaving only
Delete (and dropping its now-orphan leading divider).

* feat(dashboard-v2): surface legacy dashboards in the v2 list

Flag legacy rows with a badge, block navigation, and open the legacy
dialog on click instead. Disable the pin action with an explanatory
tooltip and gate the row's actions menu to legacy-safe items.

---------

Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-07-10 14:41:30 +00:00
Nikhil Mantri
1e257a5b26 feat(infra-monitoring): daemonsets ready and misscheduled nodes count (#12008)
* chore: types and constants added

* chore: update buildDaemonSetRecords

* chore: updated orderBy:

* chore: wired in checks API:

* chore: generated api spec:

* chore: update integration tests

* fix: checks constants updated in integration tests
2026-07-10 13:10:50 +00:00
Ashwin Bhatkal
51036d6cc4 feat(public-dashboard): integrate v2 (Perses-spec) public dashboards (#12032)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat(public-dashboard): detect v1 vs v2 schema for the public viewer

Anonymous public viewers have no feature flags, so the schema can't be read from
use_dashboard_v2. Probe the v2 model endpoint first and fall back to v1 only on the
'dashboard_invalid_data' (HTTP 501) schema-mismatch signal. Probing v2 first also stops
the v1 endpoint from serving v2 dashboards with un-redacted queries.

* feat(public-dashboard): fetch v2 public panel data by key

Adds a by-key fetcher over the anonymous /api/v2/public/dashboards/{id}/panels/{key}/query_range
endpoint (the generated client omits the startTime/endTime params) and a store-free
usePublicPanelQuery that mirrors usePanelQuery's PanelQueryData shape. No variables and no
pagination — the public endpoint supports neither.

* feat(public-dashboard): render v2 public dashboards read-only

Adds a read-only v2 viewer that reuses the authenticated V2 panel renderers
(PanelHeader with hideActions, PanelBody, panel registry) and the pure layoutsToSections
util, with a forked read-only grid. The public page branches on the resolved schema:
v1 keeps the existing container, v2 renders the new viewer. Dashboard variables are not
rendered — the public endpoint does not substitute them.

* feat(public-dashboard): match the standard auto-refresh control

Replace the hand-rolled 'Off' select (which was styled inconsistently and clipped its
options) with a PublicAutoRefresh that mirrors the app's DateTimeSelectionV2 refresh cluster:
a grouped refresh button + auto-refresh popover (Auto Refresh checkbox + full interval list),
portal-rendered so nothing clips. It's prop-driven — the public viewer keeps managing its own
time window — so the container now tracks enabled + interval and exposes a manual refresh.
Also nudge the header-right gap 8→12px.

* feat(public-dashboard): declare v2 query_range params, drop the wrapper, address review

Declare startTime/endTime as query params on the v2 public query_range endpoint via
RequestQuery and regenerate the OpenAPI spec + orval client, so the generated
getPublicDashboardPanelQueryRangeV2 carries them. usePublicPanelQuery now calls the
generated fetcher directly and the hand-written wrapper is removed.

Also from review: drop the defensive panelDefinition guard so an unsupported kind
surfaces loudly, use lodash noop, and trim excessive comments across the v2 files.

* fix: bind query params from PublicWidgetQueryRangeParams

---------

Co-authored-by: Naman Verma <naman.verma@signoz.io>
2026-07-09 13:27:17 +00:00
Nikhil Mantri
da250bfd3e fix: commented out container metrics to surface k8s.pod.start_time (#12042) 2026-07-09 12:09:53 +00:00
Nikhil Mantri
67007e2902 feat(infra-monitoring): Kubernetes Container Monitoring List API (#11946)
* chore: added types for containers

* chore: added querier query and constants

* chore: added helper queries

* chore: added wiring

* chore: added open api spec

* chore: endpoint and variable rename

* chore: use recency for container status.reason as well

* chore: pass orgID to getMetadata for containers (adapt to rebased main)

* chore: add metrics to the containers list for metadata and earliest time

* chore: corrected metrics list for metadata lookup

* chore: added changes to the checks API for new kube containers section

* chore: containers query modified

* chore: added integration tests

* chore: integration tests

* chore: goroutines in container monitoring

* chore: constants deduplication

* chore: inlined requests.Post call for this PR instead of wrapping in a function
2026-07-09 11:42:24 +00:00
Tushar Vats
cd8bf27da7 chore: refactor qb to lossen tight coupling of querybuilder with other modules (#11911)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
2026-07-08 18:07:03 +00:00
Nikhil Mantri
652a9044f4 feat(infra-monitoring): goroutines wrapping custom queries for concurrency (#11981)
* chore: added types and open api spec changes

* chore: added method to calculate reason

* chore: per group pod status counts with req metric checks method added

* chore: wired up pod status counts

* chore: pod restarts type added

* chore: added restart counts for the group

* chore: bug in query fix

* chore: onboarding API changes

* chore: integration tests added

* chore: added podcountsbyphase in other entities

* chore: added pod status counts for other entities

* chore: added integration tests for other entities

* chore: added checks api changes for other entities

* chore: rearrangement

* chore: removed succeeded status and mark it as completed

* chore: query beautified

* chore: corrected metrics list for metadata lookup

* chore: removed dead constants

* chore: goroutines for ListHosts

* chore: goroutines for ListPods

* chore: goroutines for ListNodes

* chore: goroutines for ListNamespaces

* chore: goroutines for ListClusters

* chore: goroutines for ListDeployments

* chore: goroutines for ListStatefulsets

* chore: added goroutines for ListStatefulsets, ListJobs and ListDaemonsets
2026-07-07 04:39:08 +00:00
Nityananda Gohain
d4646657ca fix: initalize querier in llmpricing (#11993) 2026-07-07 04:04:29 +00:00
Nikhil Mantri
29681572db feat(infra-monitoring): pod status and restart counts. (#11885)
Some checks failed
build-staging / js-build (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* chore: added types and open api spec changes

* chore: added method to calculate reason

* chore: per group pod status counts with req metric checks method added

* chore: wired up pod status counts

* chore: pod restarts type added

* chore: added restart counts for the group

* chore: bug in query fix

* chore: onboarding API changes

* chore: integration tests added

* chore: added podcountsbyphase in other entities

* chore: added pod status counts for other entities

* chore: added integration tests for other entities

* chore: added checks api changes for other entities

* chore: rearrangement

* chore: removed succeeded status and mark it as completed

* chore: query beautified

* chore: corrected metrics list for metadata lookup

* chore: removed dead constants
2026-07-06 20:11:40 +00:00
Vikrant Gupta
1644e35d9c feat(authz): make serviceaccount_role and user_role first class citizens (#11977)
* feat(authz): add serviceaccount and user role handlers

* feat(authz): add support for role and service account extractors

* feat(authz): make the user apis v2

* feat(authz): return the existing service account role id
2026-07-06 13:48:46 +00:00
Naman Verma
832a6cc794 feat: v2 versions of public dashboard APIs (#11725)
* feat: add first draft of v2 public dashboard apis

* fix: remove duplicate call to GetDashboardByPublicIDV2 in GetPublicWidgetQueryRangeV2

* fix: fill fields that were in the data blob in v1

* chore: trim comments

* fix: remove fields that v1 also removes when redacting

* chore: rename method name

* test: unit tests for GetPanelQuery

* fix: add fill gaps to query

* fix: generate api specs

* test: add integration tests for new v2 public apis

* fix: add query validation and aggregation validation

* fix: remove unneeded tags db call from public query range api

* fix: redact variable queries as well

* fix: move regex out of method so that it is only compiled once per package load

* chore: remove empty line

* fix: use pointer to specs during redaction

* fix: move single expression validation to dashboard package, use chparser for it

* test: add integration test for variable query redaction

* test: add integration test for expression with many parens

* test: use valid query in integration test

* test: use realistic query in variable
2026-07-06 09:16:42 +00:00
Nityananda Gohain
84834b087b feat: add endpoint for fetching unpriced models (#11763)
* feat: add endpoint for fetching unpriced models

* fix: more updates

* fix: response model
2026-07-06 08:50:15 +00:00
Naman Verma
766a05e24b chore: fix issues found in bug bash (#11949)
* fix: return list of all tags sorted alphabetically

* chore: return reserved keys in list api response for easy filtering

* fix: add length limit to dashboard display name

* test: check error message as well

* chore: increase the length limits

* chore: add copy suffix on cloning dashboards

* fix: increase limit to 64 for dashboard view name

* fix: send user friendly err message on length check fail

* fix: add path to error message

* fix: include path in main error message directly

* fix: move regex out so that it only compiles once per init

* fix: format integration test properly
2026-07-03 08:10:37 +00:00
Srikanth Chekuri
66f03d5912 fix(metrics): use local table for fingerprint ctes (#11931)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* fix(metrics): use local table for fingerprint ctes

* chore: remove queries file

* chore: update reduced_test.go
2026-07-01 16:14:11 +00:00