## Summary
- Saved views now persist a versioned, typed spec (`schemaVersion` +
`spec{compositeQuery, selectedFields, display}`) instead of a bare
composite-query blob plus an opaque, frontend-owned `extraData` string
-- mirroring the pattern dashboards already use for their v2/perses
schema.
- `/api/v1/explorer/views` keeps working exactly as before: a thin
conversion layer translates to/from the legacy wire format, including
folding `extraData`'s ad hoc JSON into the typed spec and back for
backward compatibility.
- A one-time migration rewrites existing rows into the new shape and
drops the now-unused `extra_data`/`category`/`tags` columns.
### Scaffolding decisions
- Using v2 for new handlers instead of renaming old handlers to
something else for these reasons - keep the diff minimum for easier
reviews, avoiding any git history or last updated at change in old route
registration.
- Keeping the conversion to old saved view type in handler itself rather
than `savedviewtypes` package to keep it un-exported and not let them be
available anywhere else to be used. It also enables `savedviewtypes` to
be independent on query-service models.
- Modified the existing handler and it's interface to include the v2
methods instead of adding another handlerV2 since apiserver already had
handler wired in, so don't want to pass on 2 version simultaneously.
### Breaking change
- Any unknown key in the `ExtraData` will be rejected and dropped
silently in the old APIs and give error in new version.
- If there was any way to add tag or category in saved view earlier,
that data will be lost.
- Old APIs will not support the old QB request payload, only v5 format
is supported.
---
ClosesSigNoz/engineering-pod#4651
Alternative discarded https://github.com/SigNoz/signoz/pull/12208
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
> **Stack** (review in order; each PR's diff is against its
predecessor):
> 1. #12323 `v2-read-path` — v2 native read path (leaf package)
> 2. #12324 `v2-wiring` — wiring, shadow/pin rollout machinery, dual-leg
conformance
> 3. #12325 `v2-transpiler` — PromQL→ClickHouse transpiler +
classification golden
> 4. #12093 `issue-4293` — the /prometheus API move (breaking slice,
last)
### What
The performance half of the v2 provider: an allowlist compiler
(`classify`/`rewrite`) that evaluates proven PromQL shapes entirely
inside
ClickHouse on the `timeSeries*ToGrid` aggregate functions (CH ≥ 25.6),
so one
row per output series comes back instead of every raw sample. Everything
not
provably equivalent falls back to the engine over the PR-1 querier; a
transpilable subtree under a non-transpilable node runs hybrid (subtree
materialized as synthetic series, engine on top). `TryExecuteRange`
slots
into the PR-2 serve/shadow paths (until now engine-only) through the new
`prometheus.RangeExecutor` capability interface — the provider stays
unexported and pkg/querier keeps holding `prometheus.Prometheus`;
providers
without the capability (v1) simply never transpile. The capability folds
into the main interface once v1 is removed.
Highlights (docs/contributing/prometheus.md carries the full correctness
story):
- Range functions map to verified grid aggregates; `increase` is
`rate × range` exactly (same extrapolated delta, factor algebra).
- Instant selectors reproduce stale-marker shadowing with a
three-aggregate
compare — skipping stale rows in WHERE would resurrect the sample the
marker buried.
- `*_over_time` at range = k·step aggregates whole step buckets
(`groupArrayInsertAt` + slide) — no per-window fan-out, no prefix-sum
differencing.
- **Window-sliver filtering** (the headline perf commit, folded here):
when
the window is narrower than the step, only window/step of the timeline
can
influence any grid point; a lattice predicate in WHERE cuts the
aggregate's
input by the coverage ratio — measured 74s/28GiB → 16s/4.3GiB on a
36k-series 1w rate, and a 2.67B-sample case that exceeded 150GiB now
completes in 19s/17GiB. Over sliver-filtered rows the last-style gates
lift
(instant selectors and `last_over_time` transpile at window < step), and
disjoint-window `*_over_time` forms drop the divisibility gate.
- Scalar-op pipelines apply in Go, slot by slot — same float64 ops, same
order the AST dictates.
Two guards land with it:
- **Classification golden** (`classification_golden_test.go` +
`testdata/classification_golden.json`): freezes the route
(full/hybrid(n)/fallback + reason) of every conformance-corpus
expression,
one line each — 317 expressions: 132 full, 39 hybrid, 146 fallback. The
test also requires each expression to route the same on every corpus
grid;
if a classifier change ever makes the route grid-dependent, the test
fails
and the key must grow. Routing is its own
correctness surface — silently falling back costs the pushdown, silently
transpiling an unproven shape risks wrong numbers; both now show up in
review as a golden diff, with the corpus suite's v2 leg judging the
numbers.
- **Workload coverage reporter** (`TestClassifyCorpus`, env-gated):
classifies
a JSON-lines corpus of real dashboard/alert queries and buckets
fallbacks
by reason, to steer future allowlist work.
**What the dual-leg suite caught on its first transpiled run** (evidence
the
PR-2 guard works, worth stating in review):
- The classifier read a duration expression's offset (`x offset step()`)
as
zero and transpiled it — offset expressions parse *without* the
experimental-parser flag, so they reach production. 20 corpus cases
served
silently wrong numbers. Fixed by refusing `OriginalOffsetExpr` /
`RangeExpr` / `StepExpr` at classification (engine evaluates them
exactly);
regression cases added, golden regenerated (30 routings flipped to
fallback).
- Name-drop assembly treated temporally-disjoint same-labelset twins as
separate series: `-{job="api"}` spanning `http_requests`/`http_errors`
returned a 400 the engine would not raise, and hybrid
`-metric_a or -metric_b` returned duplicate `{}` series. The engine's
actual rule is: assemble the matrix by labelset, merging elements that
never share an evaluation timestamp; error only on a same-timestamp
conflict. Both the full-plan path (`mergeSameLabelsetSeries`) and the
hybrid post-strip path (`mergeMatrixByLabelset`) now reproduce it, with
unit tests pinning the corpus scenarios.
- 12 remaining divergences, all one class, recorded in
`known_divergences_v2.json` with causes: the engine aggregates with
Kahan
compensated summation (sum, sum_over_time) and an overflow-free
incremental
mean (avg); ClickHouse's `sumForEach`/`avgForEach`/`arraySum` are naive,
so
±1e100 cancellation returns 0/residue and near-max-float64 `avg`
overflows
to ±Inf. Burn-down note: `sumKahanForEach` for the cancellation class;
the
overflow class needs an incremental-mean aggregate ClickHouse doesn't
have.
### Alternatives considered and discarded
- **General PromQL→SQL translation.** An allowlist inverts the failure
mode:
an overlooked construct becomes a fallback instead of a wrong number.
Every
shape on the list was validated slot-for-slot against the vendored
engine
on live data before entering it.
- **ClickHouse's own PromQL dialect** (ClickHouse#57545,
`dialect='promql'`).
Emits the same grid functions, but currently covers only
rate/irate/delta/idelta/last_over_time, has no fallback engine, and ties
us
to their TimeSeries table engine. We use the same primitives with our
own
classifier and our own exactness gates.
- **Prefix-sum differencing for `*_over_time` windows.**
Large-minus-large
cancellation drifts past the shadow tolerance on counter-sized values;
direct per-slot combination of at most W bucket partials adds the way
the
engine adds.
- **Fanning each sample into every window that covers it.** Multiplies
rows
by W — billions of rows for a long range over a short step; the bucketed
form's row count is series × buckets, the size of the output.
- **Handling staleness by filtering stale rows in WHERE (instant
units).**
Resurrects the older real sample the marker was written to bury; hence
the
last-overall vs last-non-stale timestamp comparison.
- **Transpiling @-modifier and default-resolution subqueries.** Their
evaluation grid depends on server runtime settings the transpiler cannot
see; they stay on the (exact) engine path.
### Test plan
- `go test ./pkg/prometheus/clickhouseprometheusv2` — transpiler unit
tests
(SQL forms, classification, scalar ops, subquery grids), golden.
- `pytest integration/tests/promqlconformance/` — the v2 leg now
exercises
transpiled serving for every routable corpus case; ledger unchanged
(empty).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Pandey <vibhupandey28@gmail.com>
* feat: enable FGA for dashboards and their public config
* test: add integration test for dashboard FGA
* fix: fix permissions for public dashboards, pinning, views
* fix: allow viewers to manage views
* fix: remove edits to the public dashboard line
* chore: add api to retry migration for a dashboard
* feat(dashboard): add retry migration action for legacy dashboards
The legacy-dashboard dialog only offered the dashboard ID and a link to
support. Now that the v1->v2 migration can be re-run on demand, let an
editor trigger it from there and fall back to support only if it still
fails.
Retrying needs edit access (the endpoint is EDITOR-gated), so viewers
keep the ID-and-support dialog unchanged.
---------
Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
* perf(tests): cache go and pnpm stores across integration image builds
Add BuildKit cache mounts for GOCACHE/GOMODCACHE and the pnpm store to the
integration Dockerfiles, and build the image via the docker CLI (docker-py,
used by testcontainers' DockerImage, does not support BuildKit). Embed the
go build command directly so Makefile changes do not invalidate the build
layer, and pin HOME/GOCACHE/GOMODCACHE/PNPM_HOME explicitly so cache-mount
targets match tool defaults by contract. The with-web node stage fetches
dependencies from the lockfile before the source copy, so frontend edits
only re-run the offline install and build.
* feat(tests): add --clean flag to prune buildkit cache mounts
The go and pnpm caches introduced for the integration image build survive
--teardown since they belong to the docker builder, not to any container.
--clean runs docker builder prune with a type=exec.cachemount filter at
session start, forcing the next image build to start cold. Documented in
the integration testing guide.
* feat(tests): add --rebuild flag to refresh the signoz container under --reuse
--reuse keeps the running signoz container, so backend source changes are
never picked up without tearing down the whole stack. --rebuild deletes the
cached signoz container and recreates it from the current sources (an
incremental image build), while databases, mocks and migrations stay reused.
Requires --reuse; combining with --teardown or --clean is a usage error.
* chore(tests): prune comments to non-obvious constraints
* docs(tests): make py-test-setup rebuild signoz and audit the integration guide
py-test-setup now passes --rebuild so re-running it after backend changes
transparently swaps in a signoz container built from the current sources.
The integration guide documents the iteration loop and fixes stale content:
option defaults (clickhouse 25.12.5, schema migrator v0.144.6), the
nonexistent --zookeeper-version option, Zookeeper vs ClickHouse Keeper, the
e2e doc path, and the lint toolchain (ruff).
* docs(tests): wire --rebuild into the e2e setup flow
The e2e bootstrap shares the signoz fixture, so --rebuild already applies;
with --with-web it also picks up frontend changes since the image bakes the
built frontend in. The setup command now passes --rebuild, and the guide
documents the iteration loop, the --rebuild/--clean flags, ClickHouse Keeper
instead of Zookeeper, and the corrected integration doc path.
* docs(tests): qualify --rebuild workflow for suites with custom signoz variants
make py-test-setup only rebuilds the default signoz instance; suites that
create their own via create_signoz(cache_key=...) keep a separately cached
container. Passing --rebuild on the suite run itself rebuilds every variant
that run instantiates.
* docs(tests): describe --clean behaviour instead of its exact command
Keeps the docs from drifting if the prune invocation behind --clean changes.
Second-generation ClickHouse-backed Prometheus provider: the stock engine
evaluates over a native storage.Querier instead of the v1 remote-read
adapter. Per-selector fetch windows, last-sample-per-step reduction for
subquery-free instant selectors (gated on prometheus.QueryTraits),
identical-labelset merge, per-type __name__ matchers and anchored regexes,
inclusive series-lookup bounds for the exporter's hour-floored registration
rows.
Not wired: no factory registration, no config selection, nothing serves
from this package yet. Fetch budgets (series/sample ceilings) are
deliberately left out for now and will come separately.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(ruletypes): expose above_or_equal and below_or_equal in CompareOperator enum
The operators are accepted by Validate(), normalized, evaluated and
returned by the rules API, but were commented out of Enum(), so the
generated OpenAPI spec (and clients generated from it, e.g.
terraform-provider-signoz) rejected rules the server itself creates.
* fix(alerts): support above_or_equal and below_or_equal operators in CreateAlertV2
Adds the two inclusive operators to the v2 alert form: selectable in the
threshold operator dropdown, normalized from all backend aliases
(5/6, above_or_eq/below_or_eq, >=/<=), rendered with their symbols in
threshold rows and match-type tooltips, and prefilled losslessly from
dashboard panel thresholds instead of collapsing onto the strict
variants. The v1 form is left untouched.
* feat: adding gcp memorystore redis service
* refactor: updating dashboard title
* refactor: extending width of uptime gauge panel
* refactor: updating cpu utilization panel
* refactor: updating dashboard panel to use rate function instead of hack
* feat: adding compute engine service
* refactor: updating dashboard panels to use rate aggregation
* fix: correct typo and unit in compute engine dashboard
* refactor: migrating dashboard to v6
* feat(rulestatehistory): populate related logs/traces links in v2 history APIs
The v2 rule history timeline dropped the relatedLogsLink/relatedTracesLink
that v1 (getRuleStateHistory) returned per entry, which the alert history
page uses to jump from a state change to the explorer with the rule's
filter and the entry's labels. Load the rule from the rule store in the
module and build the links with contextlinks, scoped to the entry's
evaluation window like v1.
Extract the builder-query filter/group-by selection that the v1 handler
and threshold rule notifications each inlined into
contextlinks.BuilderQueryForSignal and reuse it from both the module and
ThresholdRule.
Also populate the links for top contributors, which both v1 (since #10760)
and v2 returned as always-empty fields even though the UI renders them;
contributor links span the queried range since the counts aggregate it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(contextlinks): remove unused v3 link helpers
PrepareLinksToTraces, PrepareLinksToLogs and PrepareFilters lost their
last callers when the deprecated v3/v4 rule support was removed in #10760;
the v5 equivalents (PrepareParamsFor*V5 and PrepareFilterExpression) are
what all remaining callers use.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: end doc comments with a period to satisfy godot
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(alerts): cover v2 rule history related links for logs and traces
Each test fires a rule with a filter and a service.name group-by, then
asserts the recorded firing entry and top contributor carry a related
explorer link for the rule's signal only, with the label-rewritten filter
expression, the evaluation window on timeline entries and the queried
range on contributors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(contextlinks): shrink explorer links to the minimal payload
The explorer pages read only the data source and filter expression from a
shared link and fill in the rest of the query shape with defaults, so stop
shipping the v3 builder-query ceremony (queryName, aggregateOperator,
aggregateAttribute, stepInterval, paging fields) and the timeRange and
options params nothing reads. Links shrink from ~1.2k to ~450 chars and
contextlinks no longer depends on the v3 model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(rulestatehistory): derive link windows from the evaluation envelope
Rules created through the current UI store the window in the v2alpha1
evaluation envelope with no top-level evalWindow, so the previous 5m
fallback produced wrong link windows for any non-default rolling window
and could not represent cumulative windows at all. Use the envelope's
NextWindowFor like the rule engine does, keeping the top-level
evalWindow (default 5m) as the fallback for rules without an envelope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(contextlinks): simplify double-encoding comment
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(alerts): use literal matchType/op in fixtures and drop link unit tests
Replace the numeric matchType/op codes in all alert scenario fixtures
with their literal forms (at_least_once, above, ...) which the API
normalizes to the same canonical values, and remove the rule history
link unit tests since the integration tests cover the behavior
end-to-end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(alerts): move rule history helpers into the shared alerts fixtures
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* chore: fix schema based on migration errors
* test: add rejection based integration tests for new validations
* fix: remove datasource field from schema
* fix: remove requirement for links to be always present
* fix: make links use defined type
* test: add integration test for link omission roundtrip
* test: change not in to None for links in panel
---------
Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
* fix(dashboards-v2): make panel/dashboard links required and non-nullable
* fix: validate links on read from db as well
* fix: allow all as value for signal
* fix: dont allow empty string for signal
* fix(dashboards-v2): dedicated `all` dynamic-variable signal (frontend + client)
* test: add empty links to new payloads in integration tests
---------
Co-authored-by: Naman Verma <naman.verma@signoz.io>
* feat: adding gcp memorystore redis service
* refactor: updating dashboard title
* refactor: extending width of uptime gauge panel
* refactor: updating cpu utilization panel
* refactor: updating dashboard panel to use rate function instead of hack
* fix(querybuildertypesv5): omit unset stepInterval on the wire
Step is a struct (struct{ time.Duration }), so omitempty had no effect — an
unset stepInterval serialized as 0 instead of being omitted, so a typed
client reading a query back saw a 0 it never sent (create -> GET drift).
Tag stepInterval with ,omitzero so an unset value is dropped while a set
value still serializes (as seconds), on all three sites: builder query,
trace-operator, and secondary aggregation. Schema-invisible (no OpenAPI /
client change). source and the metric enums were already handled in #12164.
* fix(telemetrytypes): round-trip empty fieldContext/fieldDataType on field keys
A TelemetryFieldKey can deliberately leave fieldContext/fieldDataType empty
to match across any context / data type, but ,omitzero dropped that empty
value on serialize, so a typed client that sent "" read it back as absent
(create -> GET drift).
Make both fields always serialize and add the empty member to their Enum()s
so "" is a valid schema value that round-trips verbatim — the same approach
#12164 used for source. Signal keeps ,omitzero: its empty value is invalid
for the query/variable signal contexts that share the enum (adding "" there
breaks those consumers), and a field key's signal is not deliberately empty.
Regenerate the OpenAPI spec + client (fieldContext/fieldDataType enums gain
"") and update the ScalarData marshal test (column keys now echo the fields).
* fix(telemetrytypes): round-trip empty signal on field keys
Extend the field-key round-trip fix to Signal: add the empty member to
Signal.Enum() and make TelemetryFieldKey.Signal always serialize, so an
empty ("any") field-key signal round-trips as a valid value instead of
being dropped — matching the fieldContext/fieldDataType treatment.
The Signal enum is shared with query/variable signals, where "" is invalid.
Narrow the frontend's TelemetrySignal type to logs/traces/metrics (so the
variable/panel signal selectors stay exhaustive), label the empty member in
the panel type switcher's map, and fold an empty drilldown signal into "all".
Regenerate the OpenAPI spec + client (Signal enum gains ""), and update the
ScalarData marshal test and the querierlogs aggregation label assertions to
include the now-serialized empty signal.
* fix(dashboardtypes): round-trip zero-valued variable/display fields
omitempty dropped explicit zero values from the create -> GET response, so
a typed client (Terraform/SDK) that sent them read back null and reported
drift. Remove the tag so these always serialize:
- Display.Description ("" round-trips; applies to dashboard/panel/variable
displays)
- TextVariableSpec.Constant (constant: false, like the disabled fix)
- ListVariableSpec.CustomAllValue / CapturingRegexp ("" round-trips)
Scalars carry no nullability, so the OpenAPI spec and generated client are
unchanged. Sort stays omitzero: its "no sort" value is "none", not "", so
omitzero only omits the invalid unset state.
* fix(dashboardtypes): round-trip panel and dashboard links
`links` used omitempty (dropped an explicit []) and its element type was the
imported perses dashboard.Link, whose own fields tag name/tooltip/
renderVariables/targetBlank omitempty — so a link's false/"" were dropped
too, and a typed client read them back as null.
- Replicate dashboard.Link as a SigNoz Link type (same pattern as
ListVariableSpec/TextVariableSpec) with every field always serialized.
- Use ,omitzero on PanelSpec.Links and DashboardSpec.Links so an explicit
[] round-trips while an unset list stays omitted (never null).
Regenerate the OpenAPI spec and frontend client: the element schema is now
DashboardtypesLink (was the perses DashboardLink) and links is nullable.
Update the frontend consumers to the renamed type and coalesce the now
type-nullable spec.links (never null on the wire) at its two boundaries.
* test(dashboard): cover variable/display/link round-trip cases
Extend the v2 dashboard round-trip test with the spec-wide zero values this
PR fixes: a display description "", a text variable's constant false, a list
variable's customAllValue/capturingRegexp "", an explicit [] of panel links
that round-trips, a link whose own zero-valued fields (name/tooltip "",
renderVariables/targetBlank false) echo back, and a linkless panel whose
links stay omitted (never null).
* test(dashboard): accept null-or-absent for unset panel links
A panel with no links round-trips as "links": null rather than being
omitted (the panel serialization path differs from the query slices, which
omit). Both mean "no links" and neither drifts for a typed client, so assert
the value is None (null or absent) instead of strictly absent. The explicit
[] case still asserts a verbatim round-trip, which is the guarantee the fix
provides.
* fix(dashboardtypes): round-trip remaining zero-valued spec fields
Complete the dashboards-v2 create -> GET round-trip audit:
- DashboardSpec.Datasources: ,omitempty -> ,omitzero so an explicit {}
round-trips (omitempty dropped it) while an unset map stays omitted.
- DashboardV2 Image, DashboardSpec.Duration/RefreshInterval: drop ,omitempty
so an explicit "" round-trips (same class as Display.Description). The
server accepts "": DurationString.validate() returns nil for len 0, and
Image/Duration/RefreshInterval have no create-time validation, so a
GET-then-PUT of "" is not rejected.
Scalars carry no nullability (no spec change); the datasources map is now
nullable: true in the regenerated OpenAPI spec and client.
* test(dashboard): cover datasources/image/duration/refreshInterval round-trip
Extend the round-trip test with the spec-wide zero values just fixed: a
dashboard-level image "", spec duration/refreshInterval "", and an explicit
empty datasources {} that must echo back as {} (omitzero) rather than being
dropped.
* fix(querybuildertypesv5): omit unset metric enum fields on the wire
A metric builder query serialized empty strings for its enum fields
because omitempty has no effect on struct-backed valuer types:
"source":"", "aggregations":[{"temporality":"","timeAggregation":"","spaceAggregation":""}]
Those "" values are not members of the corresponding OpenAPI enums
(source=[meter], temporality=[delta,cumulative,unspecified], etc.), so a
typed client reading a rule back rejected it (create -> GET round-trip
drift; terraform-provider-signoz generate-config failed schema validation).
Tag Source/Temporality/TimeAggregation/SpaceAggregation with ,omitzero so
an unset value is dropped instead of emitted as an invalid "", matching the
existing convention (dashboardtypes Sort, telemetrytypes field keys). Valid
values still serialize. The OpenAPI spec regenerates byte-identical, which
confirms the enums were already correct.
* test(querybuildertypesv5): cover client-sent empty enum values
A client (e.g. terraform) may send explicit source:"" / temporality:"" for
an unset enum. Assert unmarshaling accepts them, normalizes to the zero
value, and re-marshaling drops them so the round-trip never echoes an
invalid "" back.
* fix(querybuildertypesv5): allow empty metric enum values in the spec
The server accepts and echoes back an unset source, temporality, and
timeAggregation for a metric query (a create -> GET returns "" for them), but
their OpenAPI enums omitted "". A typed client (terraform-provider-signoz
generate-config) therefore rejected the config generated for an imported rule.
Add "" as a valid member of the Source, Temporality, and TimeAggregation enums
so the spec matches what the server actually accepts and returns. spaceAggregation
is left unchanged: an empty value is rejected with 400 at creation (IsValid), so
it is never stored or echoed and "" must stay out of its enum.
Drop the earlier ,omitzero tags: these fields already always-serialize, so an
accepted "" round-trips faithfully instead of being silently dropped (silent
mutation is itself drift). source loses its no-op omitempty for the same reason.
Regenerate the OpenAPI spec and frontend client (both git-diff gated).
* test(querybuildertypesv5): assert accepted empty enums round-trip
Empty source/temporality/timeAggregation are echoed back (not dropped) and are
stable across marshal -> unmarshal -> marshal; spaceAggregation carries a valid
value since an empty one is 400'd at creation.
* test(querybuildertypesv5): merge and rename metric enum round-trip test
Fold the unmarshal-echo case into the table-driven marshal round-trip test
(its marshal -> unmarshal -> marshal check already covers the client-sends-""
path) and rename to TestQueryBuilderQuery_MetricAggregation_MarshalJSONEnumRoundTrip.
* style(metrictypes): drop explanatory comments on enum changes
Remove the comments added to Temporality/TimeAggregation Enum() and the metric
enum round-trip test case.
* style(telemetrytypes): drop explanatory comment on Source enum change
Remove the comment added to Source.Enum(), keeping the pre-existing doc/TODO.
* fix(dashboardtypes): accept threshold value of 0 on create
A NumberPanel/TimeSeries/Table threshold with `value: 0` (a legitimate
value the SigNoz UI emits by default) was rejected on create with
`dashboard_invalid_input` "Field validation for 'Value' failed on the
'required' tag".
go-playground/validator's `required` treats a numeric field equal to its
zero value as "missing", so `validate:"required"` on the float `Value`
wrongly rejected 0. Drop `validate:"required"` from `Value` on
ThresholdWithLabel and ComparisonThreshold; keep `required:"true"` since
the field is always present in the schema (0 is a valid present value, not
an absent one), so the OpenAPI/generated client are unaffected. `Color`
keeps both tags — an empty colour is genuinely invalid.
Drop the two "missing value" cases from TestValidateRequiredFields, which
asserted the removed invariant.
* fix(querybuildertypesv5): round-trip zero-valued query spec fields
A dashboard/alert query that sets a zero-valued field — `disabled: false`,
`legend: ""`, or an explicit empty `groupBy`/`order`/`selectFields`/etc. —
created fine but the GET response omitted it, so a typed client that echoes
what it sent (Terraform, SDKs, PUT-after-GET) read back `null`/absent and
reported drift. `,omitempty` dropped these zero values on the way out.
Fix the create -> GET asymmetry:
- Slice fields use `,omitzero` instead of `,omitempty`. `omitzero` omits a
nil slice (field never set stays absent) but keeps an explicit non-nil
`[]`, so an empty array round-trips as `[]` and there is no `null`
regression. Applied to groupBy, order, selectFields, aggregations,
functions, secondaryAggregations and function args across the builder,
formula, trace-operator and join specs, plus ListPanelSpec.selectFields.
- Scalars `disabled` (bool) and `legend` (string) drop the tag entirely;
`omitzero`/`omitempty` both suppress false/"", so the only way to
round-trip them is to always serialize.
Result types in resp.go keep `,omitempty` — they are server-computed and
never round-tripped. Regenerate docs/api/openapi.yml and the frontend
client: the omitzero slices are now `nullable: true` in the schema (never
null on the wire, but the generated types gain `| null`, which existing
consumers already handle via `?? []`).
* test(dashboard): round-trip serialization for zero-valued fields
Add a v2 dashboards integration test that creates one minimal dashboard
(stripped from SigNoz/dashboards cicd-perses.json) and asserts the
create -> GET round-trip preserves every zero-valued field the fix targets:
- threshold value 0 (ComparisonThreshold + ThresholdWithLabel) is accepted
on create and echoed back
- builder slices set to an explicit [] (groupBy/order/selectFields/functions)
round-trip as [], while a bare builder's unset slices stay absent (never
null) on read
- scalars disabled/legend always echo false/""
Table-driven: one equality table for round-tripped values and one absence
table for omitted slices.
* test(dashboard): fold round-trip test into 03_v2_dashboard
Move test_dashboard_v2_roundtrip_preserves_zero_values alongside the other
v2 dashboard tests (test_create_rejects_*, lifecycle, ...) instead of a
standalone file, with the dashboard payload inlined per this suite's style.
* feat(authz): enable FGA for telemetry resources on v5 query_range
Authorize /api/v5/query_range and /preview at the telemetry-resource level,
derived from the request body:
- coretypes: ResourceWithID + ResourceExtractor as the resource-level analogue
of the id extractors; NewResolvedResourceWithID/NewResolvedResourceWithError;
telemetryresource selector regex widened to query-type selectors with up to
two hashed segments (metric name, where clause) or wildcards
- telemetrytypes: QueryRangeResources maps each query to its telemetry
resource (signal/source aware: audit-logs, meter-metrics) with a hierarchical
selector id (query_type/<hash(metric)>/<hash(where)>); PrefixSelector expands
the id into the grant ladder [exact, prefix/*..., *]
- handler: generic TelemetryResourceDef fans out an injected ResourceExtractor;
fails closed when extraction errors or resolves nothing
- audit: log and skip resolved resources that carry a resolution error
- querier routes: ViewAccess -> CheckResources with telemetry read scopes;
substitute_vars stays ViewAccess (no telemetry access)
- sqlmigration 099: backfill telemetry read tuples for existing orgs
(admin: logs/traces/metrics/audit-logs/meter-metrics; editor/viewer:
logs/traces/metrics)
* feat(authz): widen telemetry selector segments to 128 bits
64-bit truncation permits chosen-collision attacks at ~2^32 work; 128 bits
pushes this to 2^64. No hashed selector is persisted yet, so the change is
free.
* chore(docs): regenerate openapi spec with telemetry read scopes
* feat(telemetry): add where clause visitor
* refactor(telemetry): restructure normalizer file and quote bare values
* feat(authz): gate v5 query_range on service.name telemetry selectors
* feat(authz): encode telemetry grants as query-type qualified atom selectors
* feat(authz): move telemetry grant key to plaintext selector segment
* feat(authz): use escaped plaintext telemetry selectors with mechanical ladder
* revert(authz): restore transaction group diff in role update
* test(authz): add querierauthz integration suite for telemetry query_range gating
* test(authz): seed logs so service.name resolves in allowed querierauthz cases
* feat(authz): backfill telemetry read tuples for existing orgs
* chore(authz): reword empty composite query error message
* feat(authz): add meter metrics and audit logs to clickhouse sql
* Revert "feat(authz): add meter metrics and audit logs to clickhouse sql"
This reverts commit c9d870e0ee.
* feat(authz): grant meter-metrics to editor/viewer, keep clickhouse admin-only
* feat(authz): remove the audit logs from clickhouse check altogether until it's introduced
* fix(ruletypes): always serialize notificationSettings.usePolicy
usePolicy is a plain bool tagged omitempty, so a false value is dropped
from the GET response. Clients that pin usePolicy to false read it back as
absent/null. Drop omitempty (matching Renotify.Enabled) so false always
serializes and round-trips.
* test(ruletypes): assert usePolicy round-trips in minimal read shape
usePolicy now always serializes, so the minimal read shape includes it with a false value rather than omitting it.
* fix(ruletypes): always serialize notificationSettings.groupBy
Like usePolicy, groupBy dropped its omitempty so an explicitly-set empty value round-trips instead of reading back as absent. A nil groupBy now serializes as null.
* fix(ruletypes): use omitzero for notificationSettings.groupBy
omitzero omits groupBy only when nil (unset) while preserving an explicitly-set empty array as [], so unset and empty stay distinguishable instead of every response carrying groupBy: null. Matches the sibling newGroupEvalDelay tag.
* fix(ruletypes): use omitzero for renotify.alertStates
Symmetric to groupBy: omitzero omits alertStates only when nil (unset renotify configs stay absent, no regression) while echoing an explicitly-set empty array as [].
* chore(ruletypes): regenerate openapi spec and api client
groupBy and renotify.alertStates switched to omitzero, so the generated spec marks both slices nullable and the orval client types them as [] | null.
* fix: set correct opapi response model for span mapper list
* fix: change group_id to groupId in response
* fix: format properly
* fix: update fixtures
* chore: added types and open api spec changes
* chore: added method to calculate reason
* chore: per group pod status counts with req metric checks method added
* chore: wired up pod status counts
* chore: pod restarts type added
* chore: added restart counts for the group
* chore: bug in query fix
* chore: onboarding API changes
* chore: integration tests added
* chore: added podcountsbyphase in other entities
* chore: added pod status counts for other entities
* chore: added integration tests for other entities
* chore: added checks api changes for other entities
* chore: rearrangement
* chore: removed succeeded status and mark it as completed
* chore: query beautified
* chore: corrected metrics list for metadata lookup
* chore: removed dead constants
* chore: goroutines for ListHosts
* chore: goroutines for ListPods
* chore: goroutines for ListNodes
* chore: goroutines for ListNamespaces
* chore: goroutines for ListClusters
* chore: goroutines for ListDeployments
* chore: goroutines for ListStatefulsets
* chore: added goroutines for ListStatefulsets, ListJobs and ListDaemonsets
* chore: added function
* chore: added struct changes
* chore: added count attr keys
* chore: wired counts to the response fields
* chore: regenerated API spec
* chore: merged main, resolved conflicts
* chore: nodes count surfacing
* chore: integration tests added
* fix: use tuple mapping to uniquely identify attrs:
* chore: integration tests update
* feat(authz): store role transaction groups as document of record
Persist a role's transaction groups as JSON on the role row so the role
details page is reconstructed deterministically from SQL instead of being
rebuilt from OpenFGA tuples (which will soon carry opaque hashed telemetry
selectors):
- authtypes: TransactionGroups gains Value/Scan (validated via
NewTransactionGroups) and MarshalJSON (nil renders as []); NewRole takes
transactionGroups; NewManagedRoles fills managed docs from the registry;
RoleWithTransactionGroups removed - Role carries the wire field and the
AuthZ interface, handler, and OpenAPI responses use *Role; GettableRole
(without transactionGroups) is the list response
- sqlmigration 099: add role.transaction_groups, backfill custom roles from
their permission tuples (dual dialect) and managed roles from the registry
- sqlmigration 059: pin insert columns so the live Role model addition does
not break fresh installs (059 runs before 099)
- ee provider: writes persist the doc alongside FGA tuples (FGA first, SQL
second, as before); GetWithTransactionGroups reads the doc; the per-type
ReadTuples fan-out (readAllTuplesForRole) is removed
- audit middleware: log and skip resolved resources carrying a resolution
error
- frontend: regenerated OpenAPI spec and API types; role list consumers
retyped to GettableRole; role GET keeps transactionGroups
* fix(authz): reconcile role tuples from openfga state, decouple migration 059
- Update and Delete derive their diff/deletion base from the tuples openfga
actually holds for the role (readAllTuplesForRole) instead of the stored
JSON record, so every mutation sweeps drift and residue; the record stays
a display-only artifact written after the tuple write
- ReadTuples restored on the AuthZ interface with plain passthroughs in both
providers and the ee server
- TransactionGroups.Value marshals unconditionally (nil renders as [] via
MarshalJSON) instead of returning a nil driver.Value
- migration 059 uses a migration-local role struct and constructor so live
Role model changes cannot alter its insert; migration 099 drops the manual
column-exists guard (AddColumn emits IF NOT EXISTS)
* refactor(authz): split role into domain Role and StorableRole
Replace the Scan/Value/MarshalJSON codecs on TransactionGroups with the
storable pattern: StorableRole is the bun model carrying transaction groups
as raw JSON text, Role is the pure domain/wire type, and
NewStorableRoleFromRole/NewRoleFromStorableRole convert at the store
boundary (nil groups persist as [], reads parse through the validating
constructor). RoleStore and sqlauthzstore speak StorableRole; both
providers convert; handlers and the wire contract are unchanged.
* revert(authz): restore TransactionGroups codecs over the storable split
Role is a bun relation target (UserRole.Role, ServiceAccountRole.Role), so
splitting it into StorableRole/Role cascaded: relations must point at the
bun model, which broke the user-roles join and leaked the storable shape
into user and service account responses. Keep the single Role model with
Scan/Value/MarshalJSON on TransactionGroups; the storable split fits leaf
models only.
This reverts commit 73aa7d32b1 and keeps transaction_test.go deleted.
* refactor(authz): use bun models in migration 099, wire oss role get
- migration 099 follows the migration-local row struct pattern: bun
NewSelect/NewUpdate for the role table reads and backfill writes; the
openfga store and tuple lookups stay raw like 081/083
- oss provider Get reads the role from the store instead of returning
unsupported
* refactor(authz): org-scoped backfill in migration 099, empty groups on null scan
- migration 099 iterates organizations: per org it backfills custom roles
from their permission tuples (readRoleTuples helper) and managed roles
from the registry (JSON precomputed per role name)
- TransactionGroups scans SQL NULL as an empty slice so the api always
renders transactionGroups as []; nullzero keeps writing NULL for nil
* fix(authz): pass unique constraints to add column in migration 099
* fix: don't return error in v2 list dashboard api if there is a v1 dashboard
* chore: generate api specs
* feat(dashboard-v2): surface legacy dashboards in the v2 list (#12024)
* feat(dashboard-v2): add legacy dashboard dialog
Surfaces a copyable dashboard ID and a Contact Support action for a
pre-v2 (legacy) dashboard that has no v2 spec to render.
* feat(dashboard-v2): hide v2-only actions for legacy dashboards
A legacy dashboard has no v2 spec, so view/open/copy-link/rename/
duplicate/lock don't apply. Gate them behind !isLegacy, leaving only
Delete (and dropping its now-orphan leading divider).
* feat(dashboard-v2): surface legacy dashboards in the v2 list
Flag legacy rows with a badge, block navigation, and open the legacy
dialog on click instead. Disable the pin action with an explanatory
tooltip and gate the row's actions menu to legacy-safe items.
---------
Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* fix(ruletypes): tag rule threshold targets as format: double
BasicRuleThreshold.{TargetValue,RecoveryTarget} and RuleCondition.Target are
*float64 but emitted a bare 'type: number' (swaggest sets format: double only for
non-pointer floats). Bare number makes oapi-codegen clients generate float32, so a
value like 0.8 loses precision on round-trip. Tag them format:double to match
non-pointer float64 fields (e.g. MetrictypesComparisonSpaceAggregationParam).
* chore(frontend): regenerate API client for rule threshold format: double
Reflects the format: double schema change on the rule threshold/condition
targets in the orval-generated client (oxfmt + oxlint applied).
* feat(public-dashboard): detect v1 vs v2 schema for the public viewer
Anonymous public viewers have no feature flags, so the schema can't be read from
use_dashboard_v2. Probe the v2 model endpoint first and fall back to v1 only on the
'dashboard_invalid_data' (HTTP 501) schema-mismatch signal. Probing v2 first also stops
the v1 endpoint from serving v2 dashboards with un-redacted queries.
* feat(public-dashboard): fetch v2 public panel data by key
Adds a by-key fetcher over the anonymous /api/v2/public/dashboards/{id}/panels/{key}/query_range
endpoint (the generated client omits the startTime/endTime params) and a store-free
usePublicPanelQuery that mirrors usePanelQuery's PanelQueryData shape. No variables and no
pagination — the public endpoint supports neither.
* feat(public-dashboard): render v2 public dashboards read-only
Adds a read-only v2 viewer that reuses the authenticated V2 panel renderers
(PanelHeader with hideActions, PanelBody, panel registry) and the pure layoutsToSections
util, with a forked read-only grid. The public page branches on the resolved schema:
v1 keeps the existing container, v2 renders the new viewer. Dashboard variables are not
rendered — the public endpoint does not substitute them.
* feat(public-dashboard): match the standard auto-refresh control
Replace the hand-rolled 'Off' select (which was styled inconsistently and clipped its
options) with a PublicAutoRefresh that mirrors the app's DateTimeSelectionV2 refresh cluster:
a grouped refresh button + auto-refresh popover (Auto Refresh checkbox + full interval list),
portal-rendered so nothing clips. It's prop-driven — the public viewer keeps managing its own
time window — so the container now tracks enabled + interval and exposes a manual refresh.
Also nudge the header-right gap 8→12px.
* feat(public-dashboard): declare v2 query_range params, drop the wrapper, address review
Declare startTime/endTime as query params on the v2 public query_range endpoint via
RequestQuery and regenerate the OpenAPI spec + orval client, so the generated
getPublicDashboardPanelQueryRangeV2 carries them. usePublicPanelQuery now calls the
generated fetcher directly and the hand-written wrapper is removed.
Also from review: drop the defensive panelDefinition guard so an unsupported kind
surfaces loudly, use lodash noop, and trim excessive comments across the v2 files.
* fix: bind query params from PublicWidgetQueryRangeParams
---------
Co-authored-by: Naman Verma <naman.verma@signoz.io>
* chore: added types for containers
* chore: added querier query and constants
* chore: added helper queries
* chore: added wiring
* chore: added open api spec
* chore: endpoint and variable rename
* chore: use recency for container status.reason as well
* chore: pass orgID to getMetadata for containers (adapt to rebased main)
* chore: add metrics to the containers list for metadata and earliest time
* chore: corrected metrics list for metadata lookup
* chore: added changes to the checks API for new kube containers section
* chore: containers query modified
* chore: added integration tests
* chore: integration tests
* chore: goroutines in container monitoring
* chore: constants deduplication
* chore: inlined requests.Post call for this PR instead of wrapping in a function
* chore: added types and open api spec changes
* chore: added method to calculate reason
* chore: per group pod status counts with req metric checks method added
* chore: wired up pod status counts
* chore: pod restarts type added
* chore: added restart counts for the group
* chore: bug in query fix
* chore: onboarding API changes
* chore: integration tests added
* chore: added podcountsbyphase in other entities
* chore: added pod status counts for other entities
* chore: added integration tests for other entities
* chore: added checks api changes for other entities
* chore: rearrangement
* chore: removed succeeded status and mark it as completed
* chore: query beautified
* chore: corrected metrics list for metadata lookup
* chore: removed dead constants
* feat(authz): add serviceaccount and user role handlers
* feat(authz): add support for role and service account extractors
* feat(authz): make the user apis v2
* feat(authz): return the existing service account role id
* feat: add first draft of v2 public dashboard apis
* fix: remove duplicate call to GetDashboardByPublicIDV2 in GetPublicWidgetQueryRangeV2
* fix: fill fields that were in the data blob in v1
* chore: trim comments
* fix: remove fields that v1 also removes when redacting
* chore: rename method name
* test: unit tests for GetPanelQuery
* fix: add fill gaps to query
* fix: generate api specs
* test: add integration tests for new v2 public apis
* fix: add query validation and aggregation validation
* fix: remove unneeded tags db call from public query range api
* fix: redact variable queries as well
* fix: move regex out of method so that it is only compiled once per package load
* chore: remove empty line
* fix: use pointer to specs during redaction
* fix: move single expression validation to dashboard package, use chparser for it
* test: add integration test for variable query redaction
* test: add integration test for expression with many parens
* test: use valid query in integration test
* test: use realistic query in variable
* fix: return list of all tags sorted alphabetically
* chore: return reserved keys in list api response for easy filtering
* fix: add length limit to dashboard display name
* test: check error message as well
* chore: increase the length limits
* chore: add copy suffix on cloning dashboards
* fix: increase limit to 64 for dashboard view name
* fix: send user friendly err message on length check fail
* fix: add path to error message
* fix: include path in main error message directly
* fix: move regex out so that it only compiles once per init
* fix: format integration test properly
The reflector saw Source's unexported valuer.String field and emitted
type: object. Add a JSONSchema exposer that pins type: string, deriving
the enum values from the existing Enum() method so the list of sources
lives in exactly one place.
* chore(authz): delete the deprecated authz apis
* test(authz): rework role integration tests onto the new CRUD APIs
Migrate the role integration suite off the deprecated PATCH endpoints and
onto the current declarative role CRUD APIs (Create/Get/List/Update/Delete
with full transactionGroups).
- role/01_register.py: verify managed roles via GetRole's transactionGroups
against a golden matrix in testdata/role/managed_role_grants.json (no more
DB tuple assertions).
- role/02_crud.py (new): custom-role CRUD lifecycle, declarative update,
validation (naming, invalid verb/type/kind/selector, duplicate, managed
immutability, delete-with-assignee), and license gating.
- role/03_fga.py: resource FGA allow/deny via declarative grant sets.
- role/02_user.py: deleted; user role-membership is covered by the
passwordauthn suite.
- serviceaccount/06_fga.py: migrated to declarative grant PUTs.
- fixtures/role.py: pure data helpers + find_role_id fixture; tests make
their HTTP calls directly.
* test(authz): scope role/SA FGA tests to fine-grained selectors
- role FGA: grant read/update/delete on a specific role name (not "*") and
assert allowed-on-granted vs forbidden-on-other; create is collection-scoped;
list on "*" returns every role.
- serviceaccount FGA: grant on a specific SA id (with a second SA to prove
cross-instance denial); dual attach/detach scoped to SA id + role name.
- add create_role fixture (alongside find_role_id) for happy-path role creation;
validation/failure cases stay inline.
- underscore-prefix file-local constants in both FGA modules.
* test(authz): rename grants terminology to transactions in role tests
* chore(metrics): review follow ups for volume control
* chore: the reduced metrics show up in summary page
* chore: 1h; 6h window changes
* chore: address some gaps
* chore: asset warning gap
* chore: address lint
* chore: regenerate api
* fix: change schema properties based on UI integration review
* fix: check that panels referred in layouts exist
* chore: extract out validate panels method
* test: add test for missing spec prefix in layout
* fix: reject dashbaords that have vars with the same name
* fix: add additional error info on patch application error
* fix: add validations to list variable that text variable has
* fix: replicate text variable spec in signoz to make name required
* chore: replicate variable.sort into signoz
* chore: remove unsupported enum values (causing errors right now)
* chore: fix variable sort type errors
* fix: add back enum values
* fix: reject single-element list default when allowMultiple is false in list variables
* fix: remove unused import
* fix: make display required
* chore: make queries non-nullable
* fix: properly define default value and datasource plugin spec's api specs
* fix: promote variable defaultValue to a named oneOf component
The list variable defaultValue was an inline string | []string oneOf,
which downstream codegen can't canonicalize: tfplugingen-openapi rejects
the inline scalar-or-array multi-type, and oapi-codegen has no named type
to attach the union's Marshal/UnmarshalJSON to.
Shape the vendored variable.DefaultValue as the named VariableDefaultValue
oneOf via a reflector InterceptSchema hook and let defaultValue $ref it,
instead of overriding the property inline. Regenerate the OpenAPI spec and
frontend client accordingly.
* refactor: move VariableDefaultValue oneOf into dashboardtypes
Define VariableDefaultValue in dashboardtypes as a subclass of the perses
variable.DefaultValue and attach the string | []string oneOf via its own
PrepareJSONSchema, instead of shaping the perses type from an openapi.go
InterceptSchema hook. This keeps the union's schema next to its type.
The named component is now DashboardtypesVariableDefaultValue; regenerate
the OpenAPI spec and frontend client accordingly.
---------
Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
Co-authored-by: grandwizard28 <vibhupandey28@gmail.com>
* feat: add api to fetch v2 dashboards for a metric name
* chore: switch to query param
* chore: generate API specs
* chore: use proper struct in return type of GetByMetricNamesV2
* chore: add method for escaping like patterns in sqlstore formatter
* fix: use only one db call in GetByMetricNamesV2
* chore: dont use type alias for list of references
