Commit Graph

2141 Commits

Author SHA1 Message Date
Nikhil Soni
94de5cf72b refactor(promote): move index creation into the metadata store 2026-10-05 21:56:43 +05:30
Nikhil Soni
6f24594cf7 feat(promote): per-path indexes for trace attributes and bare paths in every domain
- the traces attributes target now supports per-path skip indexes; their
  expression is a bare type cast, CAST(dynamicElement(col.path, 'T'), 'T'),
  with no lower/assumeNotNull folding. The cast also unwraps the Nullable
  that dynamicElement returns, which bloom filter indexes reject.
- the body. path prefix is dropped for logs body: the API URL already
  names the context, so paths are bare attribute names in every domain;
  prefixed paths are rejected with a guiding error.
- ListLogsJSONIndexes generalizes to ListJSONIndexes(source) driven by the
  target, and the index expression unfolding accepts both the folded and
  the bare cast forms.
2026-10-05 21:56:43 +05:30
Nikhil Soni
8e607aa6d7 refactor(promote): drop comments that restate the code 2026-10-05 21:56:40 +05:30
Nikhil Soni
e35ad7b1cd refactor(promote): validate the listing filters in the handler 2026-10-05 21:56:40 +05:30
Nikhil Soni
3498eb00b1 feat(promote): add filters to the promoted paths listing
GET /api/v1/promoted_path accepts signal, context, promoted and indexes
query parameters narrowing the listing. The signal, context and path
fields of PromotePath are now marked required in the API contract.
2026-10-05 21:56:40 +05:30
Nikhil Soni
fc1d0cc504 refactor(promote)!: move the promotion domain from the URL into the request
The promote paths API collapses to /api/v1/promoted_path. Each PromotePath
carries its signal and context, so a single request can span domains and
the list endpoint returns every domain's paths annotated with theirs.
2026-10-05 21:56:40 +05:30
Nikhil Soni
a6ab2f3583 refactor(promote)!: rename the API path to promoted_path
A noun and singular, like the other API URLs.
2026-10-05 21:56:40 +05:30
Nikhil Soni
1525a334c2 refactor(promote): group target constructors; drop redundant and unsupported-feature tests
- move NewTargetFromPath next to the other target constructors
- drop TestNewTargetFromPath: thin glue over SignalFromText/FieldContextFromText/TargetFor
- drop traces index rejection cases: per-path indexes are simply not supported for traces yet
2026-10-05 21:56:40 +05:30
Nikhil Soni
378717a3c0 test(promote): cover the per-path skip index creation of the logs body domain 2026-10-05 21:56:40 +05:30
Nikhil Soni
41427714fe refactor(promote): move the path resolution to types with a validate method, table-drive the tests 2026-10-05 21:56:40 +05:30
Nikhil Soni
cadad6d61d test: align the subtest names with the table format rule 2026-10-05 21:56:40 +05:30
Nikhil Soni
d0eb63073e fix(promote): rename the signal path variable to telemetry_signal
orval generates an AbortSignal parameter named signal for every client
method, so a {signal} path variable produced a duplicate identifier in
the generated client (tsc error). The URL itself is unchanged in
behavior: /api/v1/promote_paths/{telemetry_signal}/{context}.
2026-10-05 21:56:40 +05:30
Nikhil Soni
d9f2ebbf0e refactor(promote): inline the promote and list helpers into their sole callers 2026-10-05 21:56:40 +05:30
Nikhil Soni
cb683a82b6 refactor(promote)!: drop the legacy logs promote_paths routes
There are no consumers of /api/v1/logs/promote_paths, so no backward
compatibility is needed: the logs body domain is served by the generic
/api/v1/promote_paths/{signal}/{context} routes and the legacy routes
and handler methods are removed.
2026-10-05 21:56:40 +05:30
Nikhil Soni
66fc58054b refactor(promote): move Target into target.go, enum-style SignalFromText, rename handler method
- Target type definition moves from types.go to target.go alongside its
  constructors, with inline comments
- SignalFromText follows the codebase enum pattern (switch over the
  declared values + Enum method) instead of a string-to-signal map
- generic route handler method renamed HandlePromotePaths -> PromotePaths
2026-10-05 21:56:40 +05:30
Nikhil Soni
e482ad78b4 refactor(promote): centralize domain construction and generalize routes
- target construction moves to promotetypes: a generic NewTarget plus
  per-domain constructors (NewLogsBodyTarget, NewTracesAttributesTarget)
  and a TargetFor registry keyed by (signal, context); implpromote and
  telemetrymetadata no longer hand-roll domain literals
- routes generalize to /api/v1/promote_paths/{signal}/{context}: the
  legacy logs body route (/api/v1/logs/promote_paths) is kept for
  compatibility but the domain now travels in the path, so a future logs
  attribute domain does not collide with the logs body route; supersedes
  the /api/v1/traces/promote_paths routes
- add telemetrytypes.SignalFromText for parsing the signal path variable
2026-10-05 21:56:40 +05:30
Nikhil Soni
d26cc928bf refactor(promote): template the promotion record with EvolutionEntry
Target now carries an EvolutionEntry template (signal, promoted column
name and type, field context) instead of loose signal/context/column
fields, so the store write is exactly row template + field names +
release time and the hardcoded JSON() column type moves to the domain
definitions. DBName/LocalTableName stay on Target explicitly as index
DDL config, used only by targets with index support.
2026-10-05 21:56:40 +05:30
Nikhil Soni
17b8b6504b refactor(promote): collapse module interface to target-parameterized methods
The per-domain methods were pure delegates; the promotion domain now
travels as promotetypes.Target through Module.ListPromotedPaths /
Module.PromotePaths, with the handler methods (one per route) passing
their domain's target.
2026-10-05 21:56:40 +05:30
Nikhil Soni
bcd3937c4c feat(promote): add traces attributes promotion API
Refactor the promote module into a target-parameterized core so the logs
body_v2 flow and future promotion domains share one implementation, and
add the spans attributes JSON column (attributes -> attributes_promoted)
as a second domain behind POST/GET /api/v1/traces/promote_paths.

- promotetypes.Target describes a promotion domain: signal, field
  context, db/table, base/promoted columns, path prefix rule and whether
  per-path skip indexes are supported
- index support is optional per target; traces starts promotion-only
  since the traces query builder does not consume per-path skip indexes
- metadata store GetPromotedPaths/PromotePaths take (signal, column,
  context) instead of being hardcoded to the logs body column
- fix the list response never attaching indexes to promoted entries
  (aggregated by unprefixed name but looked up by prefixed path) and
  reporting indexed+promoted paths twice
2026-10-05 21:56:40 +05:30
Naman Verma
e8c22cb259 fix(promql): serve transpiled series without their synthetic __name__ (#13031)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

`max_over_time` expects a range vector as input, which is `labels ->
[(timestamp,value),...]`. it turns each entry into
`labelsWith__name__removed -> maxOfAllValues`.

However, if two entries have `labelsWith__name__removed` as the same,
then `max_over_time` throws an error. For eg if the input is:
1. {"host":"a", "__name__": "transpiled_1"} -> ....
2. {"host":"a", "__name__": "transpiled_2"} -> ....
then `max_over_time` will break.

Currently, `executeHybrid` in
`pkg/prometheus/clickhouseprometheusv2/transpiler_exec.go` always puts
in the `__name__` label as `signoz_transpiled_*`, which can lead to the
above scenario.

Removing this `__name__` label fixes that issue. Also, nothing ever
needs this label. When the engine asks for `signoz_transpiled_0`, our
storage finds the data with a map lookup on that name and returns it.
The series' own labels play no part in the lookup.

After this change, no synthetic `__name__` exists anymore. So, the code
that stripped it after the engine ran, and `mergeMatrixByLabelset`,
which re-merged the rows those names had split, are deleted. The
engine's own merging handles this now.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/pulse-pod/issues/508

<!--If applicable, include screenshots or screen recordings that clearly
show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-10-05 09:56:02 +00:00
Tushar Vats
7663ab02d4 feat(logparsingpipeline): add json_body_dual_ingestion flag for dual body ingestion (#12831)
#### Description

Server counterpart to SigNoz/signoz-otel-collector#891, which makes the
collector write each log body to both the legacy `body` column and
`body_v2` while a `json_body_dual_ingestion` flag is on.

- Adds the `json_body_dual_ingestion` feature flag (experimental, off by
default).
- Normalize is placed by read mode, so user pipelines always see the
body the explorer shows. With `use_json_body` on it stays ahead of user
pipelines. With dual ingestion alone, reads are still on the legacy
`body`, so it runs after them and only feeds `body_v2`.
- Whenever dual ingestion is on, the operator carries
`json_body_dual_ingestion: true` so it stashes the original body for the
exporter to restore. Running last under dual makes that stash the
post-pipeline body, exactly what legacy ingestion stores today.
- The pipeline preview follows the same rule and normalizes only under
`use_json_body`.

Design notes: [Normalize Operator and
Pipelines](https://app.notion.com/p/signoz/Normalize-Operator-and-Pipelines-3d7fcc6bcd19802396b9e8e817e30381),
[Dual JSON body
ingestion](https://app.notion.com/p/signoz/Dual-JSON-body-ingestion-3c6fcc6bcd198049963bce6ce2648af8)

#### Additional Information

- Collectors must run a build containing
SigNoz/signoz-otel-collector#891 before this flag is turned on. Verified
locally against v0.144.9: an older collector does not reject the unknown
operator key, it silently ignores it (operator configs are decoded with
`confmap.WithIgnoreUnused()`), runs normalize, and writes the normalized
body into the legacy `body` column until it is upgraded. No collector
release includes #891 yet.
- The exporter's own `json_body_dual_ingestion` key is collector deploy
config and is flipped together with this flag; the server does not set
it.
- Toggling either flag does not bump the pipeline config version, so
connected agents need a new pipeline save to pick up the operator or its
position. Same caveat as `use_json_body` today.
- Under dual, normalize is last among the SigNoz pipelines; custom
collector processors placed after them still see the normalized map.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-10-02 10:56:38 +00:00
Srikanth Chekuri
1643df620b feat: resolve semconv families across logs and metrics (#12870)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

Phase 2 of #6143: semantic-convention families resolve on logs and
metrics, behind the `resolve_semconv_families` flag (default off), on
the storage contract of #12802.

- Registry: each member carries the scope of its own rename edges, so a
fan-out keeps one membership per target and an ambiguous name stays
literal.
- Logs and metrics families need no family code of their own. The gate
applies to every signal, and `LogicalRead` merges members through each
storage's `Read`.
- Metric-name families union the storage names in every `metric_name`
filter, and the querier reads type, temporality, and the reduced flag
across the family.
- Span-metrics labels: the metrics the processor emits, listed by name,
also read each family member with the `resource_` prefix. Requested
names are never rewritten.
- Values suggestions and related values cover every spelling of the
family.
- `deployment.environment.name` resolves on all three signals.
`db.system.name` stays off until a value-mapping reader exists.

#### Additional Information

- `pkg/semconv.Family` fields are now unexported, and `transition.go` is
removed. #12446 reads the old API and needs an update when stacked.
- A target that emits both names of a metric-name family double-counts
in `sum()` during the overlap window. Reading both names is the feature.
Pinned by a test.
- A family of metrics labels keeps the keyless contract of a single
label: no guard, no NULL group.
2026-10-01 20:28:15 +00:00
Vikrant Gupta
572345be63 feat(authz): enable FGA for users and reset password tokens (#13020)
#### Description

- Moves the users API off the legacy `AdminAccess` gate onto
`CheckResources` + `ResourceDef`s; `me` and anonymous password flows
stay `OpenAccess`.
- Invite checks `role:attach` per requested role; an empty role list
resolves to no link, so the sibling def skips the check.
- Migration `131_add_user_tuples` backfills admin `user` and
`factor-password` tuples for existing orgs.

#### Issues closed by this PR

Closes: SigNoz/keystone-pod#38
2026-10-01 11:26:18 +00:00
Nikhil Soni
bcf96f2816 feat(traces-qb): gate JSON span attribute reads behind a feature flag (#12966)
#### Description

- The new feature flag `use_trace_attributes_json` (default off) now
gates the JSON columns in the traces `getColumn`, alongside the
evolution entry.
- `SelectEvolutionsForColumns` now ignores evolutions of columns the
mapper didn't return instead of erroring, so a flag-off attribute
resolves to its map even though the key carries the JSON evolution.

Part of https://github.com/SigNoz/signoz/pull/12966

#### Additional Information

- Evolution entry migration: SigNoz/signoz-otel-collector#928
- Original QB PR: #4781
2026-10-01 10:21:10 +00:00
Vikrant Gupta
e374d03e54 revert(authz): restore gjson-based body extraction in resource middleware (#13024)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- Reverts #13014 and #13015. The resource middleware goes back to
reading body-derived resource ids with `BodyJSONPath` / `BodyJSONArray`
over the raw body, and handlers decode their own request bodies again.
- Authz should not own request decoding; that ownership stays with the
handlers.

#### Additional Information

- Contributes to: https://github.com/SigNoz/keystone-pod/issues/37
2026-09-30 13:53:21 +00:00
Swapnil Nakade
d445b6c296 chore: bumping cloud integration agent version to v0.0.15 (#13023)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Bumping the cloud integration agent's version to latest v0.0.15

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Contributes to https://github.com/SigNoz/keystone-pod/issues/101
2026-09-30 12:38:24 +00:00
Swapnil Nakade
fd8aaac300 feat: adding sync state in cloud integration (#12991)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
The agent only saw the current list of enabled regions, so it couldn’t
tell which regions had been removed. To find stacks to clean up, it
checked unrelated AWS regions, causing unnecessary calls and permission
errors. Sync state keeps track of regions sent to the agent and pending
removals until the agent acknowledges cleanup.

Please check
[comment](https://github.com/SigNoz/keystone-pod/issues/101#issuecomment-5832865898)
for approach

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Contributes to https://github.com/SigNoz/keystone-pod/issues/101

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
This PR should be merged before changes for cloud-integration repo.

<!--Please delete paragraphs that you did not use before submitting.-->
2026-09-30 11:39:46 +00:00
Naman Verma
e4cd8dbf10 chore: store v2 config for notification channels in db (#12984)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

1. add a config column to notification channels table where the channel
config goes without dealing with receiver at all. this helps in cleaning
up all round trip issues caused by dealing with receiver in the storage
layer. v2 apis treat receiver as a side effect now
1a. for applicable fields, defaults are filled in create/update api if
fields are omitted
1b. explicit [] and {} are no longer dropped, and omitted [] and {} are
returned as explicit null
2. add integration tests for all notification channel round trip issues
3. code cleanup of v2 channels types 
4. migration to fill the config column from receiver column, which logs
results like dashboards migration did
4a. it fails for receivers that cannot be modeled in v2. repair api can
be used for them
4b. for types that v2 supports, any fields that v1 supports but v2
doesn't, this migration drops those fields
5. make v1 API reject anything that v2 apis do not support, and also
fill the new config column added
6. add integration tests for v1<>v2 interaction to ensure that alert
manager doesn't break because of new changes added

What breaks/changes for v1:
1. types that v2 does not support can no longer be created
2. channels with multiple receivers cannot be created
3. channels with fields that v2 does not support cannot be created
4. existing channels of types that v2 does not support can no longer be
edited via v1 API. They can be deleted though. Also, they keep on
sending notifications as before (sigh).

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/pulse-pod/issues/376
Closes https://github.com/SigNoz/pulse-pod/issues/378
2026-09-30 11:30:33 +00:00
praneeth-signoz
81d024dfc8 chore(channel-receivers): relax channel validations (#13008)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

- Relax all the strict validations like https, host name...for all the
channels. And why this is needed ?

1. Channel URLs were pinned to the vendor's own host —
`chat.googleapis.com`, `*.atlassian.net`— which blocked deployments that
send notifications through a proxy or relay.
2. Provider's contract is not ours to hardcode — so we check the field
is there and let the provider reject what it doesn't accept

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

closes https://github.com/SigNoz/pulse-pod/issues/374
2026-09-30 10:20:41 +00:00
Vikrant Gupta
8dba9a13ea fix(authz): read every body-derived resource id from the decoded request (#13015)
#### Description

- Follows #13014. Moves the remaining body-derived resource ids (gateway
limits, zeus hosts, cloud integration check-ins, auth domains, query
range) off gjson and onto the decoded request, with the handlers reading
the same value. Part of SigNoz/keystone-pod#37.
- Removes `BodyJSONPath`, `BodyJSONArray`, and
`ExtractorContext.RequestBody`.

#### Issues closed by this PR 

- Closes: https://github.com/SigNoz/keystone-pod/issues/37
2026-09-30 09:34:32 +00:00
Vikrant Gupta
8d80f98710 fix(authz): decode the request body once in the resource middleware (#13014)
Some checks failed
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
#### Description

- The resource middleware now decodes the body once into the route's
declared `OpenAPIDef.Request` type, rejects a malformed body before any
check, and carries the decoded value on `ExtractorContext.Body`.
`BodyField` / `BodyFields` read ids off that value, and handlers read
the same value via `coretypes.BodyFromContext`.
- `handler.Handler` exposes `Request()`, and `handler.New` panics when a
route with resource defs declares a non-pointer request, since the
middleware instantiates it.
- Only `POST /api/v1/service_account_roles` is wired to the new
extractors in this PR to keep the review small. The remaining body
routes still use the gjson extractors and decode again in their
handlers.

#### Issues closed by this PR
- Contributes to: https://github.com/SigNoz/keystone-pod/issues/37
2026-09-29 21:32:25 +00:00
Swapnil Nakade
254758942e fix: using allLogs categoryGroups for azure container apps (#13013)
#### Description

- The Container Apps logs pipeline failed because the definition passed
log category names (`ContainerAppConsoleLogs`, `ContainerAppSystemLogs`)
as `categoryGroups`. Azure accepts only `allLogs` or `audit` there, so
it rejected the diagnostic setting.
- Switched to `allLogs`, matching the other Azure services. The agent
picks this up on its next config sync, so no agent release is needed.

#### Issues closed by this PR

Closes SigNoz/keystone-pod#45

#### Additional Information
- Not tested on live Azure. Microsoft's built-in policy for
`Microsoft.App/managedEnvironments` sends the same `allLogs` setting to
Event Hub.
2026-09-29 14:24:00 +00:00
praneeth-signoz
47dd1fabf3 chore(channel-specs): Move channel specs to separate files (#12989)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

- Moved channel specs to separate files under alert manager types
- Channel receivers are also moved the same file

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Closes
https://github.com/orgs/SigNoz/projects/34/views/26?pane=issue&itemId=252814150&issue=SigNoz%7Cpulse-pod%7C374
2026-09-28 17:44:35 +00:00
Vikrant Gupta
270988fb48 fix(tokenizer): persist last_observed_at on postgres (#12982)
#### Description

- The flush CTE rendered `last_observed_at` as an untyped literal, which
postgres resolves to `text` and refuses to assign to the `timestamptz`
column. The column never populated, so the idle expiry never applied.
- Build the CTE from the token model with only `id`, `last_observed_at`
and `updated_at`, so bun casts per dialect and no token secrets land in
the statement.
- Flush now applies cached times through `Token.UpdateLastObservedAt`,
which also skips rows with a newer stored value.
- Integration test in `passwordauthn` runs with a short GC interval and
asserts the column populates on both sql stores.
2026-09-28 14:23:01 +00:00
Naman Verma
39badeb591 fix: remove rules types package import from migration #049 (#12998)
Some checks failed
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Migration package ideally should have have things from types package
imported. Given that rules v1->v2 will (most probably) update/remove
some of the types, such as removing `PreferredChannels` from
`PostableRule`, better not to have this type imported in migrations
package.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Part of https://github.com/SigNoz/pulse-pod/issues/225
2026-09-28 11:39:07 +00:00
Nityananda Gohain
ec05bfe755 fix: empty patterns in pricing are rejected (#12995)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Empty patterns were not rejected because of which corrupt config was
created, rejecting them at the handler layer.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/nerve-pod/issues/282
2026-09-28 08:09:38 +00:00
Nityananda Gohain
8371a70801 perf(querybuilder): compare materialized exists columns explicitly (#12978)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Materialized existence checks now render as an explicit comparison
instead of a bare bool column. Results are unchanged; only skip-index
usage improves.

  ```sql
  -- before
  WHERE `attribute_string_gen_ai$$request$$model_exists`
     OR `attribute_string_gen_ai$$provider$$name` = 'anthropic'

  -- after
  WHERE `attribute_string_gen_ai$$request$$model_exists` = true
     OR `attribute_string_gen_ai$$provider$$name` = 'anthropic'
  ```

  <details>
<summary>EXPLAIN indexes = 1 (trace-matching phase, 123M
spans)</summary>

  Before: bare `col_exists`
  ```
  Name: idx_gen_ai_span_exists
  Granules: 15193/15193
  Name: <Combined skip indexes>
  Granules: 15193/15193
  ```

  After: `col_exists = true`
  ```
  Name: idx_gen_ai_span_exists
  Granules: 15193/15193
  Name: <Combined skip indexes>
  Granules: 488/15193
  ```
  </details>

----
- ClickHouse can use a different skip index for each side of an OR and
union the results, but it can't when one side is a bare bool column.
Comparing with `= true` fixes that.
- This shape comes from the AI explorer trace list with a span filter: a
trace qualifies when it has a gen_ai span *and* a span matching the
filter (possibly different spans), so the WHERE is `(gen_ai gate) OR
<filter>` followed by a HAVING.
- Needs the gen_ai materialized columns and `idx_gen_ai_span_exists`
from SigNoz/signoz-otel-collector#929; without them there's no index to
combine.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/nerve-pod/issues/282

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
- Benchmarked the AI trace list filtered on `gen_ai.provider.name`
against a 123M-span table (direct I/O, caches off): from ~30M spans in
the window, latency drops 16–17% and CPU 35–38%, with ~25x fewer rows
read (123M spans: 510 → 427 ms, 1.5 → 0.9 sCPU). The saved time and CPU
keep growing with span count, so larger windows save more.
- Single-condition filters (`gen_ai.request.model EXISTS` in dashboard
panels, the AND-ed gate in AI aggregations) already pruned with the bare
form; no change there.
2026-09-24 13:15:32 +00:00
Naman Verma
9d9b0e194a chore: add ability to mark API stability as beta/alpha (#12957)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

If an API that is already deployed is currently being tested via UI
integration or any other means, we should mark such APIs as under
development so that other external clients know that these APIs aren't
fully stable. This is especially required if we are working on v2
versions of APIs for any entity.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Part of https://github.com/SigNoz/pulse-pod/issues/369

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information

This PR adds the development flag on the v2 notification channel APIs

<!--Please delete paragraphs that you did not use before submitting.-->
2026-09-24 12:19:02 +00:00
Nikhil Mantri
ee35fc351f feat(alerts): New list API for alert rules (powers filters, sorting, pagination) (#12780)
#### Description

- New `GET /api/v3/rules` list API for alert rules: filter query DSL,
`states` filter, sort, and offset pagination (design discussion:
SigNoz/pulse-pod#324).
- Based on #12806, which extracts the shared list filter SQL compiler;
this PR adds only the rules key-policy resolver
(`sqlrulestore/filterquery_resolver.go`) on top of it.
- Rule state lives only in the rule manager's memory, so state
filtering, total, sort and pagination run in code after the SQL fetch;
total always equals what is pageable.
- Sorting is deterministic on ties: equal rows break on name then id,
always ascending, so pages never overlap or drop rows between requests.
- Response rows carry only list-page fields, deliberately excluding
`condition`, `annotations` and `notificationSettings`. The envelope also
returns the org's distinct label pairs and the reserved filter keys for
suggestions.
- Also guards previously unlocked reads of the rules map
(`ListRuleStates`, `GetRule`, `TriggeredAlerts`).

**Filter keys and operators**

| Key | Operators | Notes |
|---|---|---|
| `name`, `created_by`, `updated_by` | `=`, `!=`, `CONTAINS`, `LIKE`,
`ILIKE`, `IN` and negations | string search |
| `labels.<key>` | string operators plus `EXISTS`, `NOT EXISTS` |
missing label evaluates as empty string; keys are case-sensitive |
| `severity` | same as `labels.<key>` | alias for `labels.severity` |
| `created_at`, `updated_at` | `=`, `!=`, `<`, `<=`, `>`, `>=`,
`BETWEEN`, `NOT BETWEEN` | quoted RFC3339 values |
| `alert_type` | `=`, `!=`, `IN`, `NOT IN` | enum: `METRIC_BASED_ALERT`,
`TRACES_BASED_ALERT`, `LOGS_BASED_ALERT`, `EXCEPTIONS_BASED_ALERT` |
| `rule_type` | `=`, `!=`, `IN`, `NOT IN` | enum: `threshold_rule`,
`promql_rule`, `anomaly_rule` |

- A bare word is free text: a case-insensitive substring match over
name, description and labels.
- `state` is not a DSL key. It is the repeated `states=` query param:
`firing`, `pending`, `recovering`, `inactive`, `nodata`, `disabled`.
- An unknown key or `REGEXP` returns a 400.

#### Issues closed by this PR

Closes SigNoz/pulse-pod#226

#### Additional Information

- A missing label evaluates as the empty string for every value
operator, one uniform rule instead of the querier's per-operator split
([`AddDefaultExistsFilter`](https://github.com/SigNoz/signoz/blob/e0da06f76d/pkg/types/querybuildertypes/querybuildertypesv5/builder_elements.go#L160));
presence is asked with `EXISTS` / `NOT EXISTS`.
- Integration tests
(`tests/integration/tests/alerts/06_list_rules_v3.py`) cover filters,
states, sorting, pagination, totals and the error contract, run against
both sqlite and postgres.
- Found while testing: the stock `create_notification_channel` fixture
teardown silently fails and leaks channels; follow-up fix needed.

---------

Co-authored-by: Naman Verma <naman.verma@signoz.io>
2026-09-24 07:16:12 +00:00
Nityananda Gohain
5a1be60745 fix(ai-o11y): scope overview dashboard to gen_ai spans and move message attributes (#12967)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- Top span names and Cost by service use `builder_ai_query`, so non-AI
spans no longer show up.
- LLM cost/token panels filter on `gen_ai.request.model EXISTS`; with
variables on "All" they scanned every span.
- Default span mappers now move (not copy) vendor message keys into
`gen_ai.input.messages` / `gen_ai.output.messages`, as documented.
- Bumped versions: dashboard to 3, `gen_ai.llm` mapper to 3,
`gen_ai.agent` mapper to 2.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-23 14:56:04 +00:00
Nityananda Gohain
6b66ab64c8 fix: add ai-o11y quick filter migration (#12964)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Added migration to update old instances where quick filters for ai-o11y
is not present.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-23 11:15:42 +00:00
Naman Verma
362d3a4fdf fix: backfill notification channel tuples (#12960)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
cacheci / tests (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Same migration logic as number 128. Needed for enterprise servers as
these tuples decide whether a role may create, list, read, update or
delete channels at all. An existing org with zero channels still needs
them, otherwise its admin can't create the first one or even list the
empty page.
2026-09-23 09:48:11 +00:00
Nityananda Gohain
5aca8b0d3c chore: remove ai-o11y ff (#12947)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Remove ai-o11y FF and enable it by default

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-23 06:39:16 +00:00
Pandey
f2229a1064 fix(analytics): format segment logger messages before passing to slog (#12950)
#### Description

- segment's `Logger` interface is printf-style, but the adapter passed
`format` as the slog message and `args` as key-value pairs.
- slog never substituted the `%d` placeholders and rendered each
positional arg as a `!BADKEY` attr.
- `Logf` and `Errorf` now `fmt.Sprintf` the message first, matching the
opamp logger adapter.
2026-09-22 19:18:36 +00:00
Vikrant Gupta
e2e9173986 fix(user): revoke sessions when a user is deleted (#12943)
#### Description

- `DeleteUser` never told the tokenizer about the deletion.
`SoftDeleteUser` removed the `auth_token` rows with raw SQL, so the
opaque tokenizer kept serving the deleted user's session from cache
until rotation forced a DB read, up to `rotation.interval` later.
- The tokenizer eviction now runs before the soft delete, inside one
transaction; `SoftDeleteUser` joins the caller's transaction instead of
opening its own.
- The hourly last-observed-at flush returned an error for any org with
nothing to flush because bun rejects an empty `VALUES` slice. It now
returns early.
- Adds an integration test asserting a deleted user's held token is
rejected on the next request.

#### Additional Information

Only affects the opaque tokenizer; under the JWT tokenizer
`DeleteTokensByUserID` is a no-op.
2026-09-22 12:14:51 +00:00
Nityananda Gohain
905e935658 fix: use db upsert for model pricing (#12942)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- Each pricing rule ran a SELECT then an INSERT or UPDATE. Rules are now
written with `INSERT ... ON CONFLICT DO UPDATE`, two statements per
request at most.
- Rules without `isOverride` match on `source_id` and skip rows the user
has overridden. Rules with it match on `id`.
- Dropped the `default:` bun tags. bun turns zero values into SQL
`DEFAULT` on insert, so a rule created disabled was stored as enabled.
- Added an integration suite for sync, override, hand-back and bulk
writes.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-22 11:15:20 +00:00
Vikrant Gupta
e8324581b3 fix(tokenizer): accept the previous token pair only within the rotation duration (#12941)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- `Token.Rotate` accepted the previous token pair only when the rotation
was older than `rotation.duration` and rejected it inside the window,
the inverse of the documented intent.
- With the opaque tokenizer, two holders of the same pair (browser tabs,
the axios interceptor and the SSE wrapper) racing at the rotation
boundary meant the loser got 401 on `/sessions/rotate` and the frontend
logged the user out. It also left a stale pair exchangeable for the live
session until the next rotation.
- `RotateToken` now detects that `Rotate` left the stored row untouched
and returns the current pair without rewriting it; the previous check
compared against the input and could never match.
- Adds a unit test for the previous-pair path inside and outside the
window.

#### Additional Information

The JWT tokenizer is unaffected since its rotation is stateless.
2026-09-22 09:58:52 +00:00
Naman Verma
13a57ebb9c chore: remove v1 dashboards code from backend (#12932)
<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/pulse-pod/issues/321
2026-09-22 08:46:28 +00:00
Nityananda Gohain
64fff60d7e chore: update dashboard for ai observability (#12875)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description
Adds the actual dashboard to the backend.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/4501

### Additional information
- Draft until the dashboard DTO changes merge; the query fixes below
land on top of them.
- will update the dashboard based on ai query builder
2026-09-21 17:57:58 +00:00
Nityananda Gohain
dd3b99f19c fix: update attribute mappings for ai (#12925)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

Version 2 of the `gen_ai.llm` and `gen_ai.tool` defaults, fixes only:

- Tool condition narrowed to `tool.name`, `ai.toolCall.`,
`tool_call_args`, `tool_response`; the bare `tool` substring was firing
on LLM spans.
- Dropped the `gen_ai.operation.name` mapper; `llm.request.type`
overwrote native values with non-semconv ones.
- Added `ai.response.toolCalls` as an output messages source for Vercel
tool-call turns.
- Renamed `gen_ai.response.finish_reason` to
`gen_ai.response.finish_reasons`.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-21 17:14:18 +00:00