Commit Graph

331 Commits

Author SHA1 Message Date
Nikhil Soni
94de5cf72b refactor(promote): move index creation into the metadata store 2026-10-05 21:56:43 +05:30
Nikhil Soni
6f24594cf7 feat(promote): per-path indexes for trace attributes and bare paths in every domain
- the traces attributes target now supports per-path skip indexes; their
  expression is a bare type cast, CAST(dynamicElement(col.path, 'T'), 'T'),
  with no lower/assumeNotNull folding. The cast also unwraps the Nullable
  that dynamicElement returns, which bloom filter indexes reject.
- the body. path prefix is dropped for logs body: the API URL already
  names the context, so paths are bare attribute names in every domain;
  prefixed paths are rejected with a guiding error.
- ListLogsJSONIndexes generalizes to ListJSONIndexes(source) driven by the
  target, and the index expression unfolding accepts both the folded and
  the bare cast forms.
2026-10-05 21:56:43 +05:30
Nikhil Soni
e35ad7b1cd refactor(promote): validate the listing filters in the handler 2026-10-05 21:56:40 +05:30
Nikhil Soni
3498eb00b1 feat(promote): add filters to the promoted paths listing
GET /api/v1/promoted_path accepts signal, context, promoted and indexes
query parameters narrowing the listing. The signal, context and path
fields of PromotePath are now marked required in the API contract.
2026-10-05 21:56:40 +05:30
Nikhil Soni
fc1d0cc504 refactor(promote)!: move the promotion domain from the URL into the request
The promote paths API collapses to /api/v1/promoted_path. Each PromotePath
carries its signal and context, so a single request can span domains and
the list endpoint returns every domain's paths annotated with theirs.
2026-10-05 21:56:40 +05:30
Nikhil Soni
1525a334c2 refactor(promote): group target constructors; drop redundant and unsupported-feature tests
- move NewTargetFromPath next to the other target constructors
- drop TestNewTargetFromPath: thin glue over SignalFromText/FieldContextFromText/TargetFor
- drop traces index rejection cases: per-path indexes are simply not supported for traces yet
2026-10-05 21:56:40 +05:30
Nikhil Soni
378717a3c0 test(promote): cover the per-path skip index creation of the logs body domain 2026-10-05 21:56:40 +05:30
Nikhil Soni
41427714fe refactor(promote): move the path resolution to types with a validate method, table-drive the tests 2026-10-05 21:56:40 +05:30
Nikhil Soni
cadad6d61d test: align the subtest names with the table format rule 2026-10-05 21:56:40 +05:30
Nikhil Soni
d0eb63073e fix(promote): rename the signal path variable to telemetry_signal
orval generates an AbortSignal parameter named signal for every client
method, so a {signal} path variable produced a duplicate identifier in
the generated client (tsc error). The URL itself is unchanged in
behavior: /api/v1/promote_paths/{telemetry_signal}/{context}.
2026-10-05 21:56:40 +05:30
Nikhil Soni
d9f2ebbf0e refactor(promote): inline the promote and list helpers into their sole callers 2026-10-05 21:56:40 +05:30
Nikhil Soni
cb683a82b6 refactor(promote)!: drop the legacy logs promote_paths routes
There are no consumers of /api/v1/logs/promote_paths, so no backward
compatibility is needed: the logs body domain is served by the generic
/api/v1/promote_paths/{signal}/{context} routes and the legacy routes
and handler methods are removed.
2026-10-05 21:56:40 +05:30
Nikhil Soni
66fc58054b refactor(promote): move Target into target.go, enum-style SignalFromText, rename handler method
- Target type definition moves from types.go to target.go alongside its
  constructors, with inline comments
- SignalFromText follows the codebase enum pattern (switch over the
  declared values + Enum method) instead of a string-to-signal map
- generic route handler method renamed HandlePromotePaths -> PromotePaths
2026-10-05 21:56:40 +05:30
Nikhil Soni
e482ad78b4 refactor(promote): centralize domain construction and generalize routes
- target construction moves to promotetypes: a generic NewTarget plus
  per-domain constructors (NewLogsBodyTarget, NewTracesAttributesTarget)
  and a TargetFor registry keyed by (signal, context); implpromote and
  telemetrymetadata no longer hand-roll domain literals
- routes generalize to /api/v1/promote_paths/{signal}/{context}: the
  legacy logs body route (/api/v1/logs/promote_paths) is kept for
  compatibility but the domain now travels in the path, so a future logs
  attribute domain does not collide with the logs body route; supersedes
  the /api/v1/traces/promote_paths routes
- add telemetrytypes.SignalFromText for parsing the signal path variable
2026-10-05 21:56:40 +05:30
Nikhil Soni
d26cc928bf refactor(promote): template the promotion record with EvolutionEntry
Target now carries an EvolutionEntry template (signal, promoted column
name and type, field context) instead of loose signal/context/column
fields, so the store write is exactly row template + field names +
release time and the hardcoded JSON() column type moves to the domain
definitions. DBName/LocalTableName stay on Target explicitly as index
DDL config, used only by targets with index support.
2026-10-05 21:56:40 +05:30
Nikhil Soni
17b8b6504b refactor(promote): collapse module interface to target-parameterized methods
The per-domain methods were pure delegates; the promotion domain now
travels as promotetypes.Target through Module.ListPromotedPaths /
Module.PromotePaths, with the handler methods (one per route) passing
their domain's target.
2026-10-05 21:56:40 +05:30
Nikhil Soni
bcd3937c4c feat(promote): add traces attributes promotion API
Refactor the promote module into a target-parameterized core so the logs
body_v2 flow and future promotion domains share one implementation, and
add the spans attributes JSON column (attributes -> attributes_promoted)
as a second domain behind POST/GET /api/v1/traces/promote_paths.

- promotetypes.Target describes a promotion domain: signal, field
  context, db/table, base/promoted columns, path prefix rule and whether
  per-path skip indexes are supported
- index support is optional per target; traces starts promotion-only
  since the traces query builder does not consume per-path skip indexes
- metadata store GetPromotedPaths/PromotePaths take (signal, column,
  context) instead of being hardcoded to the logs body column
- fix the list response never attaching indexes to promoted entries
  (aggregated by unprefixed name but looked up by prefixed path) and
  reporting indexed+promoted paths twice
2026-10-05 21:56:40 +05:30
Srikanth Chekuri
1643df620b feat: resolve semconv families across logs and metrics (#12870)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

Phase 2 of #6143: semantic-convention families resolve on logs and
metrics, behind the `resolve_semconv_families` flag (default off), on
the storage contract of #12802.

- Registry: each member carries the scope of its own rename edges, so a
fan-out keeps one membership per target and an ambiguous name stays
literal.
- Logs and metrics families need no family code of their own. The gate
applies to every signal, and `LogicalRead` merges members through each
storage's `Read`.
- Metric-name families union the storage names in every `metric_name`
filter, and the querier reads type, temporality, and the reduced flag
across the family.
- Span-metrics labels: the metrics the processor emits, listed by name,
also read each family member with the `resource_` prefix. Requested
names are never rewritten.
- Values suggestions and related values cover every spelling of the
family.
- `deployment.environment.name` resolves on all three signals.
`db.system.name` stays off until a value-mapping reader exists.

#### Additional Information

- `pkg/semconv.Family` fields are now unexported, and `transition.go` is
removed. #12446 reads the old API and needs an update when stacked.
- A target that emits both names of a metric-name family double-counts
in `sum()` during the overlap window. Reading both names is the feature.
Pinned by a test.
- A family of metrics labels keeps the keyless contract of a single
label: no guard, no NULL group.
2026-10-01 20:28:15 +00:00
Vikrant Gupta
572345be63 feat(authz): enable FGA for users and reset password tokens (#13020)
#### Description

- Moves the users API off the legacy `AdminAccess` gate onto
`CheckResources` + `ResourceDef`s; `me` and anonymous password flows
stay `OpenAccess`.
- Invite checks `role:attach` per requested role; an empty role list
resolves to no link, so the sibling def skips the check.
- Migration `131_add_user_tuples` backfills admin `user` and
`factor-password` tuples for existing orgs.

#### Issues closed by this PR

Closes: SigNoz/keystone-pod#38
2026-10-01 11:26:18 +00:00
Vikrant Gupta
e374d03e54 revert(authz): restore gjson-based body extraction in resource middleware (#13024)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- Reverts #13014 and #13015. The resource middleware goes back to
reading body-derived resource ids with `BodyJSONPath` / `BodyJSONArray`
over the raw body, and handlers decode their own request bodies again.
- Authz should not own request decoding; that ownership stays with the
handlers.

#### Additional Information

- Contributes to: https://github.com/SigNoz/keystone-pod/issues/37
2026-09-30 13:53:21 +00:00
Swapnil Nakade
d445b6c296 chore: bumping cloud integration agent version to v0.0.15 (#13023)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Bumping the cloud integration agent's version to latest v0.0.15

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Contributes to https://github.com/SigNoz/keystone-pod/issues/101
2026-09-30 12:38:24 +00:00
Swapnil Nakade
fd8aaac300 feat: adding sync state in cloud integration (#12991)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
The agent only saw the current list of enabled regions, so it couldn’t
tell which regions had been removed. To find stacks to clean up, it
checked unrelated AWS regions, causing unnecessary calls and permission
errors. Sync state keeps track of regions sent to the agent and pending
removals until the agent acknowledges cleanup.

Please check
[comment](https://github.com/SigNoz/keystone-pod/issues/101#issuecomment-5832865898)
for approach

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Contributes to https://github.com/SigNoz/keystone-pod/issues/101

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
This PR should be merged before changes for cloud-integration repo.

<!--Please delete paragraphs that you did not use before submitting.-->
2026-09-30 11:39:46 +00:00
Vikrant Gupta
8dba9a13ea fix(authz): read every body-derived resource id from the decoded request (#13015)
#### Description

- Follows #13014. Moves the remaining body-derived resource ids (gateway
limits, zeus hosts, cloud integration check-ins, auth domains, query
range) off gjson and onto the decoded request, with the handlers reading
the same value. Part of SigNoz/keystone-pod#37.
- Removes `BodyJSONPath`, `BodyJSONArray`, and
`ExtractorContext.RequestBody`.

#### Issues closed by this PR 

- Closes: https://github.com/SigNoz/keystone-pod/issues/37
2026-09-30 09:34:32 +00:00
Vikrant Gupta
8d80f98710 fix(authz): decode the request body once in the resource middleware (#13014)
Some checks failed
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
#### Description

- The resource middleware now decodes the body once into the route's
declared `OpenAPIDef.Request` type, rejects a malformed body before any
check, and carries the decoded value on `ExtractorContext.Body`.
`BodyField` / `BodyFields` read ids off that value, and handlers read
the same value via `coretypes.BodyFromContext`.
- `handler.Handler` exposes `Request()`, and `handler.New` panics when a
route with resource defs declares a non-pointer request, since the
middleware instantiates it.
- Only `POST /api/v1/service_account_roles` is wired to the new
extractors in this PR to keep the review small. The remaining body
routes still use the gjson extractors and decode again in their
handlers.

#### Issues closed by this PR
- Contributes to: https://github.com/SigNoz/keystone-pod/issues/37
2026-09-29 21:32:25 +00:00
Swapnil Nakade
254758942e fix: using allLogs categoryGroups for azure container apps (#13013)
#### Description

- The Container Apps logs pipeline failed because the definition passed
log category names (`ContainerAppConsoleLogs`, `ContainerAppSystemLogs`)
as `categoryGroups`. Azure accepts only `allLogs` or `audit` there, so
it rejected the diagnostic setting.
- Switched to `allLogs`, matching the other Azure services. The agent
picks this up on its next config sync, so no agent release is needed.

#### Issues closed by this PR

Closes SigNoz/keystone-pod#45

#### Additional Information
- Not tested on live Azure. Microsoft's built-in policy for
`Microsoft.App/managedEnvironments` sends the same `allLogs` setting to
Event Hub.
2026-09-29 14:24:00 +00:00
Nityananda Gohain
5a1be60745 fix(ai-o11y): scope overview dashboard to gen_ai spans and move message attributes (#12967)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- Top span names and Cost by service use `builder_ai_query`, so non-AI
spans no longer show up.
- LLM cost/token panels filter on `gen_ai.request.model EXISTS`; with
variables on "All" they scanned every span.
- Default span mappers now move (not copy) vendor message keys into
`gen_ai.input.messages` / `gen_ai.output.messages`, as documented.
- Bumped versions: dashboard to 3, `gen_ai.llm` mapper to 3,
`gen_ai.agent` mapper to 2.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-23 14:56:04 +00:00
Vikrant Gupta
e2e9173986 fix(user): revoke sessions when a user is deleted (#12943)
#### Description

- `DeleteUser` never told the tokenizer about the deletion.
`SoftDeleteUser` removed the `auth_token` rows with raw SQL, so the
opaque tokenizer kept serving the deleted user's session from cache
until rotation forced a DB read, up to `rotation.interval` later.
- The tokenizer eviction now runs before the soft delete, inside one
transaction; `SoftDeleteUser` joins the caller's transaction instead of
opening its own.
- The hourly last-observed-at flush returned an error for any org with
nothing to flush because bun rejects an empty `VALUES` slice. It now
returns early.
- Adds an integration test asserting a deleted user's held token is
rejected on the next request.

#### Additional Information

Only affects the opaque tokenizer; under the JWT tokenizer
`DeleteTokensByUserID` is a no-op.
2026-09-22 12:14:51 +00:00
Nityananda Gohain
905e935658 fix: use db upsert for model pricing (#12942)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- Each pricing rule ran a SELECT then an INSERT or UPDATE. Rules are now
written with `INSERT ... ON CONFLICT DO UPDATE`, two statements per
request at most.
- Rules without `isOverride` match on `source_id` and skip rows the user
has overridden. Rules with it match on `id`.
- Dropped the `default:` bun tags. bun turns zero values into SQL
`DEFAULT` on insert, so a rule created disabled was stored as enabled.
- Added an integration suite for sync, override, hand-back and bulk
writes.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-22 11:15:20 +00:00
Naman Verma
13a57ebb9c chore: remove v1 dashboards code from backend (#12932)
<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/pulse-pod/issues/321
2026-09-22 08:46:28 +00:00
Nityananda Gohain
64fff60d7e chore: update dashboard for ai observability (#12875)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description
Adds the actual dashboard to the backend.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/4501

### Additional information
- Draft until the dashboard DTO changes merge; the query fixes below
land on top of them.
- will update the dashboard based on ai query builder
2026-09-21 17:57:58 +00:00
Nityananda Gohain
dd3b99f19c fix: update attribute mappings for ai (#12925)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

Version 2 of the `gen_ai.llm` and `gen_ai.tool` defaults, fixes only:

- Tool condition narrowed to `tool.name`, `ai.toolCall.`,
`tool_call_args`, `tool_response`; the bare `tool` substring was firing
on LLM spans.
- Dropped the `gen_ai.operation.name` mapper; `llm.request.type`
overwrote native values with non-semconv ones.
- Added `ai.response.toolCalls` as an output messages source for Vercel
tool-call turns.
- Renamed `gen_ai.response.finish_reason` to
`gen_ai.response.finish_reasons`.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Part of https://github.com/SigNoz/engineering-pod/issues/6107
2026-09-21 17:14:18 +00:00
Nityananda Gohain
ea8f95ee08 chore: enable ai 011y processors by default (#12912)
Some checks failed
build-staging / prepare (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Enable the processors by default so that metadata is populated

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
No issue
2026-09-19 13:02:43 +00:00
Nityananda Gohain
b64116d67d feat: support for default attribute mapping (#12809)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Until now every org started with no span mapper groups and we had to
create `llm`, `agent` and `tool` by hand. This PR ships them as
defaults.

- The three groups live as JSON in the binary. On startup and when a new
org is created, they get seeded. When we change a definition and bump
its version, the next release updates the org's copy in place.
- Anything SigNoz ships is marked `origin: system` and can only be
switched on or off. Users can't rename or delete these groups and
mappers, and can't take their names. Anything the user adds is theirs to
edit or remove, including new mappers in a shipped group or new sources
on a shipped mapper.
  - Upgrades keep the user's on/off choices and never touch their items.
- Condition substrings and sources now carry `enabled` and `origin`, so
a substring is an object instead of a plain string. Disabled ones are
left out of the collector config.
- - Migration 127 only adds the `origin` and `version` columns. Stored
JSON is not rewritten because these tables are empty on every instance.
- Fixes creating a group or mapper with `enabled: false` being saved as
true (the bun `default:true` tag turned false into SQL `DEFAULT`)
  
Frontend: no UI changes. Generated client regenerated; drafts carry
`enabled` and `origin` so saves from the existing screens round-trip
shipped items intact.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Closes https://github.com/SigNoz/engineering-pod/issues/5329


<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
* Frontend follow-up: toggles for sources and substrings, "Default"
badge and read-only rows for shipped items, hide rename/delete on system
groups and mappers.
* Deferred: per-group upgrade changelog ( will come back to this later)

---------

Co-authored-by: Gaurav Tewari <gauravtewari111@gmail.com>
Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-09-19 11:22:46 +00:00
Pandey
8286e787b2 fix(tracefunnel): quote step names in slow and error trace queries (#12886)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- `#12593` moved the n-step trace-funnel query builders onto
`clickhousesql.StringLiteral`, but the two-step `slow-traces` and
`error-traces` builders still interpolated `service_name`/`span_name`
into the SQL string literal raw.
- Route those four values through the same helper, so every funnel query
builder quotes step names consistently.

#### Additional Information

- No behaviour change for ordinary names; the
`slow-traces`/`error-traces` funnel queries now handle names containing
a quote the same way the rest of the module already does.
2026-09-17 07:01:21 +00:00
Swapnil Nakade
e1acfc94ba feat: adding dashboard delete unsafe v2 method (#12876)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- Adding Dashboard's DeleteUnsafeV2 method
- Using this method in cloud integration dashboard clean up as well

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
https://github.com/SigNoz/platform-pod/issues/3153


<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
https://signoz-team.slack.com/archives/C08UBUBV68Z/p1789459747045369
2026-09-16 10:20:22 +00:00
Srikanth Chekuri
9c886be120 chore(querybuilder): compile every signal through one storage contract (#12802)
The semantic convention family as first call citizen revealed that the
current state of the query builder needs a bit refactoring for long term
maintenance.

The `FieldMapper` and `ConditionBuilder` are now one abstraction
`Storage`.

A storage now answers
- what the compiler cannot know i.e one read per field key (the bare
SQL, the membership present or absent, what an absent row reads, and
whether the read keeps its type or filters only).
- the fallback for a key metadata does not report
- its traits
- and one Condition compilation part.

And we introduce a new type to use in the system, `Resolved`

```
// Resolved is what resolution produces for one key: its meanings, and how
// they came to be. It is the only thing the compilers receive. Compile it
// with the operator and value it was resolved with.
type Resolved struct {
	Key    *telemetrytypes.TelemetryFieldKey
	Fields []*telemetrytypes.LogicalField
	// FromFallback: the fields came from the storage's fallback, not from
	// metadata matches.
	FromFallback bool
	// Ambiguous: the matches held several interpretations.
	Ambiguous bool
	// Skipped: the storage contributes nothing for this key.
	Skipped  bool
	Warnings []string
}
```

The prepared SQL has no changes, where it changed, it specifically made
the expression better by removing the redundant part.

- The prepared SQL remains identical with this refactoring
- No changes to integration tests


Assisted-by: Claude Fable 5.1
2026-09-15 11:36:06 +00:00
Nityananda Gohain
c41899f2eb feat: support ai trace alerts (#12783)
#### Description
- Add the `AI_TRACES_BASED_ALERT` alert type so alerts can be built with
the AI explorer's `builder_ai_query`.
- Treat AI trace queries like trace queries for new-series filtering and
related links.
- Related links for AI alerts open the AI observability explorer and tag
the shared query as `builder_ai_query`, so `trace.*` aggregate fields in
the filter resolve. Existing logs and traces links are unchanged.
- Rule history timeline and top-contributor responses gain
`relatedAITracesLink`; AI alerts populate it and leave
`relatedTracesLink` empty so the frontend can route without
  checking the alert type.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Closes https://github.com/SigNoz/engineering-pod/issues/6021


#### Additional Information
Notifications keep the existing `related_traces` annotation, now
carrying the AI explorer URL, so every channel and `$trace.url` template
keeps working.
2026-09-11 09:27:49 +00:00
Srikanth Chekuri
8e00c04056 refactor(qb): quote field names with the ClickHouse quoting helpers (#12593)
#### Description

- Every user-controlled field name that reaches generated SQL goes
through the new `pkg/clickhousesql` package (`Identifier`,
`StringLiteral`, `Literal`, `LikePattern`): map reads and `mapContains`,
JSON sub-column paths and the JSON body access plan, labels, fingerprint
labels, materialized column names, select aliases, group-by and order-by
references, the legacy string-body JSONPath, and the raw SQL in the
trace funnel, trace detail and infra monitoring modules. Filter
expressions built from request or telemetry values use
`querybuilder.FilterStringLiteral`. The same package now also renders
dashboard variable values in the querier, LIKE patterns in the metadata
store and label lists in the PromQL transpiler, which each had their own
escaping.
- A `$` followed by a digit, `{` or `?` is written as `\x24`, which
ClickHouse decodes in identifiers and literals. Those are the forms the
tools react to: go-sqlbuilder resolves `$0` in a compiled fragment to
its own WHERE clause and recurses until the stack overflows, and
clickhouse-go rejects a query mixing `$<digits>` with `?` arguments. Any
other `$` stays literal, so materialized column names keep their `$$`
and render exactly as before; a key like `http.2xx` becomes ``
`attribute_string_http$\x242xx` `` instead of failing in the driver.
- Compiled sqlbuilder fragments (Select, GroupBy, OrderBy, raw Where
text) are wrapped with `sqlbuilder.Escape`; the metrics builder escapes
its compiled time-series subquery, which is compiled a second time when
joined.
- The raw statement validator (`ErrIfStatementIsNotValid`,
`LogIfStatementIsNotValid`) moves from
`pkg/querybuilder/clickhouse_sql.go` to
`pkg/clickhousesql/statement.go`. Its `Code*` identifiers drop the
`ClickHouseSQL` prefix; the code strings are unchanged.
- Unit tests round-trip the helpers over hostile names and drive them
through the modules' raw SQL;
`tests/integration/tests/queriercommon/08_field_name_quoting.py` and
`querier_json_body/07_field_name_quoting.py` query such names through
the logs, traces and metrics builders against a real ClickHouse.

#### Additional Information

- `docs/contributing/go/clickhousesql.md` documents the quoting
functions, where `sqlbuilder.Escape` belongs, the `$` rule and the
statement validator; `.claude/rules/go-contrib.md` points at it.
- `pkg/clickhousesql` is a leaf package so `telemetrytypes` (JSON access
plan) and `querybuilder` share one implementation without a cycle.
- For names without special characters the generated SQL is byte
identical.
- Not covered here: the legacy v3/v4 query_range builders and the
`pkg/query-service/utils` quoting helpers (`QuoteEscapedString`,
`QuoteEscapedStringForContains`, `ClickHouseFormattedValue`,
`AddBackTickToFormatTag`), the collector's `JSONSubColumnIndexExpr`, and
aggregation arguments naming a key that contains a backtick (rejected by
the SQL parser, a 500 as before).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-09-10 13:19:23 +00:00
Nikhil Mantri
ad68a1b991 refactor: extract shared list filter SQL compiler from the dashboards visitor (#12806)
#### Description

- Extracts the dashboards list filter visitor's generic core into a
shared package, `pkg/parser/filterquery/sqlcompiler`: boolean
composition, operator dispatch, value extraction, the LIKE family
builders and the `Compiled` output type, behind a small per-feature
`FieldResolver`.
- Dashboards keeps only a key-policy resolver (`listfilter_resolver.go`)
and an error-code wrap; the old 631-line visitor is deleted. Emitted SQL
is unchanged, pinned by the existing exact-SQL unit suite.
- Scope note in the package doc: this compiler serves list pages over
the relational sql DB. Telemetry queries stay on querybuilder's
ClickHouse visitor.
- Also fixes a pre-existing bug shared with the copy: a `LIKE`/`ILIKE`
pattern ending in an unescaped backslash never matches on sqlite and
aborts the query on Postgres (a data-dependent 500); it is now rejected
as a 400, with a unit case.
- Split out of #12780 per review so the commoning-out merges first; the
alert rules list API consumes this next (#12780 will be rebased onto
this branch). Part of SigNoz/pulse-pod#226.
2026-09-10 13:09:05 +00:00
Srikanth Chekuri
d5af6f6d6b feat(telemetrymetadata): fields keys and values for the quick filters migration (#12794)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- `fields/values` returns `boolValues` for bool fields: static span
fields such as `has_error`, stored bool attributes (decided by a
row-existence check, so they take no limit slots), and keys the caller
declares bool. `isRoot` and `isEntryPoint` return `true` only, matching
the condition builder.
- `fields/values` honors `startUnixMilli` for logs and traces against
the tag tables. The predicate is `toDate(unix_milli / 1000) >=
toDate(start)`, the partition expression, because the tag tables
deduplicate a value per day and keep whichever hour was inserted last.
For traces this means "span started on or after the start's day", for
logs "ingested on or after the start's day". `endUnixMilli` is ignored.
- `fields/keys` honors `fieldContext` and `fieldDataType` for the static
intrinsic and calculated keys, matches them case-insensitively, types
`isRoot`/`isEntryPoint` as bool, gives meter keys a data type, filters
the traces keys table by tag type for the scope context, and no longer
returns early for the log and scope contexts (logs and audit).
- The span context keeps returning attribute keys, since queries spell
span attributes as `span.<name>`.
- Default quick filters name the span and log fields the way the fields
API serves them (`has_error` as a span bool, `name`, `duration_nano`,
`response_status_code`, `http_host`, `http_url`, `trace_id` as span
fields, `severity_text` as a log field). Migration 126 rewrites the
static fields of every stored quick filter the same way, whatever
context the legacy seeds gave them. Deprecated names such as `hasError`
are not served.
- The API-backed checkbox hook merges `boolValues` into its options; the
`has_error` title is kept.
- The fields/values handler no longer drops `boolValues` from the
response.


🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-09-08 22:11:27 +00:00
Nityananda Gohain
a75442f31e fix: add quick filters v2 api to support TelemetryFieldKey (#12698)
Some checks failed
build-staging / staging (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description
The old API didn't support telemetryFieldKey, so adding a new v2 API to
support it.

This PR
* Migrates old data to the new one.
* Existing API's now internally stores it in the new struct so that they
don't break the UI.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Closes https://github.com/SigNoz/engineering-pod/issues/5947

## Additional details
* the old api is safe with new field as it is just a subset of it.
2026-09-03 07:49:31 +00:00
Nityananda Gohain
52588c4582 fix: support for related values in ai field values (#12716)
#### Description

Adds support for related values in ai observability field values.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/engineering-pod/issues/5975
2026-09-03 07:12:22 +00:00
Nityananda Gohain
7eb610287e feat: system dashboards (#12620)
#### Description
Adding support for system dashboards.
* as of now updates are only through new versions in the file.
* user cannot update the dashboard
* For now kept the dashboard content empty and will raise it separately.


Closes https://github.com/SigNoz/engineering-pod/issues/4501
2026-09-01 10:17:10 +00:00
Swapnil Nakade
aca35a1309 chore: bumping cloud integration agent version to v0.0.14 (#12737)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Bumping cloud integration agent version from v0.0.13 to v0.0.14

#### Contributes to
https://github.com/SigNoz/platform-pod/issues/3038
2026-09-01 09:26:50 +00:00
Vikrant Gupta
dca6aa497d chore(serviceaccount): remove deprecated nested role endpoints (#12591)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- Removes the deprecated `POST /api/v1/service_accounts/{id}/roles` and
`DELETE /api/v1/service_accounts/{id}/roles/{rid}` routes, their HTTP
handlers, and the `DeprecatedPostableServiceAccountRole` type, now that
all consumers use `/api/v1/service_account_roles`.
- Keeps the `GET /api/v1/service_accounts/{id}/roles` listing endpoint.
- Regenerates `docs/api/openapi.yml` and the frontend client.

#### Issues closed by this PR

Closes SigNoz/platform-pod#2919

#### Additional Information

- Final step of the migration; the frontend (#12589) and
integration-test (#12590) consumer moves are already merged.
2026-08-19 20:03:16 +00:00
Swapnil Nakade
0f3b3dfb07 refactor: adding FunctionName variable in Lambda dashboard (#12599)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
AWS Cloud Integration's Lambda dashboard was missing FunctionName
variable, this PR adds that variable for better UX.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Closes https://github.com/SigNoz/platform-pod/issues/2988

<!--If applicable, include screenshots or screen recordings that clearly
show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings
<img width="1503" height="815" alt="image"
src="https://github.com/user-attachments/assets/56203f8d-1f06-4c39-b3c1-c1dabf028045"
/>

---------

Co-authored-by: Vikrant Gupta <vikrant@signoz.io>
2026-08-18 19:39:25 +00:00
Srikanth Chekuri
cfc7a04bc8 feat: resolve semantic convention names in trace queries (#12442)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- A referenced name in a trace query now resolves to a `LogicalField`
(#12499): one field, addressed by the requested spelling, backed by its
physical member keys. A semantic-convention family
(`deployment.environment.name` / `deployment.environment`) merges into
one expression with current-wins precedence; the response keeps the
requested spelling.
- `FieldMapper` gets one new method, `ExistsFor` (the per-key presence
primitive). `LogicalValueExpr` and `LogicalExistsExpr` build all family
SQL in one place from `FieldFor` and `ExistsFor`; no signal implements
family logic.
- Statement builders prefetch sibling spellings; the metadata store
stays family-blind and autocomplete stays literal. Traces and the
resource filter compile per logical field; logs, metrics, and the other
signals keep their SQL unchanged.
- The `resolve_semconv_families` feature flag (default: disabled) gates
all family behavior. With the flag off, the generated SQL is the same as
main; tests pin this. Part of #6143.

#### Additional Information

- Stack: #12441 (merged) → **#12442** → #12443 → #12444 → #12445 →
#12446 → #12447. This layer bases on main.
- Rollback: turn the flag off; stored telemetry is untouched.
2026-08-17 21:59:26 +00:00
Vikrant Gupta
81d1716c7e fix(user): clear role assignments when a user is deleted (#12554)
#### Description

- Soft-deleting a user revoked the FGA grant but left the `user_role`
rows behind. The role-delete guard (`OnBeforeRoleDelete` →
`GetUsersByOrgIDAndRoleID`) still counted the deleted user, so the role
could never be deleted — and detaching the assignment was also blocked
because the user is deleted. That left the role permanently undeletable.
- `SoftDeleteUser` now deletes the user's `user_role` rows in the same
transaction that already clears its password, tokens, and preferences,
so the SQL side matches the FGA revoke.
- Migration `delete_orphan_user_roles` clears the orphan `user_role`
rows left by users deleted before this change.

#### Additional Information

- Regression test in `role/02_crud.py`: assign a custom role to a user,
delete the user, then delete the role → now `204` (was the deadlock).
2026-08-17 19:11:43 +00:00
Nityananda Gohain
edb63ae7be feat[ai-011y]: fields API for ai query builder (#12140)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
## Pull Request

---

### 📄 Summary
- Add a `type` param to `/api/v1/fields/keys`; for type=builder_ai_query
(flag-gated) the metadata store returns the per-trace aggregate columns
(llm_call_count, input_tokens, …) as
trace-context keys — they're computed at query time, never ingested, so
the attribute scan can't serve them.
- Split `TraceColumn.Orderable` into `Orderable + Filterable`: ORDER BY
uses orderable, the trace-level filter validates against filterable, and
the API only returns keys that are both. `last_activity_time` is
order-only and now rejected in filters with a targeted error.**
- UI note: last_activity_time should be added to client-side list (it's
the default sort).

#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/5714

---

### ✅ Change Type
_Select all that apply_

- [x] ✨ Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🧪 Testing Strategy
> How was this change validated?

- Tests added/updated: ✅ 
- Manual verification: ✅ 
- Edge cases covered: ✅ 

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius: None
- Potential regressions:
- Rollback plan:
2026-08-17 07:57:57 +00:00
Pandey
5b3b2865d1 fix(authtypes): restructure auth domain payload into a kind/spec envelope (#12472)
#### Description

- Moves the endpoints to `/api/v2/auth_domains` and removes the
`/api/v1/domains` routes — the request/response shapes changed, so they
live behind new paths instead of breaking v1 in place.
- Restructures the auth domain payload: `config` is now a `{kind, spec}`
discriminated envelope (same pattern as `RuleThresholdData` /
`EvaluationEnvelope`), replacing the old `ssoType` discriminator with
`samlConfig` / `googleAuthConfig` / `oidcConfig` sibling fields;
`ssoEnabled` and `roleMapping` move to the root as `enabled` and
`roleMapping`.
- Renames the provider kind `google_auth` → `google`, and the SAML keys
to metadata-consistent ones: `samlEntity` → `entityId`, `samlIdp` →
`location`, `samlCert` → `certificate`.
- Migrates the persisted documents too: a new sqlmigration rewrites
`auth_domain.data` into `{enabled, config: {kind, spec}, roleMapping}`,
so all legacy-shape code (storable twins, `google_auth` translation,
per-kind conversion switches) is deleted; the remaining per-kind wiring
lives in a single variant registry that `UnmarshalJSON`,
`JSONSchemaOneOf` and the discriminator mapping derive from.
- `AuthDomain` exposes the domain shape (`Enabled()`, `Kind()`,
`Config()`, `RoleMapping()`, typed spec accessors) instead of the
persisted document; `config` presence is enforced explicitly on
Postable/Updatable (the old PUT path never enforced it and could poison
a row).
- Secret fields (`clientSecret`, `serviceAccountJson`) are `format:
password` in the schema, and `GoogleConfig` loses the unused
`redirectURI` (the migration strips it from persisted documents).
- Frontend: regenerated client is a clean discriminated union; both
directions of the envelope↔form translation live in
`CreateEdit.utils.ts` with an explicit kind→provider mapping (no
cross-enum casts).
- The generated OpenAPI spec carries a real `discriminator`; the
kind/spec envelope pattern itself is documented generically in #12494,
and this PR only keeps the auth domain worked example in `types.md` in
step with the refactored types.
- Updates the google authn integration tests (#12486) to the new API,
and adds parametrized POST→GET roundtrip cases pinning the response
contract per kind (server-side defaulting, role-name normalization, null
maps) plus enforcement-toggle update coverage.

#### Issues closed by this PR

Closes SigNoz/platform-pod#2268

#### Additional Information

- Breaking change: `/api/v1/domains` is gone; the resource is now
`/api/v2/auth_domains` with the new shape. Login and SSO callback flows
are behaviorally unchanged, and existing rows are migrated in place at
startup.
- The `AuthNProvider` rename also surfaces in `/api/v2/sessions/context`
responses (`provider: "google"`) — the login page only consumes the
callback `url` — and in the reported stats key, which changes from
`authdomain.google_auth.count` to `authdomain.google.count`.
- Verified: `make go-test`, Go lint, frontend jest suites for
AuthDomain, `pnpm build`, `pnpm tsgo --noEmit`, and the full
`callbackauthn` domain suites (17 tests: roundtrip pins, the enforcement
toggle, and the google E2E flows) against a container rebuilt from this
branch — including a live run of the data migration over legacy-format
rows.
2026-08-13 16:39:27 +00:00