Compare commits

..

7 Commits

Author SHA1 Message Date
Nityananda Gohain
a75442f31e fix: add quick filters v2 api to support TelemetryFieldKey (#12698)
Some checks are pending
build-staging / staging (push) Blocked by required conditions
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
cacheci / tests (push) Waiting to run
Release Drafter / update_release_draft (push) Waiting to run
#### Description
The old API didn't support telemetryFieldKey, so adding a new v2 API to
support it.

This PR
* Migrates old data to the new one.
* Existing API's now internally stores it in the new struct so that they
don't break the UI.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Closes https://github.com/SigNoz/engineering-pod/issues/5947

## Additional details
* the old api is safe with new field as it is just a subset of it.
2026-09-03 07:49:31 +00:00
Nityananda Gohain
52588c4582 fix: support for related values in ai field values (#12716)
#### Description

Adds support for related values in ai observability field values.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/engineering-pod/issues/5975
2026-09-03 07:12:22 +00:00
Nityananda Gohain
e485f50221 feat: support ai trace aggregate filtering in ai span list (#12122)
## Pull Request

---

### 📄 Summary
* Span list (raw) queries can now leverage trace-level `trace.` filter
conditions leading to the trace-level component being qualified with
`__trace_scope`.
* An error is now raised if the span list is ordered by the trace level
key.


The following constraint is known: in case of ordering by `timestamp`,
the querier processes the span list by time bucket thus performing
trace-level aggregates calculation per bucket not within a time window.
The records are kept separately.


#### Issues closed by this PR

Fixes https://github.com/SigNoz/engineering-pod/issues/5976

---

###  Change Type
_Select all that apply_

- [ ]  Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🧪 Testing Strategy
> How was this change validated?

- Tests added/updated: 
- Manual verification: 
- Edge cases covered:

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius: None
- Potential regressions:
- Rollback plan:
2026-09-03 07:10:24 +00:00
Vikrant Gupta
b3b547f34a feat(gateway): add first-class ingestion limit APIs (#12625)
#### Description

- Adds first-class ingestion limit APIs under
`/api/v2/gateway/ingestion_limits`: create (`keyId` in body), get,
update, and delete by `{limitId}`. Get proxies the new upstream `GET
/v1/workspaces/me/limits/{limitID}`.
- Adds key read APIs: `GET /api/v2/gateway/ingestion_keys/{keyId}` (key
by id — upstream does not embed limits here) and `GET
/api/v2/gateway/ingestion_keys/{keyId}/limits` (limits for a key, with
current-period usage metrics).
- Marks the existing limit routes (`POST
/ingestion_keys/{keyId}/limits`, `PATCH/DELETE
/ingestion_keys/limits/{limitId}`) as deprecated; they keep working
unchanged.
- Renames the old create body to `DeprecatedPostableIngestionKeyLimit`;
`PostableIngestionKeyLimit` is now the first-class body carrying
`keyId`. Handlers decode via `binding.JSON` and the create response is
`types.Identifiable`.

Part of SigNoz/platform-pod#2651.

#### Additional Information

- OpenAPI spec and the generated frontend client are regenerated; the UI
stays on the deprecated routes for now.
- Requires the upstream get-by-id endpoints from
SigNoz/opentelemetry-gateway#96 (merged and deployed).
2026-09-03 07:07:10 +00:00
Naman Verma
be9a045455 chore: add internal name to data layer of notification channels (#12754)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Convert the existing `name` to store an immutable DNS1123 internal name
of a notification_channel, and add a display name column where the data
from the existing `name` column will go.

This is just a database level change. The internal name is not being
used by any consumer, be it the API or rules or route policies. All that
will come in subsequent PRs

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Part of https://github.com/SigNoz/pulse-pod/issues/296

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information

Eventually references (rules, routing policies) migrate onto the
internal name, freeing the display name to become a user-editable. But
that will happen post rules migration so that all rules are on v2.


<!--Please delete paragraphs that you did not use before submitting.-->
2026-09-03 06:52:58 +00:00
Nikhil Mantri
297d3dd44b feat(alert-channel-integrations): incidentio frontend (#12645)
## Description

Frontend for the **incident.io** alert channel (backend in #12644 — this
PR is stacked on it).

- Adds **incident.io** to the channel-type dropdown with a settings
form: alert source **URL** + **token** (both required), and
**title/description** template fields prefilled with the backend
defaults — same UX as Jira/JSM.
- A tip above the form links to the setup docs (create an HTTP alert
source in incident.io, copy URL + token).
- Client-side validation mirrors the backend for a nicer error
experience: both fields required, URL must be an alert events URL
(`…/v2/alert_events/http/<source_config_id>`).
- `send_resolved` is seeded **on** so incident.io alerts resolve with
the rule (backend can't default it — same reasoning as JSM).
- **Additional metadata** section: key-value rows merged into every
alert's metadata on top of the alert's labels (channel wins on clash);
values may use templates.
- Create, edit and test-channel flows all wired; editing prefills from
the stored `incidentio_configs`.

Notes for the reviewer:

- Follows the JSM Ops form/handler pattern file-for-file; no new
patterns introduced.
- Tests: 4 create-flow cases (fields render, required-field error, URL
validation error, payload shape with defaults) + 1 edit-flow payload
case.

## Issues closed by this PR

Closes SigNoz/pulse-pod#173

---------

Co-authored-by: Naman Verma <naman.verma@signoz.io>
2026-09-03 06:20:47 +00:00
Nikhil Mantri
e84a61d43f feat(alert-channel-integrations): incidentio channel integration (#12644)
## Description

Adds **incident.io** as a native alert notification channel, using
incident.io's HTTP alert source (Alert Events V2 API)

- A channel is configured with the alert source's **URL + token**; title
and description templates are prefilled with the same defaults as
Jira/JSM.
- Alerts fire and auto-resolve in incident.io; the description is
markdown (incident.io renders it natively) and carries the usual deep
links — **View in SigNoz, related logs, related traces**.
- All rule labels (severity, team, custom labels) are sent as
**metadata**, so users can map them to incident.io attributes and
route/escalate on them.

Notes and decisions for the reviewer (full details in the [discussion
ticket and doc](https://github.com/SigNoz/pulse-pod/issues/171)):

- **Dedup:** one incident.io alert per notification group, keyed by the
group key hash (same identity Jira uses). A resolve targets the same
key; re-fires after resolve correctly open a fresh alert — no key
rotation needed.
- **Repeat notifications are no-ops on incident.io** (it drops duplicate
firing events) — unlike Jira, we cannot append updated values to an open
alert; operators click through to SigNoz for current values.
- **Limits:** description capped client-side under incident.io's
documented 512 KB payload limit; retries only on 429/5xx (documented
limit: 120 events/min per source).
- **Channel-level metadata:** optional key-value pairs on the channel
config, merged into every event's metadata on top of the alert's labels
(channel wins on key clash — Opsgenie precedent). Values are
template-expanded; a value that fails to expand is sent raw with a
warning logged, so delivery never breaks on a bad template.
- Upstream alertmanager ships its own basic incident.io notifier — our
config **shadows it** so the SigNoz notifier (templates, dedup,
metadata) handles delivery.
- Frontend (channel form) follows in a stacked PR.

## Issues closed by this PR

Closes SigNoz/pulse-pod#172

---------

Co-authored-by: Naman Verma <naman.verma@signoz.io>
2026-09-03 05:32:51 +00:00
97 changed files with 5725 additions and 4467 deletions

View File

@@ -50,6 +50,7 @@ jobs:
- logspipelines
- passwordauthn
- preference
- quickfilter
- querierlogs
- queriertraces
- queriermetrics

View File

@@ -109,6 +109,25 @@ components:
webhook_url:
$ref: '#/components/schemas/ConfigSecretURL'
type: object
AlertmanagertypesIncidentIOReceiverConfig:
properties:
description:
type: string
http_config:
$ref: '#/components/schemas/ConfigHTTPClientConfig'
metadata:
additionalProperties:
type: string
type: object
send_resolved:
type: boolean
title:
type: string
token:
type: string
url:
type: string
type: object
AlertmanagertypesJSMOpsReceiverConfig:
properties:
api_key:
@@ -217,12 +236,12 @@ components:
- jira_configs
- required:
- jsmops_configs
- required:
- incidentio_configs
- required:
- discord_configs
- required:
- email_configs
- required:
- incidentio_configs
- required:
- pagerduty_configs
- required:
@@ -266,7 +285,7 @@ components:
type: array
incidentio_configs:
items:
$ref: '#/components/schemas/ConfigIncidentioConfig'
$ref: '#/components/schemas/AlertmanagertypesIncidentIOReceiverConfig'
type: array
jira_configs:
items:
@@ -397,7 +416,7 @@ components:
type: array
incidentio_configs:
items:
$ref: '#/components/schemas/ConfigIncidentioConfig'
$ref: '#/components/schemas/AlertmanagertypesIncidentIOReceiverConfig'
type: array
jira_configs:
items:
@@ -3045,58 +3064,6 @@ components:
- tags
- spec
type: object
DashboardtypesHeatmapColorMode:
enum:
- scheme
- opacity
type: string
DashboardtypesHeatmapColorScale:
enum:
- log
- sqrt
- linear
type: string
DashboardtypesHeatmapColors:
properties:
fill:
type: string
max:
nullable: true
type: number
min:
nullable: true
type: number
mode:
$ref: '#/components/schemas/DashboardtypesHeatmapColorMode'
reverse:
type: boolean
scale:
$ref: '#/components/schemas/DashboardtypesHeatmapColorScale'
scheme:
type: string
steps:
type: integer
type: object
DashboardtypesHeatmapPanelSpec:
properties:
colors:
$ref: '#/components/schemas/DashboardtypesHeatmapColors'
formatting:
$ref: '#/components/schemas/DashboardtypesPanelFormatting'
legend:
$ref: '#/components/schemas/DashboardtypesLegend'
showOverflow:
type: boolean
visualization:
$ref: '#/components/schemas/DashboardtypesHeatmapVisualization'
type: object
DashboardtypesHeatmapVisualization:
properties:
showVisualMap:
type: boolean
timePreference:
$ref: '#/components/schemas/DashboardtypesTimePreference'
type: object
DashboardtypesHistogramBuckets:
properties:
bucketCount:
@@ -3449,7 +3416,6 @@ components:
discriminator:
mapping:
signoz/BarChartPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBarChartPanelSpec'
signoz/HeatmapPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec'
signoz/HistogramPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec'
signoz/ListPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec'
signoz/NumberPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesNumberPanelSpec'
@@ -3465,7 +3431,6 @@ components:
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTablePanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec'
type: object
DashboardtypesPanelPluginKind:
enum:
@@ -3476,7 +3441,6 @@ components:
- signoz/TablePanel
- signoz/HistogramPanel
- signoz/ListPanel
- signoz/HeatmapPanel
type: string
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBarChartPanelSpec:
properties:
@@ -3490,18 +3454,6 @@ components:
- kind
- spec
type: object
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec:
properties:
kind:
enum:
- signoz/HeatmapPanel
type: string
spec:
$ref: '#/components/schemas/DashboardtypesHeatmapPanelSpec'
required:
- kind
- spec
type: object
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec:
properties:
kind:
@@ -4138,6 +4090,18 @@ components:
nullable: true
type: object
type: object
GatewaytypesDeprecatedPostableIngestionKeyLimit:
properties:
config:
$ref: '#/components/schemas/GatewaytypesLimitConfig'
signal:
type: string
tags:
items:
type: string
nullable: true
type: array
type: object
GatewaytypesGettableCreatedIngestionKey:
properties:
id:
@@ -4281,6 +4245,8 @@ components:
properties:
config:
$ref: '#/components/schemas/GatewaytypesLimitConfig'
keyId:
type: string
signal:
type: string
tags:
@@ -4288,6 +4254,8 @@ components:
type: string
nullable: true
type: array
required:
- keyId
type: object
GatewaytypesUpdatableIngestionKeyLimit:
properties:
@@ -7017,7 +6985,10 @@ components:
$ref: '#/components/schemas/Querybuildertypesv5TimeSeries'
type: array
meta:
$ref: '#/components/schemas/Querybuildertypesv5AggregationMeta'
properties:
unit:
type: string
type: object
predictedSeries:
items:
$ref: '#/components/schemas/Querybuildertypesv5TimeSeries'
@@ -7032,51 +7003,12 @@ components:
$ref: '#/components/schemas/Querybuildertypesv5TimeSeries'
type: array
type: object
Querybuildertypesv5AggregationMeta:
Querybuildertypesv5Bucket:
properties:
buckets:
items:
format: double
type: number
type: array
unit:
type: string
step:
format: double
type: number
type: object
Querybuildertypesv5BucketOptions:
discriminator:
mapping:
linear: '#/components/schemas/Querybuildertypesv5BucketOptionsLinear'
log: '#/components/schemas/Querybuildertypesv5BucketOptionsLog'
propertyName: kind
oneOf:
- $ref: '#/components/schemas/Querybuildertypesv5BucketOptionsLinear'
- $ref: '#/components/schemas/Querybuildertypesv5BucketOptionsLog'
type: object
Querybuildertypesv5BucketOptionsLinear:
properties:
kind:
$ref: '#/components/schemas/Querybuildertypesv5BucketsKind'
spec:
$ref: '#/components/schemas/Querybuildertypesv5LinearBucketsSpec'
required:
- kind
- spec
type: object
Querybuildertypesv5BucketOptionsLog:
properties:
kind:
$ref: '#/components/schemas/Querybuildertypesv5BucketsKind'
spec:
$ref: '#/components/schemas/Querybuildertypesv5LogBucketsSpec'
required:
- kind
- spec
type: object
Querybuildertypesv5BucketsKind:
enum:
- linear
- log
type: string
Querybuildertypesv5BuilderQuerySpec:
discriminator:
mapping:
@@ -7257,16 +7189,6 @@ components:
value:
type: string
type: object
Querybuildertypesv5LinearBucketsSpec:
properties:
maxValue:
format: double
type: number
numBuckets:
type: integer
required:
- maxValue
type: object
Querybuildertypesv5LogAggregation:
properties:
alias:
@@ -7274,12 +7196,6 @@ components:
expression:
type: string
type: object
Querybuildertypesv5LogBucketsSpec:
properties:
scale:
nullable: true
type: integer
type: object
Querybuildertypesv5MetricAggregation:
properties:
comparisonSpaceAggregationParam:
@@ -7738,8 +7654,6 @@ components:
queries (traces, logs, metrics), formulas, joins, trace operators, PromQL,
and ClickHouse SQL queries.
properties:
bucketOptions:
$ref: '#/components/schemas/Querybuildertypesv5BucketOptions'
compositeQuery:
$ref: '#/components/schemas/Querybuildertypesv5CompositeQuery'
end:
@@ -7839,7 +7753,6 @@ components:
- raw
- raw_stream
- trace
- heatmap
type: string
Querybuildertypesv5ScalarData:
properties:
@@ -7914,6 +7827,8 @@ components:
type: object
Querybuildertypesv5TimeSeriesValue:
properties:
bucket:
$ref: '#/components/schemas/Querybuildertypesv5Bucket'
partial:
type: boolean
timestamp:
@@ -7958,6 +7873,48 @@ components:
- custom
- text
type: string
QuickfiltertypesSource:
enum:
- traces
- logs
- api_monitoring
- exceptions
- meter
- ai_observability
type: string
QuickfiltertypesSourceFilters:
properties:
createdAt:
format: date-time
type: string
filters:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
type: array
id:
type: string
orgId:
type: string
source:
$ref: '#/components/schemas/QuickfiltertypesSource'
updatedAt:
format: date-time
type: string
required:
- id
- orgId
- source
- filters
type: object
QuickfiltertypesUpdatableQuickFilters:
properties:
filters:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
type: array
required:
- filters
type: object
RenderErrorResponse:
properties:
error:
@@ -9798,6 +9755,10 @@ paths:
name: name
schema:
type: string
- in: query
name: existingQuery
schema:
type: string
responses:
"200":
content:
@@ -16262,6 +16223,63 @@ paths:
summary: Delete ingestion key for workspace
tags:
- gateway
get:
deprecated: false
description: This endpoint returns an ingestion key for the workspace
operationId: GetIngestionKey
parameters:
- in: path
name: keyId
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/GatewaytypesIngestionKey'
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Get ingestion key for workspace
tags:
- gateway
patch:
deprecated: false
description: This endpoint updates an ingestion key for the workspace
@@ -16307,8 +16325,68 @@ paths:
tags:
- gateway
/api/v2/gateway/ingestion_keys/{keyId}/limits:
post:
get:
deprecated: false
description: This endpoint returns the ingestion limits for an ingestion key
operationId: GetIngestionKeyLimits
parameters:
- in: path
name: keyId
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/GatewaytypesLimit'
nullable: true
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Get limits for the ingestion key
tags:
- gateway
post:
deprecated: true
description: This endpoint creates an ingestion key limit
operationId: CreateIngestionKeyLimit
parameters:
@@ -16321,7 +16399,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/GatewaytypesPostableIngestionKeyLimit'
$ref: '#/components/schemas/GatewaytypesDeprecatedPostableIngestionKeyLimit'
responses:
"201":
content:
@@ -16365,7 +16443,7 @@ paths:
- gateway
/api/v2/gateway/ingestion_keys/limits/{limitId}:
delete:
deprecated: false
deprecated: true
description: This endpoint deletes an ingestion key limit
operationId: DeleteIngestionKeyLimit
parameters:
@@ -16404,7 +16482,7 @@ paths:
tags:
- gateway
patch:
deprecated: false
deprecated: true
description: This endpoint updates an ingestion key limit
operationId: UpdateIngestionKeyLimit
parameters:
@@ -16507,6 +16585,205 @@ paths:
summary: Search ingestion keys for workspace
tags:
- gateway
/api/v2/gateway/ingestion_limits:
post:
deprecated: false
description: This endpoint creates an ingestion limit for the ingestion key
referenced by keyId
operationId: CreateIngestionLimit
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GatewaytypesPostableIngestionKeyLimit'
responses:
"201":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/TypesIdentifiable'
status:
type: string
required:
- status
- data
type: object
description: Created
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Create ingestion limit
tags:
- gateway
/api/v2/gateway/ingestion_limits/{limitId}:
delete:
deprecated: false
description: This endpoint deletes an ingestion limit
operationId: DeleteIngestionLimit
parameters:
- in: path
name: limitId
required: true
schema:
type: string
responses:
"204":
description: No Content
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Delete ingestion limit
tags:
- gateway
get:
deprecated: false
description: This endpoint returns an ingestion limit
operationId: GetIngestionLimit
parameters:
- in: path
name: limitId
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/GatewaytypesLimit'
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Get ingestion limit
tags:
- gateway
patch:
deprecated: false
description: This endpoint updates an ingestion limit
operationId: UpdateIngestionLimit
parameters:
- in: path
name: limitId
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GatewaytypesUpdatableIngestionKeyLimit'
responses:
"204":
description: No Content
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Update ingestion limit
tags:
- gateway
/api/v2/healthz:
get:
operationId: Healthz
@@ -18961,6 +19238,170 @@ paths:
summary: Get query range result (v2)
tags:
- dashboard
/api/v2/quick_filters:
get:
deprecated: false
description: Returns the org's quick filters for every source, each filter as
a telemetry field key.
operationId: ListQuickFilters
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/QuickfiltertypesSourceFilters'
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- quick-filter:list
- tokenizer:
- quick-filter:list
summary: List quick filters
tags:
- quick_filter
/api/v2/quick_filters/{source}:
get:
deprecated: false
description: Returns the org's quick filters for one source, each filter as
a telemetry field key.
operationId: GetQuickFilters
parameters:
- in: path
name: source
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/QuickfiltertypesSourceFilters'
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- quick-filter:read
- tokenizer:
- quick-filter:read
summary: Get a source's quick filters
tags:
- quick_filter
put:
deprecated: false
description: Replaces the org's quick filters for the source named in the path.
operationId: UpdateQuickFilters
parameters:
- in: path
name: source
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/QuickfiltertypesUpdatableQuickFilters'
responses:
"204":
description: No Content
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- quick-filter:update
- tokenizer:
- quick-filter:update
summary: Update quick filters
tags:
- quick_filter
/api/v2/readyz:
get:
operationId: Readyz

View File

@@ -108,6 +108,20 @@ func (provider *Provider) SearchIngestionKeysByName(ctx context.Context, orgID v
}, nil
}
func (provider *Provider) GetIngestionKey(ctx context.Context, orgID valuer.UUID, keyID string) (*gatewaytypes.IngestionKey, error) {
responseBody, err := provider.do(ctx, orgID, http.MethodGet, "/v1/workspaces/me/keys/"+keyID, nil, nil)
if err != nil {
return nil, err
}
var ingestionKey gatewaytypes.IngestionKey
if err := json.Unmarshal([]byte(gjson.GetBytes(responseBody, "data").String()), &ingestionKey); err != nil {
return nil, err
}
return &ingestionKey, nil
}
func (provider *Provider) CreateIngestionKey(ctx context.Context, orgID valuer.UUID, name string, tags []string, expiresAt time.Time) (*gatewaytypes.GettableCreatedIngestionKey, error) {
requestBody := gatewaytypes.PostableIngestionKey{
Name: name,
@@ -161,7 +175,7 @@ func (provider *Provider) DeleteIngestionKey(ctx context.Context, orgID valuer.U
}
func (provider *Provider) CreateIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, keyID string, signal string, limitConfig gatewaytypes.LimitConfig, tags []string) (*gatewaytypes.GettableCreatedIngestionKeyLimit, error) {
requestBody := gatewaytypes.PostableIngestionKeyLimit{
requestBody := gatewaytypes.DeprecatedPostableIngestionKeyLimit{
Signal: signal,
Config: limitConfig,
Tags: tags,
@@ -184,6 +198,34 @@ func (provider *Provider) CreateIngestionKeyLimit(ctx context.Context, orgID val
return &createdIngestionKeyLimitResponse, nil
}
func (provider *Provider) GetIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, limitID string) (*gatewaytypes.Limit, error) {
responseBody, err := provider.do(ctx, orgID, http.MethodGet, "/v1/workspaces/me/limits/"+limitID, nil, nil)
if err != nil {
return nil, err
}
var limit gatewaytypes.Limit
if err := json.Unmarshal([]byte(gjson.GetBytes(responseBody, "data").String()), &limit); err != nil {
return nil, err
}
return &limit, nil
}
func (provider *Provider) GetIngestionKeyLimits(ctx context.Context, orgID valuer.UUID, keyID string) ([]gatewaytypes.Limit, error) {
responseBody, err := provider.do(ctx, orgID, http.MethodGet, "/v1/workspaces/me/keys/"+keyID+"/limits", nil, nil)
if err != nil {
return nil, err
}
var limits []gatewaytypes.Limit
if err := json.Unmarshal([]byte(gjson.GetBytes(responseBody, "data").String()), &limits); err != nil {
return nil, err
}
return limits, nil
}
func (provider *Provider) UpdateIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, limitID string, limitConfig gatewaytypes.LimitConfig, tags []string) error {
requestBody := gatewaytypes.UpdatableIngestionKeyLimit{
Config: limitConfig,

View File

@@ -75,6 +75,23 @@
"field_jsmops_tags": "Tags",
"placeholder_jsmops_tags": "Type a tag and press Enter",
"help_jsmops_tags": "Tags added to every alert.",
"incidentio_tip": "Create an HTTP alert source in incident.io (On-call \u2192 Alert routing \u2192 Sources) and paste its URL and token below.",
"incidentio_tip_link": "Learn how",
"field_incidentio_url": "Alert source URL",
"help_incidentio_url": "The alert events URL from the source's setup page, e.g. https://api.incident.io/v2/alert_events/http/<source_config_id>.",
"field_incidentio_token": "Token",
"help_incidentio_token": "The alert source's secret token, from the same setup page.",
"field_incidentio_title": "Title",
"help_incidentio_title": "Template for the alert title. Kept stable while the alert fires \u2014 incident.io ignores content updates on repeat events.",
"field_incidentio_description": "Description",
"help_incidentio_description": "Template for the alert description. Markdown, rendered natively by incident.io.",
"incidentio_required_fields": "Alert source URL and token are required",
"incidentio_url_invalid": "URL must be an incident.io alert events URL (https://api.incident.io/v2/alert_events/http/<source_config_id>)",
"field_incidentio_metadata": "Additional metadata",
"help_incidentio_metadata": "Key-value pairs added to every alert's metadata, on top of the alert's labels (these win on a key clash). Values may use templates, e.g. {{ .CommonLabels.severity }}.",
"placeholder_incidentio_metadata_key": "Key",
"placeholder_incidentio_metadata_value": "Value",
"button_incidentio_add_metadata": "Add metadata",
"field_slack_recipient": "Recipient",
"field_slack_title": "Title",

View File

@@ -75,6 +75,23 @@
"field_jsmops_tags": "Tags",
"placeholder_jsmops_tags": "Type a tag and press Enter",
"help_jsmops_tags": "Tags added to every alert.",
"incidentio_tip": "Create an HTTP alert source in incident.io (On-call \u2192 Alert routing \u2192 Sources) and paste its URL and token below.",
"incidentio_tip_link": "Learn how",
"field_incidentio_url": "Alert source URL",
"help_incidentio_url": "The alert events URL from the source's setup page, e.g. https://api.incident.io/v2/alert_events/http/<source_config_id>.",
"field_incidentio_token": "Token",
"help_incidentio_token": "The alert source's secret token, from the same setup page.",
"field_incidentio_title": "Title",
"help_incidentio_title": "Template for the alert title. Kept stable while the alert fires \u2014 incident.io ignores content updates on repeat events.",
"field_incidentio_description": "Description",
"help_incidentio_description": "Template for the alert description. Markdown, rendered natively by incident.io.",
"incidentio_required_fields": "Alert source URL and token are required",
"incidentio_url_invalid": "URL must be an incident.io alert events URL (https://api.incident.io/v2/alert_events/http/<source_config_id>)",
"field_incidentio_metadata": "Additional metadata",
"help_incidentio_metadata": "Key-value pairs added to every alert's metadata, on top of the alert's labels (these win on a key clash). Values may use templates, e.g. {{ .CommonLabels.severity }}.",
"placeholder_incidentio_metadata_key": "Key",
"placeholder_incidentio_metadata_value": "Value",
"button_incidentio_add_metadata": "Add metadata",
"field_slack_recipient": "Recipient",
"field_slack_title": "Title",
"field_slack_description": "Description",

View File

@@ -21,18 +21,28 @@ import type {
CreateIngestionKey201,
CreateIngestionKeyLimit201,
CreateIngestionKeyLimitPathParameters,
CreateIngestionLimit201,
DeleteIngestionKeyLimitPathParameters,
DeleteIngestionKeyPathParameters,
DeleteIngestionLimitPathParameters,
GatewaytypesDeprecatedPostableIngestionKeyLimitDTO,
GatewaytypesPostableIngestionKeyDTO,
GatewaytypesPostableIngestionKeyLimitDTO,
GatewaytypesUpdatableIngestionKeyLimitDTO,
GetIngestionKey200,
GetIngestionKeyLimits200,
GetIngestionKeyLimitsPathParameters,
GetIngestionKeyPathParameters,
GetIngestionKeys200,
GetIngestionKeysParams,
GetIngestionLimit200,
GetIngestionLimitPathParameters,
RenderErrorResponseDTO,
SearchIngestionKeys200,
SearchIngestionKeysParams,
UpdateIngestionKeyLimitPathParameters,
UpdateIngestionKeyPathParameters,
UpdateIngestionLimitPathParameters,
} from '../sigNoz.schemas';
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
@@ -300,6 +310,108 @@ export const useDeleteIngestionKey = <
> => {
return useMutation(getDeleteIngestionKeyMutationOptions(options));
};
/**
* This endpoint returns an ingestion key for the workspace
* @summary Get ingestion key for workspace
*/
export const getIngestionKey = (
{ keyId }: GetIngestionKeyPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetIngestionKey200>({
url: `/api/v2/gateway/ingestion_keys/${keyId}`,
method: 'GET',
signal,
});
};
export const getGetIngestionKeyQueryKey = ({
keyId,
}: GetIngestionKeyPathParameters) => {
return [`/api/v2/gateway/ingestion_keys/${keyId}`] as const;
};
export const getGetIngestionKeyQueryOptions = <
TData = Awaited<ReturnType<typeof getIngestionKey>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ keyId }: GetIngestionKeyPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getIngestionKey>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getGetIngestionKeyQueryKey({ keyId });
const queryFn: QueryFunction<Awaited<ReturnType<typeof getIngestionKey>>> = ({
signal,
}) => getIngestionKey({ keyId }, signal);
return {
queryKey,
queryFn,
enabled: !!keyId,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof getIngestionKey>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetIngestionKeyQueryResult = NonNullable<
Awaited<ReturnType<typeof getIngestionKey>>
>;
export type GetIngestionKeyQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Get ingestion key for workspace
*/
export function useGetIngestionKey<
TData = Awaited<ReturnType<typeof getIngestionKey>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ keyId }: GetIngestionKeyPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getIngestionKey>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetIngestionKeyQueryOptions({ keyId }, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary Get ingestion key for workspace
*/
export const invalidateGetIngestionKey = async (
queryClient: QueryClient,
{ keyId }: GetIngestionKeyPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetIngestionKeyQueryKey({ keyId }) },
options,
);
return queryClient;
};
/**
* This endpoint updates an ingestion key for the workspace
* @summary Update ingestion key for workspace
@@ -399,20 +511,123 @@ export const useUpdateIngestionKey = <
> => {
return useMutation(getUpdateIngestionKeyMutationOptions(options));
};
/**
* This endpoint returns the ingestion limits for an ingestion key
* @summary Get limits for the ingestion key
*/
export const getIngestionKeyLimits = (
{ keyId }: GetIngestionKeyLimitsPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetIngestionKeyLimits200>({
url: `/api/v2/gateway/ingestion_keys/${keyId}/limits`,
method: 'GET',
signal,
});
};
export const getGetIngestionKeyLimitsQueryKey = ({
keyId,
}: GetIngestionKeyLimitsPathParameters) => {
return [`/api/v2/gateway/ingestion_keys/${keyId}/limits`] as const;
};
export const getGetIngestionKeyLimitsQueryOptions = <
TData = Awaited<ReturnType<typeof getIngestionKeyLimits>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ keyId }: GetIngestionKeyLimitsPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getIngestionKeyLimits>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getGetIngestionKeyLimitsQueryKey({ keyId });
const queryFn: QueryFunction<
Awaited<ReturnType<typeof getIngestionKeyLimits>>
> = ({ signal }) => getIngestionKeyLimits({ keyId }, signal);
return {
queryKey,
queryFn,
enabled: !!keyId,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof getIngestionKeyLimits>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetIngestionKeyLimitsQueryResult = NonNullable<
Awaited<ReturnType<typeof getIngestionKeyLimits>>
>;
export type GetIngestionKeyLimitsQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Get limits for the ingestion key
*/
export function useGetIngestionKeyLimits<
TData = Awaited<ReturnType<typeof getIngestionKeyLimits>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ keyId }: GetIngestionKeyLimitsPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getIngestionKeyLimits>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetIngestionKeyLimitsQueryOptions({ keyId }, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary Get limits for the ingestion key
*/
export const invalidateGetIngestionKeyLimits = async (
queryClient: QueryClient,
{ keyId }: GetIngestionKeyLimitsPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetIngestionKeyLimitsQueryKey({ keyId }) },
options,
);
return queryClient;
};
/**
* This endpoint creates an ingestion key limit
* @deprecated
* @summary Create limit for the ingestion key
*/
export const createIngestionKeyLimit = (
{ keyId }: CreateIngestionKeyLimitPathParameters,
gatewaytypesPostableIngestionKeyLimitDTO?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>,
gatewaytypesDeprecatedPostableIngestionKeyLimitDTO?: BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<CreateIngestionKeyLimit201>({
url: `/api/v2/gateway/ingestion_keys/${keyId}/limits`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: gatewaytypesPostableIngestionKeyLimitDTO,
data: gatewaytypesDeprecatedPostableIngestionKeyLimitDTO,
signal,
});
};
@@ -426,7 +641,7 @@ export const getCreateIngestionKeyLimitMutationOptions = <
TError,
{
pathParams: CreateIngestionKeyLimitPathParameters;
data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>;
data?: BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>;
},
TContext
>;
@@ -435,7 +650,7 @@ export const getCreateIngestionKeyLimitMutationOptions = <
TError,
{
pathParams: CreateIngestionKeyLimitPathParameters;
data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>;
data?: BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>;
},
TContext
> => {
@@ -452,7 +667,7 @@ export const getCreateIngestionKeyLimitMutationOptions = <
Awaited<ReturnType<typeof createIngestionKeyLimit>>,
{
pathParams: CreateIngestionKeyLimitPathParameters;
data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>;
data?: BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
@@ -467,12 +682,13 @@ export type CreateIngestionKeyLimitMutationResult = NonNullable<
Awaited<ReturnType<typeof createIngestionKeyLimit>>
>;
export type CreateIngestionKeyLimitMutationBody =
| BodyType<GatewaytypesPostableIngestionKeyLimitDTO>
| BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>
| undefined;
export type CreateIngestionKeyLimitMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Create limit for the ingestion key
*/
export const useCreateIngestionKeyLimit = <
@@ -484,7 +700,7 @@ export const useCreateIngestionKeyLimit = <
TError,
{
pathParams: CreateIngestionKeyLimitPathParameters;
data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>;
data?: BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>;
},
TContext
>;
@@ -493,7 +709,7 @@ export const useCreateIngestionKeyLimit = <
TError,
{
pathParams: CreateIngestionKeyLimitPathParameters;
data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>;
data?: BodyType<GatewaytypesDeprecatedPostableIngestionKeyLimitDTO>;
},
TContext
> => {
@@ -501,6 +717,7 @@ export const useCreateIngestionKeyLimit = <
};
/**
* This endpoint deletes an ingestion key limit
* @deprecated
* @summary Delete limit for the ingestion key
*/
export const deleteIngestionKeyLimit = (
@@ -559,6 +776,7 @@ export type DeleteIngestionKeyLimitMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Delete limit for the ingestion key
*/
export const useDeleteIngestionKeyLimit = <
@@ -581,6 +799,7 @@ export const useDeleteIngestionKeyLimit = <
};
/**
* This endpoint updates an ingestion key limit
* @deprecated
* @summary Update limit for the ingestion key
*/
export const updateIngestionKeyLimit = (
@@ -653,6 +872,7 @@ export type UpdateIngestionKeyLimitMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Update limit for the ingestion key
*/
export const useUpdateIngestionKeyLimit = <
@@ -779,3 +999,370 @@ export const invalidateSearchIngestionKeys = async (
return queryClient;
};
/**
* This endpoint creates an ingestion limit for the ingestion key referenced by keyId
* @summary Create ingestion limit
*/
export const createIngestionLimit = (
gatewaytypesPostableIngestionKeyLimitDTO?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<CreateIngestionLimit201>({
url: `/api/v2/gateway/ingestion_limits`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: gatewaytypesPostableIngestionKeyLimitDTO,
signal,
});
};
export const getCreateIngestionLimitMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof createIngestionLimit>>,
TError,
{ data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof createIngestionLimit>>,
TError,
{ data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO> },
TContext
> => {
const mutationKey = ['createIngestionLimit'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof createIngestionLimit>>,
{ data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO> }
> = (props) => {
const { data } = props ?? {};
return createIngestionLimit(data);
};
return { mutationFn, ...mutationOptions };
};
export type CreateIngestionLimitMutationResult = NonNullable<
Awaited<ReturnType<typeof createIngestionLimit>>
>;
export type CreateIngestionLimitMutationBody =
| BodyType<GatewaytypesPostableIngestionKeyLimitDTO>
| undefined;
export type CreateIngestionLimitMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Create ingestion limit
*/
export const useCreateIngestionLimit = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof createIngestionLimit>>,
TError,
{ data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof createIngestionLimit>>,
TError,
{ data?: BodyType<GatewaytypesPostableIngestionKeyLimitDTO> },
TContext
> => {
return useMutation(getCreateIngestionLimitMutationOptions(options));
};
/**
* This endpoint deletes an ingestion limit
* @summary Delete ingestion limit
*/
export const deleteIngestionLimit = (
{ limitId }: DeleteIngestionLimitPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/gateway/ingestion_limits/${limitId}`,
method: 'DELETE',
signal,
});
};
export const getDeleteIngestionLimitMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof deleteIngestionLimit>>,
TError,
{ pathParams: DeleteIngestionLimitPathParameters },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof deleteIngestionLimit>>,
TError,
{ pathParams: DeleteIngestionLimitPathParameters },
TContext
> => {
const mutationKey = ['deleteIngestionLimit'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof deleteIngestionLimit>>,
{ pathParams: DeleteIngestionLimitPathParameters }
> = (props) => {
const { pathParams } = props ?? {};
return deleteIngestionLimit(pathParams);
};
return { mutationFn, ...mutationOptions };
};
export type DeleteIngestionLimitMutationResult = NonNullable<
Awaited<ReturnType<typeof deleteIngestionLimit>>
>;
export type DeleteIngestionLimitMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Delete ingestion limit
*/
export const useDeleteIngestionLimit = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof deleteIngestionLimit>>,
TError,
{ pathParams: DeleteIngestionLimitPathParameters },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof deleteIngestionLimit>>,
TError,
{ pathParams: DeleteIngestionLimitPathParameters },
TContext
> => {
return useMutation(getDeleteIngestionLimitMutationOptions(options));
};
/**
* This endpoint returns an ingestion limit
* @summary Get ingestion limit
*/
export const getIngestionLimit = (
{ limitId }: GetIngestionLimitPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetIngestionLimit200>({
url: `/api/v2/gateway/ingestion_limits/${limitId}`,
method: 'GET',
signal,
});
};
export const getGetIngestionLimitQueryKey = ({
limitId,
}: GetIngestionLimitPathParameters) => {
return [`/api/v2/gateway/ingestion_limits/${limitId}`] as const;
};
export const getGetIngestionLimitQueryOptions = <
TData = Awaited<ReturnType<typeof getIngestionLimit>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ limitId }: GetIngestionLimitPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getIngestionLimit>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getGetIngestionLimitQueryKey({ limitId });
const queryFn: QueryFunction<
Awaited<ReturnType<typeof getIngestionLimit>>
> = ({ signal }) => getIngestionLimit({ limitId }, signal);
return {
queryKey,
queryFn,
enabled: !!limitId,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof getIngestionLimit>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetIngestionLimitQueryResult = NonNullable<
Awaited<ReturnType<typeof getIngestionLimit>>
>;
export type GetIngestionLimitQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Get ingestion limit
*/
export function useGetIngestionLimit<
TData = Awaited<ReturnType<typeof getIngestionLimit>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ limitId }: GetIngestionLimitPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getIngestionLimit>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetIngestionLimitQueryOptions({ limitId }, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary Get ingestion limit
*/
export const invalidateGetIngestionLimit = async (
queryClient: QueryClient,
{ limitId }: GetIngestionLimitPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetIngestionLimitQueryKey({ limitId }) },
options,
);
return queryClient;
};
/**
* This endpoint updates an ingestion limit
* @summary Update ingestion limit
*/
export const updateIngestionLimit = (
{ limitId }: UpdateIngestionLimitPathParameters,
gatewaytypesUpdatableIngestionKeyLimitDTO?: BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/gateway/ingestion_limits/${limitId}`,
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
data: gatewaytypesUpdatableIngestionKeyLimitDTO,
signal,
});
};
export const getUpdateIngestionLimitMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof updateIngestionLimit>>,
TError,
{
pathParams: UpdateIngestionLimitPathParameters;
data?: BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>;
},
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof updateIngestionLimit>>,
TError,
{
pathParams: UpdateIngestionLimitPathParameters;
data?: BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>;
},
TContext
> => {
const mutationKey = ['updateIngestionLimit'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof updateIngestionLimit>>,
{
pathParams: UpdateIngestionLimitPathParameters;
data?: BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
return updateIngestionLimit(pathParams, data);
};
return { mutationFn, ...mutationOptions };
};
export type UpdateIngestionLimitMutationResult = NonNullable<
Awaited<ReturnType<typeof updateIngestionLimit>>
>;
export type UpdateIngestionLimitMutationBody =
| BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>
| undefined;
export type UpdateIngestionLimitMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Update ingestion limit
*/
export const useUpdateIngestionLimit = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof updateIngestionLimit>>,
TError,
{
pathParams: UpdateIngestionLimitPathParameters;
data?: BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>;
},
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof updateIngestionLimit>>,
TError,
{
pathParams: UpdateIngestionLimitPathParameters;
data?: BodyType<GatewaytypesUpdatableIngestionKeyLimitDTO>;
},
TContext
> => {
return useMutation(getUpdateIngestionLimitMutationOptions(options));
};

View File

@@ -0,0 +1,316 @@
/**
* ! Do not edit manually
* * The file has been auto-generated using Orval for SigNoz
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useMutation, useQuery } from 'react-query';
import type {
InvalidateOptions,
MutationFunction,
QueryClient,
QueryFunction,
QueryKey,
UseMutationOptions,
UseMutationResult,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type {
GetQuickFilters200,
GetQuickFiltersPathParameters,
ListQuickFilters200,
QuickfiltertypesUpdatableQuickFiltersDTO,
RenderErrorResponseDTO,
UpdateQuickFiltersPathParameters,
} from '../sigNoz.schemas';
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
/**
* Returns the org's quick filters for every source, each filter as a telemetry field key.
* @summary List quick filters
*/
export const listQuickFilters = (signal?: AbortSignal) => {
return GeneratedAPIInstance<ListQuickFilters200>({
url: `/api/v2/quick_filters`,
method: 'GET',
signal,
});
};
export const getListQuickFiltersQueryKey = () => {
return [`/api/v2/quick_filters`] as const;
};
export const getListQuickFiltersQueryOptions = <
TData = Awaited<ReturnType<typeof listQuickFilters>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listQuickFilters>>,
TError,
TData
>;
}) => {
const { query: queryOptions } = options ?? {};
const queryKey = queryOptions?.queryKey ?? getListQuickFiltersQueryKey();
const queryFn: QueryFunction<Awaited<ReturnType<typeof listQuickFilters>>> = ({
signal,
}) => listQuickFilters(signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof listQuickFilters>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListQuickFiltersQueryResult = NonNullable<
Awaited<ReturnType<typeof listQuickFilters>>
>;
export type ListQuickFiltersQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary List quick filters
*/
export function useListQuickFilters<
TData = Awaited<ReturnType<typeof listQuickFilters>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listQuickFilters>>,
TError,
TData
>;
}): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListQuickFiltersQueryOptions(options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary List quick filters
*/
export const invalidateListQuickFilters = async (
queryClient: QueryClient,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListQuickFiltersQueryKey() },
options,
);
return queryClient;
};
/**
* Returns the org's quick filters for one source, each filter as a telemetry field key.
* @summary Get a source's quick filters
*/
export const getQuickFilters = (
{ source }: GetQuickFiltersPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetQuickFilters200>({
url: `/api/v2/quick_filters/${source}`,
method: 'GET',
signal,
});
};
export const getGetQuickFiltersQueryKey = ({
source,
}: GetQuickFiltersPathParameters) => {
return [`/api/v2/quick_filters/${source}`] as const;
};
export const getGetQuickFiltersQueryOptions = <
TData = Awaited<ReturnType<typeof getQuickFilters>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ source }: GetQuickFiltersPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getQuickFilters>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getGetQuickFiltersQueryKey({ source });
const queryFn: QueryFunction<Awaited<ReturnType<typeof getQuickFilters>>> = ({
signal,
}) => getQuickFilters({ source }, signal);
return {
queryKey,
queryFn,
enabled: !!source,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof getQuickFilters>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetQuickFiltersQueryResult = NonNullable<
Awaited<ReturnType<typeof getQuickFilters>>
>;
export type GetQuickFiltersQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Get a source's quick filters
*/
export function useGetQuickFilters<
TData = Awaited<ReturnType<typeof getQuickFilters>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ source }: GetQuickFiltersPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getQuickFilters>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetQuickFiltersQueryOptions({ source }, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary Get a source's quick filters
*/
export const invalidateGetQuickFilters = async (
queryClient: QueryClient,
{ source }: GetQuickFiltersPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetQuickFiltersQueryKey({ source }) },
options,
);
return queryClient;
};
/**
* Replaces the org's quick filters for the source named in the path.
* @summary Update quick filters
*/
export const updateQuickFilters = (
{ source }: UpdateQuickFiltersPathParameters,
quickfiltertypesUpdatableQuickFiltersDTO?: BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/quick_filters/${source}`,
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
data: quickfiltertypesUpdatableQuickFiltersDTO,
signal,
});
};
export const getUpdateQuickFiltersMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof updateQuickFilters>>,
TError,
{
pathParams: UpdateQuickFiltersPathParameters;
data?: BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>;
},
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof updateQuickFilters>>,
TError,
{
pathParams: UpdateQuickFiltersPathParameters;
data?: BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>;
},
TContext
> => {
const mutationKey = ['updateQuickFilters'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof updateQuickFilters>>,
{
pathParams: UpdateQuickFiltersPathParameters;
data?: BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
return updateQuickFilters(pathParams, data);
};
return { mutationFn, ...mutationOptions };
};
export type UpdateQuickFiltersMutationResult = NonNullable<
Awaited<ReturnType<typeof updateQuickFilters>>
>;
export type UpdateQuickFiltersMutationBody =
| BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>
| undefined;
export type UpdateQuickFiltersMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Update quick filters
*/
export const useUpdateQuickFilters = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof updateQuickFilters>>,
TError,
{
pathParams: UpdateQuickFiltersPathParameters;
data?: BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>;
},
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof updateQuickFilters>>,
TError,
{
pathParams: UpdateQuickFiltersPathParameters;
data?: BodyType<QuickfiltertypesUpdatableQuickFiltersDTO>;
},
TContext
> => {
return useMutation(getUpdateQuickFiltersMutationOptions(options));
};

View File

@@ -385,6 +385,38 @@ export interface AlertmanagertypesGoogleChatReceiverConfigDTO {
webhook_url?: ConfigSecretURLDTO;
}
export type AlertmanagertypesIncidentIOReceiverConfigDTOMetadata = {
[key: string]: string;
};
export interface AlertmanagertypesIncidentIOReceiverConfigDTO {
/**
* @type string
*/
description?: string;
http_config?: ConfigHTTPClientConfigDTO;
/**
* @type object
*/
metadata?: AlertmanagertypesIncidentIOReceiverConfigDTOMetadata;
/**
* @type boolean
*/
send_resolved?: boolean;
/**
* @type string
*/
title?: string;
/**
* @type string
*/
token?: string;
/**
* @type string
*/
url?: string;
}
export interface AlertmanagertypesJSMOpsReceiverConfigDTO {
/**
* @type string
@@ -685,39 +717,6 @@ export interface ConfigEmailConfigDTO {
to?: string;
}
export type TimeDurationDTO = number;
export interface ConfigURLType2DTO {
[key: string]: unknown;
}
export interface ConfigIncidentioConfigDTO {
/**
* @type string
*/
alert_source_token?: string;
/**
* @type string
*/
alert_source_token_file?: string;
http_config?: ConfigHTTPClientConfigDTO;
/**
* @type integer
* @minimum 0
*/
max_alerts?: number;
/**
* @type boolean
*/
send_resolved?: boolean;
timeout?: TimeDurationDTO;
url?: ConfigURLType2DTO;
/**
* @type string
*/
url_file?: string;
}
export interface ConfigMattermostFieldDTO {
/**
* @type boolean,null
@@ -903,6 +902,10 @@ export interface ConfigMSTeamsV2ConfigDTO {
webhook_url_file?: string;
}
export interface ConfigURLType2DTO {
[key: string]: unknown;
}
export interface ConfigOpsGenieConfigResponderDTO {
/**
* @type string
@@ -1011,6 +1014,8 @@ export interface ConfigPagerdutyLinkDTO {
text?: string;
}
export type TimeDurationDTO = number;
export type ConfigPagerdutyConfigDTODetails = { [key: string]: unknown };
export interface ConfigPagerdutyConfigDTO {
@@ -1672,7 +1677,7 @@ export type AlertmanagertypesPostableChannelDTO = unknown & {
/**
* @type array
*/
incidentio_configs?: ConfigIncidentioConfigDTO[];
incidentio_configs?: AlertmanagertypesIncidentIOReceiverConfigDTO[];
/**
* @type array
*/
@@ -1803,7 +1808,7 @@ export interface AlertmanagertypesReceiverDTO {
/**
* @type array
*/
incidentio_configs?: ConfigIncidentioConfigDTO[];
incidentio_configs?: AlertmanagertypesIncidentIOReceiverConfigDTO[];
/**
* @type array
*/
@@ -3300,6 +3305,33 @@ export interface CommonJSONRefDTO {
$ref?: string;
}
export interface ConfigIncidentioConfigDTO {
/**
* @type string
*/
alert_source_token?: string;
/**
* @type string
*/
alert_source_token_file?: string;
http_config?: ConfigHTTPClientConfigDTO;
/**
* @type integer
* @minimum 0
*/
max_alerts?: number;
/**
* @type boolean
*/
send_resolved?: boolean;
timeout?: TimeDurationDTO;
url?: ConfigURLType2DTO;
/**
* @type string
*/
url_file?: string;
}
export type ConfigJiraConfigDTOCustomFields = { [key: string]: unknown };
export interface ConfigJiraFieldConfigDTO {
@@ -5461,6 +5493,34 @@ export interface FeaturetypesGettableFeatureDTO {
variants?: FeaturetypesGettableFeatureDTOVariants;
}
export interface GatewaytypesLimitValueDTO {
/**
* @type integer,null
*/
count?: number | null;
/**
* @type integer,null
*/
size?: number | null;
}
export interface GatewaytypesLimitConfigDTO {
day?: GatewaytypesLimitValueDTO;
second?: GatewaytypesLimitValueDTO;
}
export interface GatewaytypesDeprecatedPostableIngestionKeyLimitDTO {
config?: GatewaytypesLimitConfigDTO;
/**
* @type string
*/
signal?: string;
/**
* @type array,null
*/
tags?: string[] | null;
}
export interface GatewaytypesGettableCreatedIngestionKeyDTO {
/**
* @type string
@@ -5498,22 +5558,6 @@ export interface GatewaytypesPaginationDTO {
total?: number;
}
export interface GatewaytypesLimitValueDTO {
/**
* @type integer,null
*/
count?: number | null;
/**
* @type integer,null
*/
size?: number | null;
}
export interface GatewaytypesLimitConfigDTO {
day?: GatewaytypesLimitValueDTO;
second?: GatewaytypesLimitValueDTO;
}
export interface GatewaytypesLimitMetricValueDTO {
/**
* @type integer
@@ -5631,6 +5675,10 @@ export interface GatewaytypesPostableIngestionKeyDTO {
export interface GatewaytypesPostableIngestionKeyLimitDTO {
config?: GatewaytypesLimitConfigDTO;
/**
* @type string
*/
keyId: string;
/**
* @type string
*/
@@ -8978,6 +9026,47 @@ export enum Querybuildertypesv5QueryTypeDTO {
clickhouse_sql = 'clickhouse_sql',
promql = 'promql',
}
export enum QuickfiltertypesSourceDTO {
traces = 'traces',
logs = 'logs',
api_monitoring = 'api_monitoring',
exceptions = 'exceptions',
meter = 'meter',
ai_observability = 'ai_observability',
}
export interface QuickfiltertypesSourceFiltersDTO {
/**
* @type string
* @format date-time
*/
createdAt?: string;
/**
* @type array
*/
filters: TelemetrytypesTelemetryFieldKeyDTO[];
/**
* @type string
*/
id: string;
/**
* @type string
*/
orgId: string;
source: QuickfiltertypesSourceDTO;
/**
* @type string
* @format date-time
*/
updatedAt?: string;
}
export interface QuickfiltertypesUpdatableQuickFiltersDTO {
/**
* @type array
*/
filters: TelemetrytypesTelemetryFieldKeyDTO[];
}
export interface RenderErrorResponseDTO {
error: ErrorsJSONDTO;
/**
@@ -10790,6 +10879,11 @@ export type GetAIObservabilityFieldsValuesParams = {
* @description undefined
*/
name?: string;
/**
* @type string
* @description undefined
*/
existingQuery?: string;
};
export type GetAIObservabilityFieldsValues200 = {
@@ -11908,9 +12002,34 @@ export type CreateIngestionKey201 = {
export type DeleteIngestionKeyPathParameters = {
keyId: string;
};
export type GetIngestionKeyPathParameters = {
keyId: string;
};
export type GetIngestionKey200 = {
data: GatewaytypesIngestionKeyDTO;
/**
* @type string
*/
status: string;
};
export type UpdateIngestionKeyPathParameters = {
keyId: string;
};
export type GetIngestionKeyLimitsPathParameters = {
keyId: string;
};
export type GetIngestionKeyLimits200 = {
/**
* @type array,null
*/
data: GatewaytypesLimitDTO[] | null;
/**
* @type string
*/
status: string;
};
export type CreateIngestionKeyLimitPathParameters = {
keyId: string;
};
@@ -11954,6 +12073,31 @@ export type SearchIngestionKeys200 = {
status: string;
};
export type CreateIngestionLimit201 = {
data: TypesIdentifiableDTO;
/**
* @type string
*/
status: string;
};
export type DeleteIngestionLimitPathParameters = {
limitId: string;
};
export type GetIngestionLimitPathParameters = {
limitId: string;
};
export type GetIngestionLimit200 = {
data: GatewaytypesLimitDTO;
/**
* @type string
*/
status: string;
};
export type UpdateIngestionLimitPathParameters = {
limitId: string;
};
export type Healthz200 = {
data: FactoryResponseDTO;
/**
@@ -12379,6 +12523,31 @@ export type GetPublicDashboardPanelQueryRangeV2200 = {
status: string;
};
export type ListQuickFilters200 = {
/**
* @type array
*/
data: QuickfiltertypesSourceFiltersDTO[];
/**
* @type string
*/
status: string;
};
export type GetQuickFiltersPathParameters = {
source: string;
};
export type GetQuickFilters200 = {
data: QuickfiltertypesSourceFiltersDTO;
/**
* @type string
*/
status: string;
};
export type UpdateQuickFiltersPathParameters = {
source: string;
};
export type Readyz200 = {
data: FactoryResponseDTO;
/**

View File

@@ -2,6 +2,7 @@ import CreateAlertChannels from 'container/CreateAlertChannels';
import { ChannelType } from 'container/CreateAlertChannels/config';
import {
GoogleChatInitialConfig,
IncidentIOInitialConfig,
JiraInitialConfig,
JsmOpsInitialConfig,
} from 'container/CreateAlertChannels/defaults';
@@ -585,7 +586,7 @@ describe('Create Alert Channel', () => {
description: 'jira_site_invalid',
}),
);
});
}, 15000);
it('Should send a jira_configs payload with basic auth', async () => {
let requestBody: unknown;
@@ -737,6 +738,122 @@ describe('Create Alert Channel', () => {
});
});
});
describe('incident.io', () => {
const incidentIOURL =
'https://api.incident.io/v2/alert_events/http/01M0D1JNVBGBGVTWX053EM12XV';
beforeEach(() => {
render(<CreateAlertChannels preType={ChannelType.IncidentIO} />);
});
it('Should display the URL and token fields with the docs tip', () => {
testLabelInputAndHelpValue({
labelText: 'field_incidentio_url',
testId: 'incidentio-url-textbox',
});
testLabelInputAndHelpValue({
labelText: 'field_incidentio_token',
testId: 'incidentio-token-textbox',
});
expect(screen.getByTestId('incidentio-tip')).toBeInTheDocument();
expect(
screen.getByRole('link', { name: 'incidentio_tip_link' }),
).toHaveAttribute(
'href',
'https://signoz.io/docs/alerts-management/notification-channel/incidentio/',
);
});
it('Should block save when the URL or token is missing', async () => {
const user = userEvent.setup();
await user.type(
screen.getByTestId('channel-name-textbox'),
'incidentio-channel',
);
await user.click(screen.getByTestId('save-channel-button'));
await waitFor(() =>
expect(errorNotification).toHaveBeenCalledWith({
message: 'Error',
description: 'incidentio_required_fields',
}),
);
});
it('Should display an error when the URL is not an alert events URL', async () => {
const user = userEvent.setup();
await user.type(
screen.getByTestId('channel-name-textbox'),
'incidentio-channel',
);
await user.type(
screen.getByTestId('incidentio-url-textbox'),
'https://api.incident.io/v2/incidents',
);
await user.type(screen.getByTestId('incidentio-token-textbox'), 'tok-abc');
await user.click(screen.getByTestId('save-channel-button'));
await waitFor(() =>
expect(errorNotification).toHaveBeenCalledWith({
message: 'Error',
description: 'incidentio_url_invalid',
}),
);
}, 15000);
it('Should send an incidentio_configs payload with prefilled defaults', async () => {
let requestBody: unknown;
server.use(
rest.post('http://localhost/api/v1/channels', async (req, res, ctx) => {
requestBody = await req.json();
return res(
ctx.status(201),
ctx.json({ status: 'success', data: 'channel created' }),
);
}),
);
const user = userEvent.setup();
await user.type(
screen.getByTestId('channel-name-textbox'),
'incidentio-channel',
);
await user.type(
screen.getByTestId('incidentio-url-textbox'),
incidentIOURL,
);
await user.type(screen.getByTestId('incidentio-token-textbox'), 'tok-abc');
await user.click(screen.getByTestId('incidentio-metadata-add'));
await user.type(screen.getByTestId('incidentio-metadata-key-0'), 'team');
await user.type(screen.getByTestId('incidentio-metadata-value-0'), 'core');
await user.click(screen.getByTestId('save-channel-button'));
await waitFor(() =>
expect(successNotification).toHaveBeenCalledWith({
message: 'Success',
description: 'channel_creation_done',
}),
);
expect(requestBody).toStrictEqual({
name: 'incidentio-channel',
incidentio_configs: [
{
url: incidentIOURL,
token: 'tok-abc',
send_resolved: true,
title: IncidentIOInitialConfig.title,
description: IncidentIOInitialConfig.description,
metadata: { team: 'core' },
},
],
});
}, 15000);
});
describe('Changing the channel type', () => {
async function selectType(
user: ReturnType<typeof userEvent.setup>,

View File

@@ -90,6 +90,40 @@ describe('EditAlertChannels save', () => {
await waitFor(() => expect(edit.calls).toHaveLength(1));
});
it('sends an incidentio_configs payload when editing an incident.io channel', async () => {
const edit = mockEditChannel();
render(
<EditAlertChannels
channelId="4"
initialValue={{
type: 'incidentio',
name: 'incidentio-channel',
url: 'https://api.incident.io/v2/alert_events/http/01M0D1JNVBGBGVTWX053EM12XV',
token: 'tok-abc',
send_resolved: true,
metadata: { env: 'prod' },
}}
/>,
);
const user = userEvent.setup();
await user.click(screen.getByTestId('save-channel-button'));
await waitFor(() => expect(edit.calls).toHaveLength(1));
expect(edit.calls[0].id).toBe('4');
expect(edit.calls[0].body).toStrictEqual({
name: 'incidentio-channel',
incidentio_configs: [
{
url: 'https://api.incident.io/v2/alert_events/http/01M0D1JNVBGBGVTWX053EM12XV',
token: 'tok-abc',
send_resolved: true,
metadata: { env: 'prod' },
},
],
});
});
it('persists send_resolved toggle in the edit request', async () => {
const edit = mockEditChannel();
render(

View File

@@ -107,6 +107,7 @@ export enum ChannelType {
GoogleChat = 'googlechat',
Jira = 'jira',
JsmOps = 'jsmops',
IncidentIO = 'incidentio',
}
// LabelFilterStatement will be used for preparing filter conditions / matchers
@@ -159,6 +160,22 @@ export interface JiraChannel extends Channel {
reopen_duration?: string;
}
// IncidentIOChannel configures the incident.io alert channel, backed by an
// incident.io HTTP alert source (Alert Events V2 API).
export interface IncidentIOChannel extends Channel {
// per-source alert events URL, e.g.
// https://api.incident.io/v2/alert_events/http/<source_config_id>
url: string;
// the alert source's secret token
token: string;
// alert title template
title?: string;
// alert body template (markdown, rendered natively by incident.io)
description?: string;
// extra metadata pairs merged over the alert's labels (channel wins on clash)
metadata?: Record<string, string>;
}
// JsmOpsChannel configures the Jira Service Management Ops alert channel
// (ex-Opsgenie alert API). Auth is the JSM integration API key.
export interface JsmOpsChannel extends Channel {

View File

@@ -2,6 +2,7 @@ import {
ChannelType,
EmailChannel,
GoogleChatChannel,
IncidentIOChannel,
JiraChannel,
JsmOpsChannel,
MsTeamsChannel,
@@ -144,6 +145,29 @@ export const JsmOpsInitialConfig: Partial<JsmOpsChannel> = {
tags: ['signoz-alert'],
};
// mirrors DefaultIncidentIOTitleTemplate / DefaultIncidentIODescriptionTemplate
// in pkg/types/alertmanagertypes/incidentio.go, applied by the backend when
// title / description are left empty. send_resolved is seeded on so incident.io
// alerts resolve with the rule (the backend cannot default it).
export const IncidentIOInitialConfig: Partial<IncidentIOChannel> = {
send_resolved: true,
title: `[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }}`,
description: `{{ range .Alerts -}}
**Alert:** {{ .Labels.alertname }}{{ if .Labels.severity }} ({{ .Labels.severity }}){{ end }}
{{ if .Annotations.summary }}**Summary:** {{ .Annotations.summary }}
{{ end }}{{ if .Annotations.description }}**Description:** {{ .Annotations.description }}
{{ end }}{{ if .GeneratorURL }}[View in SigNoz]({{ .GeneratorURL }})
{{ end }}{{ if .Annotations.related_logs }}[View related logs]({{ .Annotations.related_logs }})
{{ end }}{{ if .Annotations.related_traces }}[View related traces]({{ .Annotations.related_traces }})
{{ end }}{{ end }}`,
};
export const EmailInitialConfig: Partial<EmailChannel> = {
send_resolved: true,
html: `<!--
@@ -553,7 +577,8 @@ export const ChannelInitialConfig: Record<
EmailChannel &
GoogleChatChannel &
JiraChannel &
JsmOpsChannel
JsmOpsChannel &
IncidentIOChannel
>
> = {
[ChannelType.Slack]: SlackInitialConfig,
@@ -561,6 +586,7 @@ export const ChannelInitialConfig: Record<
[ChannelType.GoogleChat]: GoogleChatInitialConfig,
[ChannelType.Jira]: JiraInitialConfig,
[ChannelType.JsmOps]: JsmOpsInitialConfig,
[ChannelType.IncidentIO]: IncidentIOInitialConfig,
[ChannelType.Pagerduty]: PagerInitialConfig,
[ChannelType.Opsgenie]: OpsgenieInitialConfig,
[ChannelType.Email]: EmailInitialConfig,

View File

@@ -32,6 +32,7 @@ import {
ChannelType,
EmailChannel,
GoogleChatChannel,
IncidentIOChannel,
JiraChannel,
JsmOpsChannel,
MsTeamsChannel,
@@ -45,9 +46,11 @@ import { ChannelInitialConfig } from './defaults';
import {
isChannelType,
isValidGoogleChatWebhookURL,
isValidIncidentIOURL,
isValidJiraReopenDuration,
isValidJiraSiteURL,
prepareGoogleChatRequest,
prepareIncidentIORequest,
prepareJiraRequest,
prepareJsmOpsRequest,
} from './utils';
@@ -77,7 +80,8 @@ function CreateAlertChannels({
EmailChannel &
GoogleChatChannel &
JiraChannel &
JsmOpsChannel
JsmOpsChannel &
IncidentIOChannel
>
>(() => ({
send_resolved: true,
@@ -550,6 +554,56 @@ function CreateAlertChannels({
showErrorModal,
]);
const validateIncidentIOConfig = useCallback((): boolean => {
if (!selectedConfig.url || !selectedConfig.token) {
notifications.error({
message: 'Error',
description: t('incidentio_required_fields'),
});
return false;
}
if (!isValidIncidentIOURL(selectedConfig.url)) {
notifications.error({
message: 'Error',
description: t('incidentio_url_invalid'),
});
return false;
}
return true;
}, [selectedConfig.url, selectedConfig.token, notifications, t]);
const onIncidentIOHandler = useCallback(async () => {
if (!validateIncidentIOConfig()) {
return { status: 'failed', statusMessage: t('channel_creation_failed') };
}
setSavingState(true);
try {
await createChannel({ data: prepareIncidentIORequest(selectedConfig) });
notifications.success({
message: 'Success',
description: t('channel_creation_done'),
});
history.replace(ROUTES.ALL_CHANNELS);
return { status: 'success', statusMessage: t('channel_creation_done') };
} catch (error) {
showErrorModal(toAPIError(error as ErrorType<RenderErrorResponseDTO>));
return { status: 'failed', statusMessage: t('channel_creation_failed') };
} finally {
setSavingState(false);
}
}, [
validateIncidentIOConfig,
createChannel,
selectedConfig,
notifications,
t,
showErrorModal,
]);
const onSaveHandler = useCallback(
async (value: ChannelType) => {
if (!selectedConfig.name) {
@@ -570,6 +624,7 @@ function CreateAlertChannels({
[ChannelType.GoogleChat]: onGoogleChatHandler,
[ChannelType.Jira]: onJiraHandler,
[ChannelType.JsmOps]: onJsmOpsHandler,
[ChannelType.IncidentIO]: onIncidentIOHandler,
};
if (isChannelType(value)) {
@@ -604,6 +659,7 @@ function CreateAlertChannels({
onGoogleChatHandler,
onJiraHandler,
onJsmOpsHandler,
onIncidentIOHandler,
notifications,
t,
],
@@ -662,6 +718,13 @@ function CreateAlertChannels({
}
await testChannel({ data: prepareJsmOpsRequest(selectedConfig) });
break;
case ChannelType.IncidentIO:
if (!validateIncidentIOConfig()) {
setTestingState(false);
return;
}
await testChannel({ data: prepareIncidentIORequest(selectedConfig) });
break;
default:
notifications.error({
message: 'Error',
@@ -712,6 +775,7 @@ function CreateAlertChannels({
validateGoogleChatConfig,
validateJiraConfig,
validateJsmOpsConfig,
validateIncidentIOConfig,
testChannel,
notifications,
],

View File

@@ -1,4 +1,5 @@
import {
AlertmanagertypesIncidentIOReceiverConfigDTO,
AlertmanagertypesJiraReceiverConfigDTO,
AlertmanagertypesJSMOpsReceiverConfigDTO,
AlertmanagertypesPostableChannelDTO,
@@ -9,6 +10,7 @@ import {
import {
ChannelType,
GoogleChatChannel,
IncidentIOChannel,
JiraChannel,
JsmOpsChannel,
} from './config';
@@ -168,3 +170,50 @@ export const prepareJsmOpsRequest = (
jsmops_configs: [jsmops],
};
};
const INCIDENTIO_EVENTS_PATH_PREFIX = '/v2/alert_events/http/';
// the backend enforces the same rule, this is only for a nicer error experience
export const isValidIncidentIOURL = (url: string): boolean => {
try {
const { protocol, pathname } = new URL(url);
const idx = pathname.indexOf(INCIDENTIO_EVENTS_PATH_PREFIX);
return (
protocol === 'https:' &&
idx !== -1 &&
pathname.length > idx + INCIDENTIO_EVENTS_PATH_PREFIX.length
);
} catch {
return false;
}
};
// create, update and test all send the same body shape. Optional fields are
// omitted when empty so the backend applies its defaults.
export const prepareIncidentIORequest = (
config: Partial<IncidentIOChannel>,
): AlertmanagertypesPostableChannelDTO => {
const incidentio: AlertmanagertypesIncidentIOReceiverConfigDTO = {
url: config.url || '',
token: config.token || '',
send_resolved: config.send_resolved || false,
};
if (config.title) {
incidentio.title = config.title;
}
if (config.description) {
incidentio.description = config.description;
}
const metadata = Object.fromEntries(
Object.entries(config.metadata || {}).filter(([key]) => key.trim() !== ''),
);
if (Object.keys(metadata).length > 0) {
incidentio.metadata = metadata;
}
return {
name: config.name || '',
incidentio_configs: [incidentio],
};
};

View File

@@ -25,6 +25,7 @@ import {
ChannelType,
EmailChannel,
GoogleChatChannel,
IncidentIOChannel,
JiraChannel,
JsmOpsChannel,
MsTeamsChannel,
@@ -36,9 +37,11 @@ import {
} from 'container/CreateAlertChannels/config';
import {
isValidGoogleChatWebhookURL,
isValidIncidentIOURL,
isValidJiraReopenDuration,
isValidJiraSiteURL,
prepareGoogleChatRequest,
prepareIncidentIORequest,
prepareJiraRequest,
prepareJsmOpsRequest,
} from 'container/CreateAlertChannels/utils';
@@ -66,7 +69,8 @@ function EditAlertChannels({
EmailChannel &
GoogleChatChannel &
JiraChannel &
JsmOpsChannel
JsmOpsChannel &
IncidentIOChannel
>
>({
...initialValue,
@@ -578,6 +582,61 @@ function EditAlertChannels({
t,
]);
const validateIncidentIOConfig = useCallback((): string => {
if (!selectedConfig.url || !selectedConfig.token) {
return t('incidentio_required_fields');
}
if (!isValidIncidentIOURL(selectedConfig.url)) {
return t('incidentio_url_invalid');
}
return '';
}, [selectedConfig, t]);
const onIncidentIOEditHandler = useCallback(async () => {
const validationError = validateIncidentIOConfig();
if (validationError !== '') {
notifications.error({
message: 'Error',
description: validationError,
});
return { status: 'failed', statusMessage: validationError };
}
setSavingState(true);
try {
await updateChannel({
pathParams: { id },
data: prepareIncidentIORequest(selectedConfig),
});
notifications.success({
message: 'Success',
description: t('channel_edit_done'),
});
history.replace(ROUTES.ALL_CHANNELS);
return { status: 'success', statusMessage: t('channel_edit_done') };
} catch (error) {
const apiError = notifyError(error);
return {
status: 'failed',
statusMessage: apiError.getErrorMessage() || t('channel_edit_failed'),
};
} finally {
setSavingState(false);
}
}, [
validateIncidentIOConfig,
updateChannel,
id,
selectedConfig,
notifications,
notifyError,
t,
]);
const onSaveHandler = useCallback(
async (value: ChannelType) => {
let result;
@@ -599,6 +658,8 @@ function EditAlertChannels({
result = await onJiraEditHandler();
} else if (value === ChannelType.JsmOps) {
result = await onJsmOpsEditHandler();
} else if (value === ChannelType.IncidentIO) {
result = await onIncidentIOEditHandler();
}
logEvent('Alert Channel: Save channel', {
type: value,
@@ -620,6 +681,7 @@ function EditAlertChannels({
onGoogleChatEditHandler,
onJiraEditHandler,
onJsmOpsEditHandler,
onIncidentIOEditHandler,
],
);
@@ -701,6 +763,19 @@ function EditAlertChannels({
await testChannel({ data: prepareJsmOpsRequest(selectedConfig) });
break;
}
case ChannelType.IncidentIO: {
const validationError = validateIncidentIOConfig();
if (validationError !== '') {
notifications.error({
message: 'Error',
description: validationError,
});
setTestingState(false);
return;
}
await testChannel({ data: prepareIncidentIORequest(selectedConfig) });
break;
}
default:
notifications.error({
message: 'Error',
@@ -740,6 +815,7 @@ function EditAlertChannels({
validateGoogleChatConfig,
validateJiraConfig,
validateJsmOpsConfig,
validateIncidentIOConfig,
testChannel,
prepareWebhookRequest,
preparePagerRequest,

View File

@@ -0,0 +1,172 @@
import { Dispatch, SetStateAction, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Minus, Plus } from '@signozhq/icons';
import { Button, Form, Input } from 'antd';
import { Typography } from '@signozhq/ui/typography';
import { IncidentIOChannel } from '../../CreateAlertChannels/config';
interface MetadataRow {
key: string;
value: string;
}
function IncidentIOSettings({
setSelectedConfig,
initialMetadata,
}: IncidentIOProps): JSX.Element {
const { t } = useTranslation('channels');
const [metadataRows, setMetadataRows] = useState<MetadataRow[]>(() =>
Object.entries(initialMetadata || {}).map(([key, value]) => ({
key,
value,
})),
);
const update = (patch: Partial<IncidentIOChannel>): void =>
setSelectedConfig((value) => ({ ...value, ...patch }));
const syncMetadata = (rows: MetadataRow[]): void => {
setMetadataRows(rows);
update({
metadata: Object.fromEntries(
rows
.filter((row) => row.key.trim() !== '')
.map((row) => [row.key, row.value]),
),
});
};
return (
<>
<Typography.Text
color="muted"
size="sm"
testId="incidentio-tip"
style={{ display: 'block', marginBottom: 16 }}
>
{t('incidentio_tip')}{' '}
<Typography.Link
href="https://signoz.io/docs/alerts-management/notification-channel/incidentio/"
target="_blank"
rel="noopener noreferrer"
>
{t('incidentio_tip_link')}
</Typography.Link>
</Typography.Text>
<Form.Item
name="url"
label={t('field_incidentio_url')}
help={t('help_incidentio_url')}
required
>
<Input
onChange={(event): void => update({ url: event.target.value })}
data-testid="incidentio-url-textbox"
/>
</Form.Item>
<Form.Item
name="token"
label={t('field_incidentio_token')}
help={t('help_incidentio_token')}
required
>
<Input
type="password"
onChange={(event): void => update({ token: event.target.value })}
data-testid="incidentio-token-textbox"
/>
</Form.Item>
<Form.Item
name="title"
label={t('field_incidentio_title')}
help={t('help_incidentio_title')}
>
<Input.TextArea
rows={2}
onChange={(event): void => update({ title: event.target.value })}
data-testid="incidentio-title-textarea"
/>
</Form.Item>
<Form.Item
name="description"
label={t('field_incidentio_description')}
help={t('help_incidentio_description')}
>
<Input.TextArea
rows={6}
onChange={(event): void => update({ description: event.target.value })}
data-testid="incidentio-description-textarea"
/>
</Form.Item>
<Form.Item
label={t('field_incidentio_metadata')}
help={t('help_incidentio_metadata')}
>
{metadataRows.map((row, index) => (
// rows have no stable identity beyond their position
// eslint-disable-next-line react/no-array-index-key
<div key={index} style={{ display: 'flex', gap: 8, marginBottom: 8 }}>
<Input
placeholder={t('placeholder_incidentio_metadata_key')}
value={row.key}
onChange={(event): void =>
syncMetadata(
metadataRows.map((r, i) =>
i === index ? { ...r, key: event.target.value } : r,
),
)
}
data-testid={`incidentio-metadata-key-${index}`}
/>
<Input
placeholder={t('placeholder_incidentio_metadata_value')}
value={row.value}
onChange={(event): void =>
syncMetadata(
metadataRows.map((r, i) =>
i === index ? { ...r, value: event.target.value } : r,
),
)
}
data-testid={`incidentio-metadata-value-${index}`}
/>
<Button
icon={<Minus size={14} />}
onClick={(): void =>
syncMetadata(metadataRows.filter((_, i) => i !== index))
}
data-testid={`incidentio-metadata-remove-${index}`}
/>
</div>
))}
<Button
type="dashed"
icon={<Plus size={14} />}
onClick={(): void =>
syncMetadata([...metadataRows, { key: '', value: '' }])
}
data-testid="incidentio-metadata-add"
>
{t('button_incidentio_add_metadata')}
</Button>
</Form.Item>
</>
);
}
interface IncidentIOProps {
setSelectedConfig: Dispatch<SetStateAction<Partial<IncidentIOChannel>>>;
initialMetadata?: Record<string, string>;
}
IncidentIOSettings.defaultProps = {
initialMetadata: undefined,
};
export default IncidentIOSettings;

View File

@@ -10,6 +10,7 @@ import {
ChannelType,
EmailChannel,
GoogleChatChannel,
IncidentIOChannel,
JiraChannel,
JsmOpsChannel,
OpsgenieChannel,
@@ -21,6 +22,7 @@ import history from 'lib/history';
import EmailSettings from './Settings/Email';
import GoogleChatSettings from './Settings/GoogleChat';
import IncidentIOSettings from './Settings/IncidentIo';
import JiraSettings from './Settings/Jira';
import JsmOpsSettings from './Settings/JsmOps';
import MsTeamsSettings from './Settings/MsTeams';
@@ -61,6 +63,13 @@ function FormAlertChannels({
return <JiraSettings setSelectedConfig={setSelectedConfig} />;
case ChannelType.JsmOps:
return <JsmOpsSettings setSelectedConfig={setSelectedConfig} />;
case ChannelType.IncidentIO:
return (
<IncidentIOSettings
setSelectedConfig={setSelectedConfig}
initialMetadata={initialValue?.metadata as Record<string, string>}
/>
);
case ChannelType.Opsgenie:
return <OpsgenieSettings setSelectedConfig={setSelectedConfig} />;
case ChannelType.Email:
@@ -157,6 +166,14 @@ function FormAlertChannels({
<Select.Option value="jsmops" key="jsmops" data-testid="select-option">
Jira Service Management Ops
</Select.Option>
<Select.Option
value="incidentio"
key="incidentio"
data-testid="select-option"
>
incident.io
</Select.Option>
</Select>
</Form.Item>
@@ -207,7 +224,8 @@ interface FormAlertChannelsProps {
EmailChannel &
GoogleChatChannel &
JiraChannel &
JsmOpsChannel
JsmOpsChannel &
IncidentIOChannel
>
>
>;

View File

@@ -11,6 +11,7 @@ import ROUTES from 'constants/routes';
import {
ChannelType,
GoogleChatChannel,
IncidentIOChannel,
JiraChannel,
JsmOpsChannel,
MsTeamsChannel,
@@ -69,7 +70,8 @@ function ChannelsEdit(): JSX.Element {
MsTeamsChannel &
GoogleChatChannel &
JiraChannel &
JsmOpsChannel
JsmOpsChannel &
IncidentIOChannel
>;
} => {
let channel: Partial<
@@ -79,7 +81,8 @@ function ChannelsEdit(): JSX.Element {
MsTeamsChannel &
GoogleChatChannel &
JiraChannel &
JsmOpsChannel
JsmOpsChannel &
IncidentIOChannel
> = {
name: '',
};
@@ -135,6 +138,15 @@ function ChannelsEdit(): JSX.Element {
};
}
if (value && 'incidentio_configs' in value) {
const [incidentIOConfig] = value.incidentio_configs;
channel = incidentIOConfig;
return {
type: ChannelType.IncidentIO,
channel,
};
}
if (value && 'jsmops_configs' in value) {
const [jsmopsConfig] = value.jsmops_configs;
channel = jsmopsConfig;

View File

@@ -0,0 +1,188 @@
package incidentio
import (
"bytes"
"context"
"encoding/json"
"log/slog"
"maps"
"net/http"
"strings"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/prometheus/alertmanager/notify"
"github.com/prometheus/alertmanager/template"
"github.com/prometheus/alertmanager/types"
)
const (
Integration = "incidentio"
// incident.io rejects payloads over 512 KB with a 413. Runes cap the
// description at 4 bytes each worst case (~400 KB), leaving headroom for
// the other fields.
maxDescriptionLenRunes = 100000
statusFiring = "firing"
statusResolved = "resolved"
)
// alertEvent is the body of incident.io's HTTP alert source endpoint
// (Alert Events V2 API). Title, status and deduplication_key are required;
// metadata values must be flat scalars. Repeat events for a firing key are
// dropped server-side and resolves for unknown keys are safe no-ops, so
// events are sent unconditionally.
type alertEvent struct {
Title string `json:"title"`
Description string `json:"description,omitempty"`
Status string `json:"status"`
DeduplicationKey string `json:"deduplication_key"`
SourceURL string `json:"source_url,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
}
type Notifier struct {
conf *alertmanagertypes.IncidentIOReceiverConfig
tmpl *template.Template
logger *slog.Logger
client *http.Client
retrier *notify.Retrier
templater alertmanagertypes.Templater
}
func New(conf *alertmanagertypes.IncidentIOReceiverConfig, t *template.Template, l *slog.Logger, templater alertmanagertypes.Templater) (*Notifier, error) {
if conf.HTTPConfig == nil {
return nil, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "incidentio http_config is nil")
}
client, err := notify.NewClientWithTracing(*conf.HTTPConfig, Integration)
if err != nil {
return nil, err
}
return &Notifier{
conf: conf,
tmpl: t,
logger: l,
client: client,
retrier: &notify.Retrier{RetryCodes: []int{http.StatusTooManyRequests}},
templater: templater,
}, nil
}
func (n *Notifier) Notify(ctx context.Context, as ...*types.Alert) (bool, error) {
key, err := notify.ExtractGroupKey(ctx)
if err != nil {
return false, err
}
firing := types.Alerts(as...).HasFiring()
n.logger.DebugContext(ctx, "sending incidentio notification", slog.String("group_key", key.String()), slog.Bool("firing", firing))
customTitle, customBody := alertmanagertemplate.ExtractTemplatesFromAnnotations(as)
result, err := n.templater.Expand(ctx, alertmanagertypes.ExpandRequest{
TitleTemplate: customTitle,
BodyTemplate: customBody,
DefaultTitleTemplate: n.conf.Title,
DefaultBodyTemplate: n.conf.Description,
}, as)
if err != nil {
return false, err
}
// title is required by the API; a channel title template can render empty,
// so fall back to the rule name, then to a static last resort.
title := result.Title
if strings.TrimSpace(title) == "" && len(as) > 0 {
title = string(as[0].Labels[ruletypes.LabelAlertName])
}
if strings.TrimSpace(title) == "" {
title = "SigNoz alert"
}
var parts []string
for _, body := range result.Body {
if body != "" {
parts = append(parts, body)
}
}
description := truncateRunes(strings.Join(parts, "\n\n---\n\n"), maxDescriptionLenRunes)
status := statusFiring
if !firing {
status = statusResolved
}
event := alertEvent{
Title: title,
Description: description,
Status: status,
DeduplicationKey: key.Hash(),
SourceURL: sourceURL(as),
Metadata: n.metadata(ctx, as),
}
var buf bytes.Buffer
if err := json.NewEncoder(&buf).Encode(event); err != nil {
return false, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, n.conf.URL, &buf)
if err != nil {
return false, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+string(n.conf.Token))
resp, err := n.client.Do(req) //nolint:bodyclose // notify.Drain closes the body
if err != nil {
return true, notify.RedactURL(err)
}
defer notify.Drain(resp)
shouldRetry, err := n.retrier.Check(resp.StatusCode, resp.Body)
if err != nil {
return shouldRetry, notify.NewErrorWithReason(notify.GetFailureReasonFromStatusCode(resp.StatusCode), err)
}
return shouldRetry, nil
}
// metadata copies the group's common labels wholesale (the Opsgenie details
// precedent), so severity, ruleId and any user-defined rule labels arrive as
// flat strings ready for incident.io attribute mapping. Channel-configured
// pairs are template-expanded and laid on top (channel wins on key clash);
// a value that fails to expand is sent raw so delivery never breaks on it.
func (n *Notifier) metadata(ctx context.Context, as []*types.Alert) map[string]string {
data := notify.GetTemplateData(ctx, n.tmpl, as, n.logger)
out := make(map[string]string, len(data.CommonLabels)+len(n.conf.Metadata))
maps.Copy(out, data.CommonLabels)
for k, v := range n.conf.Metadata {
expanded, err := n.tmpl.ExecuteTextString(v, data)
if err != nil {
n.logger.WarnContext(ctx, "failed to expand incidentio metadata value, sending it raw", slog.String("metadata_key", k))
expanded = v
}
out[k] = expanded
}
if len(out) == 0 {
return nil
}
return out
}
// sourceURL returns the per-rule SigNoz link from the ruleSource label, which
// is identical for every alert in the group.
func sourceURL(as []*types.Alert) string {
if len(as) == 0 {
return ""
}
return string(as[0].Labels[ruletypes.LabelRuleSource])
}
func truncateRunes(s string, max int) string {
runes := []rune(s)
if len(runes) <= max {
return s
}
return string(runes[:max-1]) + "…"
}

View File

@@ -0,0 +1,216 @@
package incidentio
import (
"context"
"encoding/json"
"log/slog"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/prometheus/alertmanager/notify"
"github.com/prometheus/alertmanager/notify/test"
"github.com/prometheus/alertmanager/types"
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type mockIncidentIO struct {
srv *httptest.Server
mu sync.Mutex
events []alertEvent
auths []string
status int
}
func newMockIncidentIO(t *testing.T) *mockIncidentIO {
t.Helper()
m := &mockIncidentIO{status: http.StatusAccepted}
m.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var ev alertEvent
_ = json.NewDecoder(r.Body).Decode(&ev)
m.mu.Lock()
m.events = append(m.events, ev)
m.auths = append(m.auths, r.Header.Get("Authorization"))
status := m.status
m.mu.Unlock()
w.WriteHeader(status)
if status == http.StatusAccepted {
_, _ = w.Write([]byte(`{"status":"accepted","message":"Event accepted for processing","deduplication_key":"` + ev.DeduplicationKey + `"}`))
} else {
_, _ = w.Write([]byte(`{"type":"validation_error","status":422,"errors":[{"code":"is_required","message":"Deduplication key is required"}]}`))
}
}))
t.Cleanup(m.srv.Close)
return m
}
func (m *mockIncidentIO) lastEvent(t *testing.T) alertEvent {
t.Helper()
m.mu.Lock()
defer m.mu.Unlock()
require.NotEmpty(t, m.events)
return m.events[len(m.events)-1]
}
func newNotifier(t *testing.T, m *mockIncidentIO) *Notifier {
t.Helper()
tmpl := test.CreateTmpl(t)
n, err := New(&alertmanagertypes.IncidentIOReceiverConfig{
URL: m.srv.URL + "/v2/alert_events/http/src-1",
Token: "tok-1",
Title: alertmanagertypes.DefaultIncidentIOTitleTemplate,
Description: alertmanagertypes.DefaultIncidentIODescriptionTemplate,
HTTPConfig: &commoncfg.HTTPClientConfig{},
}, tmpl, slog.New(slog.DiscardHandler), alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler)))
require.NoError(t, err)
return n
}
func alert(firing bool) *types.Alert {
a := &types.Alert{Alert: model.Alert{
Labels: model.LabelSet{
"alertname": "HighCPU",
"severity": "critical",
"ruleSource": "https://signoz.example/alerts/edit?ruleId=1",
},
Annotations: model.LabelSet{"summary": "cpu high", "related_logs": "https://signoz.example/logs?q=1"},
GeneratorURL: "https://signoz.example/alerts/edit?ruleId=1",
StartsAt: time.Now().Add(-time.Minute),
}}
if firing {
a.EndsAt = time.Now().Add(time.Hour)
} else {
a.EndsAt = time.Now().Add(-time.Minute)
}
return a
}
func ctx() context.Context {
return notify.WithGroupKey(context.Background(), "test-incidentio")
}
func TestNotifyFiringEvent(t *testing.T) {
m := newMockIncidentIO(t)
retry, err := newNotifier(t, m).Notify(ctx(), alert(true))
require.NoError(t, err)
assert.False(t, retry)
ev := m.lastEvent(t)
assert.Equal(t, "[FIRING:1] HighCPU", ev.Title)
assert.Equal(t, "firing", ev.Status)
assert.NotEmpty(t, ev.DeduplicationKey)
assert.Equal(t, "https://signoz.example/alerts/edit?ruleId=1", ev.SourceURL)
assert.Contains(t, ev.Description, "**Alert:** HighCPU (critical)")
assert.Contains(t, ev.Description, "**Summary:** cpu high")
assert.Contains(t, ev.Description, "[View in SigNoz](https://signoz.example/alerts/edit?ruleId=1)")
assert.Contains(t, ev.Description, "[View related logs](https://signoz.example/logs?q=1)")
assert.Equal(t, map[string]string{
"alertname": "HighCPU",
"severity": "critical",
"ruleSource": "https://signoz.example/alerts/edit?ruleId=1",
}, ev.Metadata)
assert.Equal(t, "Bearer tok-1", m.auths[0])
}
func TestNotifyResolvedEventReusesDedupKey(t *testing.T) {
m := newMockIncidentIO(t)
n := newNotifier(t, m)
_, err := n.Notify(ctx(), alert(true))
require.NoError(t, err)
firingKey := m.lastEvent(t).DeduplicationKey
_, err = n.Notify(ctx(), alert(false))
require.NoError(t, err)
ev := m.lastEvent(t)
assert.Equal(t, "resolved", ev.Status)
assert.Equal(t, firingKey, ev.DeduplicationKey)
}
func TestNotifyPermanentFailureDoesNotRetry(t *testing.T) {
m := newMockIncidentIO(t)
m.status = http.StatusUnprocessableEntity
retry, err := newNotifier(t, m).Notify(ctx(), alert(true))
require.Error(t, err)
assert.False(t, retry)
assert.Contains(t, err.Error(), "Deduplication key is required") // response body surfaces to the user
}
func TestNotifyRateLimitRetries(t *testing.T) {
m := newMockIncidentIO(t)
m.status = http.StatusTooManyRequests
retry, err := newNotifier(t, m).Notify(ctx(), alert(true))
require.Error(t, err)
assert.True(t, retry)
}
func TestNotifyMergesChannelMetadata(t *testing.T) {
m := newMockIncidentIO(t)
tmpl := test.CreateTmpl(t)
n, err := New(&alertmanagertypes.IncidentIOReceiverConfig{
URL: m.srv.URL + "/v2/alert_events/http/src-1",
Token: "tok-1",
Title: alertmanagertypes.DefaultIncidentIOTitleTemplate,
Description: alertmanagertypes.DefaultIncidentIODescriptionTemplate,
HTTPConfig: &commoncfg.HTTPClientConfig{},
Metadata: map[string]string{
"env": "prod",
"sev": "{{ .CommonLabels.severity }}",
"alertname": "channel-wins",
"broken": "{{ .Nope",
},
}, tmpl, slog.New(slog.DiscardHandler), alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler)))
require.NoError(t, err)
_, err = n.Notify(ctx(), alert(true))
require.NoError(t, err) // a broken metadata template must not fail delivery
md := m.lastEvent(t).Metadata
assert.Equal(t, "prod", md["env"])
assert.Equal(t, "critical", md["sev"]) // values are template-expanded
assert.Equal(t, "channel-wins", md["alertname"]) // channel overrides the rule label
assert.Equal(t, "{{ .Nope", md["broken"]) // unexpandable value sent raw
assert.Equal(t, "critical", md["severity"]) // rule labels still present
}
func TestNotifyEmptyTitleFallsBackToRuleName(t *testing.T) {
m := newMockIncidentIO(t)
tmpl := test.CreateTmpl(t)
n, err := New(&alertmanagertypes.IncidentIOReceiverConfig{
URL: m.srv.URL + "/v2/alert_events/http/src-1",
Token: "tok-1",
Title: `{{ .CommonLabels.nonexistent }}`,
Description: alertmanagertypes.DefaultIncidentIODescriptionTemplate,
HTTPConfig: &commoncfg.HTTPClientConfig{},
}, tmpl, slog.New(slog.DiscardHandler), alertmanagertemplate.New(tmpl, slog.New(slog.DiscardHandler)))
require.NoError(t, err)
_, err = n.Notify(ctx(), alert(true))
require.NoError(t, err)
assert.Equal(t, "HighCPU", m.lastEvent(t).Title)
}
func TestNotifyTruncatesLongDescription(t *testing.T) {
m := newMockIncidentIO(t)
a := alert(true)
a.Annotations["description"] = model.LabelValue(strings.Repeat("x", maxDescriptionLenRunes+1000))
_, err := newNotifier(t, m).Notify(ctx(), a)
require.NoError(t, err)
desc := []rune(m.lastEvent(t).Description)
assert.LessOrEqual(t, len(desc), maxDescriptionLenRunes)
assert.Equal(t, '…', desc[len(desc)-1])
}

View File

@@ -6,6 +6,7 @@ import (
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/email"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/googlechat"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/incidentio"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/jira"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/jsmops"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/msteamsv2"
@@ -30,6 +31,7 @@ var customNotifierIntegrations = []string{
googlechat.Integration,
jira.Integration,
jsmops.Integration,
incidentio.Integration,
}
func NewReceiverIntegrations(nc *alertmanagertypes.Receiver, tmpl *template.Template, logger *slog.Logger, templater alertmanagertypes.Templater) ([]notify.Integration, error) {
@@ -95,6 +97,11 @@ func NewReceiverIntegrations(nc *alertmanagertypes.Receiver, tmpl *template.Temp
return jsmops.New(c, tmpl, l, templater, true)
})
}
for i, c := range nc.IncidentIOConfigs {
add(incidentio.Integration, i, c, func(l *slog.Logger) (notify.Notifier, error) {
return incidentio.New(c, tmpl, l, templater)
})
}
if errs.Len() > 0 {
return nil, &errs

View File

@@ -75,7 +75,7 @@ func (store *config) CreateChannel(ctx context.Context, channel *alertmanagertyp
NewInsert().
Model(channel).
Exec(ctx); err != nil {
return err
return store.sqlstore.WrapAlreadyExistsErrf(err, alertmanagertypes.ErrCodeAlertmanagerChannelAlreadyExists, "channel with name %q already exists", channel.Name)
}
return nil

View File

@@ -0,0 +1,71 @@
package sqlalertmanagerstore
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
func TestCreateChannelRejectsDuplicateNameInSameOrg(t *testing.T) {
sqlstore := newTestStore(t)
_, err := sqlstore.BunDB().NewCreateTable().
Model((*alertmanagertypes.Channel)(nil)).
IfNotExists().
Exec(t.Context())
require.NoError(t, err)
_, err = sqlstore.BunDB().NewCreateIndex().
Model((*alertmanagertypes.Channel)(nil)).
Index("notification_channel_org_id_name_idx").
Column("org_id", "name").
Unique().
Exec(t.Context())
require.NoError(t, err)
store := NewConfigStore(sqlstore)
orgID := valuer.GenerateUUID().StringValue()
now := time.Now().UTC()
firstChannel := &alertmanagertypes.Channel{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
TimeAuditable: types.TimeAuditable{CreatedAt: now, UpdatedAt: now},
Name: "shared-name",
DisplayName: "First Channel",
Type: "slack",
Data: `{"name":"First Channel","slack_configs":[{"api_url":"https://hooks.slack.com/services/first"}]}`,
OrgID: orgID,
}
require.NoError(t, store.CreateChannel(t.Context(), firstChannel))
duplicateChannel := &alertmanagertypes.Channel{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
TimeAuditable: types.TimeAuditable{CreatedAt: now, UpdatedAt: now},
Name: "shared-name",
DisplayName: "Second Channel",
Type: "slack",
Data: `{"name":"Second Channel","slack_configs":[{"api_url":"https://hooks.slack.com/services/second"}]}`,
OrgID: orgID,
}
err = store.CreateChannel(t.Context(), duplicateChannel)
require.Error(t, err)
assert.True(t, errors.Ast(err, errors.TypeAlreadyExists))
otherOrgChannel := &alertmanagertypes.Channel{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
TimeAuditable: types.TimeAuditable{CreatedAt: now, UpdatedAt: now},
Name: "shared-name",
DisplayName: "Second Channel",
Type: "slack",
Data: `{"name":"Second Channel","slack_configs":[{"api_url":"https://hooks.slack.com/services/second"}]}`,
OrgID: valuer.GenerateUUID().StringValue(),
}
assert.NoError(t, store.CreateChannel(t.Context(), otherOrgChannel))
}

View File

@@ -187,7 +187,7 @@ func (provider *provider) DeleteChannelByID(ctx context.Context, orgID string, c
}
// Check if channel is referenced by any route policy (rule-based or policy-based)
policies, err := provider.notificationManager.GetRoutePoliciesByChannel(ctx, orgID, channel.Name)
policies, err := provider.notificationManager.GetRoutePoliciesByChannel(ctx, orgID, channel.DisplayName)
if err != nil {
return err
}
@@ -198,7 +198,7 @@ func (provider *provider) DeleteChannelByID(ctx context.Context, orgID string, c
}
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"channel %q cannot be deleted because it is used by the following routing policies: %v",
channel.Name, names)
channel.DisplayName, names)
}
config, err := provider.configStore.Get(ctx, orgID)
@@ -206,7 +206,7 @@ func (provider *provider) DeleteChannelByID(ctx context.Context, orgID string, c
return err
}
if err := config.DeleteReceiver(channel.Name); err != nil {
if err := config.DeleteReceiver(channel.DisplayName); err != nil {
return err
}

View File

@@ -97,23 +97,57 @@ func (provider *provider) addGatewayRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_keys/{keyId}/limits", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.CreateIngestionKeyLimit), handler.OpenAPIDef{
if err := router.Handle("/api/v2/gateway/ingestion_keys/{keyId}", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.GetIngestionKey), handler.OpenAPIDef{
ID: "GetIngestionKey",
Tags: []string{"gateway"},
Summary: "Get ingestion key for workspace",
Description: "This endpoint returns an ingestion key for the workspace",
Request: nil,
RequestContentType: "",
Response: new(gatewaytypes.IngestionKey),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_keys/{keyId}/limits", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.DeprecatedCreateIngestionKeyLimit), handler.OpenAPIDef{
ID: "CreateIngestionKeyLimit",
Tags: []string{"gateway"},
Summary: "Create limit for the ingestion key",
Description: "This endpoint creates an ingestion key limit",
Request: new(gatewaytypes.PostableIngestionKeyLimit),
Request: new(gatewaytypes.DeprecatedPostableIngestionKeyLimit),
RequestContentType: "application/json",
Response: new(gatewaytypes.GettableCreatedIngestionKeyLimit),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{},
Deprecated: false,
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_keys/{keyId}/limits", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.GetIngestionKeyLimits), handler.OpenAPIDef{
ID: "GetIngestionKeyLimits",
Tags: []string{"gateway"},
Summary: "Get limits for the ingestion key",
Description: "This endpoint returns the ingestion limits for an ingestion key",
Request: nil,
RequestContentType: "",
Response: new([]gatewaytypes.Limit),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_keys/limits/{limitId}", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.UpdateIngestionKeyLimit), handler.OpenAPIDef{
ID: "UpdateIngestionKeyLimit",
Tags: []string{"gateway"},
@@ -125,7 +159,7 @@ func (provider *provider) addGatewayRoutes(router *mux.Router) error {
ResponseContentType: "",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{},
Deprecated: false,
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPatch).GetError(); err != nil {
return err
@@ -142,6 +176,74 @@ func (provider *provider) addGatewayRoutes(router *mux.Router) error {
ResponseContentType: "",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{},
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodDelete).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_limits", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.CreateIngestionKeyLimit), handler.OpenAPIDef{
ID: "CreateIngestionLimit",
Tags: []string{"gateway"},
Summary: "Create ingestion limit",
Description: "This endpoint creates an ingestion limit for the ingestion key referenced by keyId",
Request: new(gatewaytypes.PostableIngestionKeyLimit),
RequestContentType: "application/json",
Response: new(types.Identifiable),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_limits/{limitId}", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.GetIngestionKeyLimit), handler.OpenAPIDef{
ID: "GetIngestionLimit",
Tags: []string{"gateway"},
Summary: "Get ingestion limit",
Description: "This endpoint returns an ingestion limit",
Request: nil,
RequestContentType: "",
Response: new(gatewaytypes.Limit),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_limits/{limitId}", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.UpdateIngestionKeyLimit), handler.OpenAPIDef{
ID: "UpdateIngestionLimit",
Tags: []string{"gateway"},
Summary: "Update ingestion limit",
Description: "This endpoint updates an ingestion limit",
Request: new(gatewaytypes.UpdatableIngestionKeyLimit),
RequestContentType: "application/json",
Response: nil,
ResponseContentType: "",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPatch).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/gateway/ingestion_limits/{limitId}", handler.New(provider.authzMiddleware.EditAccess(provider.gatewayHandler.DeleteIngestionKeyLimit), handler.OpenAPIDef{
ID: "DeleteIngestionLimit",
Tags: []string{"gateway"},
Summary: "Delete ingestion limit",
Description: "This endpoint deletes an ingestion limit",
Request: nil,
RequestContentType: "",
Response: nil,
ResponseContentType: "",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodDelete).GetError(); err != nil {

View File

@@ -25,6 +25,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/preference"
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/modules/quickfilter"
"github.com/SigNoz/signoz/pkg/modules/rawdataexport"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/savedview"
@@ -84,6 +85,8 @@ type provider struct {
llmPricingRuleHandler llmpricingrule.Handler
statsHandler statsreporter.Handler
savedViewHandler savedview.Handler
quickFilterModule quickfilter.Module
quickFilterHandler quickfilter.Handler
}
func NewFactory(
@@ -124,6 +127,8 @@ func NewFactory(
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
quickFilterModule quickfilter.Module,
quickFilterHandler quickfilter.Handler,
) factory.ProviderFactory[apiserver.APIServer, apiserver.Config] {
return factory.NewProviderFactory(factory.MustNewName("signoz"), func(ctx context.Context, providerSettings factory.ProviderSettings, config apiserver.Config) (apiserver.APIServer, error) {
return newProvider(
@@ -167,6 +172,8 @@ func NewFactory(
rulerHandler,
statsHandler,
savedViewHandler,
quickFilterModule,
quickFilterHandler,
)
})
}
@@ -212,6 +219,8 @@ func newProvider(
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
savedViewHandler savedview.Handler,
quickFilterModule quickfilter.Module,
quickFilterHandler quickfilter.Handler,
) (apiserver.APIServer, error) {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/apiserver/signozapiserver")
router := mux.NewRouter().UseEncodedPath()
@@ -256,6 +265,8 @@ func newProvider(
llmPricingRuleHandler: llmPricingRuleHandler,
statsHandler: statsHandler,
savedViewHandler: savedViewHandler,
quickFilterModule: quickFilterModule,
quickFilterHandler: quickFilterHandler,
}
provider.authzMiddleware = middleware.NewAuthZ(settings.Logger(), orgGetter, authzService)
@@ -404,6 +415,10 @@ func (provider *provider) AddToRouter(router *mux.Router) error {
return err
}
if err := provider.addQuickFilterRoutes(router); err != nil {
return err
}
return nil
}

View File

@@ -0,0 +1,120 @@
package signozapiserver
import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/SigNoz/signoz/pkg/types/quickfiltertypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
func (provider *provider) addQuickFilterRoutes(router *mux.Router) error {
if err := router.Handle("/api/v2/quick_filters", handler.New(
provider.authzMiddleware.CheckResources(provider.quickFilterHandler.ListQuickFiltersV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "ListQuickFilters",
Tags: []string{"quick_filter"},
Summary: "List quick filters",
Description: "Returns the org's quick filters for every source, each filter as a telemetry field key.",
Request: nil,
RequestContentType: "",
Response: make([]*quickfiltertypes.SourceFilters, 0),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceQuickFilter.Scope(coretypes.VerbList)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceQuickFilter,
Verb: coretypes.VerbList,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}),
)).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/quick_filters/{source}", handler.New(
provider.authzMiddleware.CheckResources(provider.quickFilterHandler.GetQuickFiltersV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetQuickFilters",
Tags: []string{"quick_filter"},
Summary: "Get a source's quick filters",
Description: "Returns the org's quick filters for one source, each filter as a telemetry field key.",
Request: nil,
RequestContentType: "",
Response: new(quickfiltertypes.SourceFilters),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceQuickFilter.Scope(coretypes.VerbRead)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceQuickFilter,
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
ID: coretypes.PathParam("source"),
Selector: provider.quickFilterSelector,
}),
)).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/quick_filters/{source}", handler.New(
provider.authzMiddleware.CheckResources(provider.quickFilterHandler.UpdateQuickFiltersV2, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "UpdateQuickFilters",
Tags: []string{"quick_filter"},
Summary: "Update quick filters",
Description: "Replaces the org's quick filters for the source named in the path.",
Request: new(quickfiltertypes.UpdatableQuickFilters),
RequestContentType: "application/json",
Response: nil,
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceQuickFilter.Scope(coretypes.VerbUpdate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceQuickFilter,
Verb: coretypes.VerbUpdate,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.PathParam("source"),
Selector: provider.quickFilterSelector,
}),
)).Methods(http.MethodPut).GetError(); err != nil {
return err
}
return nil
}
func (provider *provider) quickFilterSelector(ctx context.Context, resource coretypes.Resource, source string, orgID valuer.UUID) ([]coretypes.Selector, error) {
validatedSource, err := quickfiltertypes.NewSource(source)
if err != nil {
return nil, err
}
// A source can have no stored row yet: GET serves it as empty and PUT
// creates it, so only the wildcard grant applies until the row exists.
quickFilter, err := provider.quickFilterModule.Get(ctx, orgID, validatedSource)
if err != nil {
if errors.Ast(err, errors.TypeNotFound) {
return []coretypes.Selector{resource.Type().MustSelector(coretypes.WildCardSelectorString)}, nil
}
return nil, err
}
return []coretypes.Selector{
resource.Type().MustSelector(quickFilter.ID.StringValue()),
resource.Type().MustSelector(coretypes.WildCardSelectorString),
}, nil
}

View File

@@ -22,6 +22,9 @@ type Gateway interface {
// Search Ingestion Keys by Name (this is supposed to be for the current user but for now in gateway code this is ignoring the consumer user)
SearchIngestionKeysByName(ctx context.Context, orgID valuer.UUID, name string, page, perPage int) (*gatewaytypes.GettableIngestionKeys, error)
// Get Ingestion Key
GetIngestionKey(ctx context.Context, orgID valuer.UUID, keyID string) (*gatewaytypes.IngestionKey, error)
// Create Ingestion Key
CreateIngestionKey(ctx context.Context, orgID valuer.UUID, name string, tags []string, expiresAt time.Time) (*gatewaytypes.GettableCreatedIngestionKey, error)
@@ -34,6 +37,12 @@ type Gateway interface {
// Create Ingestion Key Limit
CreateIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, keyID string, signal string, limitConfig gatewaytypes.LimitConfig, tags []string) (*gatewaytypes.GettableCreatedIngestionKeyLimit, error)
// Get Ingestion Key Limit
GetIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, limitID string) (*gatewaytypes.Limit, error)
// Get Ingestion Key Limits
GetIngestionKeyLimits(ctx context.Context, orgID valuer.UUID, keyID string) ([]gatewaytypes.Limit, error)
// Update Ingestion Key Limit
UpdateIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, limitID string, limitConfig gatewaytypes.LimitConfig, tags []string) error
@@ -46,6 +55,8 @@ type Handler interface {
SearchIngestionKeys(http.ResponseWriter, *http.Request)
GetIngestionKey(http.ResponseWriter, *http.Request)
CreateIngestionKey(http.ResponseWriter, *http.Request)
UpdateIngestionKey(http.ResponseWriter, *http.Request)
@@ -54,7 +65,13 @@ type Handler interface {
CreateIngestionKeyLimit(http.ResponseWriter, *http.Request)
GetIngestionKeyLimit(http.ResponseWriter, *http.Request)
GetIngestionKeyLimits(http.ResponseWriter, *http.Request)
UpdateIngestionKeyLimit(http.ResponseWriter, *http.Request)
DeleteIngestionKeyLimit(http.ResponseWriter, *http.Request)
DeprecatedCreateIngestionKeyLimit(http.ResponseWriter, *http.Request)
}

View File

@@ -1,11 +1,11 @@
package gateway
import (
"encoding/json"
"net/http"
"strconv"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/gatewaytypes"
@@ -111,8 +111,8 @@ func (handler *handler) CreateIngestionKey(rw http.ResponseWriter, r *http.Reque
orgID := valuer.MustNewUUID(claims.OrgID)
var req gatewaytypes.PostableIngestionKey
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid request body"))
if err := binding.JSON.BindBody(r.Body, &req); err != nil {
render.Error(rw, err)
return
}
@@ -143,8 +143,8 @@ func (handler *handler) UpdateIngestionKey(rw http.ResponseWriter, r *http.Reque
}
var req gatewaytypes.PostableIngestionKey
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid request body"))
if err := binding.JSON.BindBody(r.Body, &req); err != nil {
render.Error(rw, err)
return
}
@@ -183,7 +183,7 @@ func (handler *handler) DeleteIngestionKey(rw http.ResponseWriter, r *http.Reque
render.Success(rw, http.StatusNoContent, nil)
}
func (handler *handler) CreateIngestionKeyLimit(rw http.ResponseWriter, r *http.Request) {
func (handler *handler) DeprecatedCreateIngestionKeyLimit(rw http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(ctx)
@@ -200,9 +200,9 @@ func (handler *handler) CreateIngestionKeyLimit(rw http.ResponseWriter, r *http.
return
}
var req gatewaytypes.PostableIngestionKeyLimit
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid request body"))
var req gatewaytypes.DeprecatedPostableIngestionKeyLimit
if err := binding.JSON.BindBody(r.Body, &req); err != nil {
render.Error(rw, err)
return
}
@@ -233,8 +233,8 @@ func (handler *handler) UpdateIngestionKeyLimit(rw http.ResponseWriter, r *http.
}
var req gatewaytypes.UpdatableIngestionKeyLimit
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid request body"))
if err := binding.JSON.BindBody(r.Body, &req); err != nil {
render.Error(rw, err)
return
}
@@ -273,6 +273,115 @@ func (handler *handler) DeleteIngestionKeyLimit(rw http.ResponseWriter, r *http.
render.Success(rw, http.StatusNoContent, nil)
}
func (handler *handler) CreateIngestionKeyLimit(rw http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
var req gatewaytypes.PostableIngestionKeyLimit
if err := binding.JSON.BindBody(r.Body, &req); err != nil {
render.Error(rw, err)
return
}
if req.KeyID == "" {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "keyId is required"))
return
}
response, err := handler.gateway.CreateIngestionKeyLimit(ctx, orgID, req.KeyID, req.Signal, req.Config, req.Tags)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusCreated, response)
}
func (handler *handler) GetIngestionKeyLimit(rw http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
limitID := mux.Vars(r)["limitId"]
if limitID == "" {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "limitId is required"))
return
}
response, err := handler.gateway.GetIngestionKeyLimit(ctx, orgID, limitID)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, response)
}
func (handler *handler) GetIngestionKey(rw http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keyID := mux.Vars(r)["keyId"]
if keyID == "" {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "keyId is required"))
return
}
response, err := handler.gateway.GetIngestionKey(ctx, orgID, keyID)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, response)
}
func (handler *handler) GetIngestionKeyLimits(rw http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keyID := mux.Vars(r)["keyId"]
if keyID == "" {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "keyId is required"))
return
}
response, err := handler.gateway.GetIngestionKeyLimits(ctx, orgID, keyID)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, response)
}
func parseIntWithDefaultValue(value string, defaultValue int) (int, error) {
if value == "" {
return defaultValue, nil

View File

@@ -31,6 +31,10 @@ func (p *provider) SearchIngestionKeysByName(_ context.Context, _ valuer.UUID, _
return nil, errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}
func (p *provider) GetIngestionKey(_ context.Context, _ valuer.UUID, _ string) (*gatewaytypes.IngestionKey, error) {
return nil, errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}
func (p *provider) CreateIngestionKey(_ context.Context, _ valuer.UUID, _ string, _ []string, _ time.Time) (*gatewaytypes.GettableCreatedIngestionKey, error) {
return nil, errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}
@@ -47,6 +51,14 @@ func (p *provider) CreateIngestionKeyLimit(_ context.Context, _ valuer.UUID, _ s
return nil, errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}
func (p *provider) GetIngestionKeyLimit(_ context.Context, _ valuer.UUID, _ string) (*gatewaytypes.Limit, error) {
return nil, errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}
func (p *provider) GetIngestionKeyLimits(_ context.Context, _ valuer.UUID, _ string) ([]gatewaytypes.Limit, error) {
return nil, errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}
func (p *provider) UpdateIngestionKeyLimit(_ context.Context, _ valuer.UUID, _ string, _ gatewaytypes.LimitConfig, _ []string) error {
return errors.New(errors.TypeUnsupported, gateway.ErrCodeGatewayUnsupported, "unsupported call")
}

View File

@@ -2,10 +2,12 @@ package implaiobservability
import (
"context"
"log/slog"
"net/http"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/modules/aiobservability"
@@ -17,11 +19,13 @@ import (
)
type handler struct {
settings factory.ScopedProviderSettings
telemetryMetadataStore telemetrytypes.MetadataStore
}
func NewHandler(telemetryMetadataStore telemetrytypes.MetadataStore) aiobservability.Handler {
func NewHandler(providerSettings factory.ProviderSettings, telemetryMetadataStore telemetrytypes.MetadataStore) aiobservability.Handler {
return &handler{
settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/modules/aiobservability/implaiobservability"),
telemetryMetadataStore: telemetryMetadataStore,
}
}
@@ -66,13 +70,6 @@ func (handler *handler) GetFieldsValues(rw http.ResponseWriter, req *http.Reques
ctx, cancel := context.WithTimeout(req.Context(), 10*time.Second)
defer cancel()
// binding ignores query params the struct does not declare, so an unsupported
// existingQuery would silently return values it did not narrow
if req.URL.Query().Has("existingQuery") {
render.Error(rw, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "existingQuery is not supported"))
return
}
var params aiobservabilitytypes.PostableFieldValueParams
if err := binding.Query.BindQuery(req.URL.Query(), &params); err != nil {
render.Error(rw, err)
@@ -84,18 +81,32 @@ func (handler *handler) GetFieldsValues(rw http.ResponseWriter, req *http.Reques
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
scopedQuery, err := aitelemetryschema.ScopedExistingQuery(params.ExistingQuery)
if err != nil {
handler.settings.Logger().WarnContext(ctx, "dropping unparseable existing query", slog.String("query", params.ExistingQuery), errors.Attr(err))
}
params.ExistingQuery = scopedQuery
fieldValueSelector := aiobservabilitytypes.NewFieldValueSelectorFromPostableFieldValueParams(params)
values := &telemetrytypes.TelemetryFieldValues{}
complete := true
// the trace context names the computed per-trace aggregates, which are never ingested
if fieldValueSelector.FieldContext != telemetrytypes.FieldContextTrace {
values, complete, err = handler.telemetryMetadataStore.GetAllValues(ctx, valuer.MustNewUUID(claims.OrgID), fieldValueSelector)
values, complete, err = handler.telemetryMetadataStore.GetAllValues(ctx, orgID, fieldValueSelector)
if err != nil {
render.Error(rw, err)
return
}
// related values are best-effort: on failure the plain values still serve the filter bar
relatedValues, relatedComplete, err := handler.telemetryMetadataStore.GetRelatedValues(ctx, orgID, fieldValueSelector)
if err != nil {
relatedValues = []string{}
}
values.RelatedValues = relatedValues
complete = complete && relatedComplete
}
render.Success(rw, http.StatusOK, &telemetrytypes.GettableFieldValues{

View File

@@ -4,10 +4,13 @@ import (
"encoding/json"
"net/http"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/modules/quickfilter"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/quickfiltertypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
@@ -20,6 +23,13 @@ func NewHandler(module quickfilter.Module) quickfilter.Handler {
return &handler{module: module}
}
// legacySourceFilters is the v1 API shape: filters as v3 attribute keys,
// with the source still spelled "signal" on the wire.
type legacySourceFilters struct {
Source quickfiltertypes.Source `json:"signal"`
Filters []v3.AttributeKey `json:"filters"`
}
func (handler *handler) GetQuickFilters(rw http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
@@ -27,13 +37,41 @@ func (handler *handler) GetQuickFilters(rw http.ResponseWriter, r *http.Request)
return
}
filters, err := handler.module.GetQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID))
filters, err := handler.module.GetQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), quickfiltertypes.Source{})
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, filters)
legacyFilters := make([]*legacySourceFilters, 0, len(filters))
for _, sourceFilters := range filters {
legacyFilters = append(legacyFilters, newLegacySourceFilters(sourceFilters))
}
render.Success(rw, http.StatusOK, legacyFilters)
}
func (handler *handler) GetSourceFilters(rw http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(rw, err)
return
}
source := mux.Vars(r)["signal"]
validatedSource, err := quickfiltertypes.NewSource(source)
if err != nil {
render.Error(rw, err)
return
}
filters, err := handler.module.GetQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), validatedSource)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, newLegacySourceFilters(handler.sourceFiltersOrEmpty(filters, validatedSource)))
}
func (handler *handler) UpdateQuickFilters(rw http.ResponseWriter, r *http.Request) {
@@ -43,14 +81,19 @@ func (handler *handler) UpdateQuickFilters(rw http.ResponseWriter, r *http.Reque
return
}
var req quickfiltertypes.UpdatableQuickFilters
decodeErr := json.NewDecoder(r.Body).Decode(&req)
if decodeErr != nil {
render.Error(rw, decodeErr)
var req legacySourceFilters
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, err)
return
}
err = handler.module.UpdateQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), req.Signal, req.Filters)
fieldKeys, err := newTelemetryFieldKeysFromLegacy(req.Source, req.Filters)
if err != nil {
render.Error(rw, err)
return
}
err = handler.module.UpsertQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), req.Source, fieldKeys)
if err != nil {
render.Error(rw, err)
return
@@ -59,21 +102,14 @@ func (handler *handler) UpdateQuickFilters(rw http.ResponseWriter, r *http.Reque
render.Success(rw, http.StatusNoContent, nil)
}
func (handler *handler) GetSignalFilters(rw http.ResponseWriter, r *http.Request) {
func (handler *handler) ListQuickFiltersV2(rw http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(rw, err)
return
}
signal := mux.Vars(r)["signal"]
validatedSignal, err := quickfiltertypes.NewSignal(signal)
if err != nil {
render.Error(rw, err)
return
}
filters, err := handler.module.GetSignalFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), validatedSignal)
filters, err := handler.module.GetQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), quickfiltertypes.Source{})
if err != nil {
render.Error(rw, err)
return
@@ -81,3 +117,141 @@ func (handler *handler) GetSignalFilters(rw http.ResponseWriter, r *http.Request
render.Success(rw, http.StatusOK, filters)
}
func (handler *handler) UpdateQuickFiltersV2(rw http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(rw, err)
return
}
source := mux.Vars(r)["source"]
validatedSource, err := quickfiltertypes.NewSource(source)
if err != nil {
render.Error(rw, err)
return
}
var req quickfiltertypes.UpdatableQuickFilters
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, err)
return
}
err = handler.module.UpsertQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), validatedSource, req.Filters)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusNoContent, nil)
}
func (handler *handler) GetQuickFiltersV2(rw http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(rw, err)
return
}
source := mux.Vars(r)["source"]
validatedSource, err := quickfiltertypes.NewSource(source)
if err != nil {
render.Error(rw, err)
return
}
filters, err := handler.module.GetQuickFilters(r.Context(), valuer.MustNewUUID(claims.OrgID), validatedSource)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, handler.sourceFiltersOrEmpty(filters, validatedSource))
}
// sourceFiltersOrEmpty keeps the single-source response contract: a source
// with no stored filters is served as an empty filter list, not an error.
func (handler *handler) sourceFiltersOrEmpty(filters []*quickfiltertypes.SourceFilters, source quickfiltertypes.Source) *quickfiltertypes.SourceFilters {
if len(filters) == 0 {
return quickfiltertypes.NewSourceFiltersFromSource(source)
}
return filters[0]
}
// newTelemetryFieldKeysFromLegacy converts a v1 write payload with the same
// normalizations as the storage migration: alias contexts, numerics to number.
// The v1 shape carries no per filter signal, so meter keys get it restored.
func newTelemetryFieldKeysFromLegacy(source quickfiltertypes.Source, filters []v3.AttributeKey) ([]telemetrytypes.TelemetryFieldKey, error) {
var fieldSignal telemetrytypes.Signal
if source == quickfiltertypes.SourceMeter {
fieldSignal = telemetrytypes.SignalMetrics
}
fieldKeys := make([]telemetrytypes.TelemetryFieldKey, 0, len(filters))
for _, filter := range filters {
if err := filter.Validate(); err != nil {
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid filter: %v", err)
}
fieldContext, ok := telemetrytypes.FieldContextFromText(string(filter.Type))
if !ok {
fieldContext = telemetrytypes.FieldContextUnspecified
}
var fieldDataType telemetrytypes.FieldDataType
if err := fieldDataType.Scan(string(filter.DataType)); err != nil {
fieldDataType = telemetrytypes.FieldDataTypeUnspecified
}
if fieldDataType == telemetrytypes.FieldDataTypeInt64 {
fieldDataType = telemetrytypes.FieldDataTypeNumber
}
fieldKeys = append(fieldKeys, telemetrytypes.TelemetryFieldKey{
Name: filter.Key,
Signal: fieldSignal,
FieldContext: fieldContext,
FieldDataType: fieldDataType,
})
}
return fieldKeys, nil
}
// newLegacySourceFilters renders stored telemetry field keys
// back into the v1 shape, restoring the legacy spellings v1 clients expect.
func newLegacySourceFilters(sourceFilters *quickfiltertypes.SourceFilters) *legacySourceFilters {
filters := make([]v3.AttributeKey, 0, len(sourceFilters.Filters))
for _, fieldKey := range sourceFilters.Filters {
// Only tag and resource exist in the v3 enum; other contexts render as
// unspecified so v1 clients never see spellings their queries can't use.
var attributeType v3.AttributeKeyType
switch fieldKey.FieldContext {
case telemetrytypes.FieldContextAttribute:
attributeType = v3.AttributeKeyTypeTag
case telemetrytypes.FieldContextResource:
attributeType = v3.AttributeKeyTypeResource
default:
attributeType = v3.AttributeKeyTypeUnspecified
}
var dataType v3.AttributeKeyDataType
switch fieldKey.FieldDataType {
case telemetrytypes.FieldDataTypeNumber:
dataType = v3.AttributeKeyDataTypeFloat64
default:
dataType = v3.AttributeKeyDataType(fieldKey.FieldDataType.StringValue())
}
filters = append(filters, v3.AttributeKey{
Key: fieldKey.Name,
Type: attributeType,
DataType: dataType,
})
}
return &legacySourceFilters{
Source: sourceFilters.Source,
Filters: filters,
}
}

View File

@@ -0,0 +1,62 @@
package implquickfilter
import (
"testing"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types/quickfiltertypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewTelemetryFieldKeysFromLegacy(t *testing.T) {
fieldKeys, err := newTelemetryFieldKeysFromLegacy(quickfiltertypes.SourceTraces, []v3.AttributeKey{
{Key: "service.name", Type: v3.AttributeKeyTypeResource, DataType: v3.AttributeKeyDataTypeString},
{Key: "http.method", Type: v3.AttributeKeyTypeTag, DataType: v3.AttributeKeyDataTypeString},
{Key: "duration_nano", Type: v3.AttributeKeyTypeTag, DataType: v3.AttributeKeyDataTypeFloat64},
{Key: "code_line", Type: v3.AttributeKeyTypeTag, DataType: v3.AttributeKeyDataTypeInt64},
})
require.NoError(t, err)
require.Len(t, fieldKeys, 4)
assert.Equal(t, telemetrytypes.TelemetryFieldKey{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString}, fieldKeys[0])
assert.Equal(t, telemetrytypes.FieldContextAttribute, fieldKeys[1].FieldContext)
assert.Equal(t, telemetrytypes.FieldDataTypeNumber, fieldKeys[2].FieldDataType)
assert.Equal(t, telemetrytypes.FieldDataTypeNumber, fieldKeys[3].FieldDataType)
t.Run("meter writes restore the per-filter telemetry signal", func(t *testing.T) {
fieldKeys, err := newTelemetryFieldKeysFromLegacy(quickfiltertypes.SourceMeter, []v3.AttributeKey{
{Key: "host.name", DataType: v3.AttributeKeyDataTypeString},
})
require.NoError(t, err)
require.Len(t, fieldKeys, 1)
assert.Equal(t, telemetrytypes.SignalMetrics, fieldKeys[0].Signal)
})
t.Run("rejects a filter without a key", func(t *testing.T) {
_, err := newTelemetryFieldKeysFromLegacy(quickfiltertypes.SourceTraces, []v3.AttributeKey{{DataType: v3.AttributeKeyDataTypeString}})
require.Error(t, err)
})
}
func TestNewLegacySourceFilters(t *testing.T) {
legacy := newLegacySourceFilters(&quickfiltertypes.SourceFilters{
Source: quickfiltertypes.SourceLogs,
Filters: []telemetrytypes.TelemetryFieldKey{
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http.method", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "duration_nano", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeNumber},
{Name: "severity_text", FieldContext: telemetrytypes.FieldContextLog, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "host.name", Signal: telemetrytypes.SignalMetrics},
},
})
assert.Equal(t, quickfiltertypes.SourceLogs, legacy.Source)
require.Len(t, legacy.Filters, 5)
assert.Equal(t, v3.AttributeKey{Key: "service.name", Type: v3.AttributeKeyTypeResource, DataType: v3.AttributeKeyDataTypeString}, legacy.Filters[0])
assert.Equal(t, v3.AttributeKeyTypeTag, legacy.Filters[1].Type)
assert.Equal(t, v3.AttributeKeyDataTypeFloat64, legacy.Filters[2].DataType)
assert.Equal(t, v3.AttributeKeyTypeUnspecified, legacy.Filters[3].Type, "contexts outside the v3 enum must render as unspecified")
assert.Equal(t, v3.AttributeKey{Key: "host.name"}, legacy.Filters[4])
}

View File

@@ -2,12 +2,11 @@ package implquickfilter
import (
"context"
"encoding/json"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/quickfilter"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types/quickfiltertypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -19,91 +18,54 @@ func NewModule(store quickfiltertypes.QuickFilterStore) quickfilter.Module {
return &module{store: store}
}
// GetQuickFilters returns all quick filters for an organization.
func (module *module) GetQuickFilters(ctx context.Context, orgID valuer.UUID) ([]*quickfiltertypes.SignalFilters, error) {
storedFilters, err := module.store.Get(ctx, orgID)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error fetching organization filters")
}
result := make([]*quickfiltertypes.SignalFilters, 0, len(storedFilters))
for _, storedFilter := range storedFilters {
signalFilter, err := quickfiltertypes.NewSignalFilterFromStorableQuickFilter(storedFilter)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error processing filter for signal: %s", storedFilter.Signal)
}
result = append(result, signalFilter)
}
return result, nil
func (module *module) Get(ctx context.Context, orgID valuer.UUID, source quickfiltertypes.Source) (*quickfiltertypes.StorableQuickFilter, error) {
return module.store.GetBySource(ctx, orgID, source.StringValue())
}
// GetSignalFilters returns quick filters for a specific signal in an organization.
func (m *module) GetSignalFilters(ctx context.Context, orgID valuer.UUID, signal quickfiltertypes.Signal) (*quickfiltertypes.SignalFilters, error) {
storedFilter, err := m.store.GetBySignal(ctx, orgID, signal.StringValue())
// GetQuickFilters returns quick filters for a source, or for every source when source is zero.
func (module *module) GetQuickFilters(ctx context.Context, orgID valuer.UUID, source quickfiltertypes.Source) ([]*quickfiltertypes.SourceFilters, error) {
if source.IsZero() {
storedFilters, err := module.store.Get(ctx, orgID)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error fetching organization filters")
}
result := make([]*quickfiltertypes.SourceFilters, 0, len(storedFilters))
for _, storedFilter := range storedFilters {
sourceFilter, err := quickfiltertypes.NewSourceFilterFromStorableQuickFilter(storedFilter)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error processing filter for source: %s", storedFilter.Source)
}
result = append(result, sourceFilter)
}
return result, nil
}
storedFilter, err := module.store.GetBySource(ctx, orgID, source.StringValue())
if err != nil {
if errors.Ast(err, errors.TypeNotFound) {
return []*quickfiltertypes.SourceFilters{}, nil
}
return nil, err
}
// If no filter exists for this signal, return empty filters with the requested signal
if storedFilter == nil {
return &quickfiltertypes.SignalFilters{
Signal: signal,
Filters: []v3.AttributeKey{},
}, nil
}
// Convert stored filter to signal filter
signalFilter, err := quickfiltertypes.NewSignalFilterFromStorableQuickFilter(storedFilter)
sourceFilter, err := quickfiltertypes.NewSourceFilterFromStorableQuickFilter(storedFilter)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error processing filter for signal: %s", storedFilter.Signal)
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error processing filter for source: %s", storedFilter.Source)
}
return signalFilter, nil
return []*quickfiltertypes.SourceFilters{sourceFilter}, nil
}
// UpdateQuickFilters updates quick filters for a specific signal in an organization.
func (module *module) UpdateQuickFilters(ctx context.Context, orgID valuer.UUID, signal quickfiltertypes.Signal, filters []v3.AttributeKey) error {
// Validate each filter
for _, filter := range filters {
if err := filter.Validate(); err != nil {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid filter: %v", err)
}
}
// Marshal filters to JSON
filterJSON, err := json.Marshal(filters)
// UpsertQuickFilters replaces quick filters for a specific source in an organization, creating them if absent.
func (module *module) UpsertQuickFilters(ctx context.Context, orgID valuer.UUID, source quickfiltertypes.Source, filters []telemetrytypes.TelemetryFieldKey) error {
filter, err := quickfiltertypes.NewStorableQuickFilter(orgID, source, filters)
if err != nil {
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error marshalling filters")
}
// Check if filter exists
existingFilter, err := module.store.GetBySignal(ctx, orgID, signal.StringValue())
if err != nil {
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error checking existing filters")
}
var filter *quickfiltertypes.StorableQuickFilter
if existingFilter != nil {
// Update in place
if err := existingFilter.Update(filterJSON); err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "error updating existing filter")
}
filter = existingFilter
} else {
// Create new
filter, err = quickfiltertypes.NewStorableQuickFilter(orgID, signal, filterJSON)
if err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "error creating new filter")
}
}
// Persist filter
if err := module.store.Upsert(ctx, filter); err != nil {
return err
}
return nil
return module.store.Upsert(ctx, filter)
}
func (module *module) SetDefaultConfig(ctx context.Context, orgID valuer.UUID) error {

View File

@@ -26,7 +26,7 @@ func (s *store) Get(ctx context.Context, orgID valuer.UUID) ([]*quickfiltertypes
NewSelect().
Model(&filters).
Where("org_id = ?", orgID).
Order("signal ASC").
Order("source ASC").
Scan(ctx)
if err != nil {
@@ -36,7 +36,7 @@ func (s *store) Get(ctx context.Context, orgID valuer.UUID) ([]*quickfiltertypes
return filters, nil
}
func (s *store) GetBySignal(ctx context.Context, orgID valuer.UUID, signal string) (*quickfiltertypes.StorableQuickFilter, error) {
func (s *store) GetBySource(ctx context.Context, orgID valuer.UUID, source string) (*quickfiltertypes.StorableQuickFilter, error) {
filter := new(quickfiltertypes.StorableQuickFilter)
err := s.store.
@@ -44,12 +44,12 @@ func (s *store) GetBySignal(ctx context.Context, orgID valuer.UUID, signal strin
NewSelect().
Model(filter).
Where("org_id = ?", orgID).
Where("signal = ?", signal).
Where("source = ?", source).
Scan(ctx)
if err != nil {
if err == sql.ErrNoRows {
return nil, s.store.WrapNotFoundErrf(err, errors.CodeNotFound, "No rows found for org_id: "+orgID.StringValue()+" signal: "+signal)
return nil, s.store.WrapNotFoundErrf(err, errors.CodeNotFound, "No rows found for org_id: "+orgID.StringValue()+" source: "+source)
}
return nil, err
}
@@ -62,7 +62,7 @@ func (s *store) Upsert(ctx context.Context, filter *quickfiltertypes.StorableQui
BunDB().
NewInsert().
Model(filter).
On("CONFLICT (id) DO UPDATE").
On("CONFLICT (org_id, source) DO UPDATE").
Set("filter = EXCLUDED.filter").
Set("updated_at = EXCLUDED.updated_at").
Exec(ctx)
@@ -78,7 +78,7 @@ func (s *store) Create(ctx context.Context, filters []*quickfiltertypes.Storable
BunDBCtx(ctx).
NewInsert().
Model(&filters).
On("CONFLICT (org_id, signal) DO NOTHING").
On("CONFLICT (org_id, source) DO NOTHING").
Exec(ctx)
if err != nil {

View File

@@ -4,20 +4,27 @@ import (
"context"
"net/http"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types/quickfiltertypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type Module interface {
GetQuickFilters(ctx context.Context, orgID valuer.UUID) ([]*quickfiltertypes.SignalFilters, error)
UpdateQuickFilters(ctx context.Context, orgID valuer.UUID, signal quickfiltertypes.Signal, filters []v3.AttributeKey) error
GetSignalFilters(ctx context.Context, orgID valuer.UUID, signal quickfiltertypes.Signal) (*quickfiltertypes.SignalFilters, error)
// Get returns the stored quick filter row for a source.
Get(ctx context.Context, orgID valuer.UUID, source quickfiltertypes.Source) (*quickfiltertypes.StorableQuickFilter, error)
// GetQuickFilters returns quick filters for a source, or for every source when source is zero.
GetQuickFilters(ctx context.Context, orgID valuer.UUID, source quickfiltertypes.Source) ([]*quickfiltertypes.SourceFilters, error)
UpsertQuickFilters(ctx context.Context, orgID valuer.UUID, source quickfiltertypes.Source, filters []telemetrytypes.TelemetryFieldKey) error
SetDefaultConfig(ctx context.Context, orgID valuer.UUID) error
}
type Handler interface {
// Legacy v1 endpoints, served by converting to and from the v3 attribute key shape.
GetQuickFilters(http.ResponseWriter, *http.Request)
UpdateQuickFilters(http.ResponseWriter, *http.Request)
GetSignalFilters(http.ResponseWriter, *http.Request)
GetSourceFilters(http.ResponseWriter, *http.Request)
ListQuickFiltersV2(http.ResponseWriter, *http.Request)
GetQuickFiltersV2(http.ResponseWriter, *http.Request)
UpdateQuickFiltersV2(http.ResponseWriter, *http.Request)
}

View File

@@ -451,7 +451,7 @@ func (bc *bucketCache) mergeBuckets(ctx context.Context, buckets []*qbtypes.Cach
// Merge values based on type
var mergedValue any
switch resultType {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
mergedValue = bc.mergeTimeSeriesValues(ctx, buckets)
// Raw and Scalar types are not cached, so no merge needed
}
@@ -476,34 +476,14 @@ func (bc *bucketCache) mergeTimeSeriesValues(ctx context.Context, buckets []*qbt
}
seriesMap := make(map[seriesKey]*qbtypes.TimeSeries, estimatedSeries)
decoded := make([]*qbtypes.TimeSeriesData, 0, len(buckets))
newestOf := map[int]*qbtypes.AggregationBucket{}
newestStartOf := map[int]uint64{}
for _, bucket := range buckets {
var tsData *qbtypes.TimeSeriesData
if err := json.Unmarshal(bucket.Value, &tsData); err != nil {
bc.logger.ErrorContext(ctx, "failed to unmarshal time series data", errors.Attr(err))
continue
}
decoded = append(decoded, tsData)
// The buckets are not guaranteed to arrive in order here, and Alias and
// Unit are taken from the most recent one, so track that explicitly
// rather than relying on iteration order.
for _, aggBucket := range tsData.Aggregations {
if _, seen := newestOf[aggBucket.Index]; !seen || bucket.StartMs >= newestStartOf[aggBucket.Index] {
newestOf[aggBucket.Index] = aggBucket
newestStartOf[aggBucket.Index] = bucket.StartMs
}
}
}
mergedBoundaries := qbtypes.MergeHeatmapAxes(decoded...)
for _, tsData := range decoded {
for _, aggBucket := range tsData.Aggregations {
qbtypes.RealignHeatmapValues(aggBucket.Series, aggBucket.Meta.Buckets, mergedBoundaries[aggBucket.Index])
for _, series := range aggBucket.Series {
// Create series key from labels
key := seriesKey{
@@ -576,18 +556,10 @@ func (bc *bucketCache) mergeTimeSeriesValues(ctx context.Context, buckets []*qbt
}
}
aggBucket := &qbtypes.AggregationBucket{
result.Aggregations = append(result.Aggregations, &qbtypes.AggregationBucket{
Index: index,
Series: seriesList,
}
if newest, ok := newestOf[index]; ok {
aggBucket.Alias = newest.Alias
aggBucket.Meta = newest.Meta
}
if boundaries, ok := mergedBoundaries[index]; ok {
aggBucket.Meta.Buckets = boundaries
}
result.Aggregations = append(result.Aggregations, aggBucket)
})
}
return result
@@ -600,7 +572,7 @@ func (bc *bucketCache) isEmptyResult(result *qbtypes.Result) (isEmpty bool, isFi
}
switch result.Type {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
if tsData, ok := result.Value.(*qbtypes.TimeSeriesData); ok {
// No aggregations at all means truly empty
if len(tsData.Aggregations) == 0 {
@@ -727,19 +699,14 @@ func (bc *bucketCache) trimResultToFluxBoundary(result *qbtypes.Result, fluxBoun
}
switch result.Type {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
// Trim time series data
if tsData, ok := result.Value.(*qbtypes.TimeSeriesData); ok && tsData != nil {
trimmedData := &qbtypes.TimeSeriesData{}
for _, aggBucket := range tsData.Aggregations {
// Meta has to survive the trim: a heatmap's counts are
// positional against Meta.Buckets, so a cached bucket that
// lost its axis cannot be read back against anything.
trimmedBucket := &qbtypes.AggregationBucket{
Index: aggBucket.Index,
Alias: aggBucket.Alias,
Meta: aggBucket.Meta,
}
for _, series := range aggBucket.Series {
@@ -799,7 +766,7 @@ func (bc *bucketCache) filterResultToTimeRange(result *qbtypes.Result, startMs,
}
switch result.Type {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
if tsData, ok := result.Value.(*qbtypes.TimeSeriesData); ok {
filteredData := &qbtypes.TimeSeriesData{
Aggregations: make([]*qbtypes.AggregationBucket, 0, len(tsData.Aggregations)),

View File

@@ -92,10 +92,6 @@ func (q *builderQuery[T]) Fingerprint() string {
// This needs to include all fields that affect the query results
parts := []string{q.queryType.StringValue()}
// A heatmap and a time series query can share every spec field and still
// return different rows, so the request type has to separate their entries
parts = append(parts, fmt.Sprintf("requestType=%s", q.kind.StringValue()))
// Add signal type
parts = append(parts, fmt.Sprintf("signal=%s", q.spec.Signal.StringValue()))
@@ -134,9 +130,6 @@ func (q *builderQuery[T]) Fingerprint() string {
}
part += ":" + route
}
if a.HeatmapBucketing != nil {
part += ":" + fingerprintHeatmapBucketing(*a.HeatmapBucketing)
}
aggParts = append(aggParts, part)
}
}
@@ -192,16 +185,6 @@ func (q *builderQuery[T]) Fingerprint() string {
return strings.Join(parts, "&")
}
// fingerprintHeatmapBucketing captures only what changes the rows ClickHouse
// returns, which is why LogBucketsSpec.Scale is absent: coarsening it happens in
// postprocessing, so every scale reads one cache entry.
func fingerprintHeatmapBucketing(b qbtypes.HeatmapBucketing) string {
if b.Kind == qbtypes.BucketsKindLinear {
return fmt.Sprintf("%s:%v:%d", b.Kind.StringValue(), b.MaxValue, b.NumBuckets)
}
return b.Kind.StringValue()
}
func fingerprintGroupByKey(gb qbtypes.GroupByKey) string {
return fingerprintFieldKey(gb.TelemetryFieldKey)
}
@@ -429,7 +412,7 @@ func (q *builderQuery[T]) narrowWindowByTraceID(ctx context.Context, fromMS, toM
func emptyResultFor(kind qbtypes.RequestType, queryName string) *qbtypes.Result {
var value any
switch kind {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
value = &qbtypes.TimeSeriesData{QueryName: queryName}
case qbtypes.RequestTypeScalar:
value = &qbtypes.ScalarData{QueryName: queryName}
@@ -482,9 +465,8 @@ func (q *builderQuery[T]) executeWithContext(ctx context.Context, query string,
queryWindow := &qbtypes.TimeRange{From: q.fromMS, To: q.toMS}
kind := q.kind
// all metric queries are time series then reduced if required, except
// heatmaps, whose statement returns a row per bucket rather than per point
if q.spec.Signal == telemetrytypes.SignalMetrics && kind != qbtypes.RequestTypeHeatmap {
// all metric queries are time series then reduced if required
if q.spec.Signal == telemetrytypes.SignalMetrics {
kind = qbtypes.RequestTypeTimeSeries
}

View File

@@ -6,7 +6,6 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
@@ -121,169 +120,6 @@ func TestBuilderQueryFingerprintQueryType(t *testing.T) {
assert.Empty(t, ai.Fingerprint())
}
func TestBuilderQueryFingerprintHeatmapBucketing(t *testing.T) {
coarseLogScale := 1
testCases := []struct {
description string
left *builderQuery[qbtypes.MetricAggregation]
right *builderQuery[qbtypes.MetricAggregation]
expectedEqual bool
}{
{
// ResolveBucketOptions pins LogScale to MaxLogScale whatever the
// caller asked for, so the two are indistinguishable here by design
description: "a coarser logScale reads the same cache entry",
left: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLog, LogScale: qbtypes.MaxLogScale, NumBuckets: qbtypes.DefaultNumBuckets},
}},
},
},
right: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLog, LogScale: qbtypes.MaxLogScale, NumBuckets: qbtypes.DefaultNumBuckets},
}},
},
},
expectedEqual: true,
},
{
description: "linear separates on maxValue",
left: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLinear, MaxValue: 500, NumBuckets: 25},
}},
},
},
right: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLinear, MaxValue: 800, NumBuckets: 25},
}},
},
},
expectedEqual: false,
},
{
description: "linear separates on numBuckets",
left: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLinear, MaxValue: 500, NumBuckets: 25},
}},
},
},
right: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLinear, MaxValue: 500, NumBuckets: 40},
}},
},
},
expectedEqual: false,
},
{
description: "linear and log are separate entries",
left: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLinear, MaxValue: 500, NumBuckets: 25},
}},
},
},
right: &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
HeatmapBucketing: &qbtypes.HeatmapBucketing{Kind: qbtypes.BucketsKindLog, LogScale: qbtypes.MaxLogScale, NumBuckets: qbtypes.DefaultNumBuckets},
}},
},
},
expectedEqual: false,
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
if testCase.expectedEqual {
assert.Equal(t, testCase.left.Fingerprint(), testCase.right.Fingerprint())
return
}
assert.NotEqual(t, testCase.left.Fingerprint(), testCase.right.Fingerprint())
})
}
t.Run("a coarser scale never reaches the axis clickhouse builds", func(t *testing.T) {
finest := (&qbtypes.BucketOptions{Kind: qbtypes.BucketsKindLog, Spec: qbtypes.LogBucketsSpec{}}).ResolveBucketOptions()
coarse := (&qbtypes.BucketOptions{Kind: qbtypes.BucketsKindLog, Spec: qbtypes.LogBucketsSpec{Scale: &coarseLogScale}}).ResolveBucketOptions()
assert.Equal(t, finest, coarse)
})
t.Run("a histogram folds in no bucket options at all", func(t *testing.T) {
// resolveHeatmapBucketing leaves histograms nil, so bucketOptions sent
// alongside one must not fragment its cache
histogram := &builderQuery[qbtypes.MetricAggregation]{
queryType: qbtypes.QueryTypeBuilder,
kind: qbtypes.RequestTypeHeatmap,
spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{{
MetricName: "signoz_latency",
Type: metrictypes.HistogramType,
}},
},
}
fingerprint := histogram.Fingerprint()
assert.NotContains(t, fingerprint, qbtypes.BucketsKindLog.StringValue())
assert.NotContains(t, fingerprint, qbtypes.BucketsKindLinear.StringValue())
})
}
func TestMakeBucketsOrder(t *testing.T) {
// Test that makeBuckets returns buckets in reverse chronological order by default
// Using milliseconds as input - need > 1 hour range to get multiple buckets

View File

@@ -31,11 +31,6 @@ var (
// written clickhouse query. The column alias indcate which value is
// to be considered as final result (or target).
legacyReservedColumnTargetAliases = []string{"__result", "__value", "result", "res", "value"}
// legacyHeatmapBucketColumn is the alias a user written clickhouse query can
// give its bucket boundary column, alongside the HeatmapBucketColumn the
// statement builder emits.
legacyHeatmapBucketColumn = "bucket"
)
// stripKeyAlias removes the __SELECT_KEY_<n>_ / __GROUP_BY_KEY_<n>_ prefix from a result
@@ -88,8 +83,6 @@ func consume(rows driver.Rows, kind qbtypes.RequestType, queryWindow *qbtypes.Ti
payload, err = readAsTimeSeries(rows, queryWindow, step, queryName)
case qbtypes.RequestTypeScalar:
payload, err = readAsScalar(rows, queryName)
case qbtypes.RequestTypeHeatmap:
payload, err = readAsHeatmap(rows, queryWindow, step, queryName)
case qbtypes.RequestTypeRaw, qbtypes.RequestTypeTrace, qbtypes.RequestTypeRawStream:
payload, err = readAsRaw(rows, queryName)
// TODO: add support for other request types
@@ -119,6 +112,35 @@ func readAsTimeSeries(rows driver.Rows, queryWindow *qbtypes.TimeRange, step qbt
stepMs := uint64(step.Milliseconds())
// Helper function to check if a timestamp represents a partial value
isPartialValue := func(timestamp int64) bool {
if stepMs == 0 || queryWindow == nil {
return false
}
timestampMs := uint64(timestamp)
// For the first interval, check if query start is misaligned
// The first complete interval starts at the first timestamp >= queryWindow.From that is aligned to step
firstCompleteInterval := queryWindow.From
if queryWindow.From%stepMs != 0 {
// Round up to next step boundary
firstCompleteInterval = ((queryWindow.From / stepMs) + 1) * stepMs
}
// If timestamp is before the first complete interval, it's partial
if timestampMs < firstCompleteInterval {
return true
}
// For the last interval, check if it would extend beyond query end
if timestampMs+stepMs > queryWindow.To {
return queryWindow.To%stepMs != 0
}
return false
}
// Pre-allocate for labels based on column count
lblValsCapacity := len(colNames) - 1 // -1 for timestamp
if lblValsCapacity < 0 {
@@ -249,7 +271,7 @@ func readAsTimeSeries(rows driver.Rows, queryWindow *qbtypes.TimeRange, step qbt
series.Values = append(series.Values, &qbtypes.TimeSeriesValue{
Timestamp: ts,
Value: val,
Partial: isPartialValue(ts, queryWindow, stepMs),
Partial: isPartialValue(ts),
})
}
}
@@ -293,223 +315,6 @@ func readAsTimeSeries(rows driver.Rows, queryWindow *qbtypes.TimeRange, step qbt
}, nil
}
// heatmapSeries accumulates one group's cells while the rows are read. Counts
// are held against their boundary rather than a slice because the axis is only
// known once every row has been seen.
type heatmapSeries struct {
labels []*qbtypes.Label
counts map[int64]map[float64]float64
}
// heatmapAccumulator is shared by the readers of the two things a heatmap can
// come back as: ClickHouse rows, and a PromQL matrix.
type heatmapAccumulator struct {
seriesByKey map[string]*heatmapSeries
seriesOrder []string
boundaries map[float64]struct{}
}
func newHeatmapAccumulator() *heatmapAccumulator {
return &heatmapAccumulator{
seriesByKey: map[string]*heatmapSeries{},
boundaries: map[float64]struct{}{},
}
}
// addCell files one cell under the group labelsKey identifies, keeping the
// labels from the first cell seen for it.
func (a *heatmapAccumulator) addCell(labelsKey string, lbls []*qbtypes.Label, ts int64, boundary, count float64) {
series, ok := a.seriesByKey[labelsKey]
if !ok {
series = &heatmapSeries{labels: lbls, counts: map[int64]map[float64]float64{}}
a.seriesByKey[labelsKey] = series
a.seriesOrder = append(a.seriesOrder, labelsKey)
}
if series.counts[ts] == nil {
series.counts[ts] = map[float64]float64{}
}
series.counts[ts][boundary] += count
if !math.IsInf(boundary, 1) {
a.boundaries[boundary] = struct{}{}
}
}
// foldSeries turns the collected cells into one series per group, in the order
// the groups first appeared.
func (a *heatmapAccumulator) foldSeries(queryWindow *qbtypes.TimeRange, stepMs uint64, queryName string) *qbtypes.TimeSeriesData {
if len(a.seriesOrder) == 0 {
return &qbtypes.TimeSeriesData{QueryName: queryName}
}
boundaries := make([]float64, 0, len(a.boundaries))
for boundary := range a.boundaries {
boundaries = append(boundaries, boundary)
}
slices.Sort(boundaries)
// the band past the last boundary is where the +Inf overflow lands
bandIndexByBoundary := make(map[float64]int, len(boundaries)+1)
for band, boundary := range boundaries {
bandIndexByBoundary[boundary] = band
}
bandIndexByBoundary[math.Inf(1)] = len(boundaries)
bucket := &qbtypes.AggregationBucket{
Index: 0,
Alias: "__result_0",
Meta: qbtypes.AggregationMeta{Buckets: boundaries},
Series: make([]*qbtypes.TimeSeries, 0, len(a.seriesOrder)),
}
for _, labelsKey := range a.seriesOrder {
accumulated := a.seriesByKey[labelsKey]
timestamps := make([]int64, 0, len(accumulated.counts))
for ts := range accumulated.counts {
timestamps = append(timestamps, ts)
}
slices.Sort(timestamps)
series := &qbtypes.TimeSeries{
Labels: accumulated.labels,
Values: make([]*qbtypes.TimeSeriesValue, 0, len(timestamps)),
}
for _, ts := range timestamps {
values := make([]float64, len(boundaries)+1)
for boundary, count := range accumulated.counts[ts] {
values[bandIndexByBoundary[boundary]] = count
}
series.Values = append(series.Values, &qbtypes.TimeSeriesValue{
Timestamp: ts,
Values: values,
Partial: isPartialValue(ts, queryWindow, stepMs),
})
}
bucket.Series = append(bucket.Series, series)
}
return &qbtypes.TimeSeriesData{
QueryName: queryName,
Aggregations: []*qbtypes.AggregationBucket{bucket},
}
}
// readAsHeatmap folds one row per cell — (timestamp, group labels, bucket upper
// boundary, count) — into one series per group.
func readAsHeatmap(rows driver.Rows, queryWindow *qbtypes.TimeRange, step qbtypes.Step, queryName string) (*qbtypes.TimeSeriesData, error) {
colTypes := rows.ColumnTypes()
colNames := rows.Columns()
slots := make([]any, len(colTypes))
for i, ct := range colTypes {
slots[i] = reflect.New(ct.ScanType()).Interface()
}
stepMs := uint64(step.Milliseconds())
accumulator := newHeatmapAccumulator()
// every column that is not the timestamp, the boundary or the count is a label
lblValsCapacity := len(colNames) - 3
if lblValsCapacity < 0 {
lblValsCapacity = 0
}
for rows.Next() {
if err := rows.Scan(slots...); err != nil {
return nil, err
}
var (
ts int64
boundary float64
count float64
hasCell bool
lblVals = make([]string, 0, lblValsCapacity)
lblObjs = make([]*qbtypes.Label, 0, lblValsCapacity)
)
for idx, ptr := range slots {
name := stripKeyAlias(colNames[idx])
value := derefValue(ptr)
if t, ok := value.(time.Time); ok {
ts = t.UnixMilli()
continue
}
switch name {
case qbtypes.HeatmapBucketColumn, legacyHeatmapBucketColumn:
boundary = numericAsFloat(value)
hasCell = true
default:
if aggRe.MatchString(name) || slices.Contains(legacyReservedColumnTargetAliases, name) {
count = numericAsFloat(value)
continue
}
// a nullable label column comes back as a nil any, which would
// otherwise key the series on the literal "<nil>"
if value == nil {
value = ""
}
lblVals = append(lblVals, fmt.Sprint(value))
lblObjs = append(lblObjs, &qbtypes.Label{
Key: telemetrytypes.TelemetryFieldKey{Name: name},
Value: value,
})
}
}
if ts == 0 || !hasCell || math.IsNaN(boundary) || math.IsInf(boundary, -1) {
continue
}
if math.IsNaN(count) || math.IsInf(count, 0) {
continue
}
sort.Strings(lblVals)
labelsKey := strings.Join(lblVals, ",")
accumulator.addCell(labelsKey, lblObjs, ts, boundary, count)
}
if err := rows.Err(); err != nil {
return nil, err
}
return accumulator.foldSeries(queryWindow, stepMs, queryName), nil
}
// isPartialValue reports whether the step interval starting at timestamp is only
// partly covered by the query window, which happens when the window boundaries
// are not step-aligned.
func isPartialValue(timestamp int64, queryWindow *qbtypes.TimeRange, stepMs uint64) bool {
if stepMs == 0 || queryWindow == nil {
return false
}
timestampMs := uint64(timestamp)
// For the first interval, check if query start is misaligned
// The first complete interval starts at the first timestamp >= queryWindow.From that is aligned to step
firstCompleteInterval := queryWindow.From
if queryWindow.From%stepMs != 0 {
// Round up to next step boundary
firstCompleteInterval = ((queryWindow.From / stepMs) + 1) * stepMs
}
// If timestamp is before the first complete interval, it's partial
if timestampMs < firstCompleteInterval {
return true
}
// For the last interval, check if it would extend beyond query end
if timestampMs+stepMs > queryWindow.To {
return queryWindow.To%stepMs != 0
}
return false
}
func isNumericKind(t reflect.Type) bool {
if t == nil {
return false

View File

@@ -1,411 +0,0 @@
package querier
import (
"math"
"reflect"
"testing"
"time"
"github.com/ClickHouse/clickhouse-go/v2/lib/driver"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// fakeColumnType is the minimum of driver.ColumnType that readAsHeatmap reads:
// the scan type it allocates a slot from.
type fakeColumnType struct {
name string
scanType reflect.Type
}
func (c fakeColumnType) Name() string { return c.name }
func (c fakeColumnType) Nullable() bool { return false }
func (c fakeColumnType) ScanType() reflect.Type { return c.scanType }
func (c fakeColumnType) DatabaseTypeName() string { return c.scanType.String() }
// fakeRows replays a fixed set of rows, each holding one value per column in
// the order the columns are declared.
type fakeRows struct {
columns []fakeColumnType
rows [][]any
cursor int
}
func (r *fakeRows) Next() bool {
r.cursor++
return r.cursor <= len(r.rows)
}
func (r *fakeRows) Scan(dest ...any) error {
row := r.rows[r.cursor-1]
for i, value := range row {
reflect.ValueOf(dest[i]).Elem().Set(reflect.ValueOf(value))
}
return nil
}
func (r *fakeRows) ScanStruct(any) error { return nil }
func (r *fakeRows) ColumnTypes() []driver.ColumnType {
types := make([]driver.ColumnType, len(r.columns))
for i, column := range r.columns {
types[i] = column
}
return types
}
func (r *fakeRows) Totals(...any) error { return nil }
func (r *fakeRows) Columns() []string {
names := make([]string, len(r.columns))
for i, column := range r.columns {
names[i] = column.name
}
return names
}
func (r *fakeRows) HasData() bool { return len(r.rows) > 0 }
func (r *fakeRows) Close() error { return nil }
func (r *fakeRows) Err() error { return nil }
var _ driver.Rows = (*fakeRows)(nil)
func TestReadAsHeatmapBuildsSharedBucketAxis(t *testing.T) {
first := time.UnixMilli(1710000000000)
second := time.UnixMilli(1710000060000)
rows := &fakeRows{
columns: []fakeColumnType{
{name: "ts", scanType: reflect.TypeOf(time.Time{})},
{name: "__GROUP_BY_KEY_0_service.name", scanType: reflect.TypeOf("")},
{name: "__bucket", scanType: reflect.TypeOf(float64(0))},
{name: "__result_0", scanType: reflect.TypeOf(float64(0))},
},
rows: [][]any{
{first, "cart", 5.0, 3.0},
{first, "cart", 10.0, 7.0},
{first, "cart", math.Inf(1), 1.0},
{first, "pay", 10.0, 2.0},
{second, "cart", 5.0, 4.0},
{second, "pay", math.Inf(1), 6.0},
},
}
data, err := readAsHeatmap(rows, &qbtypes.TimeRange{From: 1710000000000, To: 1710000120000}, qbtypes.Step{Duration: time.Minute}, "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
aggregation := data.Aggregations[0]
// +Inf is not a boundary; it is the slot past the last one
assert.Equal(t, []float64{5, 10}, aggregation.Meta.Buckets)
require.Len(t, aggregation.Series, 2)
cart := aggregation.Series[0]
require.Len(t, cart.Labels, 1)
assert.Equal(t, "cart", cart.Labels[0].Value)
require.Len(t, cart.Values, 2)
assert.Equal(t, int64(1710000000000), cart.Values[0].Timestamp)
assert.Equal(t, []float64{3, 7, 1}, cart.Values[0].Values)
assert.Equal(t, []float64{4, 0, 0}, cart.Values[1].Values)
pay := aggregation.Series[1]
assert.Equal(t, "pay", pay.Labels[0].Value)
assert.Equal(t, []float64{0, 2, 0}, pay.Values[0].Values)
assert.Equal(t, []float64{0, 0, 6}, pay.Values[1].Values)
}
func TestReadAsHeatmapWithoutGroupBy(t *testing.T) {
at := time.UnixMilli(1710000000000)
rows := &fakeRows{
columns: []fakeColumnType{
{name: "ts", scanType: reflect.TypeOf(time.Time{})},
{name: "__bucket", scanType: reflect.TypeOf(float64(0))},
{name: "__result_0", scanType: reflect.TypeOf(float64(0))},
},
rows: [][]any{
{at, 2.5, 9.0},
{at, 5.0, 4.0},
},
}
data, err := readAsHeatmap(rows, nil, qbtypes.Step{Duration: time.Minute}, "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
aggregation := data.Aggregations[0]
assert.Equal(t, []float64{2.5, 5}, aggregation.Meta.Buckets)
require.Len(t, aggregation.Series, 1)
assert.Empty(t, aggregation.Series[0].Labels)
// no +Inf row, so the overflow slot is present but empty
assert.Equal(t, []float64{9, 4, 0}, aggregation.Series[0].Values[0].Values)
}
func TestReadAsHeatmapWithoutRows(t *testing.T) {
rows := &fakeRows{
columns: []fakeColumnType{
{name: "ts", scanType: reflect.TypeOf(time.Time{})},
{name: "__bucket", scanType: reflect.TypeOf(float64(0))},
{name: "__result_0", scanType: reflect.TypeOf(float64(0))},
},
}
data, err := readAsHeatmap(rows, nil, qbtypes.Step{Duration: time.Minute}, "A")
require.NoError(t, err)
assert.Equal(t, "A", data.QueryName)
assert.Empty(t, data.Aggregations)
}
func TestReadAsHeatmapMarksPartialTimestamps(t *testing.T) {
misaligned := time.UnixMilli(1710000000000)
aligned := time.UnixMilli(1710000060000)
rows := &fakeRows{
columns: []fakeColumnType{
{name: "ts", scanType: reflect.TypeOf(time.Time{})},
{name: "__bucket", scanType: reflect.TypeOf(float64(0))},
{name: "__result_0", scanType: reflect.TypeOf(float64(0))},
},
rows: [][]any{
{misaligned, 5.0, 1.0},
{aligned, 5.0, 2.0},
},
}
// The window starts mid-step, so the step the first row falls in is only
// partly covered by it.
data, err := readAsHeatmap(rows, &qbtypes.TimeRange{From: 1710000030000, To: 1710000120000}, qbtypes.Step{Duration: time.Minute}, "A")
require.NoError(t, err)
values := data.Aggregations[0].Series[0].Values
require.Len(t, values, 2)
assert.True(t, values[0].Partial)
assert.False(t, values[1].Partial)
}
func TestMergeTimeSeriesResultsUnionsHeatmapAxes(t *testing.T) {
// a log axis holds whichever bands the data reached, so a wide cached range
// and a narrow fresh one routinely disagree on which bands exist
cached := &qbtypes.TimeSeriesData{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
Index: 0,
Meta: qbtypes.AggregationMeta{Buckets: []float64{1, 4, 16}},
Series: []*qbtypes.TimeSeries{{
Labels: []*qbtypes.Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host.name"}, Value: "node-1"}},
Values: []*qbtypes.TimeSeriesValue{{Timestamp: 1710000000000, Values: []float64{1, 2, 3, 4}}},
}},
}},
}
fresh := []*qbtypes.Result{{
Value: &qbtypes.TimeSeriesData{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
Index: 0,
Meta: qbtypes.AggregationMeta{Buckets: []float64{2, 4}},
Series: []*qbtypes.TimeSeries{{
Labels: []*qbtypes.Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host.name"}, Value: "node-1"}},
Values: []*qbtypes.TimeSeriesValue{{Timestamp: 1710000060000, Values: []float64{5, 6, 7}}},
}},
}},
},
}}
merged := (&querier{}).mergeTimeSeriesResults(cached, fresh)
require.Len(t, merged.Aggregations, 1)
aggBucket := merged.Aggregations[0]
assert.Equal(t, []float64{1, 2, 4, 16}, aggBucket.Meta.Buckets)
require.Len(t, aggBucket.Series, 1)
require.Len(t, aggBucket.Series[0].Values, 2)
// the cached 16 band survives even though the fresh range never reached it
assert.Equal(t, []float64{1, 0, 2, 3, 4}, aggBucket.Series[0].Values[0].Values)
// and the fresh 2 band survives even though the cached range never had it
assert.Equal(t, []float64{0, 5, 6, 0, 7}, aggBucket.Series[0].Values[1].Values)
}
func TestReadAsHeatmapAcceptsHandWrittenColumnAliases(t *testing.T) {
at := time.UnixMilli(1710000000000)
// the aliases a user written clickhouse query would reach for, rather than
// the __bucket / __result_0 the statement builder emits
rows := &fakeRows{
columns: []fakeColumnType{
{name: "ts", scanType: reflect.TypeOf(time.Time{})},
{name: "service.name", scanType: reflect.TypeOf("")},
{name: "bucket", scanType: reflect.TypeOf(float64(0))},
{name: "value", scanType: reflect.TypeOf(float64(0))},
},
rows: [][]any{
{at, "cart", 5.0, 3.0},
{at, "cart", 10.0, 7.0},
},
}
data, err := readAsHeatmap(rows, nil, qbtypes.Step{Duration: time.Minute}, "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
aggregation := data.Aggregations[0]
assert.Equal(t, []float64{5, 10}, aggregation.Meta.Buckets)
require.Len(t, aggregation.Series, 1)
require.Len(t, aggregation.Series[0].Labels, 1)
assert.Equal(t, "cart", aggregation.Series[0].Labels[0].Value)
assert.Equal(t, []float64{3, 7, 0}, aggregation.Series[0].Values[0].Values)
}
func TestApplyFormulasBucketsTheFormulaOutput(t *testing.T) {
q := &querier{logger: instrumentationtest.New().Logger()}
seriesAt := func(labelValue string, values ...float64) *qbtypes.TimeSeries {
points := make([]*qbtypes.TimeSeriesValue, 0, len(values))
for index, value := range values {
points = append(points, &qbtypes.TimeSeriesValue{
Timestamp: 1710000000000 + int64(index)*60000,
Value: value,
})
}
return &qbtypes.TimeSeries{
Labels: []*qbtypes.Label{{
Key: telemetrytypes.TelemetryFieldKey{Name: "host.name"},
Value: labelValue,
}},
Values: points,
}
}
results := map[string]*qbtypes.Result{
"A": {Value: &qbtypes.TimeSeriesData{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{Index: 0, Series: []*qbtypes.TimeSeries{seriesAt("h1", 8, 64)}}},
}},
"B": {Value: &qbtypes.TimeSeriesData{
QueryName: "B",
Aggregations: []*qbtypes.AggregationBucket{{Index: 0, Series: []*qbtypes.TimeSeries{seriesAt("h1", 4, 16)}}},
}},
}
req := &qbtypes.QueryRangeRequest{
RequestType: qbtypes.RequestTypeHeatmap,
BucketOptions: &qbtypes.BucketOptions{Kind: qbtypes.BucketsKindLog, Spec: qbtypes.LogBucketsSpec{}},
CompositeQuery: qbtypes.CompositeQuery{Queries: []qbtypes.QueryEnvelope{
{Type: qbtypes.QueryTypeBuilder, Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{Name: "A", Disabled: true}},
{Type: qbtypes.QueryTypeBuilder, Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{Name: "B", Disabled: true}},
{Type: qbtypes.QueryTypeFormula, Spec: qbtypes.QueryBuilderFormula{Name: "F1", Expression: "A / B"}},
}},
}
results = q.applyFormulas(t.Context(), results, req)
formula, ok := results["F1"]
require.True(t, ok, "formula produced no result")
tsData, ok := formula.Value.(*qbtypes.TimeSeriesData)
require.True(t, ok)
require.Len(t, tsData.Aggregations, 1)
// 8/4 is 2 and 64/16 is 4, a doubling apart, so the filled axis carries
// every band from 2 to 4 inclusive and the two points sit at its ends
aggBucket := tsData.Aggregations[0]
require.Len(t, aggBucket.Meta.Buckets, 17)
assert.Equal(t, math.Exp2(1), aggBucket.Meta.Buckets[0])
assert.Equal(t, math.Exp2(2), aggBucket.Meta.Buckets[16])
require.Len(t, aggBucket.Series, 1)
points := aggBucket.Series[0].Values
require.Len(t, points, 2)
assert.Equal(t, float64(1), points[0].Values[0])
assert.Equal(t, float64(1), points[1].Values[16])
for index, point := range points {
require.Len(t, point.Values, 18, "point %d", index)
var total float64
for _, count := range point.Values {
total += count
}
assert.Equal(t, float64(1), total, "point %d counts the one series it came from", index)
}
}
func TestApplyFormulasCoarsensTheFormulaAxis(t *testing.T) {
q := &querier{logger: instrumentationtest.New().Logger()}
scale := 0
req := &qbtypes.QueryRangeRequest{
RequestType: qbtypes.RequestTypeHeatmap,
BucketOptions: &qbtypes.BucketOptions{Kind: qbtypes.BucketsKindLog, Spec: qbtypes.LogBucketsSpec{Scale: &scale}},
CompositeQuery: qbtypes.CompositeQuery{Queries: []qbtypes.QueryEnvelope{
{Type: qbtypes.QueryTypeBuilder, Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{Name: "A", Disabled: true}},
{Type: qbtypes.QueryTypeFormula, Spec: qbtypes.QueryBuilderFormula{Name: "F1", Expression: "A * 2"}},
}},
}
results := map[string]*qbtypes.Result{
"A": {Value: &qbtypes.TimeSeriesData{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{Index: 0, Series: []*qbtypes.TimeSeries{{
Values: []*qbtypes.TimeSeriesValue{
{Timestamp: 1710000000000, Value: 1.5},
{Timestamp: 1710000060000, Value: 2},
},
}}}},
}},
}
results = q.applyFormulas(t.Context(), results, req)
tsData := results["F1"].Value.(*qbtypes.TimeSeriesData)
aggBucket := tsData.Aggregations[0]
// 3 and 4 sit in different bands at scale 4 but the same doubling at scale 0
assert.Equal(t, []float64{math.Exp2(2)}, aggBucket.Meta.Buckets)
assert.Equal(t, []float64{1, 0}, aggBucket.Series[0].Values[0].Values)
assert.Equal(t, []float64{1, 0}, aggBucket.Series[0].Values[1].Values)
}
func TestTrimResultToFluxBoundaryKeepsTheHeatmapAxis(t *testing.T) {
cache := &bucketCache{logger: instrumentationtest.New().Logger()}
result := &qbtypes.Result{
Type: qbtypes.RequestTypeHeatmap,
Value: &qbtypes.TimeSeriesData{
Aggregations: []*qbtypes.AggregationBucket{{
Index: 0,
Alias: "__result_0",
Meta: qbtypes.AggregationMeta{Unit: "By", Buckets: []float64{1, 2, 4}},
Series: []*qbtypes.TimeSeries{{
Values: []*qbtypes.TimeSeriesValue{
{Timestamp: 1710000000000, Values: []float64{1, 2, 3, 4}},
},
}},
}},
},
}
trimmed := cache.trimResultToFluxBoundary(result, 1710000060000)
tsData, ok := trimmed.Value.(*qbtypes.TimeSeriesData)
require.True(t, ok)
require.Len(t, tsData.Aggregations, 1)
// the counts are positional against the axis, so a cached bucket that lost
// Meta.Buckets would be realigned from an empty axis and collapse into the
// overflow slot on the way back out
aggBucket := tsData.Aggregations[0]
assert.Equal(t, []float64{1, 2, 4}, aggBucket.Meta.Buckets)
assert.Equal(t, "By", aggBucket.Meta.Unit)
assert.Equal(t, "__result_0", aggBucket.Alias)
}
func TestRealignFromAnEmptyAxisCollapsesIntoTheOverflow(t *testing.T) {
// pins the behaviour the trim bug exposed: with no axis to read the counts
// against, everything lands in the overflow slot
series := []*qbtypes.TimeSeries{{
Values: []*qbtypes.TimeSeriesValue{{Timestamp: 1710000000000, Values: []float64{7, 8, 9, 10}}},
}}
qbtypes.RealignHeatmapValues(series, nil, []float64{1, 2, 4})
assert.Equal(t, []float64{0, 0, 0, 7}, series[0].Values[0].Values)
}

View File

@@ -195,17 +195,6 @@ func postProcessBuilderQuery[T any](
return result
}
// resolveHeatmapAxis brings a heatmap axis to the resolution the caller asked
// for. Coarsening runs before the fill so the empty bands land at the resolution
// being returned rather than the one ClickHouse bucketed at.
func resolveHeatmapAxis(tsData *qbtypes.TimeSeriesData, bucketing qbtypes.HeatmapBucketing, requestedScale int) {
if bucketing.Kind == qbtypes.BucketsKindLog && requestedScale < bucketing.LogScale {
qbtypes.DownscaleHeatmapAxis(tsData, bucketing.LogScale, requestedScale)
bucketing.LogScale = requestedScale
}
qbtypes.DensifyHeatmapAxis(tsData, bucketing)
}
// postProcessMetricQuery applies postprocessing to a metric query result.
func postProcessMetricQuery(
q *querier,
@@ -227,12 +216,6 @@ func postProcessMetricQuery(
}
}
if req.RequestType == qbtypes.RequestTypeHeatmap && config.HeatmapBucketing != nil {
if tsData, ok := result.Value.(*qbtypes.TimeSeriesData); ok {
resolveHeatmapAxis(tsData, *config.HeatmapBucketing, req.BucketOptions.ResolveLogScale())
}
}
result = q.applySeriesLimit(result, query.Limit, query.Order)
if len(query.Functions) > 0 {
@@ -359,19 +342,6 @@ func (q *querier) applyFormulas(ctx context.Context, results map[string]*qbtypes
result = q.applySeriesLimit(result, formula.Limit, formula.Order)
results[name] = result
}
case qbtypes.RequestTypeHeatmap:
// The queries a formula reads were run as time series, so what
// arrives here is one value per group per timestamp.
result := q.processTimeSeriesFormula(ctx, results, formula, req)
if result != nil {
if tsData, ok := result.Value.(*qbtypes.TimeSeriesData); ok {
bucketing := req.BucketOptions.ResolveBucketOptions()
qbtypes.BucketTimeSeriesValues(tsData, bucketing)
resolveHeatmapAxis(tsData, bucketing, req.BucketOptions.ResolveLogScale())
}
result = q.applySeriesLimit(result, formula.Limit, formula.Order)
results[name] = result
}
case qbtypes.RequestTypeScalar:
result := q.processScalarFormula(ctx, results, formula, req)
// For scalar results, apply limit by processScalarFormula itself since it needs to be applied before converting back to scalar format
@@ -524,7 +494,7 @@ func (q *querier) processScalarFormula(
bucket := &qbtypes.AggregationBucket{
Index: aggIdx,
Alias: scalarData.Columns[colIdx].Name,
Meta: qbtypes.AggregationMeta{Unit: scalarData.Columns[colIdx].Meta.Unit},
Meta: scalarData.Columns[colIdx].Meta,
Series: make([]*qbtypes.TimeSeries, 0),
}
@@ -697,14 +667,13 @@ func convertTimeSeriesDataToScalar(tsData *qbtypes.TimeSeriesData, queryName str
if name == "" {
name = fmt.Sprintf("__result_%d", agg.Index)
}
column := &qbtypes.ColumnDescriptor{
columns = append(columns, &qbtypes.ColumnDescriptor{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: name},
QueryName: queryName,
AggregationIndex: int64(agg.Index),
Meta: agg.Meta,
Type: qbtypes.ColumnTypeAggregation,
}
column.Meta.Unit = agg.Meta.Unit
columns = append(columns, column)
})
}
// Build rows.

View File

@@ -50,7 +50,7 @@ func (q *querier) QueryRangePreview(
env := []qbtypes.QueryEnvelope{req.CompositeQuery.Queries[idx]}
ps.Warnings = append(ps.Warnings, q.adjustStepInterval(env, req.Start, req.End)...)
missingMetricQueries, metricWarnings, mErr := q.resolveMetricMetadata(ctx, orgID, env, req.Start, req.End, req.RequestType, req.BucketOptions)
missingMetricQueries, metricWarnings, mErr := q.resolveMetricMetadata(ctx, orgID, env, req.Start, req.End)
if mErr != nil {
// Report this query's error but keep previewing the rest.
ps.Error = mErr

View File

@@ -1,145 +0,0 @@
package querier
import (
"fmt"
"math"
"slices"
"sort"
"strconv"
"strings"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/promql"
"github.com/SigNoz/signoz/pkg/errors"
qbv5 "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// promHistogramBucketLabel is the label a classic histogram carries its
// cumulative upper bound on, in PromQL as in the metric itself.
const promHistogramBucketLabel = "le"
// promHeatmapGroup accumulates one group's cumulative counts. `le` series are
// separate series in a matrix, so a group is assembled across several of them
// and the differencing can only run once they have all been read.
type promHeatmapGroup struct {
labels []*qbv5.Label
labelsKey string
cumulative map[int64]map[float64]float64
}
// foldMatrixAsHeatmap reads a classic histogram matrix as heatmap cells: one
// series per (group, `le`) carrying the cumulative count at that boundary,
// folded into one series per group whose points hold a count per band.
//
// This is readAsHeatmap's counterpart for a result the builder did not produce.
// buildHistogramHeatmapFinalSelect differences along `le` in SQL with
// lagInFrame; there is no statement here to attach that to, so it runs below
// against the same rules.
//
// Whether the expression kept `le` can only be seen in the result, so a matrix
// carrying data but no `le` anywhere is refused rather than drawn as one
// meaningless band.
func foldMatrixAsHeatmap(matrix promql.Matrix, queryWindow *qbv5.TimeRange, stepMs uint64, queryName string) (*qbv5.TimeSeriesData, error) {
groups := map[string]*promHeatmapGroup{}
groupOrder := []string{}
sawBucketLabel := false
for _, promSeries := range matrix {
boundary, ok := extractBucketBoundary(promSeries.Metric)
if !ok {
continue
}
sawBucketLabel = true
lbls, labelsKey := extractHeatmapGroup(promSeries.Metric)
group, ok := groups[labelsKey]
if !ok {
group = &promHeatmapGroup{labels: lbls, labelsKey: labelsKey, cumulative: map[int64]map[float64]float64{}}
groups[labelsKey] = group
groupOrder = append(groupOrder, labelsKey)
}
for _, point := range promSeries.Floats {
// A non-finite cumulative count has nothing to difference against.
// Skipping the point leaves the band above it differenced against
// the next boundary that does have one, which is what lagInFrame
// does with an absent row on the builder path.
if math.IsNaN(point.F) || math.IsInf(point.F, 0) {
continue
}
if group.cumulative[point.T] == nil {
group.cumulative[point.T] = map[float64]float64{}
}
group.cumulative[point.T][boundary] = point.F
}
}
if len(matrix) > 0 && !sawBucketLabel {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"promql heatmap needs a %q label to draw its bucket axis from, and %q returned none: keep it in the result, as in `sum by (%s) (increase(metric_bucket[5m]))`",
promHistogramBucketLabel, queryName, promHistogramBucketLabel)
}
accumulator := newHeatmapAccumulator()
for _, labelsKey := range groupOrder {
group := groups[labelsKey]
for ts, cumulative := range group.cumulative {
boundaries := make([]float64, 0, len(cumulative))
for boundary := range cumulative {
boundaries = append(boundaries, boundary)
}
slices.Sort(boundaries)
previous := float64(0)
for _, boundary := range boundaries {
accumulator.addCell(labelsKey, group.labels, ts, boundary, math.Max(cumulative[boundary]-previous, 0))
previous = cumulative[boundary]
}
}
}
return accumulator.foldSeries(queryWindow, stepMs, queryName), nil
}
// extractBucketBoundary reads the `le` label as a boundary. The label is a
// string, so `+Inf` arrives as one and parses to the overflow boundary. A -Inf
// or NaN label bounds nothing and is reported as absent.
func extractBucketBoundary(metric labels.Labels) (float64, bool) {
raw := metric.Get(promHistogramBucketLabel)
if raw == "" {
return 0, false
}
boundary, err := strconv.ParseFloat(raw, 64)
if err != nil || math.IsNaN(boundary) || math.IsInf(boundary, -1) {
return 0, false
}
return boundary, true
}
// extractHeatmapGroup returns the labels identifying a series' group — every
// label except `le`, which becomes the Y axis — and a key for it.
//
// The key holds names as well as values, unlike the row reader's, because two
// matrix series can carry different label sets where two rows of one result
// cannot, and values alone would collide across them.
func extractHeatmapGroup(metric labels.Labels) ([]*qbv5.Label, string) {
lbls := make([]*qbv5.Label, 0, metric.Len())
pairs := make([]string, 0, metric.Len())
metric.Range(func(l labels.Label) {
if l.Name == promHistogramBucketLabel || excludePromLabel(l.Name) {
return
}
lbls = append(lbls, &qbv5.Label{
Key: telemetrytypes.TelemetryFieldKey{Name: l.Name},
Value: l.Value,
})
pairs = append(pairs, fmt.Sprintf("%s=%s", l.Name, l.Value))
})
sort.Strings(pairs)
return lbls, strings.Join(pairs, ",")
}

View File

@@ -1,231 +0,0 @@
package querier
import (
"log/slog"
"math"
"sync"
"testing"
"time"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/promql"
qbv5 "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFoldMatrixAsHeatmapDifferencesAlongTheBucketLabel(t *testing.T) {
firstTimestamp := int64(1710000000000)
secondTimestamp := int64(1710000060000)
matrix := promql.Matrix{
{
Metric: labels.FromStrings("service.name", "cart", "le", "5"),
Floats: []promql.FPoint{{T: firstTimestamp, F: 3}, {T: secondTimestamp, F: 4}},
},
{
Metric: labels.FromStrings("service.name", "cart", "le", "10"),
Floats: []promql.FPoint{{T: firstTimestamp, F: 10}, {T: secondTimestamp, F: 4}},
},
{
Metric: labels.FromStrings("service.name", "cart", "le", "+Inf"),
Floats: []promql.FPoint{{T: firstTimestamp, F: 11}, {T: secondTimestamp, F: 4}},
},
{
Metric: labels.FromStrings("service.name", "pay", "le", "5"),
Floats: []promql.FPoint{{T: firstTimestamp, F: 0}, {T: secondTimestamp, F: 0}},
},
{
Metric: labels.FromStrings("service.name", "pay", "le", "10"),
Floats: []promql.FPoint{{T: firstTimestamp, F: 2}, {T: secondTimestamp, F: 0}},
},
{
Metric: labels.FromStrings("service.name", "pay", "le", "+Inf"),
Floats: []promql.FPoint{{T: firstTimestamp, F: 2}, {T: secondTimestamp, F: 6}},
},
}
data, err := foldMatrixAsHeatmap(matrix, &qbv5.TimeRange{From: 1710000000000, To: 1710000120000}, uint64(time.Minute.Milliseconds()), "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
aggregation := data.Aggregations[0]
// +Inf is not a boundary; it is the slot past the last one
assert.Equal(t, []float64{5, 10}, aggregation.Meta.Buckets)
require.Len(t, aggregation.Series, 2)
cart := aggregation.Series[0]
require.Len(t, cart.Labels, 1)
assert.Equal(t, "service.name", cart.Labels[0].Key.Name)
assert.Equal(t, "cart", cart.Labels[0].Value)
require.Len(t, cart.Values, 2)
assert.Equal(t, firstTimestamp, cart.Values[0].Timestamp)
assert.Equal(t, []float64{3, 7, 1}, cart.Values[0].Values)
assert.Equal(t, []float64{4, 0, 0}, cart.Values[1].Values)
pay := aggregation.Series[1]
assert.Equal(t, "pay", pay.Labels[0].Value)
assert.Equal(t, []float64{0, 2, 0}, pay.Values[0].Values)
assert.Equal(t, []float64{0, 0, 6}, pay.Values[1].Values)
}
func TestToResultShapesAHeatmapRequestAsCells(t *testing.T) {
at := int64(1710000000000)
q := &promqlQuery{
query: qbv5.PromQuery{Name: "A", Step: qbv5.Step{Duration: time.Minute}},
tr: qbv5.TimeRange{From: 1710000000000, To: 1710000060000},
requestType: qbv5.RequestTypeHeatmap,
}
matrix := promql.Matrix{
{Metric: labels.FromStrings("le", "5"), Floats: []promql.FPoint{{T: at, F: 3}}},
{Metric: labels.FromStrings("le", "+Inf"), Floats: []promql.FPoint{{T: at, F: 8}}},
}
var mu sync.Mutex
var rows, bytes uint64
result, err := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes)
require.NoError(t, err)
assert.Equal(t, qbv5.RequestTypeHeatmap, result.Type)
tsData, ok := result.Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
require.Len(t, tsData.Aggregations, 1)
assert.Equal(t, []float64{5}, tsData.Aggregations[0].Meta.Buckets)
point := tsData.Aggregations[0].Series[0].Values[0]
// counts, not a single value: the +Inf series becomes the overflow slot
assert.Equal(t, []float64{3, 5}, point.Values)
assert.Zero(t, point.Value)
}
func TestToResultRefusesAHeatmapRequestWithoutTheBucketLabel(t *testing.T) {
q := &promqlQuery{
query: qbv5.PromQuery{Name: "A", Step: qbv5.Step{Duration: time.Minute}},
tr: qbv5.TimeRange{From: 1710000000000, To: 1710000060000},
requestType: qbv5.RequestTypeHeatmap,
}
matrix := promql.Matrix{
{Metric: labels.FromStrings("service.name", "cart"), Floats: []promql.FPoint{{T: 1710000000000, F: 3}}},
}
var mu sync.Mutex
var rows, bytes uint64
result, err := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes)
require.Error(t, err)
assert.Nil(t, result)
}
// The cache key is the fingerprint alone, so two request types over one
// expression must not produce the same one — a time series payload served to a
// heatmap request has no axis and reads back as a single collapsed band.
func TestFingerprintSeparatesHeatmapFromTimeSeries(t *testing.T) {
fingerprintFor := func(requestType qbv5.RequestType) string {
q := &promqlQuery{
logger: slog.New(slog.DiscardHandler),
query: qbv5.PromQuery{Name: "A", Query: "sum by (le) (increase(signoz_latency_bucket[5m]))", Step: qbv5.Step{Duration: time.Minute}},
tr: qbv5.TimeRange{From: 1710000000000, To: 1710003600000},
requestType: requestType,
}
return q.Fingerprint()
}
heatmap := fingerprintFor(qbv5.RequestTypeHeatmap)
timeSeries := fingerprintFor(qbv5.RequestTypeTimeSeries)
assert.NotEmpty(t, heatmap, "a heatmap decomposes into time buckets like a time series")
assert.NotEqual(t, timeSeries, heatmap)
assert.Empty(t, fingerprintFor(qbv5.RequestTypeScalar), "a scalar result is its window's last point")
}
func TestFoldMatrixAsHeatmapRefusesAMatrixWithoutTheBucketLabel(t *testing.T) {
matrix := promql.Matrix{
{
Metric: labels.FromStrings("service.name", "cart"),
Floats: []promql.FPoint{{T: 1710000000000, F: 3}},
},
}
data, err := foldMatrixAsHeatmap(matrix, &qbv5.TimeRange{From: 1710000000000, To: 1710000060000}, uint64(time.Minute.Milliseconds()), "A")
require.Error(t, err)
assert.Nil(t, data)
assert.Contains(t, err.Error(), `"le"`)
}
func TestFoldMatrixAsHeatmapAcceptsAnEmptyMatrix(t *testing.T) {
data, err := foldMatrixAsHeatmap(promql.Matrix{}, &qbv5.TimeRange{From: 1710000000000, To: 1710000060000}, uint64(time.Minute.Milliseconds()), "A")
require.NoError(t, err)
assert.Equal(t, "A", data.QueryName)
assert.Empty(t, data.Aggregations)
}
func TestFoldMatrixAsHeatmapClampsADecreasingCumulativeCount(t *testing.T) {
at := int64(1710000000000)
matrix := promql.Matrix{
{
Metric: labels.FromStrings("le", "5"),
Floats: []promql.FPoint{{T: at, F: 10}},
},
{
Metric: labels.FromStrings("le", "10"),
Floats: []promql.FPoint{{T: at, F: 4}},
},
}
data, err := foldMatrixAsHeatmap(matrix, &qbv5.TimeRange{From: 1710000000000, To: 1710000060000}, uint64(time.Minute.Milliseconds()), "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
// a cumulative count that went backwards would difference to -6
assert.Equal(t, []float64{10, 0, 0}, data.Aggregations[0].Series[0].Values[0].Values)
}
func TestFoldMatrixAsHeatmapWidensTheBandOverAMissingBoundary(t *testing.T) {
at := int64(1710000000000)
matrix := promql.Matrix{
{
Metric: labels.FromStrings("le", "5"),
Floats: []promql.FPoint{{T: at, F: 3}},
},
{
Metric: labels.FromStrings("le", "10"),
Floats: []promql.FPoint{{T: at, F: math.NaN()}},
},
{
Metric: labels.FromStrings("le", "20"),
Floats: []promql.FPoint{{T: at, F: 30}},
},
}
data, err := foldMatrixAsHeatmap(matrix, &qbv5.TimeRange{From: 1710000000000, To: 1710000060000}, uint64(time.Minute.Milliseconds()), "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
aggregation := data.Aggregations[0]
// 10 carried nothing to difference against, so it is not on the axis at all
// and 20 differences against 5, holding what (5,10] and (10,20] would split
assert.Equal(t, []float64{5, 20}, aggregation.Meta.Buckets)
assert.Equal(t, []float64{3, 27, 0}, aggregation.Series[0].Values[0].Values)
}
func TestFoldMatrixAsHeatmapHidesInternalLabels(t *testing.T) {
at := int64(1710000000000)
matrix := promql.Matrix{
{
Metric: labels.FromStrings("__temporality__", "delta", "__resource.host.name", "h1", "service.name", "cart", "le", "5"),
Floats: []promql.FPoint{{T: at, F: 3}},
},
}
data, err := foldMatrixAsHeatmap(matrix, &qbv5.TimeRange{From: 1710000000000, To: 1710000060000}, uint64(time.Minute.Milliseconds()), "A")
require.NoError(t, err)
require.Len(t, data.Aggregations, 1)
require.Len(t, data.Aggregations[0].Series, 1)
series := data.Aggregations[0].Series[0]
require.Len(t, series.Labels, 1)
assert.Equal(t, "service.name", series.Labels[0].Key.Name)
}

View File

@@ -155,12 +155,7 @@ func (q *promqlQuery) Fingerprint() string {
if q.opts.serve != nil {
return ""
}
// Only a result that is one value per timestamp, or one vector of counts
// per timestamp, decomposes into cacheable time buckets. A scalar result is
// its window's last point, which says nothing about any sub-range of it.
switch q.requestType {
case qbv5.RequestTypeTimeSeries, qbv5.RequestTypeHeatmap:
default:
if q.requestType != qbv5.RequestTypeTimeSeries {
return ""
}
@@ -171,10 +166,6 @@ func (q *promqlQuery) Fingerprint() string {
}
parts := []string{
"promql",
// the cache key is the fingerprint alone, and a heatmap and a time
// series query over one expression return different shapes, so the
// request type has to separate their entries
fmt.Sprintf("requestType=%s", q.requestType.StringValue()),
query,
q.query.Step.String(),
}
@@ -378,7 +369,7 @@ func (q *promqlQuery) Execute(ctx context.Context) (*qbv5.Result, error) {
}
return nil, err
}
return q.toResult(matrix, nil, began, &statsMu, &rowsScanned, &bytesScanned)
return q.toResult(matrix, nil, began, &statsMu, &rowsScanned, &bytesScanned), nil
}
// When the serving provider has the RangeExecutor capability
@@ -394,7 +385,7 @@ func (q *promqlQuery) Execute(ctx context.Context) (*qbv5.Result, error) {
return nil, err
}
if served {
return q.toResult(matrix, nil, began, &statsMu, &rowsScanned, &bytesScanned)
return q.toResult(matrix, nil, began, &statsMu, &rowsScanned, &bytesScanned), nil
}
}
@@ -455,48 +446,20 @@ func (q *promqlQuery) Execute(ctx context.Context) (*qbv5.Result, error) {
}
warnings, _ := res.Warnings.AsStrings(query, 10, 0)
return q.toResult(matrix, warnings, began, &statsMu, &rowsScanned, &bytesScanned)
}
// excludePromLabel hides only known SigNoz storage keys: label names are user
// data and may legitimately start with "__" (e.g. __address__), so a blanket
// dunder strip mangles user labelsets. The __scope./__resource. prefixes cover
// every exporter version's keys.
func excludePromLabel(labelName string) bool {
return labelName == "__temporality__" ||
strings.HasPrefix(labelName, "__scope.") ||
strings.HasPrefix(labelName, "__resource.")
}
// collectExecStats snapshots the scan counters a query accumulated. Callers take
// it at the point they are done with the matrix, so the duration covers the
// shaping they did.
func collectExecStats(began time.Time, statsMu *sync.Mutex, rowsScanned, bytesScanned *uint64) qbv5.ExecStats {
statsMu.Lock()
defer statsMu.Unlock()
return qbv5.ExecStats{
RowsScanned: *rowsScanned,
BytesScanned: *bytesScanned,
DurationMS: uint64(time.Since(began).Milliseconds()),
}
return q.toResult(matrix, warnings, began, &statsMu, &rowsScanned, &bytesScanned), nil
}
// toResult converts an evaluated matrix into the v5 result shape, attaching
// the ClickHouse scan stats accumulated during evaluation.
func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began time.Time, statsMu *sync.Mutex, rowsScanned, bytesScanned *uint64) (*qbv5.Result, error) {
// A heatmap reads one label as its Y axis and returns a count per band, so
// the per-series copy below cannot produce it.
if q.requestType == qbv5.RequestTypeHeatmap {
tsData, err := foldMatrixAsHeatmap(matrix, &q.tr, uint64(q.query.Step.Milliseconds()), q.query.Name)
if err != nil {
return nil, err
}
return &qbv5.Result{
Type: q.requestType,
Value: tsData,
Warnings: warnings,
Stats: collectExecStats(began, statsMu, rowsScanned, bytesScanned),
}, nil
func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began time.Time, statsMu *sync.Mutex, rowsScanned, bytesScanned *uint64) *qbv5.Result {
// Hide only known SigNoz storage keys: label names are user data and may
// legitimately start with "__" (e.g. __address__), so a blanket dunder
// strip mangles user labelsets. The __scope./__resource. prefixes cover
// every exporter version's keys.
excludeLabel := func(labelName string) bool {
return labelName == "__temporality__" ||
strings.HasPrefix(labelName, "__scope.") ||
strings.HasPrefix(labelName, "__resource.")
}
var series []*qbv5.TimeSeries
@@ -504,7 +467,7 @@ func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began ti
var s qbv5.TimeSeries
lbls := make([]*qbv5.Label, 0, v.Metric.Len())
v.Metric.Range(func(l labels.Label) {
if excludePromLabel(l.Name) {
if excludeLabel(l.Name) {
return
}
lbls = append(lbls, &qbv5.Label{
@@ -532,7 +495,13 @@ func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began ti
series = append(series, &s)
}
stats := collectExecStats(began, statsMu, rowsScanned, bytesScanned)
statsMu.Lock()
stats := qbv5.ExecStats{
RowsScanned: *rowsScanned,
BytesScanned: *bytesScanned,
DurationMS: uint64(time.Since(began).Milliseconds()),
}
statsMu.Unlock()
tsData := &qbv5.TimeSeriesData{QueryName: q.query.Name}
// No bucket at all when nothing survived: a bucket holding no series reads
@@ -565,5 +534,5 @@ func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began ti
Value: payload,
Warnings: warnings,
Stats: stats,
}, nil
}
}

View File

@@ -495,8 +495,7 @@ func TestToResultDropsNonFiniteValues(t *testing.T) {
var mu sync.Mutex
var rows, bytes uint64
result, err := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes)
require.NoError(t, err)
result := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes)
tsData, ok := result.Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
@@ -527,9 +526,7 @@ func TestToResultDropsSeriesAndBucketLeftEmpty(t *testing.T) {
var mu sync.Mutex
var rows, bytes uint64
result, err := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes)
require.NoError(t, err)
tsData, ok := result.Value.(*qbv5.TimeSeriesData)
tsData, ok := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes).Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
require.Len(t, tsData.Aggregations, 1)
require.Len(t, tsData.Aggregations[0].Series, 1, "the all-NaN series is gone")
@@ -538,9 +535,7 @@ func TestToResultDropsSeriesAndBucketLeftEmpty(t *testing.T) {
allNaN := promql.Matrix{
{Metric: labels.FromStrings("job_name", "idleJob"), Floats: []promql.FPoint{{T: 1000, F: math.NaN()}}},
}
result, err = q.toResult(allNaN, nil, time.Now(), &mu, &rows, &bytes)
require.NoError(t, err)
tsData, ok = result.Value.(*qbv5.TimeSeriesData)
tsData, ok = q.toResult(allNaN, nil, time.Now(), &mu, &rows, &bytes).Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
assert.Empty(t, tsData.Aggregations)
}

View File

@@ -156,7 +156,7 @@ func (q *querier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtype
// We need to set if it is unspecified or adjust it if value is not within recommended range
intervalWarnings := q.adjustStepInterval(req.CompositeQuery.Queries, req.Start, req.End)
missingMetricQueries, metricWarnings, err := q.resolveMetricMetadata(ctx, orgID, req.CompositeQuery.Queries, req.Start, req.End, req.RequestType, req.BucketOptions)
missingMetricQueries, metricWarnings, err := q.resolveMetricMetadata(ctx, orgID, req.CompositeQuery.Queries, req.Start, req.End)
if err != nil {
return nil, err
}
@@ -177,7 +177,7 @@ func (q *querier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtype
preseededResults := make(map[string]any)
for _, name := range missingMetricQueries {
switch req.RequestType {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
preseededResults[name] = &qbtypes.TimeSeriesData{QueryName: name}
case qbtypes.RequestTypeScalar:
preseededResults[name] = &qbtypes.ScalarData{QueryName: name}
@@ -334,24 +334,15 @@ func (q *querier) buildQueries(
if missingMetricQuerySet[spec.Name] {
continue
}
// A disabled query in a heatmap request is there to feed a
// formula, and the formula evaluator reads
// TimeSeriesValue.Value, which heatmap cells leave at zero in
// favour of Values. Its inputs therefore run as time series;
// applyFormulas buckets the formula's output into cells after.
requestType := req.RequestType
if requestType == qbtypes.RequestTypeHeatmap && spec.Disabled {
requestType = qbtypes.RequestTypeTimeSeries
}
spec.ShiftBy = extractShiftFromBuilderQuery(spec)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, requestType)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, req.RequestType)
var bq *builderQuery[qbtypes.MetricAggregation]
if spec.Source == telemetrytypes.SourceMeter {
event.Source = telemetrytypes.SourceMeter.StringValue()
bq = newBuilderQuery(q.logger, q.telemetryStore, orgID, q.meterStmtBuilder, query.Type, spec, timeRange, requestType, tmplVars, builderConfig{})
bq = newBuilderQuery(q.logger, q.telemetryStore, orgID, q.meterStmtBuilder, query.Type, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
} else {
bq = newBuilderQuery(q.logger, q.telemetryStore, orgID, q.metricStmtBuilder, query.Type, spec, timeRange, requestType, tmplVars, builderConfig{})
bq = newBuilderQuery(q.logger, q.telemetryStore, orgID, q.metricStmtBuilder, query.Type, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
}
queries[spec.Name] = bq
@@ -424,7 +415,7 @@ func (q *querier) populateQBEvent(event *qbtypes.QBEvent, queries []qbtypes.Quer
// resolved: never-seen metrics and dormant metrics (seen but no data in
// the query window).
// - err: Internal when a metadata fetch fails.
func (q *querier) resolveMetricMetadata(ctx context.Context, orgID valuer.UUID, queries []qbtypes.QueryEnvelope, start, end uint64, requestType qbtypes.RequestType, bucketOptions *qbtypes.BucketOptions) (missingMetricQueries []string, metricWarnings []string, err error) {
func (q *querier) resolveMetricMetadata(ctx context.Context, orgID valuer.UUID, queries []qbtypes.QueryEnvelope, start, end uint64) (missingMetricQueries []string, metricWarnings []string, err error) {
metricNames := make([]string, 0)
for idx := range queries {
if queries[idx].Type != qbtypes.QueryTypeBuilder {
@@ -474,15 +465,6 @@ func (q *querier) resolveMetricMetadata(ctx context.Context, orgID valuer.UUID,
spec.Aggregations[i].Type = foundMetricType
}
}
// Only the enabled query draws cells, so only it needs an axis and
// the metric-type refusals that come with one. A heatmap refuses an
// unresolved type outright rather than returning an empty result for
// it, so this has to run before the drop below.
if requestType == qbtypes.RequestTypeHeatmap && !spec.Disabled {
if err := spec.Aggregations[i].ResolveHeatmapBucketing(bucketOptions); err != nil {
return nil, nil, err
}
}
if spec.Aggregations[i].Type == metrictypes.UnspecifiedType {
missingMetrics = append(missingMetrics, spec.Aggregations[i].MetricName)
continue
@@ -1018,7 +1000,7 @@ func (q *querier) mergeResults(cached *qbtypes.Result, fresh []*qbtypes.Result)
// Merge all fresh results including the first one
switch merged.Type {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
// Pass nil as cached value to ensure proper merging of all fresh results
merged.Value = q.mergeTimeSeriesResults(nil, fresh)
}
@@ -1041,7 +1023,7 @@ func (q *querier) mergeResults(cached *qbtypes.Result, fresh []*qbtypes.Result)
}
switch merged.Type {
case qbtypes.RequestTypeTimeSeries, qbtypes.RequestTypeHeatmap:
case qbtypes.RequestTypeTimeSeries:
merged.Value = q.mergeTimeSeriesResults(cached.Value.(*qbtypes.TimeSeriesData), fresh)
}
@@ -1070,23 +1052,12 @@ func (q *querier) mergeTimeSeriesResults(cachedValue *qbtypes.TimeSeriesData, fr
// Map to store aggregation bucket metadata
bucketMetadata := make(map[int]*qbtypes.AggregationBucket)
// Both halves are moved onto the union of their axes before being merged
// positionally, so a band one range never reached reads as zero there.
axes := make([]*qbtypes.TimeSeriesData, 0, len(freshResults)+1)
axes = append(axes, cachedValue)
for _, result := range freshResults {
freshTS, _ := result.Value.(*qbtypes.TimeSeriesData)
axes = append(axes, freshTS)
}
mergedBoundaries := qbtypes.MergeHeatmapAxes(axes...)
// Process cached data if available
if cachedValue != nil && cachedValue.Aggregations != nil {
for _, aggBucket := range cachedValue.Aggregations {
if seriesMap[aggBucket.Index] == nil {
seriesMap[aggBucket.Index] = make(map[string]*qbtypes.TimeSeries)
}
qbtypes.RealignHeatmapValues(aggBucket.Series, aggBucket.Meta.Buckets, mergedBoundaries[aggBucket.Index])
if bucketMetadata[aggBucket.Index] == nil {
bucketMetadata[aggBucket.Index] = aggBucket
}
@@ -1138,7 +1109,6 @@ func (q *querier) mergeTimeSeriesResults(cachedValue *qbtypes.TimeSeriesData, fr
}
for _, aggBucket := range freshTS.Aggregations {
qbtypes.RealignHeatmapValues(aggBucket.Series, aggBucket.Meta.Buckets, mergedBoundaries[aggBucket.Index])
for _, series := range aggBucket.Series {
key := qbtypes.GetUniqueSeriesKey(series.Labels)
@@ -1202,9 +1172,6 @@ func (q *querier) mergeTimeSeriesResults(cachedValue *qbtypes.TimeSeriesData, fr
bucket.Alias = metadata.Alias
bucket.Meta = metadata.Meta
}
if boundaries, ok := mergedBoundaries[index]; ok {
bucket.Meta.Buckets = boundaries
}
result.Aggregations = append(result.Aggregations, bucket)
}

View File

@@ -451,9 +451,9 @@ func (aH *APIHandler) RegisterRoutes(router *mux.Router, am *middleware.AuthZ) {
router.HandleFunc("/api/v1/disks", am.ViewAccess(aH.getDisks)).Methods(http.MethodGet)
// Quick Filters
// Quick Filters (v1 routes serve the legacy v3 shape; v2 lives in signozapiserver)
router.HandleFunc("/api/v1/orgs/me/filters", am.ViewAccess(aH.Signoz.Handlers.QuickFilter.GetQuickFilters)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/orgs/me/filters/{signal}", am.ViewAccess(aH.Signoz.Handlers.QuickFilter.GetSignalFilters)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/orgs/me/filters/{signal}", am.ViewAccess(aH.Signoz.Handlers.QuickFilter.GetSourceFilters)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/orgs/me/filters", am.AdminAccess(aH.Signoz.Handlers.QuickFilter.UpdateQuickFilters)).Methods(http.MethodPut)
router.HandleFunc("/api/v1/register", am.OpenAccess(aH.registerUser)).Methods(http.MethodPost)

View File

@@ -333,7 +333,7 @@ func (m *Manager) validateChannels(ctx context.Context, orgID string, rule *rule
known := make(map[string]struct{}, len(orgChannels))
for _, ch := range orgChannels {
known[ch.Name] = struct{}{}
known[ch.DisplayName] = struct{}{}
}
var unknown []string

View File

@@ -127,7 +127,7 @@ func NewHandlers(
FlaggerHandler: flagger.NewHandler(flaggerService),
GatewayHandler: gateway.NewHandler(gatewayService),
Fields: implfields.NewHandler(providerSettings, telemetryMetadataStore),
AIObservability: implaiobservability.NewHandler(telemetryMetadataStore),
AIObservability: implaiobservability.NewHandler(providerSettings, telemetryMetadataStore),
AuthzHandler: signozauthzapi.NewHandler(authz),
ZeusHandler: zeus.NewHandler(zeusService, licensingService),
LicensingHandler: licensing.NewHandler(licensingService),

View File

@@ -30,6 +30,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/preference"
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/modules/quickfilter"
"github.com/SigNoz/signoz/pkg/modules/rawdataexport"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/savedview"
@@ -97,6 +98,8 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
struct{ ruler.Handler }{},
struct{ statsreporter.Handler }{},
struct{ savedview.Handler }{},
struct{ quickfilter.Module }{},
struct{ quickfilter.Handler }{},
).New(ctx, instrumentation.ToProviderSettings(), apiserver.Config{})
if err != nil {
return nil, err

View File

@@ -247,6 +247,9 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddDeploymentHostTuplesFactory(sqlstore),
sqlmigration.NewAddSystemDashboardFactory(sqlstore, sqlschema),
sqlmigration.NewAddLicenseTuplesFactory(sqlstore),
sqlmigration.NewAddChannelDisplayNameFactory(sqlstore, sqlschema),
sqlmigration.NewMigrateQuickFiltersFactory(sqlstore),
sqlmigration.NewAddQuickFilterTuplesFactory(sqlstore),
)
}
@@ -352,6 +355,8 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
handlers.RulerHandler,
handlers.StatsHandler,
handlers.SavedView,
modules.QuickFilter,
handlers.QuickFilter,
),
)
}

View File

@@ -153,8 +153,23 @@ func (migration *addAlertmanager) populateOrgIDInChannels(ctx context.Context, t
return nil
}
// storableChannel is the channel table as it stood at this migration.
// alertmanagertypes.Channel has grown columns since, and reading through it here
// would put those columns into statements that run before they exist.
type storableChannel struct {
bun.BaseModel `bun:"table:notification_channel"`
ID string `bun:"id,pk,type:text"`
CreatedAt time.Time `bun:"created_at"`
UpdatedAt time.Time `bun:"updated_at"`
Name string `bun:"name"`
Type string `bun:"type"`
Data string `bun:"data"`
OrgID string `bun:"org_id"`
}
func (migration *addAlertmanager) populateAlertmanagerConfig(ctx context.Context, tx bun.Tx, orgID string) error {
var channels []*alertmanagertypes.Channel
var channels []*storableChannel
err := tx.
NewSelect().
@@ -205,7 +220,13 @@ func (migration *addAlertmanager) populateAlertmanagerConfig(ctx context.Context
}
}
config, err := alertmanagertypes.NewConfigFromChannels(alertmanagerserver.NewConfig().Global, alertmanagerserver.NewConfig().Route, channels, orgID)
// NewConfigFromChannels reads nothing off a channel but Data.
receiverChannels := make(alertmanagertypes.Channels, 0, len(channels))
for _, channel := range channels {
receiverChannels = append(receiverChannels, &alertmanagertypes.Channel{Data: channel.Data})
}
config, err := alertmanagertypes.NewConfigFromChannels(alertmanagerserver.NewConfig().Global, alertmanagerserver.NewConfig().Route, receiverChannels, orgID)
if err != nil {
return err
}
@@ -273,7 +294,7 @@ func newReceiver(input string) (config.Receiver, error) {
return receiverWithDefaults, nil
}
func (migration *addAlertmanager) msTeamsChannelToMSTeamsV2Channel(c *alertmanagertypes.Channel) error {
func (migration *addAlertmanager) msTeamsChannelToMSTeamsV2Channel(c *storableChannel) error {
if c.Type != "msteams" {
return nil
}

View File

@@ -0,0 +1,172 @@
package sqlmigration
import (
"context"
"crypto/rand"
"strings"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
)
type addChannelDisplayName struct {
sqlstore sqlstore.SQLStore
sqlschema sqlschema.SQLSchema
}
func NewAddChannelDisplayNameFactory(sqlstore sqlstore.SQLStore, sqlschema sqlschema.SQLSchema) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(
factory.MustNewName("channel_display_name"),
func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &addChannelDisplayName{sqlstore: sqlstore, sqlschema: sqlschema}, nil
},
)
}
func (migration *addChannelDisplayName) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
// Up moves the free-text name onto display_name and gives name the DNS1123
// identity, matching organizations and dashboards. Nothing outside the channel
// reads the new name yet: routing policies and rules still reference
// display_name, and the alertmanager config still keys receivers by that same
// string.
func (migration *addChannelDisplayName) Up(ctx context.Context, db *bun.DB) error {
// Adding a NOT NULL column rebuilds the whole table on SQLite (create temp,
// copy, drop, rename), and notification_channel has a foreign key on org_id
// that the copy would re-validate. Enforcement stays off for the rebuild.
if err := migration.sqlschema.ToggleFKEnforcement(ctx, db, false); err != nil {
return err
}
table, uniqueConstraints, err := migration.sqlschema.GetTable(ctx, sqlschema.TableName("notification_channel"))
if err != nil {
return err
}
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() {
_ = tx.Rollback()
}()
if _, err := migration.sqlstore.Dialect().RenameColumn(ctx, tx, "notification_channel", "name", "display_name"); err != nil {
return err
}
// The table was inspected before the rename, and the recreate-table fallback
// below rebuilds the table from this description, so it has to follow.
for _, column := range table.Columns {
if column.Name == sqlschema.ColumnName("name") {
column.Name = sqlschema.ColumnName("display_name")
}
}
nameColumn := &sqlschema.Column{
Name: sqlschema.ColumnName("name"),
DataType: sqlschema.DataTypeText,
Nullable: false,
}
sqls := migration.sqlschema.Operator().AddColumn(table, uniqueConstraints, nameColumn, "")
for _, sql := range sqls {
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
return err
}
}
type channel struct {
bun.BaseModel `bun:"table:notification_channel"`
ID valuer.UUID `bun:"id,pk"`
DisplayName string `bun:"display_name"`
}
var channels []channel
if err := tx.
NewSelect().
Model(&channels).
Column("id", "display_name").
Scan(ctx); err != nil {
return err
}
for _, existing := range channels {
if _, err := tx.
NewUpdate().
Model((*channel)(nil)).
Set("name = ?", slugifyChannelName(existing.DisplayName)).
Where("id = ?", existing.ID).
Exec(ctx); err != nil {
return err
}
}
indexSQLs := migration.sqlschema.Operator().CreateIndex(&sqlschema.UniqueIndex{
TableName: "notification_channel",
ColumnNames: []sqlschema.ColumnName{"org_id", "name"},
})
for _, sql := range indexSQLs {
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
return err
}
}
if err := tx.Commit(); err != nil {
return err
}
return migration.sqlschema.ToggleFKEnforcement(ctx, db, true)
}
func (migration *addChannelDisplayName) Down(context.Context, *bun.DB) error {
return nil
}
const migrationChannelNameSuffixLen = 8
// slugifyChannelName is a copy of dashboardtypes.generateDashboardName. The
// random suffix is what makes the unique index safe to add without a collision
// loop over the existing free-text names.
func slugifyChannelName(displayName string) string {
const dns1123LabelMaxLen = 63
suffixAlphabet := []byte("abcdefghijklmnopqrstuvwxyz0123456789")
var b strings.Builder
b.Grow(len(displayName))
prevHyphen := false
for _, r := range strings.ToLower(displayName) {
switch {
case (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9'):
b.WriteRune(r)
prevHyphen = false
case b.Len() > 0 && !prevHyphen:
b.WriteByte('-')
prevHyphen = true
}
}
prefix := strings.TrimRight(b.String(), "-")
suffix := make([]byte, migrationChannelNameSuffixLen)
if _, err := rand.Read(suffix); err != nil {
panic(err)
}
for i := range suffix {
suffix[i] = suffixAlphabet[int(suffix[i])%len(suffixAlphabet)]
}
maxPrefix := dns1123LabelMaxLen - 1 - migrationChannelNameSuffixLen
if len(prefix) > maxPrefix {
prefix = strings.TrimRight(prefix[:maxPrefix], "-")
}
if prefix == "" {
return string(suffix)
}
return prefix + "-" + string(suffix)
}

View File

@@ -0,0 +1,180 @@
package sqlmigration
import (
"context"
"encoding/json"
"log/slog"
"strings"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
)
type storableQuickFilterRow struct {
bun.BaseModel `bun:"table:quick_filter"`
ID string `bun:"id,pk"`
Filter string `bun:"filter"`
}
// legacyQuickFilterEntry carries both shapes a stored entry can be in: the
// legacy key/type/dataType shape and the current name-carrying shape.
type legacyQuickFilterEntry struct {
Name string `json:"name"`
Key string `json:"key"`
Type string `json:"type"`
DataType string `json:"dataType"`
Signal string `json:"signal"`
}
// quickFilterLegacyTypeToFieldContext maps the v3 attribute key types the v1
// write path could store. Materialized top-level fields carried no type, and
// anything unknown (e.g. "Sum" in the old meter defaults) normalizes to
// unspecified, matching what the v1 write path does at runtime.
var quickFilterLegacyTypeToFieldContext = map[string]string{
"tag": "attribute",
"resource": "resource",
"scope": "scope",
}
// quickFilterLegacyDataTypeToFieldDataType maps the v3 attribute key data
// types the v1 write path could store, with numerics collapsed to number,
// matching the fields API and the v1 write path.
var quickFilterLegacyDataTypeToFieldDataType = map[string]string{
"string": "string",
"bool": "bool",
"int64": "number",
"float64": "number",
}
type migrateQuickFilters struct {
sqlstore sqlstore.SQLStore
settings factory.ProviderSettings
}
func NewMigrateQuickFiltersFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(factory.MustNewName("migrate_quick_filters"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &migrateQuickFilters{sqlstore: sqlstore, settings: ps}, nil
})
}
func (migration *migrateQuickFilters) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
func (migration *migrateQuickFilters) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var rows []*storableQuickFilterRow
if err := tx.NewSelect().Model(&rows).Scan(ctx); err != nil {
return err
}
var migrated, skipped int
for _, row := range rows {
migratedFilter, changed, ok := migrateQuickFilterEntries(row.Filter)
if !ok {
migration.settings.Logger.WarnContext(ctx, "quick filter could not be parsed, leaving it untouched", slog.String("quick_filter_id", row.ID), slog.String("raw_filter", row.Filter))
skipped++
continue
}
if !changed {
continue
}
migrated++
if _, err := tx.NewUpdate().Model((*storableQuickFilterRow)(nil)).Set("filter = ?", migratedFilter).Where("id = ?", row.ID).Exec(ctx); err != nil {
return err
}
}
migration.settings.Logger.InfoContext(ctx, "migrated quick filters to telemetry field keys", slog.Int("total", len(rows)), slog.Int("migrated", migrated), slog.Int("skipped", skipped))
if _, err := migration.sqlstore.Dialect().RenameColumn(ctx, tx, "quick_filter", "signal", "source"); err != nil {
return err
}
for _, column := range []string{"created_by", "updated_by"} {
if err := migration.sqlstore.Dialect().DropColumn(ctx, tx, "quick_filter", column); err != nil {
return err
}
}
return tx.Commit()
}
func (migration *migrateQuickFilters) Down(context.Context, *bun.DB) error {
return nil
}
// migrateQuickFilterEntries rewrites a stored filter list from the legacy
// key/dataType/type shape to telemetry field keys; ok=false means unparseable.
func migrateQuickFilterEntries(filter string) (migrated string, changed bool, ok bool) {
var entriesRaw []json.RawMessage
if err := json.Unmarshal([]byte(filter), &entriesRaw); err != nil {
return "", false, false
}
migratedEntries := make([]json.RawMessage, 0, len(entriesRaw))
for _, rawEntry := range entriesRaw {
var entry legacyQuickFilterEntry
if err := json.Unmarshal(rawEntry, &entry); err != nil {
// Some stored entries are plain strings rather than objects; treat
// the string as the filter key name, dropping empty ones.
var name string
if err := json.Unmarshal(rawEntry, &name); err != nil {
return "", false, false
}
entry = legacyQuickFilterEntry{Key: name}
}
switch {
case entry.Name != "":
migratedEntries = append(migratedEntries, rawEntry)
case entry.Key != "":
migratedJSON, err := marshalUnescaped(telemetryFieldKeyOutput{
Name: entry.Key,
Signal: entry.Signal,
FieldContext: quickFilterFieldContext(entry.Type),
FieldDataType: quickFilterFieldDataType(entry.DataType),
})
if err != nil {
return "", false, false
}
migratedEntries = append(migratedEntries, migratedJSON)
changed = true
default:
changed = true
}
}
if !changed {
return "", false, true
}
migratedJSON, err := marshalUnescaped(migratedEntries)
if err != nil {
return "", false, false
}
return string(migratedJSON), true, true
}
// quickFilterFieldDataType resolves legacy datatype spellings, with unknowns
// normalized to unspecified.
func quickFilterFieldDataType(legacyDataType string) string {
return quickFilterLegacyDataTypeToFieldDataType[strings.ToLower(strings.TrimSpace(legacyDataType))]
}
// quickFilterFieldContext resolves legacy type spellings, with unknowns
// normalized to unspecified.
func quickFilterFieldContext(legacyType string) string {
return quickFilterLegacyTypeToFieldContext[strings.ToLower(strings.TrimSpace(legacyType))]
}

View File

@@ -0,0 +1,139 @@
package sqlmigration
import (
"context"
"database/sql"
"time"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/oklog/ulid/v2"
"github.com/uptrace/bun"
"github.com/uptrace/bun/dialect"
"github.com/uptrace/bun/migrate"
)
type addQuickFilterTuples struct {
sqlstore sqlstore.SQLStore
}
func NewAddQuickFilterTuplesFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(factory.MustNewName("add_quick_filter_tuples"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &addQuickFilterTuples{sqlstore: sqlstore}, nil
})
}
func (migration *addQuickFilterTuples) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
func (migration *addQuickFilterTuples) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var storeID string
err = tx.QueryRowContext(ctx, `SELECT id FROM store WHERE name = ? LIMIT 1`, "signoz").Scan(&storeID)
if err != nil {
return err
}
var orgIDs []string
err = tx.NewSelect().
Table("organizations").
Column("id").
Scan(ctx, &orgIDs)
if err != nil && err != sql.ErrNoRows {
return err
}
isPG := migration.sqlstore.BunDB().Dialect().Name() == dialect.PG
// quick-filter moved from the legacy ViewAccess/AdminAccess role gate to
// CheckResources, which on enterprise requires real tuples -- existing orgs
// never had these written, only new orgs get them from the registry at bootstrap.
tuples := []migrationTuple{
{authtypes.SigNozAdminRoleName, "metaresource", "quick-filter", "read"},
{authtypes.SigNozAdminRoleName, "metaresource", "quick-filter", "update"},
{authtypes.SigNozAdminRoleName, "metaresource", "quick-filter", "list"},
{authtypes.SigNozEditorRoleName, "metaresource", "quick-filter", "read"},
{authtypes.SigNozEditorRoleName, "metaresource", "quick-filter", "list"},
{authtypes.SigNozViewerRoleName, "metaresource", "quick-filter", "read"},
{authtypes.SigNozViewerRoleName, "metaresource", "quick-filter", "list"},
}
for _, orgID := range orgIDs {
for _, tuple := range tuples {
entropy := ulid.DefaultEntropy()
now := time.Now().UTC()
tupleID := ulid.MustNew(ulid.Timestamp(now), entropy).String()
objectID := "organization/" + orgID + "/" + tuple.objectName + "/*"
roleSubject := "organization/" + orgID + "/role/" + tuple.roleName
if isPG {
user := "role:" + roleSubject + "#assignee"
result, err := tx.ExecContext(ctx, `
INSERT INTO tuple (store, object_type, object_id, relation, _user, user_type, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, object_type, object_id, relation, _user) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, user, "userset", tupleID, now,
)
if err != nil {
return err
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
continue
}
_, err = tx.ExecContext(ctx, `
INSERT INTO changelog (store, object_type, object_id, relation, _user, operation, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, user, 0, tupleID, now,
)
if err != nil {
return err
}
} else {
result, err := tx.ExecContext(ctx, `
INSERT INTO tuple (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, user_type, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", "userset", tupleID, now,
)
if err != nil {
return err
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
continue
}
_, err = tx.ExecContext(ctx, `
INSERT INTO changelog (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, operation, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", 0, tupleID, now,
)
if err != nil {
return err
}
}
}
}
return tx.Commit()
}
func (migration *addQuickFilterTuples) Down(context.Context, *bun.DB) error {
return nil
}

View File

@@ -0,0 +1,133 @@
package aistatementbuilder
import (
"context"
"testing"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Mixed filter: the span-level part gates the scan, the trace-level part becomes the
// __trace_scope qualification.
func TestBuild_FullSQL_SpanList_TraceScoped(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND trace.output_tokens > 1000"},
Limit: 10,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __trace_scope AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
)
SELECT timestamp AS __SELECT_KEY_0_timestamp, trace_id AS __SELECT_KEY_1_trace_id, span_id AS __SELECT_KEY_2_span_id,
trace_state AS __SELECT_KEY_3_trace_state, parent_span_id AS __SELECT_KEY_4_parent_span_id, flags AS __SELECT_KEY_5_flags,
name AS __SELECT_KEY_6_name, kind AS __SELECT_KEY_7_kind, kind_string AS __SELECT_KEY_8_kind_string, duration_nano AS __SELECT_KEY_9_duration_nano,
status_code AS __SELECT_KEY_10_status_code, status_message AS __SELECT_KEY_11_status_message,
status_code_string AS __SELECT_KEY_12_status_code_string, events AS __SELECT_KEY_13_events, links AS __SELECT_KEY_14_links,
response_status_code AS __SELECT_KEY_15_response_status_code, external_http_url AS __SELECT_KEY_16_external_http_url,
http_url AS __SELECT_KEY_17_http_url, external_http_method AS __SELECT_KEY_18_external_http_method,
http_method AS __SELECT_KEY_19_http_method, http_host AS __SELECT_KEY_20_http_host, db_name AS __SELECT_KEY_21_db_name,
db_operation AS __SELECT_KEY_22_db_operation, has_error AS __SELECT_KEY_23_has_error, is_remote AS __SELECT_KEY_24_is_remote,
attributes_string, attributes_number, attributes_bool, resources_string
FROM signoz_traces.distributed_signoz_index_v3
WHERE trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)
AND (((mapContains(attributes_string, 'gen_ai.request.model')
OR mapContains(attributes_string, 'gen_ai.tool.name')
OR mapContains(attributes_string, 'gen_ai.agent.name')))
AND ((attributes_string['gen_ai.request.model'] = 'gpt-4o-mini'
AND mapContains(attributes_string, 'gen_ai.request.model'))))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
LIMIT 10
`, stmt)
}
// A resource attribute mixed with a trace-level condition: the resource part flows
// through the fingerprint machinery, the trace-level part through __trace_scope.
func TestBuild_SpanList_ResourcePlusTraceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: "resource.service.name = 'checkout' AND trace.output_tokens > 1000"},
Limit: 10,
}, nil)
require.NoError(t, err)
got := renderSQL(t, stmt)
assert.Contains(t, got, "__resource_filter AS (")
assert.Contains(t, got, "resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
assert.Contains(t, got, "__trace_scope AS (")
assert.Contains(t, got, "trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)")
assert.Contains(t, got, "HAVING output_tokens > 1000")
}
// Trace-level order keys are rejected — known aggregate alias or not — while a bare
// span column sharing an alias (duration_nano) stays orderable.
func TestBuild_SpanList_OrderKeyValidation(t *testing.T) {
b := newTestBuilder(t)
build := func(q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) error {
q.Signal = telemetrytypes.SignalTraces
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw, q, nil)
return err
}
err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.output_tokens"}}}},
})
require.ErrorContains(t, err, `ordering the span list by trace-level key "trace.output_tokens" is not supported`)
err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.foo"}}}},
})
require.ErrorContains(t, err, `trace-level key "trace.foo"`)
err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "duration_nano"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 10,
})
require.NoError(t, err, "bare duration_nano is a span column, not a trace-level key")
}
// Variables in a trace-level condition resolve through the standard pipeline; a
// dynamic __all__ drops the condition (no scope CTE).
func TestBuild_SpanList_TraceFilter_Variables(t *testing.T) {
b := newTestBuilder(t)
build := func(expr string, vars map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Filter: &qbtypes.Filter{Expression: expr},
Limit: 10,
}, vars)
}
stmt, err := build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Value: 700}})
require.NoError(t, err)
assert.Contains(t, renderSQL(t, stmt), "HAVING output_tokens > 700")
stmt, err = build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.DynamicVariableType, Value: "__all__"}})
require.NoError(t, err)
assert.NotContains(t, stmt.Query, "__trace_scope")
}

View File

@@ -1,8 +1,6 @@
package aistatementbuilder
import (
"strings"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/statementbuilder"
@@ -27,11 +25,8 @@ func NewFactory(
// Scope describes gen_ai for the scoped trace builder: an AI trace has >=1 gen_ai
// LLM, tool, or agent span, and its list adds AI/LLM per-trace metrics.
func Scope() scopedtraces.TraceScope {
gateKeyNames := []string{aiobservabilitytypes.GenAIRequestModel, aiobservabilitytypes.GenAIToolName, aiobservabilitytypes.GenAIAgentName}
gateExprs := make([]string, 0, len(gateKeyNames))
gateKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(gateKeyNames))
for _, name := range gateKeyNames {
gateExprs = append(gateExprs, name+" EXISTS")
gateKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(aiobservabilitytypes.GenAISpanGateKeys))
for _, name := range aiobservabilitytypes.GenAISpanGateKeys {
gateKeys = append(gateKeys, &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
@@ -79,7 +74,7 @@ func Scope() scopedtraces.TraceScope {
}
return scopedtraces.TraceScope{
FilterExpression: strings.Join(gateExprs, " OR "),
FilterExpression: aiobservabilitytypes.GenAISpanFilterExpression(),
FieldKeys: gateKeys,
Columns: columns,
DefaultOrderAlias: "last_activity_time",

View File

@@ -129,7 +129,7 @@ func (b *meterQueryStatementBuilder) buildPipelineStatement(
}
// final SELECT
return b.metricsStatementBuilder.BuildFinalSelect(cteFragments, cteArgs, qbtypes.RequestTypeTimeSeries, query)
return b.metricsStatementBuilder.BuildFinalSelect(cteFragments, cteArgs, query)
}
func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(

View File

@@ -4,13 +4,9 @@ import (
"context"
"fmt"
"log/slog"
"math"
"slices"
"strconv"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
@@ -117,7 +113,7 @@ func (b *StatementBuilder) Build(
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
_ qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
@@ -129,14 +125,13 @@ func (b *StatementBuilder) Build(
start, end = querybuilder.AdjustedMetricTimeRange(start, end, uint64(query.StepInterval.Seconds()), query)
return b.buildPipelineStatement(ctx, orgID, start, end, requestType, query, keys, variables)
return b.buildPipelineStatement(ctx, orgID, start, end, query, keys, variables)
}
func (b *StatementBuilder) buildPipelineStatement(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
@@ -149,7 +144,7 @@ func (b *StatementBuilder) buildPipelineStatement(
cteQuery := query
if query.Aggregations[0].Type == metrictypes.HistogramType {
query.GroupBy = slices.DeleteFunc(slices.Clone(query.GroupBy), isHistogramBucket)
cteQuery = histogramCTEQuery(requestType, query)
cteQuery = histogramCTEQuery(query)
}
agg := cteQuery.Aggregations[0]
@@ -221,7 +216,7 @@ func (b *StatementBuilder) buildPipelineStatement(
}
}
mainStmt, err := b.BuildFinalSelect(cteFragments, cteArgs, requestType, query)
mainStmt, err := b.BuildFinalSelect(cteFragments, cteArgs, query)
if err != nil {
return nil, err
}
@@ -229,7 +224,7 @@ func (b *StatementBuilder) buildPipelineStatement(
if reducedFragments == nil {
return mainStmt, nil
}
reducedStmt, err := b.BuildFinalSelect(reducedFragments, reducedArgs, requestType, query)
reducedStmt, err := b.BuildFinalSelect(reducedFragments, reducedArgs, query)
if err != nil {
return nil, err
}
@@ -763,9 +758,11 @@ func (b *StatementBuilder) buildSpatialAggregationCTE(
func (b *StatementBuilder) BuildFinalSelect(
cteFragments []string,
cteArgs [][]any,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (*qbtypes.Statement, error) {
metricType := query.Aggregations[0].Type
spaceAgg := query.Aggregations[0].SpaceAggregation
combined := querybuilder.CombineCTEs(cteFragments)
var args []any
@@ -773,22 +770,6 @@ func (b *StatementBuilder) BuildFinalSelect(
args = append(args, a...)
}
if requestType == qbtypes.RequestTypeHeatmap {
return buildHeatmapFinalSelect(combined, args, query)
}
return buildAggregationFinalSelect(combined, args, query)
}
// buildAggregationFinalSelect reads __spatial_aggregation_cte as one value per
// (group, timestamp), which is what every request type but heatmap wants.
func buildAggregationFinalSelect(
combined string,
args []any,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (*qbtypes.Statement, error) {
metricType := query.Aggregations[0].Type
spaceAgg := query.Aggregations[0].SpaceAggregation
sb := sqlbuilder.NewSelectBuilder()
if metricType == metrictypes.HistogramType && spaceAgg.IsPercentile() {
@@ -861,160 +842,17 @@ func buildAggregationFinalSelect(
return &qbtypes.Statement{Query: combined + q, Args: append(args, a...)}, nil
}
const (
histogramBucketKey = "le"
heatmapValueAlias = "__result_0"
heatmapWindow = "__heatmap_window"
)
const histogramBucketKey = "le"
func isHistogramBucket(k qbtypes.GroupByKey) bool { return k.Name == histogramBucketKey }
// buildHeatmapFinalSelect turns __spatial_aggregation_cte into one row per
// heatmap cell: (ts, group labels..., bucket upper boundary, count). Histograms
// already carry their boundaries as `le` labels; every other metric type has its
// axis derived from the aggregated value itself.
func buildHeatmapFinalSelect(
combined string,
args []any,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (*qbtypes.Statement, error) {
if query.Aggregations[0].Type == metrictypes.HistogramType {
return buildHistogramHeatmapFinalSelect(combined, args, query)
}
return buildValueHeatmapFinalSelect(combined, args, query)
}
// buildHistogramHeatmapFinalSelect differences the cumulative per-`le` counts in
// __spatial_aggregation_cte into a count per band.
//
// `le` labels are cumulative upper bounds, so a bucket's own count is the
// difference against the next-smallest `le` in the same (group, timestamp).
// The boundary reported is the `le` itself, which leaves the `le=+Inf` row
// carrying an infinite boundary for the reader to turn into the open-above
// overflow band.
func buildHistogramHeatmapFinalSelect(
combined string,
args []any,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (*qbtypes.Statement, error) {
groupAliases := GroupByAliases(query.GroupBy)
partitionBy := append(append([]string{}, groupAliases...), "ts")
sb := sqlbuilder.NewSelectBuilder()
sb.Select("ts")
sb.SelectMore(groupAliases...)
sb.SelectMore(fmt.Sprintf("toFloat64(%s) AS %s", histogramBucketKey, qbtypes.HeatmapBucketColumn))
// Counts across `le` should rise monotonically; partial scrapes can break
// that, and a negative cell count has no meaning on a heatmap.
sb.SelectMore(fmt.Sprintf(
"greatest(value - lagInFrame(value, 1, 0) OVER %s, 0) AS %s",
heatmapWindow, heatmapValueAlias,
))
// sqlbuilder has no WINDOW clause, and the fragment has to land between FROM
// and ORDER BY. Heatmap statements never carry a WHERE or GROUP BY here, so
// appending it to FROM puts it in the right place.
sb.From(fmt.Sprintf(
"__spatial_aggregation_cte WINDOW %s AS (PARTITION BY %s ORDER BY toFloat64(%s))",
heatmapWindow, strings.Join(partitionBy, ", "), histogramBucketKey,
))
sb.OrderBy(groupAliases...)
sb.OrderBy("ts", fmt.Sprintf("toFloat64(%s)", histogramBucketKey))
q, a := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return &qbtypes.Statement{Query: combined + q, Args: append(args, a...)}, nil
}
// buildValueHeatmapFinalSelect places each spatially aggregated value in a band
// of the requested axis. __spatial_aggregation_cte holds one row per (group,
// timestamp), so a cell counts the one group it came from; the panel sums
// across the series it is showing, which is what lets the legend select among
// them.
func buildValueHeatmapFinalSelect(
combined string,
args []any,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (*qbtypes.Statement, error) {
bucketing := query.Aggregations[0].HeatmapBucketing
if bucketing == nil {
return nil, errors.NewInternalf(errors.CodeInternal,
"heatmap over a %s metric reached the statement builder without a resolved bucket axis",
query.Aggregations[0].Type.StringValue())
}
boundary, err := heatmapBoundaryExpr(*bucketing)
if err != nil {
return nil, err
}
groupAliases := GroupByAliases(query.GroupBy)
sb := sqlbuilder.NewSelectBuilder()
sb.Select("ts")
sb.SelectMore(groupAliases...)
sb.SelectMore(fmt.Sprintf("%s AS %s", boundary, qbtypes.HeatmapBucketColumn))
sb.SelectMore(fmt.Sprintf("toFloat64(1) AS %s", heatmapValueAlias))
sb.From("__spatial_aggregation_cte")
sb.OrderBy(groupAliases...)
sb.OrderBy("ts", qbtypes.HeatmapBucketColumn)
q, a := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return &qbtypes.Statement{Query: combined + q, Args: append(args, a...)}, nil
}
// heatmapBoundaryExpr renders the upper bound of the band `value` falls in.
// Values at or below zero have no log band of their own and no linear band below
// the first, so both scalings report them at the axis's lowest boundary rather
// than dropping the row: an upper bound still describes them truthfully.
func heatmapBoundaryExpr(bucketing qbtypes.HeatmapBucketing) (string, error) {
switch bucketing.Kind {
case qbtypes.BucketsKindLinear:
maxValue := formatFloat(bucketing.MaxValue)
numBuckets := strconv.Itoa(bucketing.NumBuckets)
// Indexing on value*numBuckets/maxValue rather than on a precomputed
// width keeps the top boundary exactly maxValue instead of a rounded
// multiple of that width.
return fmt.Sprintf(
"multiIf(value > %s, toFloat64('+Inf'), least(greatest(ceil(value * %s / %s), 1), %s) * %s / %s)",
maxValue, numBuckets, maxValue, numBuckets, maxValue, numBuckets,
), nil
case qbtypes.BucketsKindLog:
// The exponential histogram mapping at a fixed scale: 2^LogScale bands
// per doubling makes a band's index a pure function of the value, so the
// data never has to be scanned to decide where the boundaries go.
//
// The two clamps keep the axis finite. Without the lower one a single
// value approaching zero runs the band index off to -inf, and filling
// the empty bands below it would then cost thousands of slots per point.
bandsPerDoubling := formatFloat(math.Exp2(float64(bucketing.LogScale)))
lowest := formatFloat(qbtypes.LowestLogBoundary)
highest := formatFloat(qbtypes.HighestLogBoundary)
return fmt.Sprintf(
"multiIf(value <= 0, toFloat64(0), value <= %s, %s, value > %s, toFloat64('+Inf'), pow(2, ceil(log2(value) * %s) / %s))",
lowest, lowest, highest, bandsPerDoubling, bandsPerDoubling,
), nil
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"unsupported bucketsScaling %q for heatmap requests", bucketing.Kind.StringValue())
}
}
// formatFloat renders a float64 as the shortest literal that reads back as the
// same value, so a boundary computed from it is identical on every row.
func formatFloat(v float64) string {
return strconv.FormatFloat(v, 'g', -1, 64)
}
func histogramCTEQuery(requestType qbtypes.RequestType, query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]) qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation] {
func histogramCTEQuery(query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]) qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation] {
query.GroupBy = append(slices.Clone(query.GroupBy), qbtypes.GroupByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: histogramBucketKey},
})
query.Aggregations = slices.Clone(query.Aggregations)
// A heatmap cell is an observation count whatever space aggregation was
// asked for, since the axis is the `le` labels rather than anything the
// space aggregation picks out. Rates would scale every cell by the step.
if query.Aggregations[0].SpaceAggregation.IsPercentile() && requestType != qbtypes.RequestTypeHeatmap {
if query.Aggregations[0].SpaceAggregation.IsPercentile() {
query.Aggregations[0].TimeAggregation = metrictypes.TimeAggregationRate
} else {
query.Aggregations[0].TimeAggregation = metrictypes.TimeAggregationIncrease

View File

@@ -284,133 +284,6 @@ func TestStatementBuilder(t *testing.T) {
},
expectedErr: nil,
},
{
name: "test_histogram_heatmap_sum",
requestType: qbtypes.RequestTypeHeatmap,
query: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "signoz_latency",
Type: metrictypes.HistogramType,
Temporality: metrictypes.Delta,
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
},
},
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "service.name",
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __spatial_aggregation_cte AS (SELECT toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(60)) AS ts, `__GROUP_BY_KEY_0_service.name`, `le`, sum(value) AS value FROM signoz_metrics.distributed_samples_v4 AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'service.name') AS `__GROUP_BY_KEY_0_service.name`, JSONExtractString(labels, 'le') AS `le` FROM signoz_metrics.time_series_v4_6hrs WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) GROUP BY fingerprint, `__GROUP_BY_KEY_0_service.name`, `le`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY ts, `__GROUP_BY_KEY_0_service.name`, `le`) SELECT ts, `__GROUP_BY_KEY_0_service.name`, toFloat64(le) AS __bucket, greatest(value - lagInFrame(value, 1, 0) OVER __heatmap_window, 0) AS __result_0 FROM __spatial_aggregation_cte WINDOW __heatmap_window AS (PARTITION BY `__GROUP_BY_KEY_0_service.name`, ts ORDER BY toFloat64(le)) ORDER BY `__GROUP_BY_KEY_0_service.name`, ts, toFloat64(le)",
Args: []any{"signoz_latency", uint64(1747936800000), uint64(1747983420000), "delta", "signoz_latency", uint64(1747947360000), uint64(1747983420000)},
},
expectedErr: nil,
},
{
name: "test_histogram_heatmap_percentile",
requestType: qbtypes.RequestTypeHeatmap,
query: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "signoz_latency",
Type: metrictypes.HistogramType,
Temporality: metrictypes.Delta,
TimeAggregation: metrictypes.TimeAggregationRate,
SpaceAggregation: metrictypes.SpaceAggregationPercentile95,
},
},
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "service.name",
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __spatial_aggregation_cte AS (SELECT toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(60)) AS ts, `__GROUP_BY_KEY_0_service.name`, `le`, sum(value) AS value FROM signoz_metrics.distributed_samples_v4 AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'service.name') AS `__GROUP_BY_KEY_0_service.name`, JSONExtractString(labels, 'le') AS `le` FROM signoz_metrics.time_series_v4_6hrs WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) GROUP BY fingerprint, `__GROUP_BY_KEY_0_service.name`, `le`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY ts, `__GROUP_BY_KEY_0_service.name`, `le`) SELECT ts, `__GROUP_BY_KEY_0_service.name`, toFloat64(le) AS __bucket, greatest(value - lagInFrame(value, 1, 0) OVER __heatmap_window, 0) AS __result_0 FROM __spatial_aggregation_cte WINDOW __heatmap_window AS (PARTITION BY `__GROUP_BY_KEY_0_service.name`, ts ORDER BY toFloat64(le)) ORDER BY `__GROUP_BY_KEY_0_service.name`, ts, toFloat64(le)",
Args: []any{"signoz_latency", uint64(1747936800000), uint64(1747983420000), "delta", "signoz_latency", uint64(1747947360000), uint64(1747983420000)},
},
expectedErr: nil,
},
{
name: "test_gauge_heatmap_log",
requestType: qbtypes.RequestTypeHeatmap,
query: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
Temporality: metrictypes.Unspecified,
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
HeatmapBucketing: &qbtypes.HeatmapBucketing{
Kind: qbtypes.BucketsKindLog,
LogScale: qbtypes.MaxLogScale,
NumBuckets: qbtypes.DefaultNumBuckets,
},
},
},
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "host.name",
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(60)) AS ts, `__GROUP_BY_KEY_0_host.name`, avg(value) AS per_series_value FROM signoz_metrics.distributed_samples_v4 AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'host.name') AS `__GROUP_BY_KEY_0_host.name` FROM signoz_metrics.time_series_v4_6hrs WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) GROUP BY fingerprint, `__GROUP_BY_KEY_0_host.name`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts, `__GROUP_BY_KEY_0_host.name` ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, `__GROUP_BY_KEY_0_host.name`, sum(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts, `__GROUP_BY_KEY_0_host.name`) SELECT ts, `__GROUP_BY_KEY_0_host.name`, multiIf(value <= 0, toFloat64(0), value <= 2.3283064365386963e-10, 2.3283064365386963e-10, value > 1.8446744073709552e+19, toFloat64('+Inf'), pow(2, ceil(log2(value) * 16) / 16)) AS __bucket, toFloat64(1) AS __result_0 FROM __spatial_aggregation_cte ORDER BY `__GROUP_BY_KEY_0_host.name`, ts, __bucket",
Args: []any{"system.memory.usage", uint64(1747936800000), uint64(1747983420000), "unspecified", "system.memory.usage", uint64(1747947360000), uint64(1747983420000), 0},
},
expectedErr: nil,
},
{
name: "test_gauge_heatmap_linear",
requestType: qbtypes.RequestTypeHeatmap,
query: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "system.memory.usage",
Type: metrictypes.GaugeType,
Temporality: metrictypes.Unspecified,
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
HeatmapBucketing: &qbtypes.HeatmapBucketing{
Kind: qbtypes.BucketsKindLinear,
LogScale: qbtypes.MaxLogScale,
MaxValue: 500,
NumBuckets: 25,
},
},
},
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "host.name",
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(60)) AS ts, `__GROUP_BY_KEY_0_host.name`, avg(value) AS per_series_value FROM signoz_metrics.distributed_samples_v4 AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'host.name') AS `__GROUP_BY_KEY_0_host.name` FROM signoz_metrics.time_series_v4_6hrs WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) GROUP BY fingerprint, `__GROUP_BY_KEY_0_host.name`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts, `__GROUP_BY_KEY_0_host.name` ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, `__GROUP_BY_KEY_0_host.name`, sum(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts, `__GROUP_BY_KEY_0_host.name`) SELECT ts, `__GROUP_BY_KEY_0_host.name`, multiIf(value > 500, toFloat64('+Inf'), least(greatest(ceil(value * 25 / 500), 1), 25) * 500 / 25) AS __bucket, toFloat64(1) AS __result_0 FROM __spatial_aggregation_cte ORDER BY `__GROUP_BY_KEY_0_host.name`, ts, __bucket",
Args: []any{"system.memory.usage", uint64(1747936800000), uint64(1747983420000), "unspecified", "system.memory.usage", uint64(1747947360000), uint64(1747983420000), 0},
},
expectedErr: nil,
},
{
name: "test_gauge_avg_sum",
requestType: qbtypes.RequestTypeTimeSeries,

View File

@@ -114,6 +114,9 @@ func (b *scopedTraceStatementBuilder) Build(
case qbtypes.RequestTypeTrace:
return b.buildTraceListQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), query, variables)
case qbtypes.RequestTypeRaw:
if err := b.validateRawOrderKeys(query); err != nil {
return nil, err
}
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
case qbtypes.RequestTypeScalar, qbtypes.RequestTypeTimeSeries:
return b.buildAggregation(ctx, orgID, start, end, requestType, query, variables)
@@ -122,27 +125,19 @@ func (b *scopedTraceStatementBuilder) Build(
}
}
// buildDelegated ANDs the base gate into the user filter and delegates to the
// standard trace builder (the span-list / raw path).
func (b *scopedTraceStatementBuilder) buildDelegated(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
gate := b.scope.FilterExpression
expr := gate
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
expr = fmt.Sprintf("(%s) AND (%s)", gate, query.Filter.Expression)
// validateRawOrderKeys rejects trace-level order keys — no per-trace value exists on
// span rows. A bare name may be a span column sharing an alias (duration_nano), so it passes.
// TODO: move this into the request validation layer (querybuildertypesv5/validation.go).
func (b *scopedTraceStatementBuilder) validateRawOrderKeys(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) error {
for _, o := range query.Order {
key := o.Key.TelemetryFieldKey
key.Normalize()
if key.FieldContext == telemetrytypes.FieldContextTrace {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"ordering the span list by trace-level key %q is not supported; order by span columns instead (e.g. timestamp, duration_nano)", o.Key.Name)
}
}
// shallow copy; only Filter is replaced, caller's query untouched
gated := query
gated.Filter = &qbtypes.Filter{Expression: expr}
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
return nil
}
// traceScopedStatementBuilder is the delegate's optional capability of constraining a
@@ -153,10 +148,10 @@ type traceScopedStatementBuilder interface {
BuildTraceScoped(ctx context.Context, orgID valuer.UUID, start, end uint64, requestType qbtypes.RequestType, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], variables map[string]qbtypes.VariableItem, traceScope, traceScopeResource *qbtypes.Statement) (*qbtypes.Statement, error)
}
// buildDelegatedAggregation serves span-level scalar/time-series through the standard
// trace builder, with the gate ANDed into the span-level filter part; a trace-level
// part becomes a qualification the delegate constrains trace_id by.
func (b *scopedTraceStatementBuilder) buildDelegatedAggregation(
// buildDelegated serves the raw span list and span-level scalar/time-series through
// the standard trace builder, with the gate ANDed into the span-level filter part; a
// trace-level part becomes a qualification the delegate constrains trace_id by.
func (b *scopedTraceStatementBuilder) buildDelegated(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,

View File

@@ -45,7 +45,7 @@ func (b *scopedTraceStatementBuilder) buildAggregation(
return nil, err
}
if len(traceAggs) == 0 {
return b.buildDelegatedAggregation(ctx, orgID, start, end, requestType, query, variables)
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
}
return b.buildTraceAggregationQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), requestType, query, variables, traceAggs)
}

View File

@@ -377,6 +377,11 @@ func (b *traceQueryStatementBuilder) buildListQuery(
cteArgs = append(cteArgs, args)
}
if scopeFrags, scopeArgs := b.attachTraceScope(sb, frag != ""); len(scopeFrags) > 0 {
cteFragments = append(cteFragments, scopeFrags...)
cteArgs = append(cteArgs, scopeArgs...)
}
for i, field := range query.SelectFields {
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &field, telemetrytypes.FieldDataTypeUnspecified, keys)
if err != nil {

View File

@@ -0,0 +1,31 @@
package aitelemetryschema
import (
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/aiobservabilitytypes"
)
var (
traceAggregateNames = func() map[string]struct{} {
names := make(map[string]struct{}, len(TraceAggregateFields))
for name := range TraceAggregateFields {
names[name] = struct{}{}
}
return names
}()
genAISpanGate = "(" + aiobservabilitytypes.GenAISpanFilterExpression() + ")"
)
// ScopedExistingQuery narrows value suggestions to gen_ai spans: the caller's
// filter minus its per-trace aggregate atoms (never ingested, so nothing can
// narrow on them), ANDed with the gen_ai span gate. An unparseable filter is
// dropped and reported through the returned error; the gate alone is still
// usable, matching how the metadata store treats a bad filter downstream.
func ScopedExistingQuery(existingQuery string) (string, error) {
spanExpr, _, err := querybuilder.SplitFilterForAggregates(existingQuery, traceAggregateNames)
if err != nil || spanExpr == "" {
return genAISpanGate, err
}
return genAISpanGate + " AND (" + spanExpr + ")", nil
}

View File

@@ -0,0 +1,103 @@
package aitelemetryschema
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestScopedExistingQuery(t *testing.T) {
gate := "(gen_ai.request.model EXISTS OR gen_ai.tool.name EXISTS OR gen_ai.agent.name EXISTS)"
testCases := []struct {
name string
existingQuery string
expected string
expectedErr string
}{
{
name: "empty query returns the gate alone",
existingQuery: "",
expected: gate,
},
{
name: "span filter is preserved under the gate",
existingQuery: "service.name = 'checkout'",
expected: gate + " AND (service.name = 'checkout')",
},
{
name: "trace aggregate filter is stripped",
existingQuery: "llm_call_count > 5",
expected: gate,
},
{
name: "mixed filter keeps only the span part",
existingQuery: "llm_call_count > 5 AND gen_ai.request.model = 'gpt-4'",
expected: gate + " AND (gen_ai.request.model = 'gpt-4')",
},
{
name: "trace context filter is stripped",
existingQuery: "trace.total_tokens > 100 AND service.name = 'checkout'",
expected: gate + " AND (service.name = 'checkout')",
},
{
name: "unparseable filter is dropped",
existingQuery: "service.name = ",
expected: gate,
expectedErr: "syntax errors while parsing the filter expression",
},
{
name: "multiple span conditions survive as one AND chain",
existingQuery: "service.name = 'checkout' AND gen_ai.request.model = 'gpt-4' AND llm_call_count > 5",
expected: gate + " AND (service.name = 'checkout' AND gen_ai.request.model = 'gpt-4')",
},
{
name: "span OR group is kept whole and parenthesized against the AND join",
existingQuery: "service.name = 'a' OR service.name = 'b'",
expected: gate + " AND ((service.name = 'a' OR service.name = 'b'))",
},
{
name: "parenthesized span OR group ANDed with an aggregate keeps only the group",
existingQuery: "(service.name = 'a' OR service.name = 'b') AND llm_call_count > 5",
expected: gate + " AND ((service.name = 'a' OR service.name = 'b'))",
},
{
name: "OR group of trace aggregates is stripped whole",
existingQuery: "llm_call_count > 5 OR total_tokens > 100",
expected: gate,
},
{
name: "OR mixing aggregate and span atoms drops the whole filter",
existingQuery: "llm_call_count > 5 OR service.name = 'checkout'",
expected: gate,
expectedErr: "trace-level and span-level filters cannot be combined within an OR/NOT group",
},
{
name: "parenthesized AND group is split, not routed whole",
existingQuery: "(llm_call_count > 5 AND service.name = 'checkout') AND gen_ai.request.model = 'gpt-4'",
expected: gate + " AND (service.name = 'checkout' AND gen_ai.request.model = 'gpt-4')",
},
{
name: "NOT over an aggregate group is stripped",
existingQuery: "NOT (llm_call_count > 5) AND service.name = 'checkout'",
expected: gate + " AND (service.name = 'checkout')",
},
{
name: "NOT over a span group is kept",
existingQuery: "NOT (service.name = 'checkout')",
expected: gate + " AND (NOT (service.name = 'checkout'))",
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
scoped, err := ScopedExistingQuery(testCase.existingQuery)
if testCase.expectedErr != "" {
assert.ErrorContains(t, err, testCase.expectedErr)
} else {
assert.NoError(t, err)
}
assert.Equal(t, testCase.expected, scoped)
})
}
}

View File

@@ -15,11 +15,12 @@ type PostableFieldKeysParams struct {
Limit int `query:"limit"`
}
// existingQuery is unsupported until the computed per-trace aggregates it may
// reference can be narrowed on.
// existingQuery may reference the computed per-trace aggregates, which are
// never ingested; those filters are stripped before narrowing values.
type PostableFieldValueParams struct {
PostableFieldKeysParams
Name string `query:"name"`
Name string `query:"name"`
ExistingQuery string `query:"existingQuery"`
}
func NewFieldKeySelectorFromPostableFieldKeysParams(params PostableFieldKeysParams) *telemetrytypes.FieldKeySelector {
@@ -30,6 +31,7 @@ func NewFieldValueSelectorFromPostableFieldValueParams(params PostableFieldValue
return telemetrytypes.NewFieldValueSelectorFromPostableFieldValueParams(telemetrytypes.PostableFieldValueParams{
PostableFieldKeysParams: params.telemetryParams(),
Name: params.Name,
ExistingQuery: params.ExistingQuery,
})
}

View File

@@ -1,5 +1,7 @@
package aiobservabilitytypes
import "strings"
// OpenTelemetry gen_ai semantic-convention attribute keys. Single source of truth
// shared by the AI query builder and the LLM pricing pipeline.
const (
@@ -26,3 +28,17 @@ const (
SignozGenAICostCacheWrite = "_signoz.gen_ai.cost_cache_write"
SignozGenAITotalCost = "_signoz.gen_ai.total_cost"
)
// GenAISpanGateKeys mark a span as gen_ai: an LLM call, a tool call, or an
// agent span. A trace belongs to the AI explorer when any span carries one.
var GenAISpanGateKeys = []string{GenAIRequestModel, GenAIToolName, GenAIAgentName}
// GenAISpanFilterExpression renders the gate as a query-builder filter
// expression: each gate key ORed on EXISTS.
func GenAISpanFilterExpression() string {
exprs := make([]string, 0, len(GenAISpanGateKeys))
for _, key := range GenAISpanGateKeys {
exprs = append(exprs, key+" EXISTS")
}
return strings.Join(exprs, " OR ")
}

View File

@@ -1,6 +1,7 @@
package alertmanagertypes
import (
"crypto/rand"
"encoding/json"
"reflect"
"regexp"
@@ -16,9 +17,10 @@ import (
)
var (
ErrCodeAlertmanagerChannelNotFound = errors.MustNewCode("alertmanager_channel_not_found")
ErrCodeAlertmanagerChannelNameMismatch = errors.MustNewCode("alertmanager_channel_name_mismatch")
ErrCodeAlertmanagerChannelInvalid = errors.MustNewCode("alertmanager_channel_invalid")
ErrCodeAlertmanagerChannelNotFound = errors.MustNewCode("alertmanager_channel_not_found")
ErrCodeAlertmanagerChannelNameMismatch = errors.MustNewCode("alertmanager_channel_name_mismatch")
ErrCodeAlertmanagerChannelInvalid = errors.MustNewCode("alertmanager_channel_invalid")
ErrCodeAlertmanagerChannelAlreadyExists = errors.MustNewCode("alertmanager_channel_already_exists")
)
var (
@@ -48,14 +50,19 @@ type Channel struct {
types.Identifiable
types.TimeAuditable
Name string `json:"name" required:"true" bun:"name"`
Type string `json:"type" required:"true" bun:"type"`
Data string `json:"data" required:"true" bun:"data"`
OrgID string `json:"orgId" required:"true" bun:"org_id"`
// Name is the DNS1123 identity references will migrate onto. Until then
// DisplayName is the receiver name inside Data and what policies and rules
// reference, so it keeps the v1 wire tag and Name stays off the v1 contract.
Name string `json:"-" bun:"name"`
DisplayName string `json:"name" required:"true" bun:"display_name"`
Type string `json:"type" required:"true" bun:"type"`
Data string `json:"data" required:"true" bun:"data"`
OrgID string `json:"orgId" required:"true" bun:"org_id"`
}
// NewChannelFromReceiver creates a new Channel from a Receiver.
// It can return nil if the receiver is the default receiver.
// A receiver carries no internal name, so one is generated from its name.
func NewChannelFromReceiver(receiver *Receiver, orgID string) (*Channel, error) {
if receiver.Name == DefaultReceiverName {
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeAlertmanagerChannelInvalid, "cannot use %s name as a channel name", receiver.Name)
@@ -70,8 +77,9 @@ func NewChannelFromReceiver(receiver *Receiver, orgID string) (*Channel, error)
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
},
Name: receiver.Name,
OrgID: orgID,
Name: generateChannelName(receiver.Name),
DisplayName: receiver.Name,
OrgID: orgID,
}
data, err := json.Marshal(receiver)
@@ -88,6 +96,60 @@ func NewChannelFromReceiver(receiver *Receiver, orgID string) (*Channel, error)
return &channel, nil
}
const channelNameSuffixLen = 8
// generateChannelName is a copy of dashboardtypes.generateDashboardName: slugify
// the display name, then append a random suffix rather than looping on collisions.
func generateChannelName(displayName string) string {
const dns1123LabelMaxLen = 63
suffixAlphabet := []byte("abcdefghijklmnopqrstuvwxyz0123456789")
var b strings.Builder
b.Grow(len(displayName))
prevHyphen := false
for _, r := range strings.ToLower(displayName) {
switch {
case (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9'):
b.WriteRune(r)
prevHyphen = false
case b.Len() > 0 && !prevHyphen:
b.WriteByte('-')
prevHyphen = true
}
}
prefix := strings.TrimRight(b.String(), "-")
suffix := make([]byte, channelNameSuffixLen)
if _, err := rand.Read(suffix); err != nil {
panic(errors.WrapInternalf(err, errors.CodeInternal, "read random for channel name suffix"))
}
for i := range suffix {
suffix[i] = suffixAlphabet[int(suffix[i])%len(suffixAlphabet)]
}
maxPrefix := dns1123LabelMaxLen - 1 - channelNameSuffixLen
if len(prefix) > maxPrefix {
prefix = strings.TrimRight(prefix[:maxPrefix], "-")
}
if prefix == "" {
return string(suffix)
}
return prefix + "-" + string(suffix)
}
// NewChannelFromReceiverWithName overrides the name that NewChannelFromReceiver
// generates.
func NewChannelFromReceiverWithName(receiver *Receiver, name string, orgID string) (*Channel, error) {
channel, err := NewChannelFromReceiver(receiver, orgID)
if err != nil {
return nil, err
}
channel.Name = name
return channel, nil
}
// receiverChannelType returns the channel.Type discriminator. Walks
// Receiver's own fields first (native), then the embed (upstream); first
// non-empty *_configs slice wins.
@@ -151,26 +213,6 @@ func NewConfigFromChannels(globalConfig GlobalConfig, routeConfig RouteConfig, c
return cfg, nil
}
func GetChannelByID(channels Channels, id valuer.UUID) (int, *Channel, error) {
for i, channel := range channels {
if channel.ID == id {
return i, channel, nil
}
}
return 0, nil, errors.Newf(errors.TypeNotFound, ErrCodeAlertmanagerChannelNotFound, "cannot find channel with id %s", id.StringValue())
}
func GetChannelByName(channels Channels, name string) (int, *Channel, error) {
for i, channel := range channels {
if channel.Name == name {
return i, channel, nil
}
}
return 0, nil, errors.Newf(errors.TypeNotFound, ErrCodeAlertmanagerChannelNotFound, "cannot find channel with name %s", name)
}
func NewStatsFromChannels(channels Channels) map[string]any {
stats := make(map[string]any)
for _, channel := range channels {
@@ -188,15 +230,21 @@ func NewStatsFromChannels(channels Channels) map[string]any {
}
func (c *Channel) Update(receiver *Receiver) error {
channel, err := NewChannelFromReceiver(receiver, c.OrgID)
channel, err := NewChannelFromReceiverWithName(receiver, c.Name, c.OrgID)
if err != nil {
return err
}
if c.Name != channel.Name {
if c.DisplayName != channel.DisplayName {
return errors.Newf(errors.TypeInvalidInput, ErrCodeAlertmanagerChannelNameMismatch, "cannot update channel name")
}
// Unreachable while the name is passed in above rather than derived from the
// receiver, which is why this is internal rather than invalid input.
if c.Name != channel.Name {
return errors.NewInternalf(ErrCodeAlertmanagerChannelNameMismatch, "cannot update channel internal name")
}
c.Type = channel.Type
c.Data = channel.Data
c.UpdatedAt = time.Now()

View File

@@ -22,9 +22,9 @@ func TestNewConfigFromChannels(t *testing.T) {
name: "OneEmailChannel",
channels: Channels{
{
Name: "email-receiver",
Type: "email",
Data: `{"name":"email-receiver","email_configs":[{"to":"test@example.com"}]}`,
DisplayName: "email-receiver",
Type: "email",
Data: `{"name":"email-receiver","email_configs":[{"to":"test@example.com"}]}`,
},
},
expectedRoutes: []map[string]any{{"receiver": "email-receiver", "continue": true, "matchers": []any{"ruleId=~\"-1\""}}},
@@ -46,9 +46,9 @@ func TestNewConfigFromChannels(t *testing.T) {
name: "OneSlackChannel",
channels: Channels{
{
Name: "slack-receiver",
Type: "slack",
Data: `{"name":"slack-receiver","slack_configs":[{"channel":"#alerts","api_url":"https://slack.com/api/test","send_resolved":true}]}`,
DisplayName: "slack-receiver",
Type: "slack",
Data: `{"name":"slack-receiver","slack_configs":[{"channel":"#alerts","api_url":"https://slack.com/api/test","send_resolved":true}]}`,
},
},
expectedRoutes: []map[string]any{{"receiver": "slack-receiver", "continue": true, "matchers": []any{"ruleId=~\"-1\""}}},
@@ -80,9 +80,9 @@ func TestNewConfigFromChannels(t *testing.T) {
name: "OnePagerdutyChannel",
channels: Channels{
{
Name: "pagerduty-receiver",
Type: "pagerduty",
Data: `{"name":"pagerduty-receiver","pagerduty_configs":[{"service_key":"test"}]}`,
DisplayName: "pagerduty-receiver",
Type: "pagerduty",
Data: `{"name":"pagerduty-receiver","pagerduty_configs":[{"service_key":"test"}]}`,
},
},
expectedRoutes: []map[string]any{{"receiver": "pagerduty-receiver", "continue": true, "matchers": []any{"ruleId=~\"-1\""}}},
@@ -112,14 +112,14 @@ func TestNewConfigFromChannels(t *testing.T) {
name: "OnePagerdutyAndOneSlackChannel",
channels: Channels{
{
Name: "pagerduty-receiver",
Type: "pagerduty",
Data: `{"name":"pagerduty-receiver","pagerduty_configs":[{"service_key":"test"}]}`,
DisplayName: "pagerduty-receiver",
Type: "pagerduty",
Data: `{"name":"pagerduty-receiver","pagerduty_configs":[{"service_key":"test"}]}`,
},
{
Name: "slack-receiver",
Type: "slack",
Data: `{"name":"slack-receiver","slack_configs":[{"channel":"#alerts","api_url":"https://slack.com/api/test","send_resolved":true}]}`,
DisplayName: "slack-receiver",
Type: "slack",
Data: `{"name":"slack-receiver","slack_configs":[{"channel":"#alerts","api_url":"https://slack.com/api/test","send_resolved":true}]}`,
},
},
expectedRoutes: []map[string]any{{"receiver": "pagerduty-receiver", "continue": true, "matchers": []any{"ruleId=~\"-1\""}}, {"receiver": "slack-receiver", "continue": true, "matchers": []any{"ruleId=~\"-1\""}}},
@@ -243,9 +243,9 @@ func TestNewChannelFromReceiver(t *testing.T) {
},
},
expected: &Channel{
Name: "test-receiver",
Type: "slack",
Data: `{"name":"test-receiver","slack_configs":[{"send_resolved":true,"api_url":"https://slack.com/api/test","channel":"#alerts","timeout":0}]}`,
DisplayName: "test-receiver",
Type: "slack",
Data: `{"name":"test-receiver","slack_configs":[{"send_resolved":true,"api_url":"https://slack.com/api/test","channel":"#alerts","timeout":0}]}`,
},
pass: true,
},
@@ -261,7 +261,7 @@ func TestNewChannelFromReceiver(t *testing.T) {
}
assert.NoError(t, err)
assert.Equal(t, testCase.expected.Name, channel.Name)
assert.Equal(t, testCase.expected.DisplayName, channel.DisplayName)
assert.Equal(t, testCase.expected.Type, channel.Type)
assert.Equal(t, testCase.expected.Data, channel.Data)
})
@@ -289,7 +289,7 @@ func TestNewChannelFromReceiverGoogleChat(t *testing.T) {
channel, err := NewChannelFromReceiver(receiver, "1")
assert.NoError(t, err)
assert.Equal(t, "googlechat-receiver", channel.Name)
assert.Equal(t, "googlechat-receiver", channel.DisplayName)
assert.Equal(t, "googlechat", channel.Type)
assert.JSONEq(t,
`{"name":"googlechat-receiver","googlechat_configs":[{"send_resolved":false,"webhook_url":"https://chat.googleapis.com/v1/spaces/test/messages","title":"Alert","text":"Body"}]}`,

View File

@@ -72,10 +72,11 @@ type customReceiverConfigs struct {
GoogleChat []*GoogleChatReceiverConfig
Jira []*JiraReceiverConfig
JSMOps []*JSMOpsReceiverConfig
IncidentIO []*IncidentIOReceiverConfig
}
func (c customReceiverConfigs) isEmpty() bool {
return len(c.GoogleChat) == 0 && len(c.Jira) == 0 && len(c.JSMOps) == 0
return len(c.GoogleChat) == 0 && len(c.Jira) == 0 && len(c.JSMOps) == 0 && len(c.IncidentIO) == 0
}
func customConfigsOf(receiver *Receiver) customReceiverConfigs {
@@ -83,6 +84,7 @@ func customConfigsOf(receiver *Receiver) customReceiverConfigs {
GoogleChat: receiver.GoogleChatConfigs,
Jira: receiver.JiraConfigs,
JSMOps: receiver.JSMOpsConfigs,
IncidentIO: receiver.IncidentIOConfigs,
}
}
@@ -193,6 +195,7 @@ func extendedReceivers(c *config.Config, customConfigs map[string]customReceiver
GoogleChatConfigs: custom.GoogleChat,
JiraConfigs: custom.Jira,
JSMOpsConfigs: custom.JSMOps,
IncidentIOConfigs: custom.IncidentIO,
}
}
@@ -370,6 +373,7 @@ func (c *Config) GetReceiver(name string) (*Receiver, error) {
GoogleChatConfigs: custom.GoogleChat,
JiraConfigs: custom.Jira,
JSMOpsConfigs: custom.JSMOps,
IncidentIOConfigs: custom.IncidentIO,
}, nil
}
}
@@ -459,6 +463,11 @@ func (c *Config) applyNativeDefaults() {
jc.HTTPConfig = httpDefault
}
}
for _, ic := range custom.IncidentIO {
if ic.HTTPConfig == nil {
ic.HTTPConfig = httpDefault
}
}
}
}

View File

@@ -0,0 +1,109 @@
package alertmanagertypes
import (
"fmt"
"net/url"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/prometheus/alertmanager/config"
commoncfg "github.com/prometheus/common/config"
)
// incidentIOEventsPathPrefix is the path of incident.io's HTTP alert source
// endpoint (Alert Events V2 API). The full URL is per-source:
// https://api.incident.io/v2/alert_events/http/<source_config_id>.
const incidentIOEventsPathPrefix = "/v2/alert_events/http/"
// The description is markdown; incident.io renders it natively. The templates
// mirror Google Chat / Jira / JSM for a consistent default across channels.
const (
DefaultIncidentIOTitleTemplate = `[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }}`
DefaultIncidentIODescriptionTemplate = `{{ range .Alerts -}}
**Alert:** {{ .Labels.alertname }}{{ if .Labels.severity }} ({{ .Labels.severity }}){{ end }}
{{ if .Annotations.summary }}**Summary:** {{ .Annotations.summary }}
{{ end }}{{ if .Annotations.description }}**Description:** {{ .Annotations.description }}
{{ end }}{{ if .GeneratorURL }}[View in SigNoz]({{ .GeneratorURL }})
{{ end }}{{ if .Annotations.related_logs }}[View related logs]({{ .Annotations.related_logs }})
{{ end }}{{ if .Annotations.related_traces }}[View related traces]({{ .Annotations.related_traces }})
{{ end }}{{ end }}`
)
// IncidentIOReceiverConfig is the SigNoz incident.io receiver, backed by an
// incident.io HTTP alert source. URL is the per-source alert events endpoint
// and Token its secret, both copied from the source's setup page.
type IncidentIOReceiverConfig struct {
config.NotifierConfig `yaml:",inline" json:",inline"`
HTTPConfig *commoncfg.HTTPClientConfig `yaml:"http_config,omitempty" json:"http_config,omitempty"`
URL string `yaml:"url,omitempty" json:"url,omitempty"`
Token config.Secret `yaml:"token,omitempty" json:"token,omitempty"`
Title string `yaml:"title,omitempty" json:"title,omitempty"`
Description string `yaml:"description,omitempty" json:"description,omitempty"`
// Metadata is merged into the event's metadata on top of the group's common
// labels (channel wins on key clash). Values are template-expanded.
Metadata map[string]string `yaml:"metadata,omitempty" json:"metadata,omitempty"`
}
// send_resolved has no omitempty upstream, so a var default here is overwritten
// by the yaml round-trip to the request value (false when omitted); the UI sends
// it explicitly, defaulted on, so incident.io alerts resolve with the rule.
var DefaultIncidentIOReceiverConfig = IncidentIOReceiverConfig{
NotifierConfig: config.NotifierConfig{
VSendResolved: false,
},
Title: DefaultIncidentIOTitleTemplate,
Description: DefaultIncidentIODescriptionTemplate,
}
func (c *IncidentIOReceiverConfig) UnmarshalYAML(unmarshal func(any) error) error {
*c = DefaultIncidentIOReceiverConfig
type plain IncidentIOReceiverConfig
if err := unmarshal((*plain)(c)); err != nil {
return err
}
if c.Title == "" {
c.Title = DefaultIncidentIOTitleTemplate
}
if c.Description == "" {
c.Description = DefaultIncidentIODescriptionTemplate
}
// Values are stored and sent exactly as configured, so anything that is
// not already canonical is rejected rather than rewritten.
if c.URL != strings.TrimSpace(c.URL) {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "incidentio url must not have leading or trailing whitespace")
}
u, err := url.Parse(c.URL)
if c.URL == "" || err != nil || u.Scheme != "https" || u.Host == "" ||
!strings.Contains(u.Path, incidentIOEventsPathPrefix) ||
strings.HasSuffix(u.Path, incidentIOEventsPathPrefix) {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, fmt.Sprintf("incidentio url must be an alert events URL (https://api.incident.io%s<source_config_id>)", incidentIOEventsPathPrefix))
}
if strings.HasSuffix(c.URL, "/") {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "incidentio url must not end with a trailing slash")
}
token := string(c.Token)
if token == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "incidentio token is required")
}
if token != strings.TrimSpace(token) {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "incidentio token must not have leading or trailing whitespace")
}
// incident.io's setup page shows the header value as "Bearer <token>"; a
// pasted prefix would be sent doubled, so reject it instead.
if strings.EqualFold(token, "bearer") || (len(token) >= 7 && strings.EqualFold(token[:7], "bearer ")) {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "incidentio token must be the source's secret token only, without the Bearer prefix")
}
return nil
}

View File

@@ -0,0 +1,65 @@
package alertmanagertypes
import (
"fmt"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const testIncidentIOURL = "https://api.incident.io/v2/alert_events/http/01M0D1JNVBGBGVTWX053EM12XV"
func TestIncidentIOReceiverConfigDefaults(t *testing.T) {
r, err := NewReceiver(fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s","token":"tok-123"}]}`, testIncidentIOURL))
require.NoError(t, err)
require.Len(t, r.IncidentIOConfigs, 1)
c := r.IncidentIOConfigs[0]
assert.Equal(t, testIncidentIOURL, c.URL)
assert.Equal(t, "tok-123", string(c.Token))
assert.Equal(t, DefaultIncidentIOTitleTemplate, c.Title)
assert.Equal(t, DefaultIncidentIODescriptionTemplate, c.Description)
assert.False(t, c.SendResolved()) // default off when omitted, like other channels
ch, err := NewChannelFromReceiver(r, "org-1")
require.NoError(t, err)
assert.Equal(t, "incidentio", ch.Type)
}
func TestIncidentIOReceiverConfigOverrides(t *testing.T) {
r, err := NewReceiver(fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s","token":"k","title":"t","description":"d","send_resolved":true}]}`, testIncidentIOURL))
require.NoError(t, err)
require.Len(t, r.IncidentIOConfigs, 1)
c := r.IncidentIOConfigs[0]
assert.Equal(t, testIncidentIOURL, c.URL)
assert.Equal(t, "t", c.Title)
assert.Equal(t, "d", c.Description)
assert.True(t, c.SendResolved())
}
func TestIncidentIOReceiverConfigValidation(t *testing.T) {
cases := []struct {
name string
json string
}{
{"missing url", `{"name":"incio","incidentio_configs":[{"token":"k"}]}`},
{"http url", `{"name":"incio","incidentio_configs":[{"url":"http://api.incident.io/v2/alert_events/http/abc","token":"k"}]}`},
{"not an alert events url", `{"name":"incio","incidentio_configs":[{"url":"https://api.incident.io/v2/incidents","token":"k"}]}`},
{"missing source config id", `{"name":"incio","incidentio_configs":[{"url":"https://api.incident.io/v2/alert_events/http/","token":"k"}]}`},
{"trailing slash", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s/","token":"k"}]}`, testIncidentIOURL)},
{"whitespace around url", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":" %s ","token":"k"}]}`, testIncidentIOURL)},
{"missing token", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s"}]}`, testIncidentIOURL)},
{"bearer prefixed token", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s","token":"Bearer tok-123"}]}`, testIncidentIOURL)},
{"lowercase bearer prefixed token", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s","token":"bearer tok-123"}]}`, testIncidentIOURL)},
{"bearer only token", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s","token":"Bearer"}]}`, testIncidentIOURL)},
{"whitespace around token", fmt.Sprintf(`{"name":"incio","incidentio_configs":[{"url":"%s","token":" tok-123 "}]}`, testIncidentIOURL)},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
_, err := NewReceiver(c.json)
assert.Error(t, err)
})
}
}

View File

@@ -28,6 +28,9 @@ type Receiver struct {
JiraConfigs []*JiraReceiverConfig `json:"jira_configs,omitempty" yaml:"jira_configs,omitempty"`
// JSM Ops (ex-Opsgenie alert API); delivered by reusing the Opsgenie notifier.
JSMOpsConfigs []*JSMOpsReceiverConfig `json:"jsmops_configs,omitempty" yaml:"jsmops_configs,omitempty"`
// Shadows upstream's incidentio_configs so our custom notifier (templater,
// group-key dedup, label metadata) handles it instead of upstream's.
IncidentIOConfigs []*IncidentIOReceiverConfig `json:"incidentio_configs,omitempty" yaml:"incidentio_configs,omitempty"`
}
// NewReceiver builds a Receiver from its JSON input, applying each notifier
@@ -72,6 +75,14 @@ func NewReceiver(input string) (*Receiver, error) {
receiver.JSMOpsConfigs[i] = defaulted
}
for i, ic := range receiver.IncidentIOConfigs {
defaulted, err := defaultedNotifierConfig(ic)
if err != nil {
return nil, err
}
receiver.IncidentIOConfigs[i] = defaulted
}
return receiver, nil
}

View File

@@ -62,7 +62,7 @@ var (
ResourceMetaResourcePipeline = NewResourceMetaResource(KindPipeline)
ResourceMetaResourceUserPreference = NewResourceMetaResource(KindUserPreference)
ResourceMetaResourceOrgPreference = NewResourceMetaResource(KindOrgPreference)
ResourceMetaResourceQuickFilter = NewResourceMetaResource(KindQuickFilter)
ResourceMetaResourceQuickFilter = NewResourceMetaResource(KindQuickFilter, VerbList, VerbRead, VerbUpdate)
ResourceMetaResourceTTLSetting = NewResourceMetaResource(KindTTLSetting)
ResourceMetaResourceRule = NewResourceMetaResource(KindRule)
ResourceMetaResourcePlannedMaintenance = NewResourceMetaResource(KindPlannedMaintenance)

View File

@@ -35,7 +35,6 @@ func (PanelPlugin) PrepareJSONSchema(s *jsonschema.Schema) error {
string(PanelKindTable): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTablePanelSpec"),
string(PanelKindHistogram): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec"),
string(PanelKindList): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec"),
string(PanelKindHeatmap): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec"),
})
}
@@ -66,7 +65,6 @@ func (PanelPlugin) JSONSchemaOneOf() []any {
PanelPluginVariant[TablePanelSpec]{Kind: string(PanelKindTable)},
PanelPluginVariant[HistogramPanelSpec]{Kind: string(PanelKindHistogram)},
PanelPluginVariant[ListPanelSpec]{Kind: string(PanelKindList)},
PanelPluginVariant[HeatmapPanelSpec]{Kind: string(PanelKindHeatmap)},
}
}
@@ -230,7 +228,6 @@ var (
PanelKindTable: func() any { return new(TablePanelSpec) },
PanelKindHistogram: func() any { return new(HistogramPanelSpec) },
PanelKindList: func() any { return new(ListPanelSpec) },
PanelKindHeatmap: func() any { return new(HeatmapPanelSpec) },
}
queryPluginSpecs = map[QueryPluginKind]func() any{
QueryKindBuilder: func() any { return new(BuilderQuerySpec) },
@@ -253,7 +250,6 @@ var (
PanelKindPieChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindTable: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindList: {QueryKindBuilder},
PanelKindHeatmap: {QueryKindBuilder},
}
)

View File

@@ -173,11 +173,10 @@ const (
PanelKindTable PanelPluginKind = "signoz/TablePanel"
PanelKindHistogram PanelPluginKind = "signoz/HistogramPanel"
PanelKindList PanelPluginKind = "signoz/ListPanel"
PanelKindHeatmap PanelPluginKind = "signoz/HeatmapPanel"
)
func (PanelPluginKind) Enum() []any {
return []any{PanelKindTimeSeries, PanelKindBarChart, PanelKindNumber, PanelKindPieChart, PanelKindTable, PanelKindHistogram, PanelKindList, PanelKindHeatmap}
return []any{PanelKindTimeSeries, PanelKindBarChart, PanelKindNumber, PanelKindPieChart, PanelKindTable, PanelKindHistogram, PanelKindList}
}
type TimeSeriesPanelSpec struct {
@@ -238,51 +237,6 @@ type ListPanelSpec struct {
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitzero" validate:"dive"`
}
type HeatmapPanelSpec struct {
Visualization HeatmapVisualization `json:"visualization"`
Formatting PanelFormatting `json:"formatting"`
Legend Legend `json:"legend"`
ShowOverflow bool `json:"showOverflow"`
Colors HeatmapColors `json:"colors"`
}
type HeatmapVisualization struct {
BasicVisualization
ShowVisualMap bool `json:"showVisualMap"`
}
type HeatmapColors struct {
Mode HeatmapColorMode `json:"mode"`
Scale HeatmapColorScale `json:"scale"`
// Min and Max clamp the colour scale; nil means derive from the data.
Min *float64 `json:"min"`
Max *float64 `json:"max"`
// Scheme, Steps and Reverse apply in scheme mode.
Scheme string `json:"scheme"`
Steps int `json:"steps" validate:"omitempty,min=2,max=128"`
Reverse bool `json:"reverse"`
// Fill applies in opacity mode; empty means the selected group's legend colour.
Fill string `json:"fill"`
}
func (c *HeatmapColors) UnmarshalJSON(data []byte) error {
type alias HeatmapColors
var tmp alias
if err := json.Unmarshal(data, &tmp); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid heatmap colors")
}
*c = HeatmapColors(tmp)
return c.validate()
}
func (c HeatmapColors) validate() error {
if c.Min != nil && c.Max != nil && *c.Min > *c.Max {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput,
"heatmap colors.min %v is greater than colors.max %v", *c.Min, *c.Max)
}
return nil
}
// ══════════════════════════════════════════════
// Panel common types
// ══════════════════════════════════════════════
@@ -755,78 +709,3 @@ func (p *PrecisionOption) UnmarshalJSON(data []byte) error {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid precision option %q: must be `0`, `1`, `2`, `3`, `4`, or `full`", v)
}
}
type HeatmapColorMode struct{ valuer.String }
var (
HeatmapColorModeScheme = HeatmapColorMode{valuer.NewString("scheme")} // default
HeatmapColorModeOpacity = HeatmapColorMode{valuer.NewString("opacity")}
)
func (HeatmapColorMode) Enum() []any {
return []any{HeatmapColorModeScheme, HeatmapColorModeOpacity}
}
func (m HeatmapColorMode) ValueOrDefault() string {
if m.IsZero() {
return HeatmapColorModeScheme.StringValue()
}
return m.StringValue()
}
func (m HeatmapColorMode) MarshalJSON() ([]byte, error) {
return json.Marshal(m.ValueOrDefault())
}
func (m *HeatmapColorMode) UnmarshalJSON(data []byte) error {
var v string
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid heatmap color mode: must be a string, one of `scheme` or `opacity`")
}
mode := HeatmapColorMode{valuer.NewString(v)}
switch mode {
case HeatmapColorModeScheme, HeatmapColorModeOpacity:
*m = mode
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid heatmap color mode %q: must be `scheme` or `opacity`", v)
}
}
type HeatmapColorScale struct{ valuer.String }
var (
HeatmapColorScaleLog = HeatmapColorScale{valuer.NewString("log")} // default
HeatmapColorScaleSqrt = HeatmapColorScale{valuer.NewString("sqrt")}
HeatmapColorScaleLinear = HeatmapColorScale{valuer.NewString("linear")}
)
func (HeatmapColorScale) Enum() []any {
return []any{HeatmapColorScaleLog, HeatmapColorScaleSqrt, HeatmapColorScaleLinear}
}
func (s HeatmapColorScale) ValueOrDefault() string {
if s.IsZero() {
return HeatmapColorScaleLog.StringValue()
}
return s.StringValue()
}
func (s HeatmapColorScale) MarshalJSON() ([]byte, error) {
return json.Marshal(s.ValueOrDefault())
}
func (s *HeatmapColorScale) UnmarshalJSON(data []byte) error {
var v string
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid heatmap color scale: must be a string, one of `log`, `sqrt`, or `linear`")
}
scale := HeatmapColorScale{valuer.NewString(v)}
switch scale {
case HeatmapColorScaleLog, HeatmapColorScaleSqrt, HeatmapColorScaleLinear:
*s = scale
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid heatmap color scale %q: must be `log`, `sqrt`, or `linear`", v)
}
}

View File

@@ -81,7 +81,7 @@ type GettableCreatedIngestionKey struct {
Value string `json:"value" required:"true"`
}
type PostableIngestionKeyLimit struct {
type DeprecatedPostableIngestionKeyLimit struct {
Signal string `json:"signal"`
Config LimitConfig `json:"config"`
Tags []string `json:"tags"`
@@ -91,6 +91,13 @@ type GettableCreatedIngestionKeyLimit struct {
ID string `json:"id" required:"true"`
}
type PostableIngestionKeyLimit struct {
KeyID string `json:"keyId" required:"true"`
Signal string `json:"signal"`
Config LimitConfig `json:"config"`
Tags []string `json:"tags"`
}
type UpdatableIngestionKeyLimit struct {
Config LimitConfig `json:"config" required:"true"`
Tags []string `json:"tags"`

View File

@@ -540,8 +540,6 @@ type MetricAggregation struct {
// reduce to operator for metric scalar requests
ReduceTo ReduceTo `json:"reduceTo,omitzero"`
HeatmapBucketing *HeatmapBucketing `json:"-"`
Reduced bool `json:"-"`
}
@@ -556,10 +554,6 @@ func (m MetricAggregation) Copy() MetricAggregation {
valueFilterCopy := *m.ValueFilter
c.ValueFilter = &valueFilterCopy
}
if m.HeatmapBucketing != nil {
bucketingCopy := *m.HeatmapBucketing
c.HeatmapBucketing = &bucketingCopy
}
return c
}

View File

@@ -1,405 +0,0 @@
package querybuildertypesv5
import (
"maps"
"math"
"slices"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
)
const (
// A positive value approaching zero runs its band index off to -inf, so
// without a clamp one near-zero sample would stretch the axis by thousands
// of bands once DensifyHeatmapAxis fills the empty ones in.
MinLogBandIndex = -512 // 2^-32, about 2.3e-10
MaxLogBandIndex = 1024 // 2^64, about 1.8e19
)
// LowestLogBoundary and HighestLogBoundary are the ends the log axis is clamped
// to. They do not vary with the requested scale.
var (
LowestLogBoundary = math.Exp2(float64(MinLogBandIndex) / math.Exp2(MaxLogScale))
HighestLogBoundary = math.Exp2(float64(MaxLogBandIndex) / math.Exp2(MaxLogScale))
)
// HeatmapBucketing is the bucket axis a heatmap statement builds in ClickHouse,
// resolved from BucketOptions once the metric type is known. It stays nil for
// histograms, whose boundaries come from their own `le` labels.
type HeatmapBucketing struct {
Kind BucketsKind
// LogScale is always MaxLogScale; LogBucketsSpec.Scale coarsens the result
// afterwards rather than changing this.
LogScale int
// MaxValue and NumBuckets are linear only.
MaxValue float64
NumBuckets int
}
// ResolveBucketOptions fills in what the caller left unset. An absent
// BucketOptions resolves to the finest log axis, the one kind that needs nothing
// from the caller.
func (b *BucketOptions) ResolveBucketOptions() HeatmapBucketing {
resolved := HeatmapBucketing{
Kind: BucketsKindLog,
LogScale: MaxLogScale,
NumBuckets: DefaultNumBuckets,
}
if b == nil {
return resolved
}
if spec, ok := b.Spec.(LinearBucketsSpec); ok {
resolved.Kind = BucketsKindLinear
resolved.MaxValue = spec.MaxValue
if spec.NumBuckets > 0 {
resolved.NumBuckets = spec.NumBuckets
}
}
return resolved
}
// ResolveLogScale returns the axis resolution the caller wants back, which
// postprocessing folds the MaxLogScale axis down to.
func (b *BucketOptions) ResolveLogScale() int {
if b == nil {
return MaxLogScale
}
if spec, ok := b.Spec.(LogBucketsSpec); ok && spec.Scale != nil {
return *spec.Scale
}
return MaxLogScale
}
// ResolveHeatmapBucketing sets a.HeatmapBucketing to the axis a heatmap draws its
// rows from, and refuses the metric types that cannot produce one. It cannot live
// in validateHeatmap: a.Type is resolved from metadata after that has run.
func (a *MetricAggregation) ResolveHeatmapBucketing(bucketOptions *BucketOptions) error {
switch a.Type {
case metrictypes.HistogramType:
if bucketOptions != nil {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"bucketOptions are not supported for histogram metrics: %q takes its bucket axis from its own `le` labels, so nothing in the spec would be applied", a.MetricName)
}
a.HeatmapBucketing = nil
return nil
// A summary carries no boundaries of its own either, and its samples reach
// the final select the same way a gauge's do, so it buckets identically.
case metrictypes.GaugeType, metrictypes.SumType, metrictypes.SummaryType:
bucketing := bucketOptions.ResolveBucketOptions()
a.HeatmapBucketing = &bucketing
return nil
case metrictypes.UnspecifiedType:
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"heatmaps need a metric whose type is known: no type is recorded for %q, so its bucket axis cannot be chosen", a.MetricName)
case metrictypes.ExpHistogramType:
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"heatmaps are not supported for exponential histograms yet: %q keeps its bucket counts in a sketch column, which needs its own reader", a.MetricName)
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"heatmaps are not supported for %s metrics", a.Type.StringValue())
}
}
// MergeHeatmapAxes collects, per aggregation index, every bucket boundary any of
// tsData reached, so that halves holding different bands can be merged onto one
// axis. Only heatmap results carry boundaries, so it comes back empty for
// everything else and the realignment it feeds is a no-op.
func MergeHeatmapAxes(tsData ...*TimeSeriesData) map[int][]float64 {
reached := map[int]map[float64]struct{}{}
for _, data := range tsData {
if data == nil {
continue
}
for _, aggBucket := range data.Aggregations {
if len(aggBucket.Meta.Buckets) == 0 {
continue
}
if reached[aggBucket.Index] == nil {
reached[aggBucket.Index] = map[float64]struct{}{}
}
for _, boundary := range aggBucket.Meta.Buckets {
reached[aggBucket.Index][boundary] = struct{}{}
}
}
}
merged := make(map[int][]float64, len(reached))
for index, boundarySet := range reached {
merged[index] = slices.Sorted(maps.Keys(boundarySet))
}
return merged
}
// regroupAxis rewrites the aggregation onto boundaries, moving the count held in
// band i to targetBandIndexes[i] and summing where several bands land together.
// A band past the end of targetBandIndexes is the overflow, which stays the
// overflow on any axis.
func regroupAxis(aggBucket *AggregationBucket, boundaries []float64, targetBandIndexes []int) {
for _, series := range aggBucket.Series {
for _, point := range series.Values {
if len(point.Values) == 0 {
continue
}
regrouped := make([]float64, len(boundaries)+1)
for band, count := range point.Values {
if band >= len(targetBandIndexes) {
regrouped[len(boundaries)] += count
continue
}
regrouped[targetBandIndexes[band]] += count
}
point.Values = regrouped
}
}
aggBucket.Meta.Buckets = boundaries
}
// RealignHeatmapValues moves every point's per-bucket counts from the axis they
// were read against onto onto, matching on boundary rather than position. Two
// ranges of one query disagree on their axes when a histogram's `le` labels
// change partway through a window, or when one range's data never reached a
// band the other did.
func RealignHeatmapValues(series []*TimeSeries, from, onto []float64) {
if len(onto) == 0 || slices.Equal(from, onto) {
return
}
bandIndexByBoundary := make(map[float64]int, len(onto))
for band, boundary := range onto {
bandIndexByBoundary[boundary] = band
}
for _, s := range series {
for _, point := range s.Values {
if len(point.Values) == 0 {
continue
}
realigned := make([]float64, len(onto)+1)
for band, count := range point.Values {
if band >= len(from) {
realigned[len(onto)] = count
break
}
if targetBand, ok := bandIndexByBoundary[from[band]]; ok {
realigned[targetBand] = count
}
}
point.Values = realigned
}
}
}
// DownscaleHeatmapAxis folds a log axis bucketed at fromScale down to toScale,
// merging every 2^(fromScale-toScale) adjacent bands into one. The coarser
// boundaries are a subset of the finer ones, so the fold is exact.
func DownscaleHeatmapAxis(tsData *TimeSeriesData, fromScale, toScale int) {
if tsData == nil || toScale >= fromScale {
return
}
for _, aggBucket := range tsData.Aggregations {
downscaleAggregationAxis(aggBucket, fromScale, toScale)
}
}
func downscaleAggregationAxis(aggBucket *AggregationBucket, fromScale, toScale int) {
if aggBucket == nil || len(aggBucket.Meta.Buckets) == 0 {
return
}
factor := int(math.Exp2(float64(fromScale - toScale)))
// Bands merge by their index in the exponential mapping, not by position in
// Meta.Buckets, which lists only the boundaries some series reached.
coarse := make([]float64, 0, len(aggBucket.Meta.Buckets))
targetBandIndexes := make([]int, len(aggBucket.Meta.Buckets))
seen := make(map[float64]int, len(aggBucket.Meta.Buckets))
for band, boundary := range aggBucket.Meta.Buckets {
merged := coarsenHeatmapBoundary(boundary, fromScale, toScale, factor)
coarseBandIndex, ok := seen[merged]
if !ok {
coarseBandIndex = len(coarse)
coarse = append(coarse, merged)
seen[merged] = coarseBandIndex
}
targetBandIndexes[band] = coarseBandIndex
}
regroupAxis(aggBucket, coarse, targetBandIndexes)
}
// coarsenHeatmapBoundary moves a boundary from the fromScale exponential axis
// onto the toScale one. The zero band has no exponent to rescale and stays put.
func coarsenHeatmapBoundary(boundary float64, fromScale, toScale, factor int) float64 {
if boundary <= 0 || math.IsInf(boundary, 0) || math.IsNaN(boundary) {
return boundary
}
index := int(math.Round(math.Log2(boundary) * math.Exp2(float64(fromScale))))
merged := int(math.Ceil(float64(index) / float64(factor)))
return math.Exp2(float64(merged) / math.Exp2(float64(toScale)))
}
// DensifyHeatmapAxis fills in the bands no series reached, which are left out of
// Meta.Buckets entirely and would otherwise render with the two sides of a gap
// touching.
//
// Only a value-derived axis can be densified: its boundaries come from an index
// that is a pure function of the value, so the ones in between are known without
// having seen them. Nothing says what sits between two `le` labels.
func DensifyHeatmapAxis(tsData *TimeSeriesData, bucketing HeatmapBucketing) {
if tsData == nil {
return
}
for _, aggBucket := range tsData.Aggregations {
densifyAggregationAxis(aggBucket, bucketing)
}
}
func densifyAggregationAxis(aggBucket *AggregationBucket, bucketing HeatmapBucketing) {
if aggBucket == nil || len(aggBucket.Meta.Buckets) == 0 {
return
}
// The zero band holds everything at or below zero. It has no index on either
// axis and sits below every other boundary, so it keeps band 0 and the fill
// runs over the rest.
offset := 0
if aggBucket.Meta.Buckets[0] <= 0 {
offset = 1
}
positive := aggBucket.Meta.Buckets[offset:]
if len(positive) == 0 {
return
}
// Only finite boundaries have a band index, and the fill sizes a slice from
// one. Nothing should put +Inf or NaN on the axis, but bail if it happens.
indexes := make([]int, len(positive))
for i, boundary := range positive {
if math.IsInf(boundary, 0) || math.IsNaN(boundary) {
return
}
indexes[i] = bucketing.calculateBandIndex(boundary)
}
lowest, highest := slices.Min(indexes), slices.Max(indexes)
dense := append([]float64{}, aggBucket.Meta.Buckets[:offset]...)
for index := lowest; index <= highest; index++ {
dense = append(dense, bucketing.calculateBandBoundary(index))
}
if len(dense) == len(aggBucket.Meta.Buckets) {
return
}
// Bands map through their index rather than by matching boundaries, so a
// regenerated boundary differing from ClickHouse's in its last bit still
// lands on the band it came from.
targetBandIndexes := make([]int, len(aggBucket.Meta.Buckets))
for i, index := range indexes {
targetBandIndexes[i+offset] = index - lowest + offset
}
regroupAxis(aggBucket, dense, targetBandIndexes)
}
// calculateBandIndex and calculateBandBoundary are inverses, and match the
// expressions the statement builder renders: k * maxValue / numBuckets on a
// linear axis, 2^(k / 2^scale) on a log one.
func (h HeatmapBucketing) calculateBandIndex(boundary float64) int {
if h.Kind == BucketsKindLinear {
return int(math.Round(boundary * float64(h.NumBuckets) / h.MaxValue))
}
return int(math.Round(math.Log2(boundary) * math.Exp2(float64(h.LogScale))))
}
func (h HeatmapBucketing) calculateBandBoundary(index int) float64 {
if h.Kind == BucketsKindLinear {
return float64(index) * h.MaxValue / float64(h.NumBuckets)
}
return math.Exp2(float64(index) / math.Exp2(float64(h.LogScale)))
}
// BucketTimeSeriesValues turns one value per (series, timestamp) into heatmap
// cells on the axis bucketing describes, which is what ClickHouse does for a
// gauge or sum. A formula has no statement to carry the boundary expression, so
// its output is bucketed here instead. Every value counts the one series it came
// from, so a point ends up with a single occupied cell.
func BucketTimeSeriesValues(tsData *TimeSeriesData, bucketing HeatmapBucketing) {
if tsData == nil {
return
}
for _, aggBucket := range tsData.Aggregations {
bucketAggregationValues(aggBucket, bucketing)
}
}
func bucketAggregationValues(aggBucket *AggregationBucket, bucketing HeatmapBucketing) {
if aggBucket == nil {
return
}
// +Inf is the open-above overflow rather than a boundary of its own, and a
// NaN value has no band at all
boundarySet := map[float64]struct{}{}
for _, series := range aggBucket.Series {
for _, point := range series.Values {
boundary := bucketing.calculateValueBoundary(point.Value)
if !math.IsNaN(boundary) && !math.IsInf(boundary, 0) {
boundarySet[boundary] = struct{}{}
}
}
}
boundaries := slices.Sorted(maps.Keys(boundarySet))
bandIndexByBoundary := make(map[float64]int, len(boundaries))
for band, boundary := range boundaries {
bandIndexByBoundary[boundary] = band
}
for _, series := range aggBucket.Series {
for _, point := range series.Values {
boundary := bucketing.calculateValueBoundary(point.Value)
point.Values = make([]float64, len(boundaries)+1)
point.Value = 0
switch {
case math.IsNaN(boundary):
case math.IsInf(boundary, 1):
point.Values[len(boundaries)] = 1
default:
point.Values[bandIndexByBoundary[boundary]] = 1
}
}
}
aggBucket.Meta.Buckets = boundaries
}
// calculateValueBoundary renders the upper bound of the band value falls in. It
// is the Go side of the expression the statement builder emits and has to stay
// identical to it: a formula heatmap and a metric heatmap that disagreed here
// would put their bands in different places.
func (h HeatmapBucketing) calculateValueBoundary(value float64) float64 {
if h.Kind == BucketsKindLinear {
if value > h.MaxValue {
return math.Inf(1)
}
numBuckets := float64(h.NumBuckets)
index := math.Min(math.Max(math.Ceil(value*numBuckets/h.MaxValue), 1), numBuckets)
return index * h.MaxValue / numBuckets
}
if value <= 0 {
return 0
}
if value <= LowestLogBoundary {
return LowestLogBoundary
}
if value > HighestLogBoundary {
return math.Inf(1)
}
bandsPerDoubling := math.Exp2(float64(h.LogScale))
return math.Exp2(math.Ceil(math.Log2(value)*bandsPerDoubling) / bandsPerDoubling)
}

View File

@@ -1,550 +0,0 @@
package querybuildertypesv5
import (
"math"
"testing"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRealignHeatmapValues(t *testing.T) {
testCases := []struct {
description string
from []float64
onto []float64
values []float64
expectedValues []float64
}{
{
description: "an unchanged axis is left alone",
from: []float64{5, 10},
onto: []float64{5, 10},
values: []float64{1, 2, 3},
expectedValues: []float64{1, 2, 3},
},
{
description: "an inserted bucket shifts the counts above it",
from: []float64{5, 10},
onto: []float64{2, 5, 10},
values: []float64{1, 2, 3},
expectedValues: []float64{0, 1, 2, 3},
},
{
description: "a dropped bucket loses its counts but the overflow survives",
from: []float64{5, 10, 25},
onto: []float64{5, 25},
values: []float64{1, 2, 3, 4},
expectedValues: []float64{1, 3, 4},
},
{
description: "an axis with nothing in common keeps only the overflow",
from: []float64{5, 10},
onto: []float64{100, 200},
values: []float64{1, 2, 3},
expectedValues: []float64{0, 0, 3},
},
{
description: "counts beyond the axis they were read against are dropped",
from: []float64{5},
onto: []float64{5, 10},
values: []float64{1, 2, 3},
expectedValues: []float64{1, 0, 2},
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
series := []*TimeSeries{{
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: testCase.values}},
}}
RealignHeatmapValues(series, testCase.from, testCase.onto)
require.Len(t, series[0].Values, 1)
assert.Equal(t, testCase.expectedValues, series[0].Values[0].Values)
})
}
}
func TestRealignHeatmapValuesLeavesNonHeatmapPointsAlone(t *testing.T) {
series := []*TimeSeries{{
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Value: 42}},
}}
RealignHeatmapValues(series, nil, []float64{5, 10})
assert.Equal(t, float64(42), series[0].Values[0].Value)
assert.Empty(t, series[0].Values[0].Values)
}
func TestRealignHeatmapValuesWithoutTargetAxis(t *testing.T) {
series := []*TimeSeries{{
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: []float64{1, 2}}},
}}
RealignHeatmapValues(series, []float64{5, 10}, nil)
assert.Equal(t, []float64{1, 2}, series[0].Values[0].Values)
}
func TestDownscaleHeatmapAxis(t *testing.T) {
testCases := []struct {
description string
fromScale int
toScale int
buckets []float64
values []float64
expectedBuckets []float64
expectedValues []float64
}{
{
description: "four scale-4 bands merge into one scale-2 band",
fromScale: 4,
toScale: 2,
buckets: []float64{
math.Exp2(0),
math.Exp2(1.0 / 16),
math.Exp2(2.0 / 16),
math.Exp2(3.0 / 16),
math.Exp2(4.0 / 16),
},
values: []float64{1, 2, 3, 4, 5, 6},
expectedBuckets: []float64{math.Exp2(0), math.Exp2(1.0 / 4)},
expectedValues: []float64{1, 14, 6},
},
{
description: "bands below 1 fold onto the same coarse boundary",
fromScale: 4,
toScale: 2,
buckets: []float64{math.Exp2(-3.0 / 16), math.Exp2(-2.0 / 16), math.Exp2(-1.0 / 16)},
values: []float64{1, 2, 3, 4},
expectedBuckets: []float64{math.Exp2(0)},
expectedValues: []float64{6, 4},
},
{
description: "the zero band keeps its own slot",
fromScale: 4,
toScale: 2,
buckets: []float64{0, math.Exp2(1.0 / 16), math.Exp2(4.0 / 16)},
values: []float64{7, 1, 2, 3},
expectedBuckets: []float64{0, math.Exp2(1.0 / 4)},
expectedValues: []float64{7, 3, 3},
},
{
// at scale 0 the whole doubling above 1 is a single band, and 2^(16/16)
// is its upper bound rather than the start of the next one
description: "a doubling's worth of bands collapses into one at scale 0",
fromScale: 4,
toScale: 0,
buckets: []float64{math.Exp2(1.0 / 16), math.Exp2(8.0 / 16), math.Exp2(16.0 / 16)},
values: []float64{1, 2, 3, 4},
expectedBuckets: []float64{math.Exp2(1)},
expectedValues: []float64{6, 4},
},
{
description: "the finest scale is left alone",
fromScale: 4,
toScale: 4,
buckets: []float64{math.Exp2(1.0 / 16), math.Exp2(2.0 / 16)},
values: []float64{1, 2, 3},
expectedBuckets: []float64{math.Exp2(1.0 / 16), math.Exp2(2.0 / 16)},
expectedValues: []float64{1, 2, 3},
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Meta: AggregationMeta{Buckets: testCase.buckets},
Series: []*TimeSeries{{
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: testCase.values}},
}},
}},
}
DownscaleHeatmapAxis(tsData, testCase.fromScale, testCase.toScale)
aggBucket := tsData.Aggregations[0]
assert.Equal(t, testCase.expectedBuckets, aggBucket.Meta.Buckets)
assert.Equal(t, testCase.expectedValues, aggBucket.Series[0].Values[0].Values)
})
}
}
func TestDownscaleHeatmapAxisKeepsTheTotalCount(t *testing.T) {
buckets := make([]float64, 0, 64)
values := make([]float64, 0, 65)
for index := range 64 {
buckets = append(buckets, math.Exp2(float64(index)/16))
values = append(values, float64(index))
}
values = append(values, 100)
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Meta: AggregationMeta{Buckets: buckets},
Series: []*TimeSeries{{
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: values}},
}},
}},
}
var before float64
for _, count := range values {
before += count
}
DownscaleHeatmapAxis(tsData, MaxLogScale, 1)
aggBucket := tsData.Aggregations[0]
// bands 0..63 fold onto ceil(k/8), so 0..8: the boundary at 2^0 keeps a band
// of its own and the four doublings above it take two each
assert.Len(t, aggBucket.Meta.Buckets, 9)
assert.Len(t, aggBucket.Series[0].Values[0].Values, 10)
var after float64
for _, count := range aggBucket.Series[0].Values[0].Values {
after += count
}
assert.Equal(t, before, after)
}
func TestDensifyHeatmapAxis(t *testing.T) {
testCases := []struct {
description string
bucketing HeatmapBucketing
buckets []float64
values []float64
expectedBuckets []float64
expectedValues []float64
}{
{
description: "an already contiguous log axis is left alone",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: 4},
buckets: []float64{math.Exp2(1.0 / 16), math.Exp2(2.0 / 16), math.Exp2(3.0 / 16)},
values: []float64{1, 2, 3, 4},
expectedBuckets: []float64{
math.Exp2(1.0 / 16),
math.Exp2(2.0 / 16),
math.Exp2(3.0 / 16),
},
expectedValues: []float64{1, 2, 3, 4},
},
{
description: "log bands nothing reached are filled in with zero",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: 4},
buckets: []float64{math.Exp2(1.0 / 16), math.Exp2(4.0 / 16)},
values: []float64{5, 7, 9},
expectedBuckets: []float64{
math.Exp2(1.0 / 16),
math.Exp2(2.0 / 16),
math.Exp2(3.0 / 16),
math.Exp2(4.0 / 16),
},
expectedValues: []float64{5, 0, 0, 7, 9},
},
{
description: "the zero band keeps the lowest slot and the fill starts above it",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: 4},
buckets: []float64{0, math.Exp2(1.0 / 16), math.Exp2(3.0 / 16)},
values: []float64{4, 5, 6, 7},
expectedBuckets: []float64{0, math.Exp2(1.0 / 16), math.Exp2(2.0 / 16), math.Exp2(3.0 / 16)},
expectedValues: []float64{4, 5, 0, 6, 7},
},
{
description: "a log axis spanning a doubling gets every band between",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: 1},
buckets: []float64{math.Exp2(0), math.Exp2(1)},
values: []float64{1, 2, 3},
expectedBuckets: []float64{math.Exp2(0), math.Exp2(0.5), math.Exp2(1)},
expectedValues: []float64{1, 0, 2, 3},
},
{
description: "linear bands nothing reached are filled in with zero",
bucketing: HeatmapBucketing{Kind: BucketsKindLinear, MaxValue: 500, NumBuckets: 25},
buckets: []float64{20, 100},
values: []float64{3, 4, 5},
expectedBuckets: []float64{20, 40, 60, 80, 100},
expectedValues: []float64{3, 0, 0, 0, 4, 5},
},
{
description: "a single band has nothing to fill in",
bucketing: HeatmapBucketing{Kind: BucketsKindLinear, MaxValue: 500, NumBuckets: 25},
buckets: []float64{100},
values: []float64{1, 2},
expectedBuckets: []float64{100},
expectedValues: []float64{1, 2},
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Meta: AggregationMeta{Buckets: testCase.buckets},
Series: []*TimeSeries{{
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: testCase.values}},
}},
}},
}
DensifyHeatmapAxis(tsData, testCase.bucketing)
aggBucket := tsData.Aggregations[0]
assert.Equal(t, testCase.expectedBuckets, aggBucket.Meta.Buckets)
assert.Equal(t, testCase.expectedValues, aggBucket.Series[0].Values[0].Values)
})
}
}
func TestDensifyHeatmapAxisKeepsTheTotalCount(t *testing.T) {
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Meta: AggregationMeta{Buckets: []float64{0, math.Exp2(2.0 / 16), math.Exp2(37.0 / 16)}},
Series: []*TimeSeries{{
Values: []*TimeSeriesValue{
{Timestamp: 1710000000000, Values: []float64{2, 3, 5, 7}},
{Timestamp: 1710000060000, Values: []float64{11, 13, 17, 19}},
},
}},
}},
}
DensifyHeatmapAxis(tsData, HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale})
aggBucket := tsData.Aggregations[0]
// the zero band plus every band from index 2 to index 37
assert.Len(t, aggBucket.Meta.Buckets, 37)
for _, point := range aggBucket.Series[0].Values {
assert.Len(t, point.Values, 38)
}
assert.Equal(t, float64(2+3+5+7), sumHeatmapCounts(aggBucket.Series[0].Values[0].Values))
assert.Equal(t, float64(11+13+17+19), sumHeatmapCounts(aggBucket.Series[0].Values[1].Values))
}
func sumHeatmapCounts(values []float64) float64 {
var total float64
for _, count := range values {
total += count
}
return total
}
func TestBucketTimeSeriesValues(t *testing.T) {
testCases := []struct {
description string
bucketing HeatmapBucketing
values []float64
expectedBuckets []float64
expectedValues [][]float64
}{
{
description: "log values land on the band above them",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale},
values: []float64{1, 2, 3},
expectedBuckets: []float64{
math.Exp2(0),
math.Exp2(1),
math.Exp2(26.0 / 16),
},
expectedValues: [][]float64{
{1, 0, 0, 0},
{0, 1, 0, 0},
{0, 0, 1, 0},
},
},
{
// the log axis has no band below zero, so both report the boundary
// that means "everything at or below zero"
description: "zero and negative values share the lowest log band",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale},
values: []float64{-5, 0, 1},
expectedBuckets: []float64{0, 1},
expectedValues: [][]float64{
{1, 0, 0},
{1, 0, 0},
{0, 1, 0},
},
},
{
description: "a linear value above maxValue lands in the overflow",
bucketing: HeatmapBucketing{Kind: BucketsKindLinear, MaxValue: 100, NumBuckets: 4},
values: []float64{30, 100, 150, 0},
expectedBuckets: []float64{25, 50, 100},
expectedValues: [][]float64{
{0, 1, 0, 0},
{0, 0, 1, 0},
{0, 0, 0, 1},
{1, 0, 0, 0},
},
},
{
description: "a value with no band occupies no cell",
bucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale},
values: []float64{math.NaN(), 1},
expectedBuckets: []float64{1},
expectedValues: [][]float64{
{0, 0},
{1, 0},
},
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
points := make([]*TimeSeriesValue, 0, len(testCase.values))
for index, value := range testCase.values {
points = append(points, &TimeSeriesValue{
Timestamp: 1710000000000 + int64(index)*60000,
Value: value,
})
}
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Series: []*TimeSeries{{Values: points}},
}},
}
BucketTimeSeriesValues(tsData, testCase.bucketing)
aggBucket := tsData.Aggregations[0]
assert.Equal(t, testCase.expectedBuckets, aggBucket.Meta.Buckets)
for index, point := range aggBucket.Series[0].Values {
assert.Equal(t, testCase.expectedValues[index], point.Values, "point %d", index)
assert.Zero(t, point.Value, "point %d keeps its scalar value", index)
}
})
}
}
func TestBucketTimeSeriesValuesSharesOneAxisAcrossSeries(t *testing.T) {
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Series: []*TimeSeries{
{
Labels: []*Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host.name"}, Value: "a"}},
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Value: 1}},
},
{
Labels: []*Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host.name"}, Value: "b"}},
Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Value: 4}},
},
},
}},
}
BucketTimeSeriesValues(tsData, HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale})
aggBucket := tsData.Aggregations[0]
assert.Equal(t, []float64{math.Exp2(0), math.Exp2(2)}, aggBucket.Meta.Buckets)
// each series counts itself, and the panel adds up whichever are selected
assert.Equal(t, []float64{1, 0, 0}, aggBucket.Series[0].Values[0].Values)
assert.Equal(t, []float64{0, 1, 0}, aggBucket.Series[1].Values[0].Values)
}
func TestBucketTimeSeriesValuesMatchesTheStatementBuilderBoundaries(t *testing.T) {
// the same expressions the statement builder renders, evaluated in Go:
// multiIf(value <= 0, 0, pow(2, ceil(log2(value) * 16) / 16)) and
// multiIf(value > max, +Inf, least(greatest(ceil(value * n / max), 1), n) * max / n)
logBucketing := HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale}
assert.Equal(t, math.Exp2(math.Ceil(math.Log2(37)*16)/16), logBucketing.calculateValueBoundary(37))
assert.Equal(t, 0.0, logBucketing.calculateValueBoundary(-1))
linearBucketing := HeatmapBucketing{Kind: BucketsKindLinear, MaxValue: 500, NumBuckets: 25}
assert.Equal(t, math.Ceil(37.0*25/500)*500/25, linearBucketing.calculateValueBoundary(37))
assert.Equal(t, 1*500.0/25, linearBucketing.calculateValueBoundary(0))
assert.True(t, math.IsInf(linearBucketing.calculateValueBoundary(501), 1))
}
func TestHeatmapBoundaryForClampsTheLogAxis(t *testing.T) {
bucketing := HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale}
// without the clamp the band index runs off to -inf as a positive value
// approaches zero, and the axis fill follows it
assert.Equal(t, LowestLogBoundary, bucketing.calculateValueBoundary(1e-30))
assert.Equal(t, LowestLogBoundary, bucketing.calculateValueBoundary(math.SmallestNonzeroFloat64))
assert.Equal(t, LowestLogBoundary, bucketing.calculateValueBoundary(LowestLogBoundary))
assert.True(t, math.IsInf(bucketing.calculateValueBoundary(1e30), 1))
assert.True(t, math.IsInf(bucketing.calculateValueBoundary(math.MaxFloat64), 1))
assert.Equal(t, HighestLogBoundary, bucketing.calculateValueBoundary(HighestLogBoundary))
// zero and negatives keep their own band below the floor
assert.Equal(t, 0.0, bucketing.calculateValueBoundary(0))
assert.Equal(t, 0.0, bucketing.calculateValueBoundary(-5))
// anything in between is untouched
assert.Equal(t, math.Exp2(math.Ceil(math.Log2(37)*16)/16), bucketing.calculateValueBoundary(37))
}
func TestLogAxisClampsStayOnTheGridAtEveryScale(t *testing.T) {
// a coarser fold must land the clamped ends on real boundaries, which holds
// because both indexes are powers of two
for scale := MinLogScale; scale <= MaxLogScale; scale++ {
bandsPerDoubling := math.Exp2(float64(scale))
for _, boundary := range []float64{LowestLogBoundary, HighestLogBoundary} {
index := math.Log2(boundary) * bandsPerDoubling
assert.Equal(t, math.Trunc(index), index, "scale %d, boundary %g", scale, boundary)
}
}
}
func TestDensifyHeatmapAxisIsBoundedByTheFloor(t *testing.T) {
bucketing := HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale}
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Series: []*TimeSeries{{
Values: []*TimeSeriesValue{
{Timestamp: 1710000000000, Value: 1e-30},
{Timestamp: 1710000060000, Value: 1000},
},
}},
}},
}
BucketTimeSeriesValues(tsData, bucketing)
DensifyHeatmapAxis(tsData, bucketing)
// 1e-30 clamps to the floor, so the fill spans MinLogBandIndex upwards
// rather than chasing that value's own index near -1594
buckets := tsData.Aggregations[0].Meta.Buckets
highest := bucketing.calculateBandIndex(bucketing.calculateValueBoundary(1000))
assert.Equal(t, LowestLogBoundary, buckets[0])
assert.Len(t, buckets, highest-MinLogBandIndex+1)
}
func TestDensifyHeatmapAxisSkipsANonFiniteBoundary(t *testing.T) {
// the overflow is the slot past the axis, never a boundary on it; a bad one
// would otherwise size the fill from a garbage band index
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Meta: AggregationMeta{Buckets: []float64{1, math.Inf(1)}},
Series: []*TimeSeries{{Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: []float64{1, 1, 0}}}}},
}},
}
DensifyHeatmapAxis(tsData, HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale})
assert.Equal(t, []float64{1, math.Inf(1)}, tsData.Aggregations[0].Meta.Buckets)
}
func TestDensifyHeatmapAxisWorstCaseSpan(t *testing.T) {
bucketing := HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale}
tsData := &TimeSeriesData{
Aggregations: []*AggregationBucket{{
Meta: AggregationMeta{Buckets: []float64{LowestLogBoundary, HighestLogBoundary}},
Series: []*TimeSeries{{Values: []*TimeSeriesValue{{Timestamp: 1710000000000, Values: []float64{1, 1, 0}}}}},
}},
}
DensifyHeatmapAxis(tsData, bucketing)
// the widest axis the bucketing can produce, whatever the data does
assert.Len(t, tsData.Aggregations[0].Meta.Buckets, MaxLogBandIndex-MinLogBandIndex+1)
}

View File

@@ -1,656 +0,0 @@
package querybuildertypesv5
import (
"encoding/json"
"math"
"testing"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestValidateHeatmapRequest(t *testing.T) {
testCases := []struct {
description string
request QueryRangeRequest
expectedErrContains string
}{
{
description: "a single metrics builder query with increase and sum is accepted",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
}}},
},
},
{
// the axis comes from the `le` labels, so the space aggregation has
// nothing left to pick out and a percentile draws the same heatmap a
// count would
description: "percentile space aggregation is accepted",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationRate,
SpaceAggregation: metrictypes.SpaceAggregationPercentile95,
}},
},
}}},
},
},
{
// the statement builder strips `le` from a histogram's groupBy before
// re-adding it for the bucket CTE, the same as any other histogram
// query, so it needs no heatmap rule of its own
description: "le in groupBy is accepted",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
GroupBy: []GroupByKey{{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "le"},
}},
},
}}},
},
},
{
description: "having is refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
Having: &Having{Expression: "sum(http.server.request.duration) > 10"},
},
}}},
},
expectedErrContains: "having is not supported",
},
{
description: "functions are refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
Functions: []Function{{Name: FunctionNameAbsolute}},
},
}}},
},
expectedErrContains: "functions are not supported",
},
{
// a promql histogram carries `le` through the matrix as an ordinary
// label, which is the same axis the builder's histogram path reads
description: "a promql query is accepted",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypePromQL,
Spec: PromQuery{Name: "A", Query: "sum by (le) (increase(signoz_latency_bucket[5m]))"},
}}},
},
},
{
description: "bucket options alongside a promql query are refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
BucketOptions: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{}},
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypePromQL,
Spec: PromQuery{Name: "A", Query: "sum by (le) (increase(signoz_latency_bucket[5m]))"},
}}},
},
expectedErrContains: "bucketOptions are not supported for promql heatmap requests",
},
{
// a clickhouse query's rows are read by request type like any other,
// so one shaped as heatmap cells renders without the builder
description: "a clickhouse query is accepted",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeClickHouseSQL,
Spec: ClickHouseQuery{Name: "A", Query: "SELECT ts, bucket, value FROM cells"},
}}},
},
},
{
description: "a formula over disabled builder queries is accepted",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Disabled: true,
Aggregations: []MetricAggregation{{
MetricName: "system.memory.usage",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
},
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "B",
Signal: telemetrytypes.SignalMetrics,
Disabled: true,
Aggregations: []MetricAggregation{{
MetricName: "system.memory.limit",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
},
{
Type: QueryTypeFormula,
Spec: QueryBuilderFormula{Name: "F1", Expression: "A / B"},
},
}},
},
},
{
description: "a formula alongside an enabled query is refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "system.memory.usage",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
},
{
Type: QueryTypeFormula,
Spec: QueryBuilderFormula{Name: "F1", Expression: "A * 2"},
},
}},
},
expectedErrContains: "exactly one enabled query",
},
{
description: "functions on a formula are refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Disabled: true,
Aggregations: []MetricAggregation{{
MetricName: "system.memory.usage",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
},
{
Type: QueryTypeFormula,
Spec: QueryBuilderFormula{
Name: "F1",
Expression: "A * 2",
Functions: []Function{{Name: FunctionNameAbsolute}},
},
},
}},
},
expectedErrContains: "functions are not supported",
},
{
// a disabled query is a formula input, so its functions still reach
// the cells the heatmap draws
description: "functions on a disabled formula input are refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Disabled: true,
Aggregations: []MetricAggregation{{
MetricName: "system.memory.usage",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
Functions: []Function{{Name: FunctionNameAbsolute}},
},
},
{
Type: QueryTypeFormula,
Spec: QueryBuilderFormula{Name: "F1", Expression: "A * 2"},
},
}},
},
expectedErrContains: "functions are not supported",
},
{
description: "a disabled clickhouse query leaves nothing to draw",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeClickHouseSQL,
Spec: ClickHouseQuery{Name: "A", Query: "SELECT 1", Disabled: true},
}}},
},
expectedErrContains: "exactly one enabled query",
},
{
description: "two enabled queries are refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
},
{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "B",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.body.size",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
},
}},
},
expectedErrContains: "exactly one enabled query",
},
{
description: "fillGaps is refused",
request: QueryRangeRequest{
Start: 1710000000000,
End: 1710003600000,
RequestType: RequestTypeHeatmap,
FormatOptions: &FormatOptions{FillGaps: true},
CompositeQuery: CompositeQuery{Queries: []QueryEnvelope{{
Type: QueryTypeBuilder,
Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []MetricAggregation{{
MetricName: "http.server.request.duration",
TimeAggregation: metrictypes.TimeAggregationIncrease,
SpaceAggregation: metrictypes.SpaceAggregationSum,
}},
},
}}},
},
expectedErrContains: "fillGaps is not supported",
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
err := testCase.request.Validate()
if testCase.expectedErrContains == "" {
require.NoError(t, err)
return
}
require.Error(t, err)
assert.Contains(t, err.Error(), testCase.expectedErrContains)
})
}
}
func TestHeatmapRequestTypeIsAccepted(t *testing.T) {
var requestType RequestType
require.NoError(t, requestType.UnmarshalJSON([]byte(`"heatmap"`)))
assert.Equal(t, RequestTypeHeatmap, requestType)
assert.True(t, requestType.IsAggregation())
}
func TestResolveBucketOptions(t *testing.T) {
coarseScale := 2
testCases := []struct {
description string
options *BucketOptions
expectedBucketing HeatmapBucketing
expectedLogScale int
}{
{
description: "an absent config defaults to the finest log axis",
options: nil,
expectedBucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale, NumBuckets: DefaultNumBuckets},
expectedLogScale: MaxLogScale,
},
{
description: "a linear spec carries its cap and count through",
options: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 1024, NumBuckets: 20}},
expectedBucketing: HeatmapBucketing{Kind: BucketsKindLinear, LogScale: MaxLogScale, MaxValue: 1024, NumBuckets: 20},
expectedLogScale: MaxLogScale,
},
{
description: "a linear spec without a count takes the default",
options: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 1024}},
expectedBucketing: HeatmapBucketing{Kind: BucketsKindLinear, LogScale: MaxLogScale, MaxValue: 1024, NumBuckets: DefaultNumBuckets},
expectedLogScale: MaxLogScale,
},
{
description: "a coarser scale is kept out of the axis clickhouse builds",
options: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{Scale: &coarseScale}},
expectedBucketing: HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale, NumBuckets: DefaultNumBuckets},
expectedLogScale: 2,
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
assert.Equal(t, testCase.expectedBucketing, testCase.options.ResolveBucketOptions())
assert.Equal(t, testCase.expectedLogScale, testCase.options.ResolveLogScale())
})
}
}
func TestUnmarshalBucketOptions(t *testing.T) {
scale := 2
testCases := []struct {
description string
body string
expectedOptions BucketOptions
expectedErrContains string
}{
{
description: "a linear kind decodes its own spec",
body: `{"kind":"linear","spec":{"maxValue":500,"numBuckets":25}}`,
expectedOptions: BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 500, NumBuckets: 25}},
},
{
description: "a log kind decodes its own spec",
body: `{"kind":"log","spec":{"scale":2}}`,
expectedOptions: BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{Scale: &scale}},
},
{
description: "an empty log spec asks for the defaults",
body: `{"kind":"log","spec":{}}`,
expectedOptions: BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{}},
},
{
description: "a kind with no spec beside it is refused",
body: `{"kind":"log"}`,
expectedErrContains: "bucketOptions spec is required",
},
{
description: "an unknown kind is refused",
body: `{"kind":"quadratic","spec":{}}`,
expectedErrContains: "invalid bucketOptions kind",
},
{
description: "a missing kind is refused",
body: `{"spec":{"maxValue":500}}`,
expectedErrContains: "invalid bucketOptions kind",
},
{
// the kind picks the spec, so a field belonging to the other one is a
// typo rather than something to quietly drop
description: "a log field under a linear kind is refused",
body: `{"kind":"linear","spec":{"maxValue":500,"scale":2}}`,
expectedErrContains: "scale",
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
var options BucketOptions
err := json.Unmarshal([]byte(testCase.body), &options)
if testCase.expectedErrContains != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), testCase.expectedErrContains)
return
}
require.NoError(t, err)
assert.Equal(t, testCase.expectedOptions, options)
})
}
}
func TestValidateBucketOptions(t *testing.T) {
tooFine := MaxLogScale + 1
tooCoarse := MinLogScale - 1
coarseScale := 2
testCases := []struct {
description string
options *BucketOptions
expectedErrContains string
}{
{
description: "an absent config is accepted",
options: nil,
},
{
description: "a full linear spec is accepted",
options: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 1024, NumBuckets: 32}},
},
{
description: "a log spec with a coarser scale is accepted",
options: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{Scale: &coarseScale}},
},
{
description: "an empty log spec is accepted",
options: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{}},
},
{
description: "a bucket count above the cap is refused",
options: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 1024, NumBuckets: MaxNumBuckets + 1}},
expectedErrContains: "numBuckets must be between",
},
{
description: "a kind with no spec behind it is refused",
options: &BucketOptions{Kind: BucketsKind{valuer.NewString("quadratic")}},
expectedErrContains: "invalid bucketOptions kind",
},
{
description: "a non-finite maxValue is refused",
options: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: math.NaN()}},
expectedErrContains: "finite maxValue greater than 0",
},
{
// a linear spec that omits maxValue decodes to zero, which is the
// same refusal
description: "a maxValue at zero is refused",
options: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{}},
expectedErrContains: "finite maxValue greater than 0",
},
{
description: "a scale finer than clickhouse buckets at is refused",
options: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{Scale: &tooFine}},
expectedErrContains: "scale must be between",
},
{
description: "a scale below the coarsest axis is refused",
options: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{Scale: &tooCoarse}},
expectedErrContains: "scale must be between",
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
err := testCase.options.validateBucketOptions()
if testCase.expectedErrContains == "" {
require.NoError(t, err)
return
}
require.Error(t, err)
assert.Contains(t, err.Error(), testCase.expectedErrContains)
})
}
}
func TestResolveHeatmapBucketing(t *testing.T) {
coarseScale := 1
testCases := []struct {
description string
aggregation MetricAggregation
bucketOptions *BucketOptions
expectedBucketing *HeatmapBucketing
expectedErrContains string
}{
{
description: "a histogram buckets on its own le labels",
aggregation: MetricAggregation{MetricName: "http.server.request.duration", Type: metrictypes.HistogramType},
bucketOptions: nil,
expectedBucketing: nil,
},
{
description: "bucketOptions alongside a histogram are refused",
aggregation: MetricAggregation{MetricName: "http.server.request.duration", Type: metrictypes.HistogramType},
bucketOptions: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 500}},
expectedErrContains: "bucketOptions are not supported for histogram metrics",
},
{
description: "a gauge with no options gets the default log axis",
aggregation: MetricAggregation{MetricName: "system.memory.usage", Type: metrictypes.GaugeType},
bucketOptions: nil,
expectedBucketing: &HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale, NumBuckets: DefaultNumBuckets},
},
{
description: "a sum takes the requested linear axis",
aggregation: MetricAggregation{MetricName: "http.server.request.count", Type: metrictypes.SumType},
bucketOptions: &BucketOptions{Kind: BucketsKindLinear, Spec: LinearBucketsSpec{MaxValue: 500, NumBuckets: 25}},
expectedBucketing: &HeatmapBucketing{Kind: BucketsKindLinear, LogScale: MaxLogScale, MaxValue: 500, NumBuckets: 25},
},
{
description: "a coarser scale does not change the axis clickhouse builds",
aggregation: MetricAggregation{MetricName: "system.memory.usage", Type: metrictypes.GaugeType},
bucketOptions: &BucketOptions{Kind: BucketsKindLog, Spec: LogBucketsSpec{Scale: &coarseScale}},
expectedBucketing: &HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale, NumBuckets: DefaultNumBuckets},
},
{
description: "an unresolved type is refused",
aggregation: MetricAggregation{MetricName: "never.seen", Type: metrictypes.UnspecifiedType},
expectedErrContains: "no type is recorded",
},
{
description: "an exponential histogram is refused",
aggregation: MetricAggregation{MetricName: "http.server.request.duration", Type: metrictypes.ExpHistogramType},
expectedErrContains: "keeps its bucket counts in a sketch column",
},
{
description: "a summary buckets like a gauge",
aggregation: MetricAggregation{MetricName: "go.gc.duration", Type: metrictypes.SummaryType},
bucketOptions: nil,
expectedBucketing: &HeatmapBucketing{Kind: BucketsKindLog, LogScale: MaxLogScale, NumBuckets: DefaultNumBuckets},
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
err := testCase.aggregation.ResolveHeatmapBucketing(testCase.bucketOptions)
if testCase.expectedErrContains != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), testCase.expectedErrContains)
assert.Contains(t, err.Error(), testCase.aggregation.MetricName)
return
}
require.NoError(t, err)
assert.Equal(t, testCase.expectedBucketing, testCase.aggregation.HeatmapBucketing)
})
}
}

View File

@@ -397,147 +397,6 @@ type QueryRangeRequest struct {
PromQLProvider string `json:"-"`
FormatOptions *FormatOptions `json:"formatOptions,omitempty"`
// BucketOptions shapes the bucket axis for heatmap requests, and is refused
// rather than ignored for the metrics that bucket on their own `le` labels.
BucketOptions *BucketOptions `json:"bucketOptions,omitempty"`
}
// BucketOptions configures how a value range is divided into heatmap buckets.
type BucketOptions struct {
Kind BucketsKind `json:"kind"`
// Spec holds the LinearBucketsSpec or LogBucketsSpec for Kind.
Spec any `json:"spec"`
}
const (
DefaultNumBuckets = 60
MaxNumBuckets = 512
// MaxLogScale is the resolution ClickHouse buckets every log heatmap at:
// 2^MaxLogScale bands per doubling. It is both the default and the finest
// available, since a coarser LogBucketsSpec.Scale folds down from it.
MaxLogScale = 4
// MinLogScale is one band per 16x, the coarsest axis worth rendering.
MinLogScale = -4
)
type BucketsKind struct {
valuer.String
}
var (
BucketsKindLinear = BucketsKind{valuer.NewString("linear")}
BucketsKindLog = BucketsKind{valuer.NewString("log")}
)
// Enum implements jsonschema.Enum.
func (BucketsKind) Enum() []any {
return []any{
BucketsKindLinear,
BucketsKindLog,
}
}
// LinearBucketsSpec divides (0, MaxValue] into NumBuckets equal bands.
type LinearBucketsSpec struct {
// Everything above MaxValue is counted in the trailing overflow band. Evenly
// spaced boundaries have no top to divide without it, so it is required.
MaxValue float64 `json:"maxValue" required:"true"`
// DefaultNumBuckets applies when unset.
NumBuckets int `json:"numBuckets,omitempty"`
}
// LogBucketsSpec spaces boundaries at 2^Scale bands per doubling, the mapping
// an exponential histogram uses.
type LogBucketsSpec struct {
// ClickHouse always buckets at MaxLogScale and the surplus is folded away
// afterwards, so every Scale reads the same cache entry. MaxLogScale applies
// when unset.
Scale *int `json:"scale,omitempty"`
}
// bucketOptionsLinear and bucketOptionsLog are the OpenAPI schemas for the two
// BucketOptions variants. They have to be named types: the reflector turns an
// anonymous one into an inline subschema, leaving the discriminator mapping in
// PrepareJSONSchema pointing at components that were never emitted. `kind` is
// required:"true" on both so oapi-codegen renders the discriminator non-pointer.
type bucketOptionsLinear struct {
Kind BucketsKind `json:"kind" required:"true" description:"How the boundaries are spaced."`
Spec LinearBucketsSpec `json:"spec" required:"true" description:"The evenly spaced bucket specification."`
}
type bucketOptionsLog struct {
Kind BucketsKind `json:"kind" required:"true" description:"How the boundaries are spaced."`
Spec LogBucketsSpec `json:"spec" required:"true" description:"The logarithmic bucket specification."`
}
var _ jsonschema.OneOfExposer = BucketOptions{}
func (BucketOptions) JSONSchemaOneOf() []any {
return []any{
bucketOptionsLinear{},
bucketOptionsLog{},
}
}
var _ jsonschema.Preparer = BucketOptions{}
// PrepareJSONSchema marks the options as a `kind`-discriminated union;
// signoz.attachDiscriminators promotes it and strips the base properties.
func (BucketOptions) PrepareJSONSchema(s *jsonschema.Schema) error {
if s.ExtraProperties == nil {
s.ExtraProperties = map[string]any{}
}
s.ExtraProperties["x-signoz-discriminator"] = map[string]any{
"propertyName": "kind",
"mapping": map[string]string{
BucketsKindLinear.StringValue(): "#/components/schemas/Querybuildertypesv5BucketOptionsLinear",
BucketsKindLog.StringValue(): "#/components/schemas/Querybuildertypesv5BucketOptionsLog",
},
}
return nil
}
func (b *BucketOptions) UnmarshalJSON(data []byte) error {
var shadow struct {
Kind BucketsKind `json:"kind"`
Spec json.RawMessage `json:"spec"`
}
if err := binding.JSON.BindBody(bytes.NewReader(data), &shadow, binding.WithDisallowUnknownFields(true)); err != nil {
return err
}
b.Kind = shadow.Kind
// An absent spec is a malformed pair rather than a request for defaults;
// `"spec": {}` asks for those.
if len(shadow.Spec) == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"bucketOptions spec is required, use an empty object for the kind's defaults")
}
switch shadow.Kind {
case BucketsKindLinear:
var spec LinearBucketsSpec
if err := binding.JSON.BindBody(bytes.NewReader(shadow.Spec), &spec, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("linear buckets spec")); err != nil {
return err
}
b.Spec = spec
case BucketsKindLog:
var spec LogBucketsSpec
if err := binding.JSON.BindBody(bytes.NewReader(shadow.Spec), &spec, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("log buckets spec")); err != nil {
return err
}
b.Spec = spec
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"invalid bucketOptions kind %q, expected one of linear, log", shadow.Kind.StringValue())
}
return nil
}
// PrepareJSONSchema adds description to the QueryRangeRequest schema.

View File

@@ -19,11 +19,11 @@ func (r *RequestType) UnmarshalJSON(data []byte) error {
}
v := RequestType{valuer.NewString(s)}
switch v {
case RequestTypeScalar, RequestTypeTimeSeries, RequestTypeRaw, RequestTypeRawStream, RequestTypeTrace, RequestTypeDistribution, RequestTypeHeatmap:
case RequestTypeScalar, RequestTypeTimeSeries, RequestTypeRaw, RequestTypeRawStream, RequestTypeTrace, RequestTypeDistribution:
*r = v
return nil
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unknown request type %q; allowed values: %s", s, "`scalar`, `time_series`, `raw`, `raw_stream`, `trace`, `distribution`, `heatmap`")
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unknown request type %q; allowed values: %s", s, "`scalar`, `time_series`, `raw`, `raw_stream`, `trace`, `distribution`")
}
}
@@ -41,9 +41,6 @@ var (
RequestTypeTrace = RequestType{valuer.NewString("trace")}
// []Bucket (struct{Lower,Upper,Count float64}), example: histogram.
RequestTypeDistribution = RequestType{valuer.NewString("distribution")}
// TimeSeriesData carrying one count per histogram bucket at each timestamp,
// with the shared bucket boundaries on the aggregation's meta.
RequestTypeHeatmap = RequestType{valuer.NewString("heatmap")}
)
// IsAggregation returns true for request types that produce aggregated results
@@ -52,7 +49,7 @@ var (
// For non-aggregation types (raw, raw_stream, trace), those fields are ignored
// and don't need to be validated.
func (r RequestType) IsAggregation() bool {
return r == RequestTypeTimeSeries || r == RequestTypeScalar || r == RequestTypeDistribution || r == RequestTypeHeatmap
return r == RequestTypeTimeSeries || r == RequestTypeScalar || r == RequestTypeDistribution
}
// Enum implements jsonschema.Enum; returns the acceptable values for RequestType.
@@ -63,7 +60,6 @@ func (RequestType) Enum() []any {
RequestTypeRaw,
RequestTypeRawStream,
RequestTypeTrace,
RequestTypeHeatmap,
// RequestTypeDistribution,
}
}

View File

@@ -138,10 +138,12 @@ type TimeSeriesData struct {
}
type AggregationBucket struct {
Index int `json:"index"` // or string Alias
Alias string `json:"alias"`
Meta AggregationMeta `json:"meta,omitempty"`
Series []*TimeSeries `json:"series"` // no extra nesting
Index int `json:"index"` // or string Alias
Alias string `json:"alias"`
Meta struct {
Unit string `json:"unit,omitempty"`
} `json:"meta,omitempty"`
Series []*TimeSeries `json:"series"` // no extra nesting
PredictedSeries []*TimeSeries `json:"predictedSeries,omitempty"`
UpperBoundSeries []*TimeSeries `json:"upperBoundSeries,omitempty"`
@@ -149,20 +151,6 @@ type AggregationBucket struct {
AnomalyScores []*TimeSeries `json:"anomalyScores,omitempty"`
}
// HeatmapBucketColumn is the alias a heatmap statement gives the column holding
// a row's bucket boundary. Every other aggregation returns a single numeric
// column the reader treats as the value; this name tells the two apart.
const HeatmapBucketColumn = "__bucket"
type AggregationMeta struct {
Unit string `json:"unit,omitempty"`
// Buckets are the ascending bucket upper bounds shared by every series here.
// Set only for heatmap results, where each point's Values holds
// len(Buckets)+1 counts: one per bucket, then the open-above overflow, whose
// bound is `le=+Inf` and so cannot be listed as a JSON number.
Buckets []float64 `json:"buckets,omitempty"`
}
type TimeSeries struct {
Labels []*Label `json:"labels,omitempty"`
Values []*TimeSeriesValue `json:"values"`
@@ -266,9 +254,13 @@ type TimeSeriesValue struct {
// on the client side, these partial values are rendered differently.
Partial bool `json:"partial,omitempty"`
// Values holds one count per histogram bucket for heatmap results, in the
// order of the aggregation's Meta.Buckets. Value is unused in that case.
// for the heatmap type chart
Values []float64 `json:"values,omitempty"`
Bucket *Bucket `json:"bucket,omitempty"`
}
type Bucket struct {
Step float64 `json:"step"`
}
type ColumnType struct {

View File

@@ -127,7 +127,7 @@ func calculateSeriesValue(series *TimeSeries) float64 {
// For single-point series, return that value directly
if len(series.Values) == 1 {
value := calculatePointValue(series.Values[0])
value := series.Values[0].Value
if math.IsNaN(value) || math.IsInf(value, 0) {
return 0.0
}
@@ -139,11 +139,10 @@ func calculateSeriesValue(series *TimeSeries) float64 {
var count float64
for _, point := range series.Values {
value := calculatePointValue(point)
if math.IsNaN(value) || math.IsInf(value, 0) {
if math.IsNaN(point.Value) || math.IsInf(point.Value, 0) {
continue
}
sum += value
sum += point.Value
count++
}
@@ -155,25 +154,6 @@ func calculateSeriesValue(series *TimeSeries) float64 {
return sum / count
}
// calculatePointValue returns what a point contributes to its series' rank.
// Heatmap points carry one count per bucket in Values and leave Value at zero,
// so they rank on the total across buckets.
func calculatePointValue(point *TimeSeriesValue) float64 {
if len(point.Values) == 0 {
return point.Value
}
var total float64
for _, value := range point.Values {
if math.IsNaN(value) || math.IsInf(value, 0) {
continue
}
total += value
}
return total
}
// convertValueToString converts various types to string for comparison.
func convertValueToString(value any) string {
switch v := value.(type) {

View File

@@ -1,12 +1,10 @@
package querybuildertypesv5
import (
"math"
"testing"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestApplySeriesLimit(t *testing.T) {
@@ -234,81 +232,3 @@ func TestApplySeriesLimit(t *testing.T) {
assert.Equal(t, 40.0, result[2].Values[0].Value)
})
}
func TestApplySeriesLimitRanksHeatmapSeriesByBucketTotals(t *testing.T) {
// A reshaped heatmap point leaves Value at zero and holds one count per
// bucket in Values, so ranking has to sum the buckets to see any difference.
series := []*TimeSeries{
{
Labels: []*Label{{
Key: telemetrytypes.TelemetryFieldKey{Name: "service.name"},
Value: "quiet",
}},
Values: []*TimeSeriesValue{
{Timestamp: 1000, Values: []float64{1, 2, 0}},
{Timestamp: 1060, Values: []float64{0, 1, 0}},
},
},
{
Labels: []*Label{{
Key: telemetrytypes.TelemetryFieldKey{Name: "service.name"},
Value: "busy",
}},
Values: []*TimeSeriesValue{
{Timestamp: 1000, Values: []float64{40, 60, 5}},
{Timestamp: 1060, Values: []float64{30, 70, 5}},
},
},
{
Labels: []*Label{{
Key: telemetrytypes.TelemetryFieldKey{Name: "service.name"},
Value: "middling",
}},
Values: []*TimeSeriesValue{
{Timestamp: 1000, Values: []float64{5, 5, 0}},
{Timestamp: 1060, Values: []float64{4, 6, 0}},
},
},
}
result := ApplySeriesLimit(series, nil, 2)
require.Len(t, result, 2)
assert.Equal(t, "busy", result[0].Labels[0].Value)
assert.Equal(t, "middling", result[1].Labels[0].Value)
}
func TestCalculatePointValue(t *testing.T) {
testCases := []struct {
description string
point *TimeSeriesValue
expectedValue float64
}{
{
description: "a plain time series point ranks on its single value",
point: &TimeSeriesValue{Timestamp: 1000, Value: 7},
expectedValue: 7,
},
{
description: "a heatmap point ranks on the total across its buckets",
point: &TimeSeriesValue{Timestamp: 1000, Values: []float64{1, 12, 14, 3}},
expectedValue: 30,
},
{
description: "non-finite bucket counts are skipped",
point: &TimeSeriesValue{Timestamp: 1000, Values: []float64{2, math.NaN(), math.Inf(1), 3}},
expectedValue: 5,
},
{
description: "an empty bucket list falls back to the single value",
point: &TimeSeriesValue{Timestamp: 1000, Value: 4, Values: []float64{}},
expectedValue: 4,
},
}
for _, testCase := range testCases {
t.Run(testCase.description, func(t *testing.T) {
assert.Equal(t, testCase.expectedValue, calculatePointValue(testCase.point))
})
}
}

View File

@@ -2,7 +2,6 @@ package querybuildertypesv5
import (
"fmt"
"math"
"slices"
"strings"
@@ -582,7 +581,7 @@ func (r *QueryRangeRequest) Validate(opts ...ValidationOption) error {
// Validate request type
switch r.RequestType {
case RequestTypeRaw, RequestTypeRawStream, RequestTypeTrace, RequestTypeTimeSeries, RequestTypeScalar, RequestTypeHeatmap:
case RequestTypeRaw, RequestTypeRawStream, RequestTypeTrace, RequestTypeTimeSeries, RequestTypeScalar:
opts = append(opts, GetValidationOptions(r.RequestType)...)
default:
return errors.NewInvalidInputf(
@@ -590,14 +589,10 @@ func (r *QueryRangeRequest) Validate(opts ...ValidationOption) error {
"invalid request type: %s",
r.RequestType,
).WithAdditional(
"Valid request types are: raw, timeseries, scalar, heatmap",
"Valid request types are: raw, timeseries, scalar",
)
}
if err := r.validateHeatmap(); err != nil {
return err
}
// raw/trace request types don't support metric queries;
// metrics are always aggregated and there is no raw form.
if r.RequestType == RequestTypeRaw || r.RequestType == RequestTypeRawStream || r.RequestType == RequestTypeTrace {
@@ -635,15 +630,11 @@ func (r *QueryRangeRequest) ValidateRequestScope() ([]ValidationOption, error) {
var opts []ValidationOption
switch r.RequestType {
case RequestTypeRaw, RequestTypeRawStream, RequestTypeTrace, RequestTypeTimeSeries, RequestTypeScalar, RequestTypeHeatmap:
case RequestTypeRaw, RequestTypeRawStream, RequestTypeTrace, RequestTypeTimeSeries, RequestTypeScalar:
opts = GetValidationOptions(r.RequestType)
default:
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid request type: %s", r.RequestType).
WithAdditional("Valid request types are: raw, timeseries, scalar, heatmap")
}
if err := r.validateHeatmap(); err != nil {
return nil, err
WithAdditional("Valid request types are: raw, timeseries, scalar")
}
if r.RequestType == RequestTypeRaw || r.RequestType == RequestTypeRawStream || r.RequestType == RequestTypeTrace {
@@ -847,129 +838,9 @@ func validateQueryEnvelope(envelope QueryEnvelope, opts ...ValidationOption) err
}
}
// validateHeatmap refuses request shapes a heatmap cannot render. Metric type is
// deliberately not checked here: MetricAggregation.Type is resolved from metadata
// after validation runs, so gauge/sum/counter and exponential histograms have to
// be refused by the querier once that resolution has happened.
func (r *QueryRangeRequest) validateHeatmap() error {
if r.RequestType != RequestTypeHeatmap {
return nil
}
if r.FormatOptions != nil && r.FormatOptions.FillGaps {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"fillGaps is not supported for heatmap requests: an absent column means collection stopped, which a zero-filled column would hide")
}
if err := r.BucketOptions.validateBucketOptions(); err != nil {
return err
}
enabled := 0
for _, envelope := range r.CompositeQuery.Queries {
switch spec := envelope.Spec.(type) {
case QueryBuilderQuery[MetricAggregation]:
if err := validateHeatmapQuery(spec.Functions, spec.Having); err != nil {
return err
}
if spec.Disabled {
continue
}
enabled++
case QueryBuilderFormula:
if err := validateHeatmapQuery(spec.Functions, spec.Having); err != nil {
return err
}
if spec.Disabled {
continue
}
enabled++
case ClickHouseQuery:
// The rows a ClickHouse query returns are read by request type, the
// same as for any other request, so one shaped as heatmap cells
// renders without the builder having produced it.
if spec.Disabled {
continue
}
enabled++
case PromQuery:
// A PromQL heatmap is a classic histogram read through its `le`
// labels, the same axis the builder's histogram path uses.
if r.BucketOptions != nil {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"bucketOptions are not supported for promql heatmap requests: the bucket axis comes from the `le` labels the query returns, so nothing in the spec would be applied")
}
if spec.Disabled {
continue
}
enabled++
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"heatmap requests support one metrics builder query, one formula over them, one clickhouse query, or one promql query, got %q", envelope.Type.StringValue())
}
}
if enabled != 1 {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"heatmap requests need exactly one enabled query, got %d", enabled)
}
return nil
}
func (b *BucketOptions) validateBucketOptions() error {
if b == nil {
return nil
}
switch spec := b.Spec.(type) {
case LinearBucketsSpec:
// Boundaries are placed at maxValue*i/numBuckets, so a cap at or below
// zero collapses every one of them onto the same point, and a non-finite
// one compares false against every value so nothing reaches the overflow.
if math.IsNaN(spec.MaxValue) || math.IsInf(spec.MaxValue, 0) || spec.MaxValue <= 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"linear buckets need a finite maxValue greater than 0, got %v", spec.MaxValue)
}
if spec.NumBuckets < 0 || spec.NumBuckets > MaxNumBuckets {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"numBuckets must be between 1 and %d, got %d", MaxNumBuckets, spec.NumBuckets)
}
case LogBucketsSpec:
if spec.Scale != nil && (*spec.Scale < MinLogScale || *spec.Scale > MaxLogScale) {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"scale must be between %d and %d, got %d", MinLogScale, MaxLogScale, *spec.Scale)
}
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"invalid bucketOptions kind %q, expected one of linear, log", b.Kind.StringValue())
}
return nil
}
// validateHeatmapQuery refuses the per-query settings that cannot mean anything
// on a heatmap. It runs on disabled queries too: a disabled query is a formula
// input, so whatever it does still reaches the cells.
func validateHeatmapQuery(functions []Function, having *Having) error {
if len(functions) > 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"functions are not supported for heatmap requests: a heatmap point is a count per bucket, not a single value")
}
if having != nil && having.Expression != "" {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"having is not supported for heatmap requests: it filters individual cells, which breaks the cumulative differencing")
}
return nil
}
func GetValidationOptions(requestType RequestType) []ValidationOption {
switch requestType {
case RequestTypeTimeSeries, RequestTypeHeatmap:
case RequestTypeTimeSeries:
return []ValidationOption{WithSkipSelectFieldValidation(), WithTimestampGroupByValidation()}
case RequestTypeScalar:
return []ValidationOption{WithSkipSelectFieldValidation(), WithReduceToValidation()}

View File

@@ -5,58 +5,69 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/aiobservabilitytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
type Signal struct {
type Source struct {
valuer.String
}
func (enum *Signal) UnmarshalJSON(data []byte) error {
func (enum *Source) UnmarshalJSON(data []byte) error {
var str string
if err := json.Unmarshal(data, &str); err != nil {
return err
}
signal, err := NewSignal(str)
source, err := NewSource(str)
if err != nil {
return err
}
*enum = signal
*enum = source
return nil
}
var (
SignalTraces = Signal{valuer.NewString("traces")}
SignalLogs = Signal{valuer.NewString("logs")}
SignalApiMonitoring = Signal{valuer.NewString("api_monitoring")}
SignalExceptions = Signal{valuer.NewString("exceptions")}
SignalMeter = Signal{valuer.NewString("meter")}
SignalAiObservability = Signal{valuer.NewString("ai_observability")}
SourceTraces = Source{valuer.NewString("traces")}
SourceLogs = Source{valuer.NewString("logs")}
SourceApiMonitoring = Source{valuer.NewString("api_monitoring")}
SourceExceptions = Source{valuer.NewString("exceptions")}
SourceMeter = Source{valuer.NewString("meter")}
SourceAiObservability = Source{valuer.NewString("ai_observability")}
)
// NewSignal creates a Signal from a string.
func NewSignal(s string) (Signal, error) {
func (Source) Enum() []any {
return []any{
SourceTraces,
SourceLogs,
SourceApiMonitoring,
SourceExceptions,
SourceMeter,
SourceAiObservability,
}
}
// NewSource creates a Source from a string.
func NewSource(s string) (Source, error) {
switch s {
case "traces":
return SignalTraces, nil
return SourceTraces, nil
case "logs":
return SignalLogs, nil
return SourceLogs, nil
case "api_monitoring":
return SignalApiMonitoring, nil
return SourceApiMonitoring, nil
case "exceptions":
return SignalExceptions, nil
return SourceExceptions, nil
case "meter":
return SignalMeter, nil
return SourceMeter, nil
case "ai_observability":
return SignalAiObservability, nil
return SourceAiObservability, nil
default:
return Signal{}, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid signal: %s", s)
return Source{}, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid source: %s", s)
}
}
@@ -65,33 +76,46 @@ type StorableQuickFilter struct {
types.Identifiable
OrgID valuer.UUID `bun:"org_id,type:text,notnull"`
Filter string `bun:"filter,type:text,notnull"`
Signal Signal `bun:"signal,type:text,notnull"`
Source Source `bun:"source,type:text,notnull"`
types.TimeAuditable
}
type SignalFilters struct {
Signal Signal `json:"signal"`
Filters []v3.AttributeKey `json:"filters"`
type SourceFilters struct {
types.Identifiable
types.TimeAuditable
OrgID valuer.UUID `json:"orgId" required:"true"`
Source Source `json:"source" required:"true"`
Filters []telemetrytypes.TelemetryFieldKey `json:"filters" required:"true" nullable:"false"`
}
type UpdatableQuickFilters struct {
Signal Signal `json:"signal"`
Filters []v3.AttributeKey `json:"filters"`
Filters []telemetrytypes.TelemetryFieldKey `json:"filters" required:"true" nullable:"false"`
}
// NewStorableQuickFilter creates a new StorableQuickFilter after validation.
func NewStorableQuickFilter(orgID valuer.UUID, signal Signal, filterJSON []byte) (*StorableQuickFilter, error) {
if orgID.StringValue() == "" {
func NewStorableQuickFilter(orgID valuer.UUID, source Source, filters []telemetrytypes.TelemetryFieldKey) (*StorableQuickFilter, error) {
if orgID.IsZero() {
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "orgID is required")
}
if _, err := NewSignal(signal.StringValue()); err != nil {
if _, err := NewSource(source.StringValue()); err != nil {
return nil, err
}
var filters []v3.AttributeKey
if err := json.Unmarshal(filterJSON, &filters); err != nil {
return nil, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid filter JSON")
if err := validateFilters(filters); err != nil {
return nil, err
}
// A nil slice marshals to the JSON literal "null"; store an empty array so
// reads never have to render a null filter list.
if filters == nil {
filters = []telemetrytypes.TelemetryFieldKey{}
}
filterJSON, err := json.Marshal(filters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error marshalling filters")
}
now := time.Now()
@@ -100,7 +124,7 @@ func NewStorableQuickFilter(orgID valuer.UUID, signal Signal, filterJSON []byte)
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Signal: signal,
Source: source,
Filter: string(filterJSON),
TimeAuditable: types.TimeAuditable{
CreatedAt: now,
@@ -109,25 +133,21 @@ func NewStorableQuickFilter(orgID valuer.UUID, signal Signal, filterJSON []byte)
}, nil
}
// Update updates an existing StorableQuickFilter with new filter data after validation.
func (quickfilter *StorableQuickFilter) Update(filterJSON []byte) error {
var filters []v3.AttributeKey
if err := json.Unmarshal(filterJSON, &filters); err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid filter JSON")
// NewSourceFiltersFromSource creates a SourceFilters with no filters for a source.
func NewSourceFiltersFromSource(source Source) *SourceFilters {
return &SourceFilters{
Source: source,
Filters: []telemetrytypes.TelemetryFieldKey{},
}
quickfilter.Filter = string(filterJSON)
quickfilter.UpdatedAt = time.Now()
return nil
}
// NewSignalFilterFromStorableQuickFilter converts a StorableQuickFilter to a SignalFilters object.
func NewSignalFilterFromStorableQuickFilter(storableQuickFilter *StorableQuickFilter) (*SignalFilters, error) {
// NewSourceFilterFromStorableQuickFilter converts a StorableQuickFilter to a SourceFilters object.
func NewSourceFilterFromStorableQuickFilter(storableQuickFilter *StorableQuickFilter) (*SourceFilters, error) {
if storableQuickFilter == nil {
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "storableQuickFilter cannot be nil")
}
var filters []v3.AttributeKey
filters := []telemetrytypes.TelemetryFieldKey{}
if storableQuickFilter.Filter != "" {
err := json.Unmarshal([]byte(storableQuickFilter.Filter), &filters)
if err != nil {
@@ -135,178 +155,114 @@ func NewSignalFilterFromStorableQuickFilter(storableQuickFilter *StorableQuickFi
}
}
return &SignalFilters{
Signal: storableQuickFilter.Signal,
Filters: filters,
// Stored filter JSON can be the literal "null" (a nil slice was upserted),
// which unmarshals to nil; the API contract requires a non-null array.
if filters == nil {
filters = []telemetrytypes.TelemetryFieldKey{}
}
return &SourceFilters{
Identifiable: storableQuickFilter.Identifiable,
OrgID: storableQuickFilter.OrgID,
Source: storableQuickFilter.Source,
Filters: filters,
TimeAuditable: storableQuickFilter.TimeAuditable,
}, nil
}
// NewDefaultQuickFilter generates default filters for all supported signals.
// NewDefaultQuickFilter generates default filters for all supported sources.
func NewDefaultQuickFilter(orgID valuer.UUID) ([]*StorableQuickFilter, error) {
tracesFilters := []map[string]interface{}{
{"key": "duration_nano", "dataType": "float64", "type": "tag"},
{"key": "deployment.environment", "dataType": "string", "type": "resource"},
{"key": "hasError", "dataType": "bool", "type": "tag"},
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": "name", "dataType": "string", "type": "tag"},
{"key": "rpc.method", "dataType": "string", "type": "tag"},
{"key": "response_status_code", "dataType": "string", "type": "tag"},
{"key": "http_host", "dataType": "string", "type": "tag"},
{"key": "http.method", "dataType": "string", "type": "tag"},
{"key": "http.route", "dataType": "string", "type": "tag"},
{"key": "http_url", "dataType": "string", "type": "tag"},
{"key": "trace_id", "dataType": "string", "type": "tag"},
tracesFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "duration_nano", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeNumber},
{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "hasError", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeBool},
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "name", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "rpc.method", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "response_status_code", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http_host", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http.method", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http.route", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "http_url", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "trace_id", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
}
logsFilters := []map[string]interface{}{
{"key": "severity_text", "dataType": "string", "type": "resource"},
{"key": "deployment.environment", "dataType": "string", "type": "resource"},
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": "host.name", "dataType": "string", "type": "resource"},
{"key": "k8s.cluster.name", "dataType": "string", "type": "resource"},
{"key": "k8s.deployment.name", "dataType": "string", "type": "resource"},
{"key": "k8s.namespace.name", "dataType": "string", "type": "resource"},
{"key": "k8s.pod.name", "dataType": "string", "type": "resource"},
logsFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "severity_text", FieldContext: telemetrytypes.FieldContextLog, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "host.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.cluster.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.deployment.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.namespace.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.pod.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
}
apiMonitoringFilters := []map[string]interface{}{
{"key": "deployment.environment", "dataType": "string", "type": "resource"},
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": "rpc.method", "dataType": "string", "type": "tag"},
apiMonitoringFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "rpc.method", FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
}
exceptionsFilters := []map[string]interface{}{
{"key": "deployment.environment", "dataType": "string", "type": "resource"},
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": "host.name", "dataType": "string", "type": "resource"},
{"key": "k8s.cluster.name", "dataType": "string", "type": "resource"},
{"key": "k8s.deployment.name", "dataType": "string", "type": "resource"},
{"key": "k8s.namespace.name", "dataType": "string", "type": "resource"},
{"key": "k8s.pod.name", "dataType": "string", "type": "resource"},
exceptionsFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "host.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.cluster.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.deployment.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.namespace.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "k8s.pod.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
}
meterFilters := []map[string]interface{}{
{"key": "deployment.environment", "dataType": "float64", "type": "Sum"},
{"key": "service.name", "dataType": "float64", "type": "Sum"},
{"key": "host.name", "dataType": "float64", "type": "Sum"},
// Meter keys are label names with no context or datatype: the meter fields
// API returns them as name+signal only, so the defaults mirror that shape.
meterFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "deployment.environment", Signal: telemetrytypes.SignalMetrics},
{Name: "service.name", Signal: telemetrytypes.SignalMetrics},
{Name: "host.name", Signal: telemetrytypes.SignalMetrics},
}
// AI observability (builder_ai_query trace explorer), ordered by expected
// usage: env scoping, the LLM identity keys, then service and the rest.
aiObservabilityFilters := []map[string]interface{}{
{"key": "deployment.environment", "dataType": "string", "type": "resource"},
{"key": aiobservabilitytypes.GenAIOperationName, "dataType": "string", "type": "tag"},
{"key": aiobservabilitytypes.GenAIProviderName, "dataType": "string", "type": "tag"},
{"key": aiobservabilitytypes.GenAIRequestModel, "dataType": "string", "type": "tag"},
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": aiobservabilitytypes.GenAIToolName, "dataType": "string", "type": "tag"},
{"key": aiobservabilitytypes.GenAIAgentName, "dataType": "string", "type": "tag"},
aiObservabilityFilters := []telemetrytypes.TelemetryFieldKey{
{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: aiobservabilitytypes.GenAIOperationName, FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: aiobservabilitytypes.GenAIProviderName, FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: aiobservabilitytypes.GenAIRequestModel, FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: aiobservabilitytypes.GenAIToolName, FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
{Name: aiobservabilitytypes.GenAIAgentName, FieldContext: telemetrytypes.FieldContextAttribute, FieldDataType: telemetrytypes.FieldDataTypeString},
}
tracesJSON, err := json.Marshal(tracesFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal traces filters")
defaults := []struct {
source Source
filters []telemetrytypes.TelemetryFieldKey
}{
{SourceTraces, tracesFilters},
{SourceLogs, logsFilters},
{SourceApiMonitoring, apiMonitoringFilters},
{SourceExceptions, exceptionsFilters},
{SourceMeter, meterFilters},
{SourceAiObservability, aiObservabilityFilters},
}
logsJSON, err := json.Marshal(logsFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal logs filters")
storableQuickFilters := make([]*StorableQuickFilter, 0, len(defaults))
for _, def := range defaults {
storableQuickFilter, err := NewStorableQuickFilter(orgID, def.source, def.filters)
if err != nil {
return nil, err
}
storableQuickFilters = append(storableQuickFilters, storableQuickFilter)
}
apiMonitoringJSON, err := json.Marshal(apiMonitoringFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal api monitoring filters")
}
exceptionsJSON, err := json.Marshal(exceptionsFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal exceptions filters")
}
meterJSON, err := json.Marshal(meterFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal meter filters")
}
aiObservabilityJSON, err := json.Marshal(aiObservabilityFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal ai observability filters")
}
timeRightNow := time.Now()
return []*StorableQuickFilter{
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(tracesJSON),
Signal: SignalTraces,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(logsJSON),
Signal: SignalLogs,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(apiMonitoringJSON),
Signal: SignalApiMonitoring,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(exceptionsJSON),
Signal: SignalExceptions,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(meterJSON),
Signal: SignalMeter,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(aiObservabilityJSON),
Signal: SignalAiObservability,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
}, nil
return storableQuickFilters, nil
}
func validateFilters(filters []telemetrytypes.TelemetryFieldKey) error {
for _, filter := range filters {
if filter.Name == "" {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "filter name is required")
}
}
return nil
}

View File

@@ -10,10 +10,10 @@ type QuickFilterStore interface {
// Get retrieves all filters for an organization
Get(ctx context.Context, orgID valuer.UUID) ([]*StorableQuickFilter, error)
// GetBySignal retrieves filters for a specific signal in an organization
GetBySignal(ctx context.Context, orgID valuer.UUID, signal string) (*StorableQuickFilter, error)
// GetBySource retrieves filters for a specific source in an organization
GetBySource(ctx context.Context, orgID valuer.UUID, source string) (*StorableQuickFilter, error)
// Upsert inserts or updates filters for an organization and signal
// Upsert inserts or updates filters for an organization and source
Upsert(ctx context.Context, filter *StorableQuickFilter) error
Create(ctx context.Context, filter []*StorableQuickFilter) error
}

View File

@@ -228,7 +228,7 @@ def test_email_channel_never_stores_or_serves_smtp_settings(
with signoz.sqlstore.conn.connect() as conn:
stored = conn.execute(
text("SELECT data FROM notification_channel WHERE name = :name"),
text("SELECT data FROM notification_channel WHERE display_name = :name"),
{"name": hostile_name},
).fetchone()
assert stored is not None

View File

@@ -177,6 +177,61 @@ def test_get_ingestion_keys(
assert data["_pagination"]["total"] == 1
def test_get_ingestion_key_by_id(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list], None],
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
gateway_url = f"/v1/workspaces/me/keys/{TEST_KEY_ID}"
make_http_mocks(
signoz.gateway,
[
Mapping(
request=MappingRequest(
method=HttpMethods.GET,
url=gateway_url,
headers=common_gateway_headers(),
),
response=MappingResponse(
status=200,
json_body={
"status": "success",
"data": {
"id": TEST_KEY_ID,
"name": "my-test-key",
"value": "secret",
"expires_at": "2030-01-01T00:00:00Z",
"tags": ["env:test"],
"created_at": "2024-01-01T00:00:00Z",
"updated_at": "2024-01-01T00:00:00Z",
"workspace_id": "ws-1",
},
},
),
persistent=False,
),
],
)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v2/gateway/ingestion_keys/{TEST_KEY_ID}"),
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.OK, f"Expected 200, got {response.status_code}: {response.text}"
data = response.json()["data"]
assert data["id"] == TEST_KEY_ID
assert data["name"] == "my-test-key"
assert data["workspace_id"] == "ws-1"
assert data["tags"] == ["env:test"]
def test_get_ingestion_keys_custom_pagination(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument

View File

@@ -65,7 +65,7 @@ def test_create_ingestion_key_limit_only_size(
status=201,
json_body={
"status": "success",
"data": {"id": "limit-created-1"},
"data": {"id": "2c5b7d3e-4f6a-4b8c-a09d-3f4a5b6c7d8e"},
},
),
persistent=False,
@@ -86,7 +86,7 @@ def test_create_ingestion_key_limit_only_size(
assert response.status_code == HTTPStatus.CREATED, f"Expected 201, got {response.status_code}: {response.text}"
assert response.json()["data"]["id"] == "limit-created-1"
assert response.json()["data"]["id"] == "2c5b7d3e-4f6a-4b8c-a09d-3f4a5b6c7d8e"
body = get_latest_gateway_request_body(signoz, "POST", gateway_url)
assert body is not None, "Expected a POST request to reach the gateway"
@@ -121,7 +121,7 @@ def test_create_ingestion_key_limit_only_count(
status=201,
json_body={
"status": "success",
"data": {"id": "limit-created-2"},
"data": {"id": "3d6c8e4f-5a7b-4c9d-b1ae-4a5b6c7d8e9f"},
},
),
persistent=False,
@@ -174,7 +174,7 @@ def test_create_ingestion_key_limit_both_size_and_count(
status=201,
json_body={
"status": "success",
"data": {"id": "limit-created-3"},
"data": {"id": "4e7d9f5a-6b8c-4dae-c2bf-5b6c7d8e9fa0"},
},
),
persistent=False,
@@ -391,3 +391,265 @@ def test_delete_ingestion_key_limit(
# Verify at least one DELETE reached the gateway
matched = get_gateway_requests(signoz, "DELETE", gateway_url)
assert len(matched) >= 1, "Expected a DELETE request to reach the gateway"
def test_create_ingestion_limit(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list], None],
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
gateway_url = f"/v1/workspaces/me/keys/{TEST_KEY_ID}/limits"
make_http_mocks(
signoz.gateway,
[
Mapping(
request=MappingRequest(
method=HttpMethods.POST,
url=gateway_url,
headers=common_gateway_headers(),
),
response=MappingResponse(
status=201,
json_body={
"status": "success",
"data": {"id": "5f8ea06b-7c9d-4ebf-a3c0-6c7d8e9fa0b1"},
},
),
persistent=False,
),
],
)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/gateway/ingestion_limits"),
json={
"keyId": TEST_KEY_ID,
"signal": "logs",
"config": {"day": {"size": 3000}},
"tags": ["test"],
},
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.CREATED, f"Expected 201, got {response.status_code}: {response.text}"
assert response.json()["data"]["id"] == "5f8ea06b-7c9d-4ebf-a3c0-6c7d8e9fa0b1"
body = get_latest_gateway_request_body(signoz, "POST", gateway_url)
assert body is not None, "Expected a POST request to reach the gateway"
assert body["signal"] == "logs"
assert body["config"]["day"]["size"] == 3000
assert body["tags"] == ["test"]
def test_create_ingestion_limit_without_key_id(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/gateway/ingestion_limits"),
json={
"signal": "logs",
"config": {"day": {"size": 3000}},
},
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, f"Expected 400, got {response.status_code}: {response.text}"
def test_get_ingestion_limit(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list], None],
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
gateway_url = f"/v1/workspaces/me/limits/{TEST_LIMIT_ID}"
make_http_mocks(
signoz.gateway,
[
Mapping(
request=MappingRequest(
method=HttpMethods.GET,
url=gateway_url,
headers=common_gateway_headers(),
),
response=MappingResponse(
status=200,
json_body={
"status": "success",
"data": {
"id": TEST_LIMIT_ID,
"key_id": TEST_KEY_ID,
"signal": "logs",
"config": {"day": {"size": 1000}},
"tags": ["test"],
},
},
),
persistent=False,
),
],
)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v2/gateway/ingestion_limits/{TEST_LIMIT_ID}"),
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.OK, f"Expected 200, got {response.status_code}: {response.text}"
data = response.json()["data"]
assert data["id"] == TEST_LIMIT_ID
assert data["key_id"] == TEST_KEY_ID
assert data["signal"] == "logs"
assert data["config"]["day"]["size"] == 1000
def test_get_ingestion_key_limits(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list], None],
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
gateway_url = f"/v1/workspaces/me/keys/{TEST_KEY_ID}/limits"
make_http_mocks(
signoz.gateway,
[
Mapping(
request=MappingRequest(
method=HttpMethods.GET,
url=gateway_url,
headers=common_gateway_headers(),
),
response=MappingResponse(
status=200,
json_body={
"status": "success",
"data": [
{
"id": TEST_LIMIT_ID,
"key_id": TEST_KEY_ID,
"signal": "logs",
"config": {"day": {"size": 1000}},
"tags": ["test"],
}
],
},
),
persistent=False,
),
],
)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v2/gateway/ingestion_keys/{TEST_KEY_ID}/limits"),
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.OK, f"Expected 200, got {response.status_code}: {response.text}"
data = response.json()["data"]
assert len(data) == 1
assert data[0]["id"] == TEST_LIMIT_ID
assert data[0]["key_id"] == TEST_KEY_ID
assert data[0]["signal"] == "logs"
assert data[0]["config"]["day"]["size"] == 1000
def test_update_ingestion_limit(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list], None],
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
gateway_url = f"/v1/workspaces/me/limits/{TEST_LIMIT_ID}"
make_http_mocks(
signoz.gateway,
[
Mapping(
request=MappingRequest(
method=HttpMethods.PATCH,
url=gateway_url,
headers=common_gateway_headers(),
),
response=MappingResponse(status=204),
persistent=False,
),
],
)
response = requests.patch(
signoz.self.host_configs["8080"].get(f"/api/v2/gateway/ingestion_limits/{TEST_LIMIT_ID}"),
json={
"config": {"day": {"size": 4000, "count": 250}},
"tags": ["test"],
},
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204, got {response.status_code}: {response.text}"
body = get_latest_gateway_request_body(signoz, "PATCH", gateway_url)
assert body is not None, "Expected a PATCH request to reach the gateway"
assert body["config"]["day"]["size"] == 4000
assert body["config"]["day"]["count"] == 250
assert body["tags"] == ["test"]
def test_delete_ingestion_limit(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list], None],
get_token: Callable[[str, str], str],
) -> None:
editor_token = get_token(GATEWAY_APIS_EDITOR_EMAIL, GATEWAY_APIS_EDITOR_PASSWORD)
gateway_url = f"/v1/workspaces/me/limits/{TEST_LIMIT_ID}"
make_http_mocks(
signoz.gateway,
[
Mapping(
request=MappingRequest(
method=HttpMethods.DELETE,
url=gateway_url,
headers=common_gateway_headers(),
),
response=MappingResponse(status=204),
persistent=False,
),
],
)
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/gateway/ingestion_limits/{TEST_LIMIT_ID}"),
headers={"Authorization": f"Bearer {editor_token}"},
timeout=10,
)
assert response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204, got {response.status_code}: {response.text}"
matched = get_gateway_requests(signoz, "DELETE", gateway_url)
assert len(matched) >= 1, "Expected a DELETE request to reach the gateway"

View File

@@ -209,6 +209,52 @@ def test_ai_span_list_excludes_non_gen_ai_spans(
assert "POST /api/chat" not in names # root span excluded
def test_ai_span_list_trace_level_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Span list (raw) with a trace-level condition returns only the gen_ai spans
of traces whose window-clipped aggregates qualify."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-spanlist-tracefilter"
small = ai_trace(now=now, service=service, user="a", in_tokens=10, out_tokens=100)
large = ai_trace(now=now, service=service, user="b", in_tokens=30, out_tokens=300)
insert_traces(small + large)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}' AND trace.output_tokens > 100",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.RAW)
assert response.status_code == HTTPStatus.OK, response.text
rows = response.json()["data"]["data"]["results"][0]["rows"]
assert len(rows) == 1, f"expected only the large trace's LLM span, got {len(rows)} rows"
body = json.dumps(rows)
assert large[0].trace_id in body
assert small[0].trace_id not in body
# a threshold no trace meets: the empty qualification yields no spans, not an error
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}' AND trace.output_tokens > 1000",
limit=10,
)
response = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.RAW)
assert response.status_code == HTTPStatus.OK, response.text
assert not (response.json()["data"]["data"]["results"][0].get("rows") or [])
def test_ai_list_having_or_aggregates(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument

View File

@@ -2,10 +2,12 @@ from collections.abc import Callable
from datetime import UTC, datetime
from http import HTTPStatus
import pytest
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.metadata import get_field_keys, get_field_values
from fixtures.querierai import ai_trace
from fixtures.metadata import AttributesMetadata, get_field_keys, get_field_values
from fixtures.querierai import ai_trace, ai_trace_mixed_spans
from fixtures.traces import Traces
AI_KEYS_PATH = "/api/v1/ai_observability/fields/keys"
@@ -106,20 +108,94 @@ def test_ai_field_values_suggests_ingested_attribute_values(
assert values["stringValues"] == ["gpt-it-values"], values
def test_ai_field_values_reject_existing_query(
@pytest.mark.parametrize(
"existing_query,search_text,expected",
[
pytest.param(None, "", {"ai-rel-a", "ai-rel-b", "ai-rel-c"}, id="no_query_scopes_to_gen_ai_spans"),
pytest.param("gen_ai.user.id = 'alice'", "", {"ai-rel-a"}, id="span_filter_narrows_under_the_gate"),
pytest.param("llm_call_count > 0", "", {"ai-rel-a", "ai-rel-b", "ai-rel-c"}, id="pure_trace_aggregate_filter_is_stripped"),
pytest.param("llm_call_count > 0 AND gen_ai.user.id = 'alice'", "", {"ai-rel-a"}, id="mixed_filter_keeps_only_the_span_part"),
pytest.param(
"llm_call_count > 0 OR gen_ai.user.id = 'alice'",
"",
{"ai-rel-a", "ai-rel-b", "ai-rel-c"},
id="class_mixing_or_drops_the_filter_not_the_request",
),
pytest.param(
"gen_ai.user.id = ",
"",
{"ai-rel-a", "ai-rel-b", "ai-rel-c"},
id="unparseable_filter_falls_back_to_the_gate",
),
pytest.param(None, "ai-rel-a", {"ai-rel-a"}, id="search_text_narrows_related_values"),
# http.request.method lives on the root span's metadata row, gen_ai.* on
# the LLM/tool/agent rows; rows are per span-shape, so the gate AND a
# cross-span attribute filter can match no single row
pytest.param("http.request.method = 'POST'", "", set(), id="cross_span_attribute_filter_matches_no_row"),
],
)
def test_ai_field_values_related_values(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
insert_attributes_metadata: Callable[[list[AttributesMetadata]], None],
existing_query: str | None,
search_text: str,
expected: set[str],
) -> None:
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
# existingQuery key resolution reads the trace keys tables, not
# attributes_metadata; a mixed trace registers the gate keys (model/tool/
# agent) plus gen_ai.user.id and http.request.method
insert_traces(ai_trace_mixed_spans(now=now, service="ai-rel-a", user="alice"))
# related values are served from attributes_metadata; one row per gate key,
# the traces row without any gate attribute and the logs row (wrong
# data_source, gate attribute present) must never surface
insert_attributes_metadata(
[
AttributesMetadata(
data_source="traces",
resource_attributes={"service.name": "ai-rel-a"},
attributes={"gen_ai.request.model": "gpt-rel", "gen_ai.user.id": "alice"},
),
AttributesMetadata(
data_source="traces",
resource_attributes={"service.name": "ai-rel-b"},
attributes={"gen_ai.tool.name": "get_weather", "gen_ai.user.id": "bob"},
),
AttributesMetadata(
data_source="traces",
resource_attributes={"service.name": "ai-rel-c"},
attributes={"gen_ai.agent.name": "chat-agent"},
),
AttributesMetadata(
data_source="traces",
resource_attributes={"service.name": "plain-rel"},
attributes={"http.request.method": "POST"},
),
AttributesMetadata(
data_source="logs",
resource_attributes={"service.name": "ai-rel-logs"},
attributes={"gen_ai.request.model": "gpt-rel"},
),
]
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = get_field_values(
signoz,
token,
{"name": "gen_ai.request.model", "existingQuery": "service.name = 'ai-it-values'"},
AI_VALUES_PATH,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text
params = {"name": "service.name", "searchText": search_text}
if existing_query is not None:
params["existingQuery"] = existing_query
response = get_field_values(signoz, token, params, AI_VALUES_PATH)
assert response.status_code == HTTPStatus.OK, response.text
assert response.json()["status"] == "success"
related = response.json()["data"]["values"].get("relatedValues") or []
assert set(related) == expected, related
def test_ai_field_values_of_computed_aggregate_are_empty(

View File

@@ -0,0 +1,276 @@
from collections.abc import Callable
from http import HTTPStatus
import requests
from sqlalchemy import sql
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
ALL_SOURCES = {
"traces",
"logs",
"api_monitoring",
"exceptions",
"meter",
"ai_observability",
}
def test_get_quick_filters_returns_defaults(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
data = response.json()["data"]
assert {source_filters["source"] for source_filters in data} == ALL_SOURCES
for source_filters in data:
assert source_filters["id"] != "00000000-0000-0000-0000-000000000000"
assert source_filters["orgId"] != "00000000-0000-0000-0000-000000000000"
assert source_filters["createdAt"] != ""
assert source_filters["updatedAt"] != ""
assert len(source_filters["filters"]) > 0
for field_key in source_filters["filters"]:
assert field_key["name"] != ""
assert "fieldContext" in field_key
assert "fieldDataType" in field_key
assert "key" not in field_key
def test_v1_get_serves_legacy_shape(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/orgs/me/filters"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
data = response.json()["data"]
assert {source_filters["signal"] for source_filters in data} == ALL_SOURCES
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/orgs/me/filters/traces"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
filters = response.json()["data"]["filters"]
assert filters[0]["key"] == "duration_nano"
assert filters[0]["type"] == "tag"
assert filters[0]["dataType"] == "float64"
assert all("name" not in legacy_filter for legacy_filter in filters)
def test_v1_update_round_trips_to_v2(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.put(
signoz.self.host_configs["8080"].get("/api/v1/orgs/me/filters"),
json={
"signal": "exceptions",
"filters": [
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": "http.method", "dataType": "string", "type": "tag"},
{"key": "code_line", "dataType": "int64", "type": "tag"},
],
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT, response.text
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/exceptions"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
filters = response.json()["data"]["filters"]
assert [(field_key["name"], field_key["fieldContext"]) for field_key in filters] == [
("service.name", "resource"),
("http.method", "attribute"),
("code_line", "attribute"),
]
assert filters[2]["fieldDataType"] == "number"
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/orgs/me/filters/exceptions"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
filters = response.json()["data"]["filters"]
assert [(legacy_filter["key"], legacy_filter["type"]) for legacy_filter in filters] == [
("service.name", "resource"),
("http.method", "tag"),
("code_line", "tag"),
]
response = requests.put(
signoz.self.host_configs["8080"].get("/api/v1/orgs/me/filters"),
json={
"signal": "meter",
"filters": [{"key": "host.name", "dataType": "string", "type": ""}],
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT, response.text
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/meter"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
assert [(field_key["name"], field_key["signal"]) for field_key in response.json()["data"]["filters"]] == [("host.name", "metrics")]
def test_update_quick_filters_round_trip(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.put(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/logs"),
json={
"filters": [
{
"name": "k8s.pod.name",
"fieldContext": "resource",
"fieldDataType": "string",
},
{
"name": "body.status",
"fieldContext": "body",
"fieldDataType": "string",
},
],
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT, response.text
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/logs"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
filters = response.json()["data"]["filters"]
assert [field_key["name"] for field_key in filters] == [
"k8s.pod.name",
"body.status",
]
assert filters[0]["fieldContext"] == "resource"
assert filters[1]["fieldContext"] == "body"
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/orgs/me/filters/logs"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
assert [(legacy_filter["key"], legacy_filter["type"]) for legacy_filter in response.json()["data"]["filters"]] == [
("k8s.pod.name", "resource"),
("body.status", ""),
]
def test_update_quick_filters_creates_row_for_source_without_one(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
with signoz.sqlstore.conn.connect() as conn:
conn.execute(
sql.text("DELETE FROM quick_filter WHERE source = :source"),
{"source": "api_monitoring"},
)
conn.commit()
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/api_monitoring"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
data = response.json()["data"]
assert data["source"] == "api_monitoring"
assert data["filters"] == []
assert data["id"] == "00000000-0000-0000-0000-000000000000"
response = requests.put(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/api_monitoring"),
json={
"filters": [
{
"name": "service.name",
"fieldContext": "resource",
"fieldDataType": "string",
},
],
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT, response.text
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/quick_filters/api_monitoring"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK, response.text
data = response.json()["data"]
assert [field_key["name"] for field_key in data["filters"]] == ["service.name"]
assert data["id"] != "00000000-0000-0000-0000-000000000000"
def test_update_quick_filters_rejects_invalid_input(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
for source, invalid_body in [
(
"traces",
{"filters": [{"key": "service.name", "dataType": "string", "type": "resource"}]},
),
("invalid", {"filters": []}),
]:
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v2/quick_filters/{source}"),
json=invalid_body,
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, response.text