Compare commits

..

11 Commits

Author SHA1 Message Date
Nikhil Mantri
a2673da1d1 Merge branch 'main' into feat/improve_alert_integration_tests 2026-08-12 16:10:10 +05:30
Swapnil Nakade
2616885d22 feat: adding mysql GCP service (#12514)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- Adding GCP integration MySQL service
- Related fix: adding formula to convert CPU utilization fraction into
percentage for Postgres dashboard

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
https://github.com/SigNoz/platform-pod/issues/2942
2026-08-12 09:53:16 +00:00
nikhilmantri0902
d5560276de chore: improved tests 2026-08-12 13:27:57 +05:30
Aditya Singh
52dd57074e feat: filter fields with no name in field selector (#12512)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Filter field selector options with name field empty

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
closes https://github.com/SigNoz/engineering-pod/issues/5890

<!--If applicable, include screenshots or screen recordings that clearly
show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings
No screen recording as this is hard to reproduce.
2026-08-12 07:54:51 +00:00
Naman Verma
fe94b817db fix(promql): remove NaN and Inf values from PromQL query range response (#12388)
## Pull Request

---

### 📄 Summary

Currently, builder and clickhouse queries remove NaN and Inf values, but
PromQL does not. This way, it ends up in the final response. While the
UI handles these values, a lot of other places in the flow do not, such
as our query response caching. This can lead to unexpected issues.

The current issue at hand is that while the first query range call shows
the correct data, the second call (that fetches from cache) does not.

Instead of fixing the caching, better to solve the problem at root level
and not return non-finite values for PromQL altogether.

#### Recordings

On local data before the change:


https://github.com/user-attachments/assets/c08ec796-a7e5-47d8-8cc5-3dfd302dba49

After the change:


https://github.com/user-attachments/assets/9162c963-1a98-4ebc-83ce-759a9127b772


#### Issues closed by this PR

Closes https://github.com/SigNoz/pulse-pod/issues/185

---

### 🧪 Testing Strategy

- Tests added/updated: Yes, integration and unit tests
- Manual verification: Added data locally to reproduce the exact
scenario

---

### ⚠️ Risk & Impact Assessment

- Blast radius: PromQL queries
- Rollback plan: Revert PR or just add a fix

---
2026-08-12 07:35:38 +00:00
Aditya Singh
ea36032d96 fix(sentry): drop benign cancellation errors from reporting on sentry (#12524)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
This PR drops Cancelation error from monaco on sentry to reduce noise



<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
Pager: https://signoz-1.pagerduty.com/incidents/Q1JG9MJ5DRA4LW
Sentry: https://signoz-io.sentry.io/issues/7491905006
2026-08-12 07:22:33 +00:00
Abhi kumar
6ecfa839f3 fix(service-map): stop the resource attribute filter bar from clearing (#12521)
#### Description

Selecting a filter on the Service Map cleared the filter bar instead of
applying it, and the same filter then turned up applied on the Services
tab. Three separate causes:

- The resource attribute context filtered its queries by the current
route, so a filter the map cannot apply vanished from the bar while
staying in state and in the `resourceAttribute` URL param — which the
sidebar carries across routes, hence it reappearing on Services. The
context now exposes whatever is in the URL, and the Service Map narrows
the queries for its own `/dependency_graph` request, so the request
payload is unchanged.
- `ServiceMap` returned early with the filter bar under a different
parent element in each branch, so React tore the bar down and rebuilt it
whenever the map flipped between having services and being empty (and it
wasn't rendered at all while loading). It now renders once, above the
loading / empty / map states. As a side effect the graph tooltip styles
in `Container` finally wrap the graph rather than only the empty state.
- The environment `Select` was keyed on its own value, remounting an
already-controlled select on every pick and closing the dropdown before
a second environment could be chosen.

#### Issues closed by this PR

Closes SigNoz/pulse-pod#199

#### Additional Information

- Related but deliberately left out of scope: `whilelistedKeys` lists
`resource_k8s_cluster_namespace`, while the backend column is
`k8s_namespace_name` (`pkg/query-service/app/services/map.go`), so that
filter is accepted by the UI and silently dropped server side.
2026-08-12 06:53:55 +00:00
Ashwin Bhatkal
62d382b3cc fix(alerts): tolerate a null channels field when editing an existing alert rule (#12510)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
## Summary

Opening an existing alert rule for editing crashes the whole page with
`Cannot read properties of null (reading 'length')`. It happens when a
rule's threshold has `channels: null`, which is the case for rules not
created through the UI.

The generated type is right — `RuletypesBasicRuleThresholdDTO.channels`
is `string[] | null`. The problem is our own `BasicThreshold` wrapper
type, which we keep because v1 and v2 alert shapes both exist. It says
`channels: string[]`, and `fromRuleDTOToPostableRuleV2` casts the DTO
straight into it with `as unknown as`. So the null reaches our code
while the compiler thinks it can't.

`getThresholdStateFromAlertDef` copied that null into state, and the
footer validator then read `.length` on it during render, which takes
down the page instead of failing one field.

This PR defaults `channels` to `[]` where the API data becomes local
state, so the validator, the payload builder and both channel dropdowns
are all safe. The validator also gets an optional chain, since a throw
there can't be recovered.

This is a guard, not the real fix. The cast in
`fromRuleDTOToPostableRuleV2` is the actual gap, and the same wrapper
also claims `spec` is non-nullable when the generated type allows null —
so `spec.map` and `spec[0].op` in the same function can still crash.
Worth fixing at the converter.

## Test plan

One test per guard. Both fail with the original error when the fix is
reverted.

- `pnpm jest src/container/CreateAlertV2/` — 420 pass, 28 suites
- `oxfmt`, `oxlint`, `tsgo --noEmit` clean

Closes https://github.com/SigNoz/pulse-pod/issues/261
2026-08-11 20:40:30 +00:00
Ashwin Bhatkal
cc07e2fa24 fix(alert-channel-integrations): de-flake the Google Chat alert channel save tests (#12509)
## Summary

The Google Chat save test fails on CI now and then with `Exceeded
timeout of 5000 ms for a test`.

The two Google Chat tests fill the form with `userEvent.type()`, which
sends one keystroke at a time. Each keystroke re-renders the whole form.
The payload test types 91 characters, so it takes ~850ms locally. CI is
about 5x slower, which puts it near the 5s limit. A busy runner then
pushes it over.

This PR pastes the values instead of typing them. One event per field,
same assertions.

| Test | Before | After |
| --- | --- | --- |
| `saving sends a googlechat_configs payload` | 847 ms | 283 ms |
| `saving with a webhook url outside chat.googleapis.com` | 590 ms | 326
ms |

Nothing regressed. The new test was added recently to an already
existing suite. It was always close to the limit.

## Test plan

- `pnpm jest
src/container/AllAlertChannels/__tests__/CreateAlertChannel.test.tsx` —
57/57 pass, 3 runs
- `oxfmt`, `oxlint`, `tsgo --noEmit` clean

Closes https://github.com/SigNoz/pulse-pod/issues/259
2026-08-11 20:17:56 +00:00
Vinicius Lourenço
d7aa63f1bc fix(infrastructure-monitoring): migrate having clause to new format (#12467)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

This fixes the bad migration I did at
https://github.com/SigNoz/signoz/pull/11060, and correctly fixes the
expressions for `having` clause inside the charts.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/platform-pod/issues/2905

Closes https://github.com/SigNoz/pulse-pod/issues/212

<!--If applicable, include screenshots or screen recordings that clearly
show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings

| Before | After |
|--------|--------|
| <img width="2156" height="1081" alt="screenshot-2026-08-07_17-15-41"
src="https://github.com/user-attachments/assets/b5617e88-2d63-4a21-915f-d21ed78f9f8b"
/> | <img width="2148" height="1075"
alt="screenshot-2026-08-07_17-12-06"
src="https://github.com/user-attachments/assets/4de3ffd0-533f-4b3b-81ac-df1515b4a076"
/> |
| <img width="2145" height="357" alt="screenshot-2026-08-07_17-16-08"
src="https://github.com/user-attachments/assets/0bfb92f3-e0c4-4cf6-9e4e-62068501da96"
/> | <img width="2146" height="360" alt="screenshot-2026-08-07_17-11-54"
src="https://github.com/user-attachments/assets/bba8ee82-68cd-4205-951f-711adaad07e0"
/> |
| <img width="1074" height="356" alt="screenshot-2026-08-07_17-15-55"
src="https://github.com/user-attachments/assets/574430b4-8547-4a1e-b53a-982ef33344ae"
/> | <img width="1074" height="363" alt="screenshot-2026-08-07_17-11-44"
src="https://github.com/user-attachments/assets/bf5698af-f7fb-45b6-886a-bc8ed4aba808"
/> |
2026-08-11 19:59:53 +00:00
Nityananda Gohain
c36b748370 feat: ai-011y quickfilters support (#12406)
## Pull Request

---

### 📄 Summary
AI 011y quickfilter

Will add the migration later.



#### Issues closed by this PR
Part of https://github.com/SigNoz/engineering-pod/issues/5714

---

###  Change Type
_Select all that apply_

- [x]  Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🧪 Testing Strategy
> How was this change validated?

- Tests added/updated:
- Manual verification:
- Edge cases covered:

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius: None
- Potential regressions:
- Rollback plan:

---
2026-08-11 19:19:35 +00:00
51 changed files with 3192 additions and 2982 deletions

View File

@@ -1499,6 +1499,7 @@ components:
- computeengine
- gke
- cloudstorage
- cloudsql_mysql
type: string
CloudintegrationtypesServiceMetadata:
properties:

View File

@@ -388,6 +388,10 @@ function App(): JSX.Element {
if (error?.name === 'AbortError') {
return null;
}
// Ignore benign Monaco cancellation errors (name 'Canceled').
if (error?.name === 'Canceled') {
return null;
}
// Drop the event if its level is 'warning' or 'info'
if (event.level === 'warning' || event.level === 'info') {

View File

@@ -2818,6 +2818,7 @@ export enum CloudintegrationtypesServiceIDDTO {
computeengine = 'computeengine',
gke = 'gke',
cloudstorage = 'cloudstorage',
cloudsql_mysql = 'cloudsql_mysql',
}
export type CloudintegrationtypesCloudIntegrationServiceDTOAnyOf = {
/**

View File

@@ -437,6 +437,17 @@ describe('Create Alert Channel', () => {
render(<CreateAlertChannels preType={ChannelType.GoogleChat} />);
});
// paste instead of type: a per-keystroke re-render of the whole form
// pushes these tests past the 5s jest timeout on slower CI runners
async function fillField(
user: ReturnType<typeof userEvent.setup>,
testId: string,
value: string,
): Promise<void> {
await user.click(screen.getByTestId(testId));
await user.paste(value);
}
it('Should check if the selected item in the type dropdown has text "Google Chat"', () => {
expect(screen.getByText('Google Chat')).toBeInTheDocument();
});
@@ -463,14 +474,8 @@ describe('Create Alert Channel', () => {
it('Should check if saving with a webhook url outside chat.googleapis.com displays error notification', async () => {
const user = userEvent.setup();
await user.type(
screen.getByTestId('channel-name-textbox'),
'gchat-channel',
);
await user.type(
screen.getByTestId('webhook-url-textbox'),
'https://example.com/webhook',
);
await fillField(user, 'channel-name-textbox', 'gchat-channel');
await fillField(user, 'webhook-url-textbox', 'https://example.com/webhook');
await user.click(screen.getByTestId('save-channel-button'));
@@ -496,11 +501,8 @@ describe('Create Alert Channel', () => {
const user = userEvent.setup();
await user.type(
screen.getByTestId('channel-name-textbox'),
'gchat-channel',
);
await user.type(screen.getByTestId('webhook-url-textbox'), validWebhookUrl);
await fillField(user, 'channel-name-textbox', 'gchat-channel');
await fillField(user, 'webhook-url-textbox', validWebhookUrl);
await user.click(screen.getByTestId('save-channel-button'));

View File

@@ -130,6 +130,28 @@ describe('Footer utils', () => {
};
expect(validateCreateAlertState(currentArgs)).toBeNull();
});
it('when threshold channels are null', () => {
const currentArgs: BuildCreateAlertRulePayloadArgs = {
...args,
basicAlertState: {
...args.basicAlertState,
name: 'test name',
},
thresholdState: {
...args.thresholdState,
thresholds: [
{
...args.thresholdState.thresholds[0],
channels: null as unknown as string[],
},
],
},
};
expect(validateCreateAlertState(currentArgs)).toBe(
'Please select at least one channel for each threshold or enable routing policies',
);
});
});
describe('getNotificationSettingsProps', () => {

View File

@@ -44,7 +44,8 @@ export function validateCreateAlertState(
if (!threshold.label) {
return 'Please enter a label for each threshold';
}
if (!notificationSettings.routingPolicies && !threshold.channels.length) {
// this runs during render, so a throw here takes down the whole page
if (!notificationSettings.routingPolicies && !threshold.channels?.length) {
return 'Please select at least one channel for each threshold or enable routing policies';
}
}

View File

@@ -316,6 +316,34 @@ describe('CreateAlertV2 utils', () => {
});
});
describe('getThresholdStateFromAlertDef null channels', () => {
it('falls back to an empty array so downstream consumers never see null', () => {
const def: PostableAlertRuleV2 = {
...defaultPostableAlertRuleV2,
condition: {
...defaultPostableAlertRuleV2.condition,
thresholds: {
kind: 'basic',
spec: [
{
name: 'critical',
target: 1,
targetUnit: UniversalYAxisUnit.MINUTES,
channels: null as unknown as string[],
matchType: AlertThresholdMatchType.AT_LEAST_ONCE,
op: AlertThresholdOperator.IS_ABOVE,
},
],
},
},
};
expect(
getThresholdStateFromAlertDef(def).thresholds[0].channels,
).toStrictEqual([]);
});
});
describe('normalizeOperator', () => {
it.each([
['1', AlertThresholdOperator.IS_ABOVE],

View File

@@ -258,7 +258,9 @@ export function getThresholdStateFromAlertDef(
recoveryThresholdValue: null,
unit: threshold.targetUnit,
color: getColorForThreshold(threshold.name),
channels: threshold.channels,
// rules created outside the UI can come back with a null channels
// field; drop the guard once the API enforces the schema
channels: threshold.channels ?? [],
})) || [],
selectedQuery: alertDef.condition.selectedQueryName || '',
operator:

View File

@@ -82,7 +82,7 @@ export function getHostMetricsQueryPayload(
start: number,
end: number,
): ReturnType<typeof getHostQueryPayload> {
return getHostQueryPayload(host.hostName, start, end);
return getHostQueryPayload(host.hostName, start, end, true);
}
export { hostWidgetInfo };

View File

@@ -562,13 +562,9 @@ export const getClusterMetricsQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `MAX(${INFRA_MONITORING_ATTR_KEYS.K8S_NODE_CONDITION_READY})`,
op: '=',
value: 1,
},
],
having: {
expression: `max(${INFRA_MONITORING_ATTR_KEYS.K8S_NODE_CONDITION_READY}) = 1`,
},
legend: `{{${INFRA_MONITORING_ATTR_KEYS.K8S_NODE_NAME}}}`,
limit: null,
orderBy: [],
@@ -648,13 +644,9 @@ export const getClusterMetricsQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `MAX(${INFRA_MONITORING_ATTR_KEYS.K8S_NODE_CONDITION_READY})`,
op: '=',
value: 0,
},
],
having: {
expression: `max(${INFRA_MONITORING_ATTR_KEYS.K8S_NODE_CONDITION_READY}) = 0`,
},
legend: `{{${INFRA_MONITORING_ATTR_KEYS.K8S_NODE_NAME}}}`,
limit: null,
orderBy: [],

View File

@@ -1208,13 +1208,9 @@ export const getNamespaceMetricsQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `MAX(${INFRA_MONITORING_ATTR_KEYS.K8S_REPLICASET_DESIRED})`,
op: '>',
value: 0,
},
],
having: {
expression: `max(${INFRA_MONITORING_ATTR_KEYS.K8S_REPLICASET_DESIRED}) > 0`,
},
legend: 'desired',
limit: null,
orderBy: [],
@@ -1261,13 +1257,9 @@ export const getNamespaceMetricsQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `MAX(${INFRA_MONITORING_ATTR_KEYS.K8S_REPLICASET_DESIRED})`,
op: '>',
value: 0,
},
],
having: {
expression: `max(${INFRA_MONITORING_ATTR_KEYS.K8S_REPLICASET_AVAILABLE}) > 0`,
},
legend: 'available',
limit: null,
orderBy: [],

View File

@@ -1,9 +1,19 @@
import { PANEL_TYPES } from 'constants/queryBuilder';
import { GetQueryResultsProps } from 'lib/dashboard/getQueryResults';
import type { Having } from 'types/api/queryBuilder/queryBuilderData';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import type { Having as HavingV5 } from 'types/api/v5/queryRange';
import { EQueryType } from 'types/common/dashboard';
import { DataSource, ReduceOperators } from 'types/common/queryBuilder';
const buildSumGreaterThanZeroHaving = (
metricKey: string,
useV5HavingFormat: boolean,
): Having[] | HavingV5 =>
useV5HavingFormat
? { expression: `sum(${metricKey}) > 0` }
: [{ columnName: `SUM(${metricKey})`, op: '>', value: 0 }];
export const getPodQueryPayload = (
clusterName: string,
podName: string,
@@ -1540,6 +1550,7 @@ export const getHostQueryPayload = (
hostName: string,
start: number,
end: number,
useV5HavingFormat = false,
): GetQueryResultsProps[] => {
const hostNameKey = 'host.name';
const cpuTimeKey = 'system.cpu.time';
@@ -1802,13 +1813,7 @@ export const getHostQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `SUM(${fsUsageKey})`,
op: '>',
value: 0,
},
],
having: buildSumGreaterThanZeroHaving(fsUsageKey, useV5HavingFormat),
legend: '{{mountpoint}}',
limit: null,
orderBy: [],
@@ -1857,13 +1862,7 @@ export const getHostQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `SUM(${fsUsageKey})`,
op: '>',
value: 0,
},
],
having: buildSumGreaterThanZeroHaving(fsUsageKey, useV5HavingFormat),
legend: '{{mountpoint}}',
limit: null,
orderBy: [],
@@ -2089,13 +2088,7 @@ export const getHostQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `SUM(${netIoKey})`,
op: '>',
value: 0,
},
],
having: buildSumGreaterThanZeroHaving(netIoKey, useV5HavingFormat),
legend: '{{device}}::{{direction}}',
limit: 30,
orderBy: [],
@@ -2551,13 +2544,7 @@ export const getHostQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `SUM(${diskOpsKey})`,
op: '>',
value: 0,
},
],
having: buildSumGreaterThanZeroHaving(diskOpsKey, useV5HavingFormat),
legend: '{{device}}::{{direction}}',
limit: null,
orderBy: [],
@@ -2626,13 +2613,7 @@ export const getHostQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `SUM(${diskPendingKey})`,
op: '>',
value: 0,
},
],
having: buildSumGreaterThanZeroHaving(diskPendingKey, useV5HavingFormat),
legend: '{{device}}',
limit: null,
orderBy: [],
@@ -2708,13 +2689,7 @@ export const getHostQueryPayload = (
type: 'tag',
},
],
having: [
{
columnName: `SUM(${diskOpTimeKey})`,
op: '>',
value: 0,
},
],
having: buildSumGreaterThanZeroHaving(diskOpTimeKey, useV5HavingFormat),
legend: '{{device}}::{{direction}}',
limit: null,
orderBy: [],

View File

@@ -387,4 +387,42 @@ describe('useOptionsMenu', () => {
expect(remaining).toHaveLength(seedColumns.length);
});
});
describe('fieldsSelector.value drops legacy columns without a name', () => {
it('excludes entries missing name while keeping valid columns', () => {
(useGetQueryKeySuggestions as jest.Mock).mockReturnValue({
data: { data: { data: { keys: {} } } },
isFetching: false,
});
(usePreferenceContext as jest.Mock).mockReturnValue({
traces: {
preferences: {
columns: [
{ name: 'body', fieldContext: 'log' },
{ key: 'legacy-key-no-name', fieldContext: 'log' },
{ name: 'timestamp', fieldContext: 'log' },
],
formatting: { format: 'table', maxLines: 1, fontSize: 'small' },
},
updateColumns: mockUpdateColumns,
updateFormatting: mockUpdateFormatting,
},
logs: {
preferences: { columns: [], formatting: {} },
updateColumns: mockUpdateColumns,
updateFormatting: mockUpdateFormatting,
},
});
const { result } = renderHook(() =>
useOptionsMenu({
dataSource: DataSource.TRACES,
aggregateOperator: 'count',
}),
);
const fields = result.current.config.fieldsSelector?.value ?? [];
expect(fields.map((f) => f.name)).toStrictEqual(['body', 'timestamp']);
});
});
});

View File

@@ -399,7 +399,7 @@ const useOptionsMenu = ({
onReorder: reorderSelectColumns,
},
fieldsSelector: {
value: preferences?.columns ?? [],
value: preferences?.columns?.filter((item) => has(item, 'name')) ?? [],
onFieldsChange: updateColumns,
},
format: {

View File

@@ -80,7 +80,6 @@ function ResourceAttributesFilter({
<div className="environment-selector">
<Select
getPopupContainer={popupContainer}
key={selectedEnvironments.join('')}
showSearch
mode="multiple"
value={selectedEnvironments}

View File

@@ -0,0 +1,175 @@
import { ReactNode } from 'react';
import { QueryClient, QueryClientProvider } from 'react-query';
import { Router } from 'react-router-dom';
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import ROUTES from 'constants/routes';
import { createMemoryHistory, MemoryHistory } from 'history';
import { ResourceProvider } from 'hooks/useResourceAttribute';
import { IResourceAttribute } from 'hooks/useResourceAttribute/types';
import { encode } from 'js-base64';
import ResourceAttributesFilter from '../ResourceAttributesFilter';
jest.mock('lib/history', () => ({
__esModule: true,
default: {
push: jest.fn(),
location: { search: '', pathname: '/' },
},
}));
jest.mock('api/metrics/getResourceAttributes', () => ({
getResourceAttributesTagKeys: jest.fn(),
getResourceAttributesTagValues: jest.fn(),
}));
// eslint-disable-next-line import/first, import/order
import {
getResourceAttributesTagKeys,
getResourceAttributesTagValues,
// eslint-disable-next-line import/newline-after-import
} from 'api/metrics/getResourceAttributes';
// eslint-disable-next-line import/first, import/order
import history from 'lib/history';
const mockTagKeys = getResourceAttributesTagKeys as jest.MockedFunction<
typeof getResourceAttributesTagKeys
>;
const mockTagValues = getResourceAttributesTagValues as jest.MockedFunction<
typeof getResourceAttributesTagValues
>;
function tagKeysPayload(keys: string[]): never {
return {
statusCode: 200,
error: null,
message: 'ok',
payload: {
data: {
attributeKeys: keys.map((key) => ({
key,
dataType: 'string',
type: 'resource',
isColumn: false,
})),
},
},
} as unknown as never;
}
function tagValuesPayload(values: string[]): never {
return {
statusCode: 200,
error: null,
message: 'ok',
payload: { data: { stringAttributeValues: values } },
} as unknown as never;
}
function seedUrl(queries: IResourceAttribute[], pathname: string): void {
const location = history.location as { search: string; pathname: string };
location.search = queries.length
? `?resourceAttribute=${encode(JSON.stringify(queries))}`
: '';
location.pathname = pathname;
}
function renderFilter(pathname: string): MemoryHistory {
const routerHistory = createMemoryHistory({
initialEntries: [`${pathname}${history.location.search}`],
});
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
function Wrapper({ children }: { children: ReactNode }): JSX.Element {
return (
<QueryClientProvider client={queryClient}>
<Router history={routerHistory}>
<ResourceProvider>{children}</ResourceProvider>
</Router>
</QueryClientProvider>
);
}
render(
<Wrapper>
<ResourceAttributesFilter />
</Wrapper>,
);
return routerHistory;
}
describe('ResourceAttributesFilter', () => {
beforeEach(() => {
mockTagKeys.mockReset();
mockTagValues.mockReset();
mockTagKeys.mockResolvedValue(
tagKeysPayload(['resource_deployment.environment']),
);
mockTagValues.mockResolvedValue(tagValuesPayload(['production', 'staging']));
seedUrl([], '/');
});
it('shows every applied filter on the service map, including ones it cannot apply', async () => {
seedUrl(
[
{
id: 'svc',
tagKey: 'resource_service_name',
operator: 'IN',
tagValue: ['frontend'],
},
{
id: 'env',
tagKey: 'resource_deployment.environment',
operator: 'IN',
tagValue: ['production'],
},
],
ROUTES.SERVICE_MAP,
);
renderFilter(ROUTES.SERVICE_MAP);
await waitFor(() =>
expect(screen.getByText(/service\.name/)).toBeInTheDocument(),
);
await waitFor(() =>
expect(
screen
.getByTestId('resource-environment-filter')
.querySelector('.ant-select-selection-item'),
).toHaveTextContent('production'),
);
});
it('keeps the environment dropdown open so more than one environment can be picked', async () => {
const user = userEvent.setup();
renderFilter('/services');
const environmentFilter = screen.getByTestId('resource-environment-filter');
await user.click(
environmentFilter.querySelector('input') as HTMLInputElement,
);
await user.click(await screen.findByTitle('production'));
await waitFor(() =>
expect(
screen.getByTitle('staging').closest('.ant-select-dropdown'),
).not.toHaveClass('ant-select-dropdown-hidden'),
);
await user.click(screen.getByTitle('staging'));
await waitFor(() => {
const selected = Array.from(
environmentFilter.querySelectorAll('.ant-select-selection-item-content'),
).map((node) => node.textContent);
expect(selected).toStrictEqual(['production', 'staging']);
});
});
});

View File

@@ -1,12 +1,10 @@
import { ReactNode, useCallback, useEffect, useMemo, useState } from 'react';
import { useLocation } from 'react-router-dom';
import { QueryParams } from 'constants/query';
import ROUTES from 'constants/routes';
import { useSafeNavigate } from 'hooks/useSafeNavigate';
import useUrlQuery from 'hooks/useUrlQuery';
import { encode } from 'js-base64';
import { whilelistedKeys } from './config';
import { ResourceContext } from './context';
import {
IResourceAttribute,
@@ -195,16 +193,9 @@ function ResourceProvider({ children }: Props): JSX.Element {
setOptionsData({ mode: undefined, options: [] });
}, [dispatchQueries]);
const getVisibleQueries = useMemo(() => {
if (pathname === ROUTES.SERVICE_MAP) {
return queries.filter((query) => whilelistedKeys.includes(query.tagKey));
}
return queries;
}, [queries, pathname]);
const value: IResourceAttributeProps = useMemo(
() => ({
queries: getVisibleQueries,
queries,
staging,
handleClearAll,
handleClose,
@@ -227,7 +218,7 @@ function ResourceProvider({ children }: Props): JSX.Element {
staging,
selectedQuery,
optionsData,
getVisibleQueries,
queries,
],
);

View File

@@ -504,22 +504,23 @@ describe('ResourceProvider', () => {
});
});
describe('getVisibleQueries (SERVICE_MAP filtering)', () => {
it('filters queries down to whitelisted keys on SERVICE_MAP', () => {
const seeded = [
{
id: 'a',
tagKey: 'resource_service_name',
operator: 'IN',
tagValue: ['frontend'],
},
{
id: 'b',
tagKey: 'resource_k8s_cluster_name',
operator: 'IN',
tagValue: ['prod'],
},
];
describe('SERVICE_MAP', () => {
const seeded = [
{
id: 'a',
tagKey: 'resource_service_name',
operator: 'IN',
tagValue: ['frontend'],
},
{
id: 'b',
tagKey: 'resource_k8s_cluster_name',
operator: 'IN',
tagValue: ['prod'],
},
];
it('exposes every query from the URL, including ones the map cannot apply', () => {
mockLibHistory(
`?resourceAttribute=${encode(JSON.stringify(seeded))}`,
ROUTES.SERVICE_MAP,
@@ -532,24 +533,10 @@ describe('ResourceProvider', () => {
wrapper: createWrapper({ routerHistory }),
});
expect(result.current.queries).toStrictEqual([seeded[1]]);
expect(result.current.queries).toStrictEqual(seeded);
});
it('returns all queries on non-SERVICE_MAP routes', () => {
const seeded = [
{
id: 'a',
tagKey: 'resource_service_name',
operator: 'IN',
tagValue: ['frontend'],
},
{
id: 'b',
tagKey: 'resource_k8s_cluster_name',
operator: 'IN',
tagValue: ['prod'],
},
];
mockLibHistory(
`?resourceAttribute=${encode(JSON.stringify(seeded))}`,
'/services',

View File

@@ -1,7 +1,10 @@
import ROUTES from 'constants/routes';
import { whilelistedKeys } from '../config';
import { mappingWithRoutesAndKeys } from '../utils';
import {
filterServiceMapSupportedQueries,
mappingWithRoutesAndKeys,
} from '../utils';
describe('useResourceAttribute config', () => {
describe('whilelistedKeys', () => {
@@ -74,4 +77,29 @@ describe('useResourceAttribute config', () => {
expect(result).toStrictEqual(allFilters);
});
});
describe('filterServiceMapSupportedQueries', () => {
const environmentQuery = {
id: 'env',
tagKey: 'resource_deployment_environment',
operator: 'IN',
tagValue: ['production'],
};
const serviceQuery = {
id: 'svc',
tagKey: 'resource_service_name',
operator: 'IN',
tagValue: ['frontend'],
};
it('should keep only the queries the service map can filter on', () => {
expect(
filterServiceMapSupportedQueries([environmentQuery, serviceQuery]),
).toStrictEqual([environmentQuery]);
});
it('should return an empty list when no query is supported', () => {
expect(filterServiceMapSupportedQueries([serviceQuery])).toStrictEqual([]);
});
});
});

View File

@@ -281,3 +281,8 @@ export const mappingWithRoutesAndKeys = (
}
return filters;
};
export const filterServiceMapSupportedQueries = (
queries: IResourceAttribute[],
): IResourceAttribute[] =>
queries.filter((query) => whilelistedKeys.includes(query.tagKey));

View File

@@ -1,6 +1,6 @@
//@ts-nocheck
import { useEffect, useRef } from 'react';
import { useEffect, useMemo, useRef } from 'react';
// eslint-disable-next-line no-restricted-imports
import { connect } from 'react-redux';
import { RouteComponentProps, withRouter } from 'react-router-dom';
@@ -11,6 +11,7 @@ import ResourceAttributesFilter from 'container/ResourceAttributesFilter';
import useResourceAttribute from 'hooks/useResourceAttribute';
import { whilelistedKeys } from 'hooks/useResourceAttribute/config';
import { IResourceAttribute } from 'hooks/useResourceAttribute/types';
import { filterServiceMapSupportedQueries } from 'hooks/useResourceAttribute/utils';
import { getDetailedServiceMapItems, ServiceMapStore } from 'store/actions';
import { AppState } from 'store/reducers';
import styled from 'styled-components';
@@ -70,32 +71,37 @@ function ServiceMap(props: ServiceMapProps): JSX.Element {
const { queries } = useResourceAttribute();
const supportedQueries = useMemo(
() => filterServiceMapSupportedQueries(queries),
[queries],
);
useEffect(() => {
/*
Call the apis only when the route is loaded.
Check this issue: https://github.com/SigNoz/signoz/issues/110
*/
getDetailedServiceMapItems(globalTime, queries);
}, [globalTime, getDetailedServiceMapItems, queries]);
getDetailedServiceMapItems(globalTime, supportedQueries);
}, [globalTime, getDetailedServiceMapItems, supportedQueries]);
useEffect(() => {
fgRef.current && fgRef.current.d3Force('charge').strength(-400);
});
if (serviceMap.loading) {
return <Spinner size="large" tip="Loading..." />;
}
const renderBody = (): JSX.Element => {
if (serviceMap.loading) {
return <Spinner size="large" tip="Loading..." />;
}
if (serviceMap.items.length === 0) {
return <Card>No Service Found</Card>;
}
return <Map fgRef={fgRef} serviceMap={serviceMap} />;
};
if (!serviceMap.loading && serviceMap.items.length === 0) {
return (
<Container>
<ResourceAttributesFilter />
<Card>No Service Found</Card>
</Container>
);
}
return (
<div className="service-map-container">
<Container className="service-map-container">
<ResourceAttributesFilter
suffixIcon={
<TextToolTip
@@ -108,8 +114,8 @@ function ServiceMap(props: ServiceMapProps): JSX.Element {
}
/>
<Map fgRef={fgRef} serviceMap={serviceMap} />
</div>
{renderBody()}
</Container>
);
}

View File

@@ -223,9 +223,7 @@ func TestEmailNotifyWithErrors(t *testing.T) {
}
c, err := loadEmailTestConfiguration(cfgFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
for _, tc := range []struct {
title string
@@ -288,10 +286,7 @@ func TestEmailNotifyWithErrors(t *testing.T) {
},
} {
t.Run(tc.title, func(t *testing.T) {
if len(tc.errMsg) == 0 {
t.Fatal("please define the expected error message")
return
}
require.NotEmpty(t, tc.errMsg, "please define the expected error message")
emailCfg := &config.EmailConfig{
Smarthost: c.Smarthost,
@@ -309,15 +304,15 @@ func TestEmailNotifyWithErrors(t *testing.T) {
_, retry, err := notifyEmail(t, emailCfg, c.Server)
require.Error(t, err)
require.Contains(t, err.Error(), tc.errMsg)
require.False(t, retry)
assert.Contains(t, err.Error(), tc.errMsg)
assert.False(t, retry)
e, err := c.Server.getLastEmail(t)
require.NoError(t, err)
if tc.hasEmail {
require.NotNil(t, e)
assert.NotNil(t, e)
} else {
require.Nil(t, e)
assert.Nil(t, e)
}
})
}
@@ -331,9 +326,7 @@ func TestEmailNotifyWithDoneContext(t *testing.T) {
}
c, err := loadEmailTestConfiguration(cfgFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
cancel()
@@ -350,7 +343,7 @@ func TestEmailNotifyWithDoneContext(t *testing.T) {
c.Server,
)
require.Error(t, err)
require.Contains(t, err.Error(), "establish connection to server")
assert.Contains(t, err.Error(), "establish connection to server")
}
// TestEmailNotifyWithoutAuthentication sends an email to an instance of
@@ -363,9 +356,7 @@ func TestEmailNotifyWithoutAuthentication(t *testing.T) {
}
c, err := loadEmailTestConfiguration(cfgFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
mail, _, err := notifyEmail(
t,
@@ -390,7 +381,7 @@ func TestEmailNotifyWithoutAuthentication(t *testing.T) {
}
headers = append(headers, k)
}
require.True(t, foundMsgID, "Couldn't find 'message-id' in %v", headers)
assert.True(t, foundMsgID, "Couldn't find 'message-id' in %v", headers)
}
// TestEmailNotifyWithSTARTTLS connects to the server, upgrades the connection
@@ -406,9 +397,7 @@ func TestEmailNotifyWithSTARTTLS(t *testing.T) {
}
c, err := loadEmailTestConfiguration(cfgFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
trueVar := true
_, _, err = notifyEmail(
@@ -437,9 +426,7 @@ func TestEmailNotifyWithAuthentication(t *testing.T) {
}
c, err := loadEmailTestConfiguration(cfgFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
td := t.TempDir()
fileWithCorrectPassword, err := os.CreateTemp(td, "smtp-password-correct")
@@ -583,13 +570,13 @@ func TestEmailNotifyWithAuthentication(t *testing.T) {
e, retry, err := notifyEmail(t, emailCfg, c.Server)
if len(tc.errMsg) > 0 {
require.Error(t, err)
require.Contains(t, err.Error(), tc.errMsg)
require.Equal(t, tc.retry, retry)
assert.Contains(t, err.Error(), tc.errMsg)
assert.Equal(t, tc.retry, retry)
return
}
require.NoError(t, err)
require.Equal(t, "1 firing alert(s)", e.Subject)
assert.Equal(t, "1 firing alert(s)", e.Subject)
getAddresses := func(addresses []map[string]string) []string {
res := make([]string, 0, len(addresses))
@@ -600,19 +587,21 @@ func TestEmailNotifyWithAuthentication(t *testing.T) {
}
to := getAddresses(e.To)
from := getAddresses(e.From)
require.Equal(t, strings.Split(emailCfg.To, ","), to)
require.Equal(t, strings.Split(emailCfg.From, ","), from)
assert.Equal(t, strings.Split(emailCfg.To, ","), to)
assert.Equal(t, strings.Split(emailCfg.From, ","), from)
if len(emailCfg.HTML) > 0 {
require.Equal(t, emailCfg.HTML, *e.HTML)
require.NotNil(t, e.HTML)
assert.Equal(t, emailCfg.HTML, *e.HTML)
} else {
require.Nil(t, e.HTML)
assert.Nil(t, e.HTML)
}
if len(emailCfg.Text) > 0 {
require.Equal(t, emailCfg.Text, *e.Text)
require.NotNil(t, e.Text)
assert.Equal(t, emailCfg.Text, *e.Text)
} else {
require.Nil(t, e.Text)
assert.Nil(t, e.Text)
}
})
}
@@ -624,7 +613,7 @@ func TestEmailConfigNoAuthMechs(t *testing.T) {
}
_, err := email.auth("")
require.Error(t, err)
require.Equal(t, "unknown auth mechanism: ", err.Error())
assert.Equal(t, "unknown auth mechanism: ", err.Error())
}
func TestEmailConfigMissingAuthParam(t *testing.T) {
@@ -634,19 +623,19 @@ func TestEmailConfigMissingAuthParam(t *testing.T) {
}
_, err := email.auth("CRAM-MD5")
require.Error(t, err)
require.Equal(t, "missing secret for CRAM-MD5 auth mechanism", err.Error())
assert.Equal(t, "missing secret for CRAM-MD5 auth mechanism", err.Error())
_, err = email.auth("PLAIN")
require.Error(t, err)
require.Equal(t, "missing password for PLAIN auth mechanism", err.Error())
assert.Equal(t, "missing password for PLAIN auth mechanism", err.Error())
_, err = email.auth("LOGIN")
require.Error(t, err)
require.Equal(t, "missing password for LOGIN auth mechanism", err.Error())
assert.Equal(t, "missing password for LOGIN auth mechanism", err.Error())
_, err = email.auth("PLAIN LOGIN")
require.Error(t, err)
require.Equal(t, "missing password for PLAIN auth mechanism\nmissing password for LOGIN auth mechanism", err.Error())
assert.Equal(t, "missing password for PLAIN auth mechanism\nmissing password for LOGIN auth mechanism", err.Error())
}
func TestEmailNoUsernameCustomError(t *testing.T) {
@@ -655,7 +644,7 @@ func TestEmailNoUsernameCustomError(t *testing.T) {
}
a, err := email.auth("CRAM-MD5")
require.ErrorIs(t, err, errNoAuthUsernameConfigured)
require.Nil(t, a)
assert.Nil(t, a)
}
// TestEmailRejected simulates the failure of an otherwise valid message submission which fails at a later point than
@@ -720,7 +709,7 @@ func TestEmailRejected(t *testing.T) {
// Send the alert to mock SMTP server.
retry, err := e.Notify(context.Background(), firingAlert)
require.ErrorContains(t, err, "501 5.5.4 Rejected!")
require.True(t, retry)
assert.True(t, retry)
require.NoError(t, srv.Shutdown(ctx))
require.Eventuallyf(t, func() bool {
@@ -789,9 +778,7 @@ func TestEmailNotifyWithThreading(t *testing.T) {
}
c, err := loadEmailTestConfiguration(cfgFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
for _, tc := range []struct {
name string
@@ -836,22 +823,22 @@ func TestEmailNotifyWithThreading(t *testing.T) {
referencesValue := mail.Headers["references"]
inReplyToValue := mail.Headers["in-reply-to"]
require.NotEmpty(t, referencesValue, "References header not found in %v", mail.Headers)
require.NotEmpty(t, inReplyToValue, "In-Reply-To header not found in %v", mail.Headers)
assert.NotEmpty(t, referencesValue, "References header not found in %v", mail.Headers)
assert.NotEmpty(t, inReplyToValue, "In-Reply-To header not found in %v", mail.Headers)
require.Equal(t, referencesValue, inReplyToValue, "References and In-Reply-To should match")
assert.Equal(t, referencesValue, inReplyToValue, "References and In-Reply-To should match")
// Verify the format: <alert-HASH-DATE@alertmanager>
require.Contains(t, referencesValue, "<alert-")
require.Contains(t, referencesValue, "@alertmanager>")
assert.Contains(t, referencesValue, "<alert-")
assert.Contains(t, referencesValue, "@alertmanager>")
if tc.wantDatePart {
today := time.Now().Format("2006-01-02")
require.Contains(t, referencesValue, today, "threading header should contain today's date")
assert.Contains(t, referencesValue, today, "threading header should contain today's date")
} else {
// With thread_by_date: none, there should be no date
// (empty string between hash and @).
require.Contains(t, referencesValue, "-@alertmanager>", "threading header should have empty date part")
assert.Contains(t, referencesValue, "-@alertmanager>", "threading header should have empty date part")
}
})
}
@@ -904,14 +891,14 @@ func TestEmailGetPassword(t *testing.T) {
require.Error(t, err)
if errors.Asc(err, errors.CodeInternal) {
_, _, errMsg, _, _, _ := errors.Unwrapb(err)
require.Contains(t, errMsg, tc.errMsg)
assert.Contains(t, errMsg, tc.errMsg)
} else {
require.Contains(t, err.Error(), tc.errMsg)
assert.Contains(t, err.Error(), tc.errMsg)
}
require.Empty(t, password)
assert.Empty(t, password)
} else {
require.NoError(t, err)
require.Equal(t, "secret", password)
assert.Equal(t, "secret", password)
}
})
}
@@ -962,11 +949,11 @@ func TestEmailGetSecret(t *testing.T) {
secret, err := email.getAuthSecret()
if len(tc.errMsg) > 0 {
require.Error(t, err)
require.Contains(t, err.Error(), tc.errMsg)
require.Empty(t, secret)
assert.Contains(t, err.Error(), tc.errMsg)
assert.Empty(t, secret)
} else {
require.NoError(t, err)
require.Equal(t, "secret", secret)
assert.Equal(t, "secret", secret)
}
})
}
@@ -1032,7 +1019,7 @@ func TestEmailImplicitTLS(t *testing.T) {
useImplicitTLS = cfg.Smarthost.Port == "465"
}
require.Equal(t, tt.expectImplicit, useImplicitTLS,
assert.Equal(t, tt.expectImplicit, useImplicitTLS,
"Expected useImplicitTLS=%v for port=%s with forceImplicitTLS=%v",
tt.expectImplicit, tt.port, tt.forceImplicitTLS)
})
@@ -1074,8 +1061,8 @@ func TestPrepareContent(t *testing.T) {
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "subj", subject)
require.Equal(t, "<div><p>line one</p>\n</div><div><p>line two</p>\n</div>", htmlBody)
assert.Equal(t, "subj", subject)
assert.Equal(t, "<div><p>line one</p>\n</div><div><p>line two</p>\n</div>", htmlBody)
})
t.Run("custom title template; default body HTML template", func(t *testing.T) {
@@ -1103,8 +1090,8 @@ func TestPrepareContent(t *testing.T) {
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "Status: firing", htmlBody)
require.Equal(t, "fixed from firing", subject)
assert.Equal(t, "Status: firing", htmlBody)
assert.Equal(t, "fixed from firing", subject)
})
t.Run("default template without HTML", func(t *testing.T) {
@@ -1125,8 +1112,8 @@ func TestPrepareContent(t *testing.T) {
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "", htmlBody)
require.Equal(t, "the email subject", subject)
assert.Equal(t, "", htmlBody)
assert.Equal(t, "the email subject", subject)
})
t.Run("custom title template; custom body template", func(t *testing.T) {
@@ -1160,11 +1147,11 @@ func TestPrepareContent(t *testing.T) {
ctx := context.Background()
subject, htmlBody, err := n.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Contains(t, htmlBody, "<!DOCTYPE html>")
require.Contains(t, htmlBody, "<p>line two</p>")
require.NotContains(t, htmlBody, "Well, what are you?")
require.Equal(t, subject, "fixed from firing")
require.NotContains(t, subject, "subject")
assert.Contains(t, htmlBody, "<!DOCTYPE html>")
assert.Contains(t, htmlBody, "<p>line two</p>")
assert.NotContains(t, htmlBody, "Well, what are you?")
assert.Equal(t, "fixed from firing", subject)
assert.NotContains(t, subject, "subject")
})
}

View File

@@ -22,6 +22,7 @@ import (
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
test "github.com/SigNoz/signoz/pkg/alertmanager/alertmanagernotify/alertmanagernotifytest"
@@ -54,7 +55,7 @@ func TestMSTeamsV2Retry(t *testing.T) {
for statusCode, expected := range test.RetryTests(test.DefaultRetryCodes()) {
actual, _ := notifier.retrier.Check(statusCode, nil)
require.Equal(t, expected, actual, "retry - error on status %d", statusCode)
assert.Equal(t, expected, actual, "retry - error on status %d", statusCode)
}
}
@@ -110,7 +111,7 @@ func TestNotifier_Notify_WithReason(t *testing.T) {
} else {
var reasonError *notify.ErrorWithReason
require.ErrorAs(t, err, &reasonError)
require.Equal(t, tt.expectedReason, reasonError.Reason)
assert.Equal(t, tt.expectedReason, reasonError.Reason)
}
})
}
@@ -188,9 +189,9 @@ func TestMSTeamsV2Templating(t *testing.T) {
require.NoError(t, err)
} else {
require.Error(t, err)
require.Contains(t, err.Error(), tc.errMsg)
assert.Contains(t, err.Error(), tc.errMsg)
}
require.Equal(t, tc.retry, ok)
assert.Equal(t, tc.retry, ok)
})
}
}
@@ -250,14 +251,14 @@ func TestPrepareContent(t *testing.T) {
}
blocks, err := notifier.prepareContent(ctx, alerts)
require.NoError(t, err)
require.NotEmpty(t, blocks)
require.Len(t, blocks, 2)
// First block should be the title with color (firing = red)
require.Equal(t, "Bolder", blocks[0].Weight)
require.Equal(t, colorRed, blocks[0].Color)
assert.Equal(t, "Bolder", blocks[0].Weight)
assert.Equal(t, colorRed, blocks[0].Color)
// verify title text
require.Equal(t, "Alertname: test", blocks[0].Text)
assert.Equal(t, "Alertname: test", blocks[0].Text)
// verify body text
require.Equal(t, "Firing alert: test", blocks[1].Text)
assert.Equal(t, "Firing alert: test", blocks[1].Text)
})
t.Run("custom template - per-alert color", func(t *testing.T) {
@@ -305,16 +306,15 @@ func TestPrepareContent(t *testing.T) {
}
blocks, err := notifier.prepareContent(ctx, alerts)
require.NoError(t, err)
require.NotEmpty(t, blocks)
// total 3 blocks: title and 2 body blocks
require.True(t, len(blocks) == 3)
require.Len(t, blocks, 3)
// First block: title color is overall color of the alerts
require.Equal(t, colorRed, blocks[0].Color)
assert.Equal(t, colorRed, blocks[0].Color)
// verify title text
require.Equal(t, "Custom Title", blocks[0].Text)
assert.Equal(t, "Custom Title", blocks[0].Text)
// Body blocks should have per-alert color
require.Equal(t, colorRed, blocks[1].Color) // firing
require.Equal(t, colorGreen, blocks[2].Color) // resolved
assert.Equal(t, colorRed, blocks[1].Color) // firing
assert.Equal(t, colorGreen, blocks[2].Color) // resolved
})
}

View File

@@ -21,6 +21,7 @@ import (
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/prometheus/alertmanager/config"
@@ -49,7 +50,7 @@ func TestOpsGenieRetry(t *testing.T) {
retryCodes := append(test.DefaultRetryCodes(), http.StatusTooManyRequests)
for statusCode, expected := range test.RetryTests(retryCodes) {
actual, _ := notifier.retrier.Check(statusCode, nil)
require.Equal(t, expected, actual, "error on status %d", statusCode)
assert.Equal(t, expected, actual, "error on status %d", statusCode)
}
}
@@ -103,9 +104,7 @@ func TestGettingOpsGegineApikeyFromFile(t *testing.T) {
func TestOpsGenie(t *testing.T) {
u, err := url.Parse("https://opsgenie/api")
if err != nil {
t.Fatalf("failed to parse URL: %v", err)
}
require.NoError(t, err)
logger := promslog.NewNopLogger()
tmpl := test.CreateTmpl(t)
@@ -236,10 +235,10 @@ func TestOpsGenie(t *testing.T) {
req, retry, err := notifier.createRequests(ctx, alert1)
require.NoError(t, err)
require.Len(t, req, 1)
require.True(t, retry)
require.Equal(t, expectedURL, req[0].URL)
require.Equal(t, "GenieKey http://am", req[0].Header.Get("Authorization"))
require.Equal(t, tc.expectedEmptyAlertBody, readBody(t, req[0]))
assert.True(t, retry)
assert.Equal(t, expectedURL, req[0].URL)
assert.Equal(t, "GenieKey http://am", req[0].Header.Get("Authorization"))
assert.Equal(t, tc.expectedEmptyAlertBody, readBody(t, req[0]))
// Fully defined alert.
alert2 := &types.Alert{
@@ -266,15 +265,15 @@ func TestOpsGenie(t *testing.T) {
}
req, retry, err = notifier.createRequests(ctx, alert2)
require.NoError(t, err)
require.True(t, retry)
assert.True(t, retry)
require.Len(t, req, 1)
require.Equal(t, tc.expectedBody, readBody(t, req[0]))
assert.Equal(t, tc.expectedBody, readBody(t, req[0]))
// Broken API Key Template.
tc.cfg.APIKey = "{{ kaput "
_, _, err = notifier.createRequests(ctx, alert2)
require.Error(t, err)
require.Equal(t, "template: :1: function \"kaput\" not defined", err.Error())
assert.Equal(t, "template: :1: function \"kaput\" not defined", err.Error())
})
}
}
@@ -307,7 +306,7 @@ func TestOpsGenieWithUpdate(t *testing.T) {
require.NoError(t, err)
requests, retry, err := notifierWithUpdate.createRequests(ctx, alert)
require.NoError(t, err)
require.True(t, retry)
assert.True(t, retry)
require.Len(t, requests, 3)
body0 := readBody(t, requests[0])
@@ -316,13 +315,13 @@ func TestOpsGenieWithUpdate(t *testing.T) {
key, _ := notify.ExtractGroupKey(ctx)
alias := key.Hash()
require.Equal(t, "https://test-opsgenie-url/v2/alerts", requests[0].URL.String())
require.NotEmpty(t, body0)
assert.Equal(t, "https://test-opsgenie-url/v2/alerts", requests[0].URL.String())
assert.NotEmpty(t, body0)
require.Equal(t, requests[1].URL.String(), fmt.Sprintf("https://test-opsgenie-url/v2/alerts/%s/message?identifierType=alias", alias))
require.JSONEq(t, `{"message":"new message"}`, body1)
require.Equal(t, requests[2].URL.String(), fmt.Sprintf("https://test-opsgenie-url/v2/alerts/%s/description?identifierType=alias", alias))
require.JSONEq(t, `{"description":"new description"}`, body2)
assert.Equal(t, requests[1].URL.String(), fmt.Sprintf("https://test-opsgenie-url/v2/alerts/%s/message?identifierType=alias", alias))
assert.JSONEq(t, `{"message":"new message"}`, body1)
assert.Equal(t, requests[2].URL.String(), fmt.Sprintf("https://test-opsgenie-url/v2/alerts/%s/description?identifierType=alias", alias))
assert.JSONEq(t, `{"description":"new description"}`, body2)
}
func TestOpsGenieApiKeyFile(t *testing.T) {
@@ -341,7 +340,8 @@ func TestOpsGenieApiKeyFile(t *testing.T) {
require.NoError(t, err)
requests, _, err := notifierWithUpdate.createRequests(ctx)
require.NoError(t, err)
require.Equal(t, "GenieKey my_secret_api_key", requests[0].Header.Get("Authorization"))
require.Len(t, requests, 1)
assert.Equal(t, "GenieKey my_secret_api_key", requests[0].Header.Get("Authorization"))
}
func TestPrepareContent(t *testing.T) {
@@ -377,8 +377,8 @@ func TestPrepareContent(t *testing.T) {
title, desc, prepErr := notifier.prepareContent(ctx, alerts)
require.NoError(t, prepErr)
require.Equal(t, "Firing alert: test", title)
require.Equal(t, "Check runbook for more details", desc)
assert.Equal(t, "Firing alert: test", title)
assert.Equal(t, "Check runbook for more details", desc)
})
t.Run("custom template", func(t *testing.T) {
@@ -431,9 +431,9 @@ func TestPrepareContent(t *testing.T) {
title, desc, err := notifier.prepareContent(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "High request throughput for payment", title)
assert.Equal(t, "High request throughput for payment", title)
// Each alert body wrapped in <div>, separated by <hr>
require.Equal(t, "<div><p>Alert firing in NS: potter-the-harry</p>\n</div><hr><div><p>Alert firing in NS: smart-the-rat</p>\n</div>", desc)
assert.Equal(t, "<div><p>Alert firing in NS: potter-the-harry</p>\n</div><hr><div><p>Alert firing in NS: smart-the-rat</p>\n</div>", desc)
})
}

View File

@@ -25,6 +25,7 @@ import (
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/prometheus/alertmanager/config"
@@ -54,7 +55,7 @@ func TestPagerDutyRetryV1(t *testing.T) {
retryCodes := append(test.DefaultRetryCodes(), http.StatusForbidden)
for statusCode, expected := range test.RetryTests(retryCodes) {
actual, _ := notifier.retrier.Check(statusCode, nil)
require.Equal(t, expected, actual, "retryv1 - error on status %d", statusCode)
assert.Equal(t, expected, actual, "retryv1 - error on status %d", statusCode)
}
}
@@ -74,7 +75,7 @@ func TestPagerDutyRetryV2(t *testing.T) {
retryCodes := append(test.DefaultRetryCodes(), http.StatusTooManyRequests)
for statusCode, expected := range test.RetryTests(retryCodes) {
actual, _ := notifier.retrier.Check(statusCode, nil)
require.Equal(t, expected, actual, "retryv2 - error on status %d", statusCode)
assert.Equal(t, expected, actual, "retryv2 - error on status %d", statusCode)
}
}
@@ -349,12 +350,12 @@ func TestPagerDutyTemplating(t *testing.T) {
require.Error(t, err)
if errors.Asc(err, errors.CodeInternal) {
_, _, errMsg, _, _, _ := errors.Unwrapb(err)
require.Contains(t, errMsg, tc.errMsg)
assert.Contains(t, errMsg, tc.errMsg)
} else {
require.Contains(t, err.Error(), tc.errMsg)
assert.Contains(t, err.Error(), tc.errMsg)
}
}
require.Equal(t, tc.retry, ok)
assert.Equal(t, tc.retry, ok)
})
}
}
@@ -393,7 +394,7 @@ func TestErrDetails(t *testing.T) {
} {
t.Run("", func(t *testing.T) {
err := errDetails(tc.status, tc.body)
require.Contains(t, err, tc.exp)
assert.Contains(t, err, tc.exp)
})
}
}
@@ -427,7 +428,7 @@ func TestEventSizeEnforcement(t *testing.T) {
encodedV1, err := notifierV1.encodeMessage(context.Background(), msgV1)
require.NoError(t, err)
require.Contains(t, encodedV1.String(), `"details":{"error":"Custom details have been removed because the original event exceeds the maximum size of 512KB"}`)
assert.Contains(t, encodedV1.String(), `"details":{"error":"Custom details have been removed because the original event exceeds the maximum size of 512KB"}`)
// V2 Messages
msgV2 := &pagerDutyMessage{
@@ -451,7 +452,7 @@ func TestEventSizeEnforcement(t *testing.T) {
encodedV2, err := notifierV2.encodeMessage(context.Background(), msgV2)
require.NoError(t, err)
require.Contains(t, encodedV2.String(), `"custom_details":{"error":"Custom details have been removed because the original event exceeds the maximum size of 512KB"}`)
assert.Contains(t, encodedV2.String(), `"custom_details":{"error":"Custom details have been removed because the original event exceeds the maximum size of 512KB"}`)
}
func TestPagerDutyEmptySrcHref(t *testing.T) {
@@ -543,8 +544,9 @@ func TestPagerDutyEmptySrcHref(t *testing.T) {
}
}
require.Equal(t, expectedImages, event.Images)
require.Equal(t, expectedLinks, event.Links)
// Handler runs on the server's goroutine — require is illegal here.
assert.Equal(t, expectedImages, event.Images)
assert.Equal(t, expectedLinks, event.Links)
},
))
defer server.Close()
@@ -644,7 +646,7 @@ func TestPagerDutyTimeout(t *testing.T) {
},
}
_, err = pd.Notify(ctx, alert)
require.Equal(t, tt.wantErr, err != nil)
assert.Equal(t, tt.wantErr, err != nil)
})
}
}
@@ -899,11 +901,12 @@ func TestRenderDetails(t *testing.T) {
tmpl: test.CreateTmpl(t),
}
got, err := n.renderDetails(tt.args.data)
if (err != nil) != tt.wantErr {
t.Errorf("renderDetails() error = %v, wantErr %v", err, tt.wantErr)
return
if tt.wantErr {
require.Error(t, err)
} else {
require.NoError(t, err)
}
require.Equal(t, tt.want, got)
assert.Equal(t, tt.want, got)
})
}
}
@@ -944,7 +947,7 @@ func TestPrepareContent(t *testing.T) {
title, err := notifier.prepareTitle(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "HighCPU for Payment service (FIRING)", title)
assert.Equal(t, "HighCPU for Payment service (FIRING)", title)
})
t.Run("custom template uses $variable annotation for title", func(t *testing.T) {
@@ -980,6 +983,6 @@ func TestPrepareContent(t *testing.T) {
title, err := notifier.prepareTitle(ctx, alerts)
require.NoError(t, err)
require.Equal(t, "HighCPU on api-server is in resolved state", title)
assert.Equal(t, "HighCPU on api-server is in resolved state", title)
})
}

View File

@@ -23,6 +23,7 @@ import (
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/prometheus/alertmanager/config"
@@ -50,7 +51,7 @@ func TestSlackRetry(t *testing.T) {
for statusCode, expected := range test.RetryTests(test.DefaultRetryCodes()) {
actual, _ := notifier.retrier.Check(statusCode, nil)
require.Equal(t, expected, actual, "error on status %d", statusCode)
assert.Equal(t, expected, actual, "error on status %d", statusCode)
}
}
@@ -232,15 +233,15 @@ func TestNotifier_Notify_WithReason(t *testing.T) {
},
}
retry, err := notifier.Notify(ctx, alert1)
require.Equal(t, tt.expectedRetry, retry)
assert.Equal(t, tt.expectedRetry, retry)
if tt.noError {
require.NoError(t, err)
} else {
var reasonError *notify.ErrorWithReason
require.ErrorAs(t, err, &reasonError)
require.Equal(t, tt.expectedReason, reasonError.Reason)
require.Contains(t, err.Error(), tt.expectedErr)
require.Contains(t, err.Error(), "channelname")
assert.Equal(t, tt.expectedReason, reasonError.Reason)
assert.Contains(t, err.Error(), tt.expectedErr)
assert.Contains(t, err.Error(), "channelname")
}
})
}
@@ -296,7 +297,7 @@ func TestSlackTimeout(t *testing.T) {
},
}
_, err = notifier.Notify(ctx, alert)
require.Equal(t, tt.wantErr, err != nil)
assert.Equal(t, tt.wantErr, err != nil)
})
}
}
@@ -350,14 +351,14 @@ func TestPrepareContent(t *testing.T) {
require.NoError(t, err)
require.Len(t, atts, 1)
require.Equal(t, "HighCPU (FIRING)", atts[0].Title)
require.Equal(t, "Alert: HighCPU - severity critical", atts[0].Text)
assert.Equal(t, "HighCPU (FIRING)", atts[0].Title)
assert.Equal(t, "Alert: HighCPU - severity critical", atts[0].Text)
// Color is templated — firing alert should be "danger"
require.Equal(t, "danger", atts[0].Color)
assert.Equal(t, "danger", atts[0].Color)
// No BlockKit blocks for default template
require.Nil(t, atts[0].Blocks)
assert.Nil(t, atts[0].Blocks)
// Default markdownIn when config has none
require.Equal(t, []string{"fallback", "pretext", "text"}, atts[0].MrkdwnIn)
assert.Equal(t, []string{"fallback", "pretext", "text"}, atts[0].MrkdwnIn)
})
t.Run("custom template produces 1+N attachments with per-alert color", func(t *testing.T) {
@@ -428,10 +429,10 @@ func TestPrepareContent(t *testing.T) {
require.Len(t, atts, 3)
// First attachment: title-only, no color, no blocks
require.Equal(t, "[firing] HighCPU — api-server", atts[0].Title)
require.Empty(t, atts[0].Color)
require.Nil(t, atts[0].Blocks)
require.Equal(t, "https://alertmanager.signoz.com", atts[0].TitleLink)
assert.Equal(t, "[firing] HighCPU — api-server", atts[0].Title)
assert.Empty(t, atts[0].Color)
assert.Nil(t, atts[0].Blocks)
assert.Equal(t, "https://alertmanager.signoz.com", atts[0].TitleLink)
expectedFiringBody := "*HighCPU*\n\n" +
"*Service:* _api-server_\n*Instance:* _i-0abc123_\n*Region:* _us-east-1_\n*Method:* _GET_\n\n" +
@@ -446,16 +447,16 @@ func TestPrepareContent(t *testing.T) {
"*Status:* resolved | *Severity:* critical\n\n"
// Second attachment: firing alert body rendered as slack mrkdwn text, red color
require.Nil(t, atts[1].Blocks)
require.Equal(t, "#FF0000", atts[1].Color)
require.Equal(t, []string{"text"}, atts[1].MrkdwnIn)
require.Equal(t, expectedFiringBody, atts[1].Text)
assert.Nil(t, atts[1].Blocks)
assert.Equal(t, "#FF0000", atts[1].Color)
assert.Equal(t, []string{"text"}, atts[1].MrkdwnIn)
assert.Equal(t, expectedFiringBody, atts[1].Text)
// Third attachment: resolved alert body rendered as slack mrkdwn text, green color
require.Nil(t, atts[2].Blocks)
require.Equal(t, "#00FF00", atts[2].Color)
require.Equal(t, []string{"text"}, atts[2].MrkdwnIn)
require.Equal(t, expectedResolvedBody, atts[2].Text)
assert.Nil(t, atts[2].Blocks)
assert.Equal(t, "#00FF00", atts[2].Color)
assert.Equal(t, []string{"text"}, atts[2].MrkdwnIn)
assert.Equal(t, expectedResolvedBody, atts[2].Text)
})
t.Run("default template with fields and actions", func(t *testing.T) {
@@ -498,49 +499,45 @@ func TestPrepareContent(t *testing.T) {
// prepareContent does not populate fields/actions — that's done by
// addFieldsAndActions which is called from Notify.
require.Nil(t, atts[0].Fields)
require.Nil(t, atts[0].Actions)
assert.Nil(t, atts[0].Fields)
assert.Nil(t, atts[0].Actions)
// Simulate what Notify does after prepareContent
notifier.addFieldsAndActions(&atts[0], tmplText)
// Verify fields
require.Len(t, atts[0].Fields, 2)
require.Equal(t, "Severity", atts[0].Fields[0].Title)
require.Equal(t, "critical", atts[0].Fields[0].Value)
require.True(t, *atts[0].Fields[0].Short)
require.Equal(t, "Service", atts[0].Fields[1].Title)
require.Equal(t, "api-server", atts[0].Fields[1].Value)
assert.Equal(t, "Severity", atts[0].Fields[0].Title)
assert.Equal(t, "critical", atts[0].Fields[0].Value)
require.NotNil(t, atts[0].Fields[0].Short)
assert.True(t, *atts[0].Fields[0].Short)
assert.Equal(t, "Service", atts[0].Fields[1].Title)
assert.Equal(t, "api-server", atts[0].Fields[1].Value)
// Verify actions
require.Len(t, atts[0].Actions, 1)
require.Equal(t, "button", atts[0].Actions[0].Type)
require.Equal(t, "View Alert", atts[0].Actions[0].Text)
require.Equal(t, "https://alertmanager.signoz.com", atts[0].Actions[0].URL)
assert.Equal(t, "button", atts[0].Actions[0].Type)
assert.Equal(t, "View Alert", atts[0].Actions[0].Text)
assert.Equal(t, "https://alertmanager.signoz.com", atts[0].Actions[0].URL)
})
}
func TestSlackMessageField(t *testing.T) {
// 1. Setup a fake Slack server
// 1. Setup a fake Slack server. The handler runs on the server's
// goroutine, so only assert (never require) is safe here.
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatal(err)
}
assert.NoError(t, json.NewDecoder(r.Body).Decode(&body))
// 2. VERIFY: Top-level text exists
if body["text"] != "My Top Level Message" {
t.Errorf("Expected top-level 'text' to be 'My Top Level Message', got %v", body["text"])
}
assert.Equal(t, "My Top Level Message", body["text"])
// 3. VERIFY: Old attachments still exist
attachments, ok := body["attachments"].([]any)
if !ok || len(attachments) == 0 {
t.Errorf("Expected attachments to exist")
} else {
first := attachments[0].(map[string]any)
if first["title"] != "Old Attachment Title" {
t.Errorf("Expected attachment title 'Old Attachment Title', got %v", first["title"])
if assert.True(t, ok, "expected attachments to exist") && assert.NotEmpty(t, attachments) {
first, ok := attachments[0].(map[string]any)
if assert.True(t, ok, "expected attachment to be an object") {
assert.Equal(t, "Old Attachment Title", first["title"])
}
}
@@ -561,21 +558,16 @@ func TestSlackMessageField(t *testing.T) {
}
tmpl, err := template.FromGlobs([]string{})
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
tmpl.ExternalURL = u
logger := slog.New(slog.DiscardHandler)
notifier, err := New(conf, tmpl, logger, newTestTemplater(tmpl))
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
ctx := context.Background()
ctx = notify.WithGroupKey(ctx, "test-group-key")
if _, err := notifier.Notify(ctx); err != nil {
t.Fatal("Notify failed:", err)
}
_, err = notifier.Notify(ctx)
require.NoError(t, err, "Notify failed")
}

View File

@@ -19,6 +19,7 @@ import (
commoncfg "github.com/prometheus/common/config"
"github.com/prometheus/common/model"
"github.com/prometheus/common/promslog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/pkg/alertmanager/alertmanagertemplate"
@@ -39,14 +40,12 @@ func TestWebhookRetry(t *testing.T) {
promslog.NewNopLogger(),
alertmanagertemplate.New(tmpl, slog.Default()),
)
if err != nil {
require.NoError(t, err)
}
require.NoError(t, err)
t.Run("test retry status code", func(t *testing.T) {
for statusCode, expected := range test.RetryTests(test.DefaultRetryCodes()) {
actual, _ := notifier.retrier.Check(statusCode, nil)
require.Equal(t, expected, actual, "error on status %d", statusCode)
assert.Equal(t, expected, actual, "error on status %d", statusCode)
}
})
@@ -73,7 +72,8 @@ func TestWebhookRetry(t *testing.T) {
} {
t.Run("", func(t *testing.T) {
_, err = notifier.retrier.Check(tc.status, tc.body)
require.Equal(t, tc.exp, err.Error())
require.Error(t, err)
assert.Equal(t, tc.exp, err.Error())
})
}
})
@@ -83,16 +83,16 @@ func TestWebhookTruncateAlerts(t *testing.T) {
alerts := make([]*types.Alert, 10)
truncatedAlerts, numTruncated := truncateAlerts(0, alerts)
require.Len(t, truncatedAlerts, 10)
require.EqualValues(t, 0, numTruncated)
assert.Len(t, truncatedAlerts, 10)
assert.EqualValues(t, 0, numTruncated)
truncatedAlerts, numTruncated = truncateAlerts(4, alerts)
require.Len(t, truncatedAlerts, 4)
require.EqualValues(t, 6, numTruncated)
assert.Len(t, truncatedAlerts, 4)
assert.EqualValues(t, 6, numTruncated)
truncatedAlerts, numTruncated = truncateAlerts(100, alerts)
require.Len(t, truncatedAlerts, 10)
require.EqualValues(t, 0, numTruncated)
assert.Len(t, truncatedAlerts, 10)
assert.EqualValues(t, 0, numTruncated)
}
func TestWebhookRedactedURL(t *testing.T) {
@@ -219,10 +219,10 @@ func TestWebhookURLTemplating(t *testing.T) {
if tc.expectError {
require.Error(t, err)
require.Contains(t, err.Error(), tc.expectedErrMsg)
assert.Contains(t, err.Error(), tc.expectedErrMsg)
} else {
require.NoError(t, err)
require.Equal(t, tc.expectedPath, calledURL)
assert.Equal(t, tc.expectedPath, calledURL)
}
})
}

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24px" height="24px" viewBox="0 0 24 24"><defs><style>.cls-1{fill:#aecbfa;}.cls-1,.cls-2,.cls-3{fill-rule:evenodd;}.cls-2{fill:#669df6;}.cls-3{fill:#4285f4;}</style></defs><title>Icon_24px_SQL_Color</title><g data-name="Product Icons"><g ><polygon class="cls-1" points="4.67 10.44 4.67 13.45 12 17.35 12 14.34 4.67 10.44"/><polygon class="cls-1" points="4.67 15.09 4.67 18.1 12 22 12 18.99 4.67 15.09"/><polygon class="cls-2" points="12 17.35 19.33 13.45 19.33 10.44 12 14.34 12 17.35"/><polygon class="cls-2" points="12 22 19.33 18.1 19.33 15.09 12 18.99 12 22"/><polygon class="cls-3" points="19.33 8.91 19.33 5.9 12 2 12 5.01 19.33 8.91"/><polygon class="cls-2" points="12 2 4.67 5.9 4.67 8.91 12 5.01 12 2"/><polygon class="cls-1" points="4.67 5.87 4.67 8.89 12 12.79 12 9.77 4.67 5.87"/><polygon class="cls-2" points="12 12.79 19.33 8.89 19.33 5.87 12 9.77 12 12.79"/></g></g></svg>

After

Width:  |  Height:  |  Size: 933 B

View File

@@ -0,0 +1,136 @@
{
"id": "cloudsql_mysql",
"title": "GCP Cloud SQL for MySQL",
"icon": "file://icon.svg",
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": true
},
"dataCollected": {
"metrics": [
{
"name": "cloudsql.googleapis.com/database/up",
"unit": "Count",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/instance_state",
"unit": "Count",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/replication/replica_lag",
"unit": "Seconds",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/cpu/utilization",
"unit": "Percent",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/memory/utilization",
"unit": "Percent",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/disk/utilization",
"unit": "Percent",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/network/connections",
"unit": "Count",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/queries",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/dml_operations_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/threads",
"unit": "Count",
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/innodb/buffer_pool_reads_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/innodb/buffer_pool_read_requests_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/slow_queries_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/aborted_connects_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/innodb/deadlocks_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/mysql/innodb/row_lock_waits_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/disk/read_ops_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/disk/write_ops_count",
"unit": "Count",
"type": "Sum",
"description": ""
}
],
"logs": []
},
"telemetryCollectionStrategy": {
"gcp": {}
},
"assets": {
"dashboards": [
{
"id": "overview",
"title": "GCP Cloud SQL for MySQL Overview",
"description": "Overview of GCP Cloud SQL for MySQL metrics",
"definition": "file://assets/dashboards/overview.json"
}
]
}
}

View File

@@ -0,0 +1,3 @@
### Monitor GCP Cloud SQL for MySQL with SigNoz
Collect key GCP Cloud SQL for MySQL metrics and view them with an out of the box dashboard.

View File

@@ -784,40 +784,57 @@
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"kind": "signoz/CompositeQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
"queries": [
{
"metricName": "cloudsql.googleapis.com/database/cpu/utilization",
"temporality": "",
"timeAggregation": "max",
"spaceAggregation": "max",
"reduceTo": "avg"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND database_id in $database_id AND gcp.resource_type = 'cloudsql_database' "
},
"groupBy": [
"type": "builder_query",
"spec": {
"name": "A",
"stepInterval": 0,
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "cloudsql.googleapis.com/database/cpu/utilization",
"temporality": "",
"timeAggregation": "max",
"spaceAggregation": "max",
"reduceTo": ""
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND database_id in $database_id AND gcp.resource_type = 'cloudsql_database' "
},
"groupBy": [
{
"name": "database_id",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": null,
"selectFields": null,
"secondaryAggregations": null,
"functions": null,
"legend": "{{database_id}}"
}
},
{
"name": "database_id",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
"type": "builder_formula",
"spec": {
"name": "F1",
"expression": "100 * A",
"disabled": false,
"order": null,
"functions": null,
"legend": "{{database_id}}"
}
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{database_id}}"
]
}
}
}
@@ -1417,4 +1434,4 @@
"refreshInterval": "",
"links": []
}
}
}

View File

@@ -5,6 +5,7 @@ import (
"context"
"fmt"
"log/slog"
"math"
"regexp"
"sort"
"strings"
@@ -478,11 +479,19 @@ func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began ti
for idx := range v.Floats {
p := v.Floats[idx]
// NaN and +/-Inf have no JSON number form and nothing to plot; the
// builder path drops them while scanning rows (see consume.go).
if math.IsNaN(p.F) || math.IsInf(p.F, 0) {
continue
}
s.Values = append(s.Values, &qbv5.TimeSeriesValue{
Timestamp: p.T,
Value: p.F,
})
}
if len(s.Values) == 0 {
continue
}
series = append(series, &s)
}
@@ -494,13 +503,11 @@ func (q *promqlQuery) toResult(matrix promql.Matrix, warnings []string, began ti
}
statsMu.Unlock()
tsData := &qbv5.TimeSeriesData{
QueryName: q.query.Name,
Aggregations: []*qbv5.AggregationBucket{
{
Series: series,
},
},
tsData := &qbv5.TimeSeriesData{QueryName: q.query.Name}
// No bucket at all when nothing survived: a bucket holding no series reads
// as "filtered to empty" to the cache, which stores it as a real result.
if len(series) > 0 {
tsData.Aggregations = []*qbv5.AggregationBucket{{Series: series}}
}
var payload any = tsData

View File

@@ -2,14 +2,21 @@ package querier
import (
"log/slog"
"math"
"strings"
"sync"
"testing"
"time"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/promql"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/prometheus/prometheustest"
qbv5 "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRemoveAllVarMatchers(t *testing.T) {
@@ -453,3 +460,82 @@ func TestFingerprint_PinnedProviderBypassesCache(t *testing.T) {
}
assert.Empty(t, q.Fingerprint())
}
func TestToResultDropsNonFiniteValues(t *testing.T) {
tests := []struct {
description string
floats []promql.FPoint
expectedTimestamps []int64
expectedValues []float64
}{
{
description: "finite values pass through untouched",
floats: []promql.FPoint{{T: 1000, F: 1.5}, {T: 2000, F: 2.5}},
expectedTimestamps: []int64{1000, 2000},
expectedValues: []float64{1.5, 2.5},
},
{
description: "a ratio's 0/0 points are dropped, the rest kept",
floats: []promql.FPoint{{T: 1000, F: 1.5}, {T: 2000, F: math.NaN()}, {T: 3000, F: 2.5}},
expectedTimestamps: []int64{1000, 3000},
expectedValues: []float64{1.5, 2.5},
},
{
description: "both infinities are dropped",
floats: []promql.FPoint{{T: 1000, F: math.Inf(1)}, {T: 2000, F: 4.5}, {T: 3000, F: math.Inf(-1)}},
expectedTimestamps: []int64{2000},
expectedValues: []float64{4.5},
},
}
for _, test := range tests {
t.Run(test.description, func(t *testing.T) {
q := &promqlQuery{query: qbv5.PromQuery{Name: "A"}, requestType: qbv5.RequestTypeTimeSeries}
matrix := promql.Matrix{{Metric: labels.FromStrings("job_name", "dbBloatMonitorJob"), Floats: test.floats}}
var mu sync.Mutex
var rows, bytes uint64
result := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes)
tsData, ok := result.Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
require.Len(t, tsData.Aggregations, 1)
require.Len(t, tsData.Aggregations[0].Series, 1)
timestamps := make([]int64, 0, len(test.expectedTimestamps))
values := make([]float64, 0, len(test.expectedValues))
for _, v := range tsData.Aggregations[0].Series[0].Values {
timestamps = append(timestamps, v.Timestamp)
values = append(values, v.Value)
}
assert.Equal(t, test.expectedTimestamps, timestamps)
assert.Equal(t, test.expectedValues, values)
})
}
}
// A series left with nothing must not surface as an empty series, and a result
// left with no series must carry no aggregation bucket at all — the cache reads
// a bucket holding no series as a real, filtered-to-empty result and stores it.
func TestToResultDropsSeriesAndBucketLeftEmpty(t *testing.T) {
q := &promqlQuery{query: qbv5.PromQuery{Name: "A"}, requestType: qbv5.RequestTypeTimeSeries}
matrix := promql.Matrix{
{Metric: labels.FromStrings("job_name", "idleJob"), Floats: []promql.FPoint{{T: 1000, F: math.NaN()}}},
{Metric: labels.FromStrings("job_name", "activeJob"), Floats: []promql.FPoint{{T: 1000, F: 7.5}}},
}
var mu sync.Mutex
var rows, bytes uint64
tsData, ok := q.toResult(matrix, nil, time.Now(), &mu, &rows, &bytes).Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
require.Len(t, tsData.Aggregations, 1)
require.Len(t, tsData.Aggregations[0].Series, 1, "the all-NaN series is gone")
assert.Equal(t, "activeJob", tsData.Aggregations[0].Series[0].Labels[0].Value)
allNaN := promql.Matrix{
{Metric: labels.FromStrings("job_name", "idleJob"), Floats: []promql.FPoint{{T: 1000, F: math.NaN()}}},
}
tsData, ok = q.toResult(allNaN, nil, time.Now(), &mu, &rows, &bytes).Value.(*qbv5.TimeSeriesData)
require.True(t, ok)
assert.Empty(t, tsData.Aggregations)
}

View File

@@ -993,8 +993,8 @@ func TestBuild_TraceList_MultiVariantGateKey(t *testing.T) {
assert.Contains(t, got, "mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_number, 'gen_ai.tool.name')")
}
// A `trace.`-prefixed aggregate in the filter box and the same condition in the
// explicit Having box build the same query; output-only aggregates are rejected.
// `trace.` marks a trace-level aggregate; `tracefield.` routes trace-level too but is
// not a rewritable alias, so the HAVING rewriter rejects it.
func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
b := newTestBuilder(t)
build := func(q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (*qbtypes.Statement, error) {
@@ -1002,14 +1002,19 @@ func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, q, nil)
}
viaTrace, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
_, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"}})
require.NoError(t, err)
viaHaving, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Having: &qbtypes.Having{Expression: "trace.output_tokens > 1000"}})
require.NoError(t, err)
assert.Equal(t, viaTrace.Query, viaHaving.Query)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
assert.Contains(t, err.Error(), "Invalid references in `Having` expression: [tracefield.output_tokens]")
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Having: &qbtypes.Having{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
assert.Contains(t, err.Error(), "Invalid references in `Having` expression: [tracefield.output_tokens]")
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"}})
@@ -1017,8 +1022,7 @@ func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
assert.Contains(t, err.Error(), "cannot be used")
}
// Query variables in a trace-level condition resolve like span filters: bound args,
// list/IN handling, dynamic __all__ dropping the condition.
// Query variables in a trace-level condition are substituted into the HAVING.
func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
b := newTestBuilder(t)
build := func(expr string, vars map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
@@ -1030,18 +1034,17 @@ func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
}, vars)
}
// scalar variable -> bound arg via the filter pipeline
// scalar variable -> literal in HAVING
stmt, err := build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Value: 700}})
require.NoError(t, err)
assert.Contains(t, stmt.Query, "HAVING output_tokens > ?")
assert.Contains(t, stmt.Args, float64(700))
assert.Contains(t, stmt.Query, "HAVING output_tokens > 700")
// list variable with IN
stmt, err = build("trace.llm_call_count IN $counts",
map[string]qbtypes.VariableItem{"counts": {Value: []any{1, 2}}})
require.NoError(t, err)
assert.Contains(t, stmt.Query, "HAVING llm_call_count IN (?, ?)")
assert.Contains(t, stmt.Query, "HAVING llm_call_count IN")
// dynamic __all__ -> condition dropped, no HAVING at all
stmt, err = build("trace.output_tokens > $threshold",
@@ -1049,7 +1052,7 @@ func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
require.NoError(t, err)
assert.NotContains(t, stmt.Query, "HAVING")
// unresolved variable -> rejected, though only as an unknown aggregate today
// unresolved variable -> rejected, not compared as a literal
_, err = build("trace.output_tokens > $missing", map[string]qbtypes.VariableItem{"other": {Value: 1}})
require.Error(t, err)
}

View File

@@ -1,781 +0,0 @@
package aistatementbuilder
import (
"context"
"testing"
"time"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// The builder assumes at least one aggregation; request validation is what enforces it.
func TestBuild_Aggregation_NoAggregations_RejectedByRequestValidation(t *testing.T) {
for _, rt := range []qbtypes.RequestType{qbtypes.RequestTypeScalar, qbtypes.RequestTypeTimeSeries} {
req := qbtypes.QueryRangeRequest{
Start: testStartMs,
End: testEndMs,
RequestType: rt,
CompositeQuery: qbtypes.CompositeQuery{
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilderAI,
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
},
}},
},
}
require.ErrorContains(t, req.Validate(), "at least one aggregation is required", rt.StringValue())
}
}
// Traces without token spans yield NULL, which the outer avg skips.
func TestBuild_FullSQL_Scalar_TraceAgg(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A span-level filter is ANDed into the per-trace scan's WHERE, next to the gate mask.
func TestBuild_FullSQL_Scalar_SpanFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini'"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A trace-level filter qualifies first: __qualified holds the trace ids whose
// whole-window value passes, and the per-trace scan is constrained to them.
func TestBuild_FullSQL_Scalar_TraceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
),
__scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A group-by column is grouped in the per-trace scan too, so a trace spanning two
// models contributes one per-trace row per model.
func TestBuild_FullSQL_Scalar_GroupBy(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model
)
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, avg(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Grouping by an intrinsic: the positional alias keeps `toString(name) AS name` (a cyclic
// alias) from forming, and an order key on the dimension resolves to that alias.
func TestBuild_FullSQL_Scalar_GroupByIntrinsic(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "name"}}},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "name"}}, Direction: qbtypes.OrderDirectionAsc}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
toString(multiIf(name <> '', toString(name), NULL)) AS __GROUP_BY_KEY_0_name,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, __GROUP_BY_KEY_0_name
)
SELECT __GROUP_BY_KEY_0_name, avg(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY __GROUP_BY_KEY_0_name
ORDER BY __GROUP_BY_KEY_0_name asc
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Every dimension at once; the HAVING on the alias is rewritten to __result_0.
func TestBuild_FullSQL_Scalar_FullCombo(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{
{Expression: "avg(trace.output_tokens)", Alias: "avg_out"},
{Expression: "count(trace.trace_id)"},
},
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND trace.total_tokens > 100"},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}}},
Having: &qbtypes.Having{Expression: "avg_out > 50"},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "avg_out"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 5,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING total_tokens > 100
),
__scoped_traces AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model
)
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, avg(output_tokens) AS __result_0, count(trace_id) AS __result_1
FROM __scoped_traces
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model
HAVING __result_0 > 50
ORDER BY __result_0 desc
LIMIT 5
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Time series: the per-trace scan buckets by span time, the outer aggregation per bucket.
func TestBuild_FullSQL_TimeSeries_TraceAgg(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, ts
)
SELECT ts, avg(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY ts
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A grouped, limited time series ranks groups on unbucketed whole-window values
// (__scoped_traces_total), so a non-composable aggregate like avg ranks exactly.
func TestBuild_FullSQL_TimeSeries_GroupLimit(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "sum(trace.output_tokens)", Alias: "total_out"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}}},
Having: &qbtypes.Having{Expression: "total_out > 500"},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "total_out"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 3,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces_total AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model
),
__limit_cte AS (
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, sum(output_tokens) AS __result_0
FROM __scoped_traces_total
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model
ORDER BY __result_0 desc
LIMIT 3
),
__scoped_traces AS (
SELECT trace_id,
toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL))) GLOBAL IN (SELECT __GROUP_BY_KEY_0_gen_ai.request.model FROM __limit_cte)
GROUP BY trace_id, ts, __GROUP_BY_KEY_0_gen_ai.request.model
)
SELECT ts, __GROUP_BY_KEY_0_gen_ai.request.model, sum(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY ts, __GROUP_BY_KEY_0_gen_ai.request.model
HAVING __result_0 > 500
ORDER BY ts desc
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A span-level scalar delegates to the trace builder, constrained by __trace_scope;
// the shape is the delegate's own, hence no SETTINGS suffix.
func TestBuild_FullSQL_Scalar_SpanAgg_TraceScoped(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "sum(gen_ai.usage.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __trace_scope AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
)
SELECT sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS __result_0
FROM signoz_traces.distributed_signoz_index_v3
WHERE trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
ORDER BY __result_0 DESC
`, stmt)
}
// Two group keys make the top-N prune a 2-tuple GLOBAL IN, and the qualification plus
// span predicate apply to the ranking scan and the main scan alike.
func TestBuild_FullSQL_TimeSeries_GroupLimit_MultiKey(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{
{Expression: "sum(trace.output_tokens)"},
{Expression: "count(trace.trace_id)"},
},
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND trace.total_tokens > 100"},
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}},
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.user.id"}},
},
Limit: 2,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING total_tokens > 100
),
__scoped_traces_total AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
toString(multiIf(mapContains(attributes_string, 'gen_ai.user.id'), attributes_string['gen_ai.user.id'], NULL)) AS __GROUP_BY_KEY_1_gen_ai.user.id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
),
__limit_cte AS (
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id, sum(output_tokens) AS __result_0, count(trace_id) AS __result_1
FROM __scoped_traces_total
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
ORDER BY __result_0 DESC
LIMIT 2
),
__scoped_traces AS (
SELECT trace_id,
toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
toString(multiIf(mapContains(attributes_string, 'gen_ai.user.id'), attributes_string['gen_ai.user.id'], NULL)) AS __GROUP_BY_KEY_1_gen_ai.user.id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
AND (toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)), toString(multiIf(mapContains(attributes_string, 'gen_ai.user.id'), attributes_string['gen_ai.user.id'], NULL))) GLOBAL IN (SELECT __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id FROM __limit_cte)
GROUP BY trace_id, ts, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
)
SELECT ts, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id, sum(output_tokens) AS __result_0, count(trace_id) AS __result_1
FROM __scoped_traces
GROUP BY ts, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A time-series limit without group-by has nothing to rank: it is ignored, matching
// the trace builder — the query equals its unlimited form.
func TestBuild_TimeSeries_LimitWithoutGroupByIgnored(t *testing.T) {
b := newTestBuilder(t)
build := func(limit int) *qbtypes.Statement {
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Limit: limit,
}, nil)
require.NoError(t, err)
return stmt
}
assert.Equal(t, build(0).Query, build(5).Query)
}
// ---------------------------------------------------------------------------
// Behavior / branch tests not covered by the goldens above
// ---------------------------------------------------------------------------
// Mixing span- and trace-level aggregations across one query is rejected.
func TestBuild_Aggregation_MixedDomainsRejected(t *testing.T) {
b := newTestBuilder(t)
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{
{Expression: "avg(trace.output_tokens)"},
{Expression: "sum(gen_ai.usage.output_tokens)"},
},
}, nil)
require.ErrorContains(t, err, "cannot be mixed")
}
// Output-only aggregates are rejected in trace-level filters on the aggregation
// path too (the raw and trace-list paths are covered elsewhere).
func TestBuild_Aggregation_OutputOnlyFilterRejected(t *testing.T) {
b := newTestBuilder(t)
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "count()"}},
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"},
}, nil)
require.ErrorContains(t, err, `aggregate "span_count" cannot be used`)
}
// Trace-level columns are rejected as group-by keys; order keys never reach the builder,
// since request validation only admits group keys and aggregation aliases/expressions.
func TestBuild_Aggregation_GroupByOrderValidation(t *testing.T) {
b := newTestBuilder(t)
ctx := context.Background()
_, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.llm_call_count"}}},
}, nil)
require.ErrorContains(t, err, `grouping by trace-level aggregate "trace.llm_call_count" is not supported`)
req := qbtypes.QueryRangeRequest{
Start: testStartMs,
End: testEndMs,
RequestType: qbtypes.RequestTypeScalar,
CompositeQuery: qbtypes.CompositeQuery{
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilderAI,
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.total_tokens"}}, Direction: qbtypes.OrderDirectionDesc}},
},
}},
},
}
require.ErrorContains(t, req.Validate(), "invalid order by key")
_, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)", Alias: "avg_out"}},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "avg_out"}}, Direction: qbtypes.OrderDirectionAsc}},
}, nil)
require.NoError(t, err)
}
// Variables in trace-level conditions resolve as bound args; a dynamic __all__ drops the
// condition, and an unresolved $var is rejected only as an unknown aggregate today.
func TestBuild_FullSQL_Aggregation_VariablesInTraceFilter(t *testing.T) {
b := newTestBuilder(t)
ctx := context.Background()
q := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > $threshold"},
}
stmt, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q,
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.TextBoxVariableType, Value: float64(1000)}})
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
),
__scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
// an unresolved $var is only rejected as an unknown aggregate today; a targeted
// "unknown variable" error is a separate concern
_, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q, nil)
require.ErrorContains(t, err, `aggregate "$threshold" cannot be used`)
// __all__ drops the condition: the query equals its unfiltered form
stmt, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q,
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.DynamicVariableType, Value: "__all__"}})
require.NoError(t, err)
unfiltered := q
unfiltered.Filter = nil
want, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, unfiltered, nil)
require.NoError(t, err)
assert.Equal(t, want.Query, stmt.Query)
// list variables render as IN with bound args; the scan selects only trace_id
// since no aggregation touches a per-trace column
stmt, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "count(trace.trace_id)"}},
Filter: &qbtypes.Filter{Expression: "trace.llm_call_count IN $counts"},
}, map[string]qbtypes.VariableItem{
"counts": {Type: qbtypes.QueryVariableType, Value: []any{float64(1), float64(2)}},
})
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING llm_call_count IN (1, 2)
),
__scoped_traces AS (
SELECT trace_id
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT count(trace_id) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Resource conditions on the native path: the __resource_filter CTE prunes the
// qualification scan and the per-trace scan by fingerprint.
func TestBuild_FullSQL_Aggregation_ResourceFilter_Native(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "service.name = 'api' AND trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __resource_filter AS (
SELECT fingerprint
FROM signoz_traces.distributed_traces_v3_resource
WHERE (simpleJSONExtractString(labels, 'service.name') = 'api' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"api%')
AND seen_at_ts_bucket_start >= 1747945619
AND seen_at_ts_bucket_start <= 1747983448
GROUP BY fingerprint
),
__qualified AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
GROUP BY trace_id
HAVING output_tokens > 1000
),
__scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// On the delegated path __trace_scope inlines its fingerprint subquery, since it is built
// without the delegate's CTEs, while the delegate keeps its own __resource_filter CTE.
func TestBuild_FullSQL_Aggregation_ResourceFilter_Delegated(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "sum(gen_ai.usage.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "service.name = 'api' AND trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __resource_filter AS (
SELECT fingerprint
FROM signoz_traces.distributed_traces_v3_resource
WHERE ((simpleJSONExtractString(labels, 'service.name') = 'api' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"api%'))
AND seen_at_ts_bucket_start >= 1747945619
AND seen_at_ts_bucket_start <= 1747983448
GROUP BY fingerprint
),
__trace_scope AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = 'api' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"api%') AND seen_at_ts_bucket_start >= 1747945619 AND seen_at_ts_bucket_start <= 1747983448 GROUP BY fingerprint))
GROUP BY trace_id
HAVING output_tokens > 1000
)
SELECT sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS __result_0
FROM signoz_traces.distributed_signoz_index_v3
WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
AND trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)
AND (((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AND ((multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = 'api' AND multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL)))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
ORDER BY __result_0 DESC
`, stmt)
}
// rate() divides by the window (scalar) / step (series). Per AggreFuncMap it counts
// per-trace rows per second; it does not sum the column.
func TestBuild_Aggregation_RateDividesByInterval(t *testing.T) {
b := newTestBuilder(t)
ctx := context.Background()
q := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "rate(trace.llm_call_count)"}},
}
stmt, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q, nil)
require.NoError(t, err)
assert.Contains(t, stmt.Query, "count(llm_call_count)/36029 AS __result_0") // (end-start) seconds
q.StepInterval = qbtypes.Step{Duration: 60 * time.Second}
stmt, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries, q, nil)
require.NoError(t, err)
assert.Contains(t, stmt.Query, "count(llm_call_count)/60 AS __result_0")
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"log/slog"
"sort"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
@@ -18,6 +19,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
qbvariables "github.com/SigNoz/signoz/pkg/variables"
"github.com/huandu/go-sqlbuilder"
)
@@ -113,8 +115,6 @@ func (b *scopedTraceStatementBuilder) Build(
return b.buildTraceListQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), query, variables)
case qbtypes.RequestTypeRaw:
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
case qbtypes.RequestTypeScalar, qbtypes.RequestTypeTimeSeries:
return b.buildAggregation(ctx, orgID, start, end, requestType, query, variables)
default:
return nil, ErrUnsupportedRequestType
}
@@ -143,61 +143,6 @@ func (b *scopedTraceStatementBuilder) buildDelegated(
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
// traceScopedStatementBuilder is the delegate's optional capability of constraining a
// query to a set of trace ids (implemented by the traces statement builder).
type traceScopedStatementBuilder interface {
BuildTraceScoped(ctx context.Context, orgID valuer.UUID, start, end uint64, requestType qbtypes.RequestType, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], variables map[string]qbtypes.VariableItem, traceScope *qbtypes.Statement) (*qbtypes.Statement, error)
}
// buildDelegatedAggregation serves span-level scalar/time-series through the standard
// trace builder, with the gate ANDed into the span-level filter part; a trace-level
// part becomes a qualification the delegate constrains trace_id by.
func (b *scopedTraceStatementBuilder) buildDelegatedAggregation(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
var spanExpr, traceExpr string
var err error
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
spanExpr, traceExpr, err = querybuilder.SplitFilterForAggregates(query.Filter.Expression, b.aggregateAliasSet())
if err != nil {
return nil, err
}
}
gate := b.scope.FilterExpression
expr := gate
if strings.TrimSpace(spanExpr) != "" {
expr = fmt.Sprintf("(%s) AND (%s)", gate, spanExpr)
}
// shallow copy; only Filter is replaced, caller's query untouched
gated := query
gated.Filter = &qbtypes.Filter{Expression: expr}
if strings.TrimSpace(traceExpr) == "" {
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
scoped, ok := b.traceStmtBuilder.(traceScopedStatementBuilder)
if !ok {
return nil, errors.NewInternalf(errors.CodeInternal, "trace statement builder does not support trace-scoped queries")
}
scope, err := b.buildQualifiedStatement(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), traceExpr, query, variables)
if err != nil {
return nil, err
}
if scope == nil {
// every trace-level condition was dropped by variable resolution
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
return scoped.BuildTraceScoped(ctx, orgID, start, end, requestType, gated, variables, scope)
}
// buildTraceListQuery wires the CTE pipeline (start/end are nanoseconds):
// matched (windowed, mask-pruned top-N trace_ids) → ranked (their [start,end] from
// the summary table) → buckets (ts_bucket_start prune) → enrichment (every per-trace
@@ -239,17 +184,22 @@ func (b *scopedTraceStatementBuilder) buildTraceListQuery(
if err != nil {
return nil, err
}
orderableSet := orderableAliasSet(resolved)
resourceFrag, resourceArgs, resourcePred, err := b.maybeAttachResourceFilter(ctx, orgID, query, start, end, variables)
if err != nil {
return nil, err
}
fp, err := b.splitFilter(ctx, orgID, query, b.aggregateAliasSet(), start, end, variables, matchedSB)
fp, err := b.splitFilter(ctx, orgID, query, b.aggregateAliasSet(), orderableSet, start, end, variables, matchedSB)
if err != nil {
return nil, err
}
matchedFrag, matchedArgs := b.buildMatchedCTE(matchedSB, start, end, startBucket, endBucket, resolved, orders, maskExpr, fp, resourcePred, limit, query.Offset)
matchedFrag, matchedArgs, err := b.buildMatchedCTE(matchedSB, start, end, startBucket, endBucket, resolved, orders, orderableSet, maskExpr, fp, resourcePred, limit, query.Offset)
if err != nil {
return nil, err
}
rankedFrag, rankedArgs := b.buildRankedCTE(start, end)
adj := querybuilder.BucketAdjustment // 30-min bucket width in seconds
@@ -430,27 +380,27 @@ func (b *scopedTraceStatementBuilder) resolveListOrders(order []qbtypes.OrderBy,
return orders, nil
}
// filterParts is the user filter split into a span-level predicate and the resolved
// trace-level HAVING (nil when there is none).
// filterParts is the user filter split into a span-level predicate and a trace-level
// HAVING expression.
type filterParts struct {
spanPred string
hasSpanFilter bool
having *traceHaving
havingExpr string
warnings []string
warningsURL string
}
// splitFilter splits query.Filter into a span-level predicate and a trace-level
// HAVING (explicit query.Having ANDed on before resolution); args bind into sb.
func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID valuer.UUID, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], classifySet map[string]struct{}, start, end uint64, variables map[string]qbtypes.VariableItem, sb *sqlbuilder.SelectBuilder) (filterParts, error) {
// splitFilter splits query.Filter into a span-level predicate (args bound into sb)
// and a trace-level HAVING (explicit query.Having ANDed on), then validates the
// trace-level part against the matched-pass aggregates.
func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID valuer.UUID, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], classifySet, orderableSet map[string]struct{}, start, end uint64, variables map[string]qbtypes.VariableItem, sb *sqlbuilder.SelectBuilder) (filterParts, error) {
var fp filterParts
havingExpr := ""
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
spanExpr, traceExpr, err := querybuilder.SplitFilterForAggregates(query.Filter.Expression, classifySet)
if err != nil {
return fp, err
}
havingExpr = traceExpr
fp.havingExpr = traceExpr
if strings.TrimSpace(spanExpr) != "" {
pred, warnings, url, err := b.resolveSpanPredicate(ctx, orgID, start, end, spanExpr, variables, sb)
if err != nil {
@@ -465,17 +415,23 @@ func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID val
}
}
if query.Having != nil && strings.TrimSpace(query.Having.Expression) != "" {
if havingExpr != "" {
havingExpr = fmt.Sprintf("(%s) AND (%s)", havingExpr, query.Having.Expression)
if fp.havingExpr != "" {
fp.havingExpr = fmt.Sprintf("(%s) AND (%s)", fp.havingExpr, query.Having.Expression)
} else {
havingExpr = query.Having.Expression
fp.havingExpr = query.Having.Expression
}
}
having, err := b.resolveTraceHaving(ctx, havingExpr, variables, sb)
if err != nil {
// the HAVING is a plain text rewrite, so substitute variables here
if strings.TrimSpace(fp.havingExpr) != "" && len(variables) > 0 {
replaced, err := qbvariables.ReplaceVariablesInExpression(fp.havingExpr, variables)
if err != nil {
return fp, err
}
fp.havingExpr = replaced
}
if err := validateAggregateFilter(fp.havingExpr, orderableSet); err != nil {
return fp, err
}
fp.having = having
return fp, nil
}
@@ -517,8 +473,8 @@ func (b *scopedTraceStatementBuilder) resolveSpanPredicate(ctx context.Context,
// span filter + HAVING + ORDER BY + LIMIT/OFFSET, selecting only the aliases ORDER BY
// / HAVING reference. Expressions carry $n markers bound to sb, so each can appear
// several times and every occurrence resolves to the same arg.
func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuilder, start, end, startBucket, endBucket uint64, resolved []resolvedColumn, orders []listOrder, maskExpr string, fp filterParts, resourcePred string, limit, offset int) (string, []any) {
needed := neededMatchedAliases(orders, fp.having)
func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuilder, start, end, startBucket, endBucket uint64, resolved []resolvedColumn, orders []listOrder, orderableSet map[string]struct{}, maskExpr string, fp filterParts, resourcePred string, limit, offset int) (string, []any, error) {
needed := neededMatchedAliases(orders, fp.havingExpr, orderableSet)
selects := []string{"trace_id"}
for _, rc := range resolved {
if _, ok := needed[rc.alias]; !ok {
@@ -555,8 +511,22 @@ func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuild
having = append(having, "countIf("+maskExpr+") > 0")
having = append(having, "countIf("+fp.spanPred+") > 0")
}
if fp.having != nil {
having = append(having, fp.having.pred)
if strings.TrimSpace(fp.havingExpr) != "" {
// the rewriter matches raw key text, so map the trace. form alongside the bare name
columnMap := make(map[string]string, len(orderableSet)*2)
for a := range orderableSet {
columnMap[a] = quoteAlias(a)
columnMap[telemetrytypes.FieldContextTrace.StringValue()+"."+a] = quoteAlias(a)
}
hv, err := querybuilder.NewHavingExpressionRewriter().Rewrite(fp.havingExpr, columnMap)
if err != nil {
return "", nil, err
}
if hv != "" {
// escape user text so a literal $ isn't read as an arg marker; the countIf
// entries hold live $n markers and must stay unescaped
having = append(having, sqlbuilder.Escape(hv))
}
}
if len(having) > 0 {
sb.Having(strings.Join(having, " AND "))
@@ -569,7 +539,7 @@ func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuild
}
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("matched AS (%s)", sql), args
return fmt.Sprintf("matched AS (%s)", sql), args, nil
}
// buildRankedCTE builds `ranked`: [start,end] bounds per matched trace from the
@@ -610,9 +580,8 @@ func (b *scopedTraceStatementBuilder) buildEnrichmentSelect(sb *sqlbuilder.Selec
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
// aggregateAliasSet recognises trace-level keys — display-only aliases included, so one
// gets a targeted error instead of falling through as a span attribute (what a predicate
// may actually use is orderableColumnSet). SpanLevel columns are filtered span-level.
// aggregateAliasSet is every trace-level column alias, used to classify filter keys;
// SpanLevel columns are filtered span-level, so skip them.
func (b *scopedTraceStatementBuilder) aggregateAliasSet() map[string]struct{} {
set := make(map[string]struct{}, len(b.scope.Columns))
for _, c := range b.scope.Columns {
@@ -623,36 +592,59 @@ func (b *scopedTraceStatementBuilder) aggregateAliasSet() map[string]struct{} {
return set
}
// orderableAliasSet is the subset of aliases computable in the matched pass.
func orderableAliasSet(resolved []resolvedColumn) map[string]struct{} {
set := make(map[string]struct{})
for _, rc := range resolved {
if rc.orderable {
set[rc.alias] = struct{}{}
}
}
return set
}
// neededMatchedAliases is the minimal alias set the matched pass must select: those
// in ORDER BY plus those the resolved trace-level HAVING touches.
func neededMatchedAliases(orders []listOrder, having *traceHaving) map[string]struct{} {
// in ORDER BY plus those in the aggregate HAVING.
func neededMatchedAliases(orders []listOrder, havingExpr string, orderableSet map[string]struct{}) map[string]struct{} {
needed := make(map[string]struct{})
for _, o := range orders {
needed[o.alias] = struct{}{}
}
if having != nil {
for name := range having.used {
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; ok {
needed[name] = struct{}{}
}
}
return needed
}
// validateAggregateFilter rejects filters on aggregates not computable in the matched
// pass (e.g. span_count) upfront, since inside the where-clause visitor the error would
// surface only as a detail of a combined one. Only unspecified- and trace-context
// selectors name aggregates.
// traceAggregateNames extracts the aggregate names a trace-level HAVING references;
// only unspecified- and trace-context selectors name aggregates.
func traceAggregateNames(havingExpr string) []string {
var names []string
for _, sel := range querybuilder.QueryStringToKeysSelectors(havingExpr) {
if sel.FieldContext == telemetrytypes.FieldContextUnspecified || sel.FieldContext == telemetrytypes.FieldContextTrace {
names = append(names, sel.Name)
}
}
return names
}
// validateAggregateFilter rejects a trace-level filter referencing an aggregate not
// computable in the matched pass.
func validateAggregateFilter(havingExpr string, orderableSet map[string]struct{}) error {
if strings.TrimSpace(havingExpr) == "" {
return nil
}
for _, sel := range querybuilder.QueryStringToKeysSelectors(havingExpr) {
if sel.FieldContext != telemetrytypes.FieldContextUnspecified && sel.FieldContext != telemetrytypes.FieldContextTrace {
continue
}
if _, ok := orderableSet[sel.Name]; !ok {
allowed := make([]string, 0, len(orderableSet))
for a := range orderableSet {
allowed = append(allowed, a)
}
sort.Strings(allowed)
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregate %q cannot be used in a trace-level filter; filterable aggregates: %s", sel.Name, strings.Join(sortedAliases(orderableSet), ", "))
"aggregate %q cannot be used in the trace-list filter; filterable aggregates: %s", name, strings.Join(allowed, ", "))
}
}
return nil

View File

@@ -1,791 +0,0 @@
package scopedtracesstatementbuilder
import (
"context"
"fmt"
"sort"
"strings"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
// The per-trace values these aggregations read are window-clipped and span-filtered,
// unlike the list's enrichment pass over every span of the whole trace, so the same
// column reads differently in each.
// traceAggregation is one aggregation rewritten to run over the per-trace scan.
type traceAggregation struct {
expr string // rewritten SQL over the per-trace column aliases
used map[string]struct{} // per-trace aliases referenced
isRate bool
}
// buildAggregation routes by aggregation domain: bare keys delegate to the standard
// trace builder, trace.-prefixed aggregates run over the per-trace scan.
func (b *scopedTraceStatementBuilder) buildAggregation(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
traceAggs, err := b.classifyAggregations(query.Aggregations)
if err != nil {
return nil, err
}
if err := b.validateGroupBy(query); err != nil {
return nil, err
}
if len(traceAggs) == 0 {
return b.buildDelegatedAggregation(ctx, orgID, start, end, requestType, query, variables)
}
return b.buildTraceAggregationQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), requestType, query, variables, traceAggs)
}
// classifyAggregations returns the rewritten trace-domain aggregations, nil when all
// are span-domain; mixing the two domains is rejected.
func (b *scopedTraceStatementBuilder) classifyAggregations(aggs []qbtypes.TraceAggregation) ([]traceAggregation, error) {
// permission, not recognition: unknown names are reported against exactly this set
traceCols := b.orderableColumnSet()
var out []traceAggregation
spanCount := 0
for _, agg := range aggs {
ta, isTrace, err := rewriteTraceAggregation(agg.Expression, traceCols)
if err != nil {
return nil, err
}
if isTrace {
out = append(out, *ta)
} else {
spanCount++
}
}
if len(out) > 0 && spanCount > 0 {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"span-level and trace-level (trace.) aggregations cannot be mixed in one query")
}
return out, nil
}
// orderableColumnSet is what a trace-level aggregation or filter predicate may use;
// recognising a key as trace-level is aggregateAliasSet's job.
func (b *scopedTraceStatementBuilder) orderableColumnSet() map[string]struct{} {
set := make(map[string]struct{})
for _, c := range b.scope.Columns {
if c.Orderable {
set[c.Alias] = struct{}{}
}
}
return set
}
// validateGroupBy rejects trace-level columns as group-by keys with a targeted error
// (not the field mapper's generic "field not found"). Order keys need no check here:
// request validation only admits group keys and aggregation aliases/expressions.
func (b *scopedTraceStatementBuilder) validateGroupBy(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) error {
// recognition, not permission: a display-only alias must be named here to be rejected
// rather than reaching the field mapper as a span attribute
aliases := b.aggregateAliasSet()
for _, gb := range query.GroupBy {
key := gb.TelemetryFieldKey
key.Normalize()
// a bare name may be a span column sharing the alias (duration_nano, timestamp)
if key.FieldContext != telemetrytypes.FieldContextTrace {
continue
}
if _, ok := aliases[key.Name]; ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"grouping by trace-level aggregate %q is not supported; group by span attributes instead (e.g. service.name)", gb.Name)
}
}
return nil
}
// rewriteTraceAggregation rewrites an aggregation over trace.-prefixed columns to run
// on the per-trace scan (trace.output_tokens → output_tokens, functions mapped via
// AggreFuncMap); a pure span-level expression returns isTrace=false for the delegate.
func rewriteTraceAggregation(expr string, traceCols map[string]struct{}) (*traceAggregation, bool, error) {
p := chparser.NewParser("SELECT " + expr)
stmts, err := p.ParseStmts()
if err != nil {
return nil, false, errors.WrapInvalidInputf(err, errors.CodeInvalidInput, "failed to parse aggregation expression %q", expr)
}
if len(stmts) == 0 {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid aggregation expression %q", expr)
}
sel, ok := stmts[0].(*chparser.SelectQuery)
if !ok || len(sel.SelectItems) == 0 {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid aggregation expression %q", expr)
}
v := &traceAggVisitor{traceCols: traceCols, used: make(map[string]struct{})}
if err := sel.SelectItems[0].Accept(v); err != nil {
return nil, false, err
}
if !v.hasTrace {
return nil, false, nil
}
if v.hasSpan {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregation %q mixes trace-level (trace.) and span-level columns; use one domain per aggregation", expr)
}
// the interval divides the rendered expression as a whole, so a second aggregation
// alongside the rate would be divided too
if v.isRate && v.aggCount > 1 {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregation %q combines a rate with another aggregation; the rate interval would divide both, so give each its own aggregation", expr)
}
return &traceAggregation{expr: chparser.Format(sel.SelectItems[0]), used: v.used, isRate: v.isRate}, true, nil
}
// traceAggVisitor classifies column references and rewrites trace.-prefixed ones in
// place; the ancestor stack tells a column identifier from a path segment, function
// name, or alias, and rejects trace. columns inside *If combinators.
type traceAggVisitor struct {
chparser.DefaultASTVisitor
traceCols map[string]struct{}
used map[string]struct{}
stack []chparser.Expr
aggCount int
hasTrace bool
hasSpan bool
isRate bool
}
func (v *traceAggVisitor) Enter(expr chparser.Expr) { v.stack = append(v.stack, expr) }
func (v *traceAggVisitor) Leave(expr chparser.Expr) { v.stack = v.stack[:len(v.stack)-1] }
// parent is the node enclosing the one currently being visited (the visited node
// itself is the stack top).
func (v *traceAggVisitor) parent() chparser.Expr {
if len(v.stack) < 2 {
return nil
}
return v.stack[len(v.stack)-2]
}
func (v *traceAggVisitor) enclosingCombinator() (string, bool) {
for _, e := range v.stack {
fn, ok := e.(*chparser.FunctionExpr)
if !ok {
continue
}
if agg, known := querybuilder.AggreFuncMap[valuer.NewString(strings.ToLower(fn.Name.Name))]; known && agg.FuncCombinator {
return fn.Name.Name, true
}
}
return "", false
}
// enclosingAggregate walks the ancestor stack; AggreFuncMap holds only aggregates and
// VisitFunctionExpr rejects any name missing from it, so a known name is enough.
func (v *traceAggVisitor) enclosingAggregate() bool {
for _, e := range v.stack {
fn, ok := e.(*chparser.FunctionExpr)
if !ok {
continue
}
if _, known := querybuilder.AggreFuncMap[valuer.NewString(strings.ToLower(fn.Name.Name))]; known {
return true
}
}
return false
}
// VisitPath classifies a dotted reference (trace.output_tokens); trace-level ones are
// rewritten in place to the bare per-trace alias.
func (v *traceAggVisitor) VisitPath(p *chparser.Path) error {
col, isTrace := traceColumnFromPath(p)
if !isTrace {
v.hasSpan = true
return nil
}
if err := v.acceptTraceColumn(chparser.Format(p), col); err != nil {
return err
}
p.Fields = p.Fields[len(p.Fields)-1:]
p.Fields[0].Name = col
return nil
}
// VisitIdent classifies a plain identifier (a backquoted `trace.output_tokens` is
// trace-level); path segments, function names, and aliases are structural, not columns.
func (v *traceAggVisitor) VisitIdent(i *chparser.Ident) error {
switch parent := v.parent().(type) {
case *chparser.Path:
return nil // segments are classified whole by VisitPath
case *chparser.FunctionExpr:
if parent.Name == i {
return nil
}
case *chparser.ColumnExpr:
if parent.Alias == i {
return nil
}
}
// the parser hands us one identifier, so the prefix is the only text to cut here
col, isTrace := strings.CutPrefix(i.Name, telemetrytypes.FieldContextTrace.StringValue()+".")
if !isTrace || col == "" {
v.hasSpan = true
return nil
}
if err := v.acceptTraceColumn(i.Name, col); err != nil {
return err
}
i.Name = col
return nil
}
func (v *traceAggVisitor) acceptTraceColumn(ref, col string) error {
if name, in := v.enclosingCombinator(); in {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"%q over trace-level (trace.) columns is not supported; put the trace-level condition in the filter expression instead", name)
}
// trace_id is always selected by the per-trace scan (count(trace.trace_id)
// counts traces); everything else must be a scope column.
if col != "trace_id" {
if _, known := v.traceCols[col]; !known {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"unknown trace-level aggregation column %q; usable columns: %s", ref, strings.Join(sortedAliases(v.traceCols), ", "))
}
v.used[col] = struct{}{}
}
// ungrouped, a bare per-trace column would make the outer SELECT emit one row per
// trace instead of one aggregated row
if !v.enclosingAggregate() {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level column %q must be inside an aggregation function (e.g. avg(%s))", ref, ref)
}
v.hasTrace = true
return nil
}
// VisitFunctionExpr validates and maps the function name. Children were already
// visited (post-order), so classification is complete for this subtree.
func (v *traceAggVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
name := strings.ToLower(fn.Name.Name)
aggFunc, ok := querybuilder.AggreFuncMap[valuer.NewString(name)]
if !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unrecognized function: %s", name)
}
if fn.Params != nil && fn.Params.Items != nil && len(fn.Params.Items.Items) > 0 && aggFunc.FuncCombinator {
// combinator predicates over span columns stay span-level (countIf(has_error=true))
v.hasSpan = true
return nil
}
fn.Name.Name = aggFunc.FuncName
v.aggCount++
if aggFunc.Rate {
v.isRate = true
}
return nil
}
// traceColumnFromPath returns the per-trace column a dotted reference names
// (trace.output_tokens -> output_tokens, trace.a.b -> a.b).
func traceColumnFromPath(p *chparser.Path) (string, bool) {
if len(p.Fields) < 2 || p.Fields[0].Name != telemetrytypes.FieldContextTrace.StringValue() {
return "", false
}
segments := make([]string, 0, len(p.Fields)-1)
for _, f := range p.Fields[1:] {
segments = append(segments, f.Name)
}
return strings.Join(segments, "."), true
}
func sortedAliases(set map[string]struct{}) []string {
out := make([]string, 0, len(set))
for a := range set {
out = append(out, a)
}
sort.Strings(out)
return out
}
// ---------------------------------------------------------------------------
// Qualification + per-trace scan
// ---------------------------------------------------------------------------
// buildQualifiedStatement selects the trace ids whose window-clipped aggregates satisfy
// the trace-level filter, with the resource prune inlined since the caller embeds this
// standalone. start/end are ns; nil when variable resolution dropped every condition.
func (b *scopedTraceStatementBuilder) buildQualifiedStatement(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
traceExpr string,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
keys, err := b.fetchKeys(ctx, orgID)
if err != nil {
return nil, err
}
sb := sqlbuilder.NewSelectBuilder()
maskExpr, resolved, err := b.resolveFor(ctx, orgID, start, end, keys, sb)
if err != nil {
return nil, err
}
having, err := b.resolveTraceHaving(ctx, traceExpr, variables, sb)
if err != nil {
return nil, err
}
if having == nil {
return nil, nil //nolint:nilnil
}
var resourcePred string
// nil when the filter has no resource-attribute conditions
if stmt, err := b.resourceFilterStmtBuilder.Build(ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables); err != nil {
return nil, err
} else if stmt != nil {
inlined, err := embedExpr(sb, stmt.Query, stmt.Args)
if err != nil {
return nil, err
}
resourcePred = fmt.Sprintf("resource_fingerprint GLOBAL IN (SELECT fingerprint FROM (%s))", inlined)
}
sql, args := b.buildPerTraceScan(sb, start, end, resolved, maskExpr, perTraceScanOpts{
needed: having.used,
havingPred: having.pred,
resourcePred: resourcePred,
})
return &qbtypes.Statement{Query: sql, Args: args}, nil
}
// embedExpr inlines a pre-built statement into sb, replacing each `?` with a builder
// Var; a count mismatch would silently shift args into the wrong slots, so error out.
func embedExpr(sb *sqlbuilder.SelectBuilder, expr string, args []any) (string, error) {
if n := strings.Count(expr, "?"); n != len(args) {
return "", errors.NewInternalf(errors.CodeInternal,
"scoped trace builder: %d placeholders != %d args embedding %q", n, len(args), expr)
}
var out strings.Builder
ai := 0
for i := 0; i < len(expr); i++ {
if expr[i] == '?' {
out.WriteString(sb.Var(args[ai]))
ai++
continue
}
out.WriteByte(expr[i])
}
return out.String(), nil
}
// groupColumn holds a resolved, arg-free span-attribute expression.
type groupColumn struct {
alias string
expr string
}
// groupByColumnAlias prefixes the i-th group-by dimension so the alias cannot shadow the
// span column its expression reads; the querier (stripKeyAlias) strips it back off.
func groupByColumnAlias(i int, name string) string {
return fmt.Sprintf("__GROUP_BY_KEY_%d_%s", i, name)
}
// orderColumn is the SQL identifier a non-aggregation order key sorts by: the
// positional alias when the key names a group-by dimension, else the key itself.
func orderColumn(orderKey string, groupBy []qbtypes.GroupByKey) string {
for i := range groupBy {
if groupBy[i].Name == orderKey {
return groupByColumnAlias(i, groupBy[i].Name)
}
}
return orderKey
}
// perTraceScanOpts parametrize one windowed, mask-pruned GROUP BY trace_id scan.
// All expressions are already resolved against the scan's builder.
type perTraceScanOpts struct {
stepSeconds int64 // >0 → bucket per-trace values by time (ts column)
groupCols []groupColumn
needed map[string]struct{} // per-trace aliases to select
spanPred string // resolved span-level filter, ANDed per span
resourcePred string // resource-fingerprint prune (CTE reference or inline subquery)
qualified bool // constrain to __qualified
limitPred string // top-N group prune (GLOBAL IN __limit_cte)
havingPred string // resolved HAVING predicate over the selected aliases
}
func (b *scopedTraceStatementBuilder) buildPerTraceScan(sb *sqlbuilder.SelectBuilder, start, end uint64, resolved []resolvedColumn, maskExpr string, o perTraceScanOpts) (string, []any) {
startBucket := start/querybuilder.NsToSeconds - querybuilder.BucketAdjustment
endBucket := end / querybuilder.NsToSeconds
selects := []string{"trace_id"}
if o.stepSeconds > 0 {
selects = append(selects, fmt.Sprintf("toStartOfInterval(timestamp, INTERVAL %d SECOND) AS ts", o.stepSeconds))
}
for _, gc := range o.groupCols {
selects = append(selects, fmt.Sprintf("toString(%s) AS `%s`", gc.expr, gc.alias))
}
for _, rc := range resolved {
if _, ok := o.needed[rc.alias]; !ok {
continue
}
selects = append(selects, rc.expr+" AS "+quoteAlias(rc.alias))
}
sb.Select(selects...)
sb.From(fmt.Sprintf("%s.%s", tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName))
where := []string{
sb.GE("timestamp", fmt.Sprintf("%d", start)),
sb.L("timestamp", fmt.Sprintf("%d", end)),
sb.GE("ts_bucket_start", startBucket),
sb.LE("ts_bucket_start", endBucket),
maskExpr,
}
if strings.TrimSpace(o.spanPred) != "" {
where = append(where, o.spanPred)
}
if o.resourcePred != "" {
where = append(where, o.resourcePred)
}
if o.qualified {
where = append(where, "trace_id GLOBAL IN (SELECT trace_id FROM __qualified)")
}
if o.limitPred != "" {
where = append(where, o.limitPred)
}
sb.Where(where...)
groupBy := []string{"trace_id"}
if o.stepSeconds > 0 {
groupBy = append(groupBy, "ts")
}
for _, gc := range o.groupCols {
groupBy = append(groupBy, "`"+gc.alias+"`")
}
sb.GroupBy(groupBy...)
if strings.TrimSpace(o.havingPred) != "" {
sb.Having(o.havingPred)
}
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
// resolveGroupColumns resolves group-by keys through the field mapper, which needs the
// metadata keys, for selection inside the per-trace scan.
func (b *scopedTraceStatementBuilder) resolveGroupColumns(ctx context.Context, orgID valuer.UUID, start, end uint64, groupBy []qbtypes.GroupByKey) ([]groupColumn, error) {
if len(groupBy) == 0 {
return nil, nil
}
selectors := make([]*telemetrytypes.FieldKeySelector, 0, len(groupBy))
for i := range groupBy {
selectors = append(selectors, &telemetrytypes.FieldKeySelector{
Name: groupBy[i].Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: groupBy[i].FieldContext,
FieldDataType: groupBy[i].FieldDataType,
SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact,
})
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
if err != nil {
return nil, err
}
out := make([]groupColumn, 0, len(groupBy))
for i := range groupBy {
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &groupBy[i].TelemetryFieldKey, telemetrytypes.FieldDataTypeString, keys)
if err != nil {
return nil, err
}
out = append(out, groupColumn{alias: groupByColumnAlias(i, groupBy[i].Name), expr: sqlbuilder.Escape(expr)})
}
return out, nil
}
// ---------------------------------------------------------------------------
// Native trace-domain aggregation query
// ---------------------------------------------------------------------------
// scanContext is one per-scan resolution: a fresh builder with the mask, columns,
// span predicate, and optionally the trace-level HAVING resolved against it.
type scanContext struct {
sb *sqlbuilder.SelectBuilder
maskExpr string
resolved []resolvedColumn
spanPred string
having *traceHaving
warnings []string
warnURL string
}
func (b *scopedTraceStatementBuilder) newScanContext(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
spanExpr, traceExpr string,
variables map[string]qbtypes.VariableItem,
) (*scanContext, error) {
sc := &scanContext{sb: sqlbuilder.NewSelectBuilder()}
var err error
sc.maskExpr, sc.resolved, err = b.resolveFor(ctx, orgID, start, end, keys, sc.sb)
if err != nil {
return nil, err
}
if strings.TrimSpace(spanExpr) != "" {
pred, warns, url, err := b.resolveSpanPredicate(ctx, orgID, start, end, spanExpr, variables, sc.sb)
if err != nil {
return nil, err
}
sc.spanPred, sc.warnings, sc.warnURL = pred, warns, url
}
if strings.TrimSpace(traceExpr) != "" {
sc.having, err = b.resolveTraceHaving(ctx, traceExpr, variables, sc.sb)
if err != nil {
return nil, err
}
}
return sc, nil
}
// buildTraceAggregationQuery aggregates over the per-trace scan: __qualified (when the
// filter has a trace-level part) → __scoped_traces → outer aggregation. start/end are ns.
func (b *scopedTraceStatementBuilder) buildTraceAggregationQuery(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
traceAggs []traceAggregation,
) (*qbtypes.Statement, error) {
keys, err := b.fetchKeys(ctx, orgID)
if err != nil {
return nil, err
}
var spanExpr, traceExpr string
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
// the broad set so a condition on a display-only alias still lands in the
// trace-level part, where resolveTraceHaving rejects it by name
spanExpr, traceExpr, err = querybuilder.SplitFilterForAggregates(query.Filter.Expression, b.aggregateAliasSet())
if err != nil {
return nil, err
}
}
resourceFrag, resourceArgs, resourcePred, err := b.maybeAttachResourceFilter(ctx, orgID, query, start, end, variables)
if err != nil {
return nil, err
}
var cteFragments []string
var cteArgs [][]any
if resourceFrag != "" {
cteFragments = append(cteFragments, resourceFrag)
cteArgs = append(cteArgs, resourceArgs)
}
// __qualified: its own scan resolution, HAVING = the trace-level filter part
qualified := false
if strings.TrimSpace(traceExpr) != "" {
qsc, err := b.newScanContext(ctx, orgID, start, end, keys, "", traceExpr, variables)
if err != nil {
return nil, err
}
if qsc.having != nil {
qsql, qargs := b.buildPerTraceScan(qsc.sb, start, end, qsc.resolved, qsc.maskExpr, perTraceScanOpts{
needed: qsc.having.used,
havingPred: qsc.having.pred,
resourcePred: resourcePred,
})
cteFragments = append(cteFragments, fmt.Sprintf("__qualified AS (%s)", qsql))
cteArgs = append(cteArgs, qargs)
qualified = true
}
}
groupCols, err := b.resolveGroupColumns(ctx, orgID, start, end, query.GroupBy)
if err != nil {
return nil, err
}
groupNames := make([]string, 0, len(groupCols))
for _, gc := range groupCols {
groupNames = append(groupNames, "`"+gc.alias+"`")
}
needed := make(map[string]struct{})
for _, ta := range traceAggs {
for a := range ta.used {
needed[a] = struct{}{}
}
}
stepSeconds := int64(0)
rateInterval := (end - start) / querybuilder.NsToSeconds
if requestType == qbtypes.RequestTypeTimeSeries {
stepSeconds = int64(query.StepInterval.Seconds())
rateInterval = uint64(stepSeconds)
}
// outer aggregation over the per-trace rows
sb := sqlbuilder.NewSelectBuilder()
selects := []string{}
if stepSeconds > 0 {
selects = append(selects, "ts")
}
selects = append(selects, groupNames...)
for i, ta := range traceAggs {
selects = append(selects, fmt.Sprintf("%s AS __result_%d", ta.rendered(rateInterval), i))
}
sb.Select(selects...)
sb.From("__scoped_traces")
// grouped, limited time series → rank groups on whole-window per-trace values
// (exact for non-composable aggregates) and prune the main scan to the top-N.
limitPred := ""
if requestType == qbtypes.RequestTypeTimeSeries && query.Limit > 0 && len(groupCols) > 0 {
tsc, err := b.newScanContext(ctx, orgID, start, end, keys, spanExpr, "", variables)
if err != nil {
return nil, err
}
totalSQL, totalArgs := b.buildPerTraceScan(tsc.sb, start, end, tsc.resolved, tsc.maskExpr, perTraceScanOpts{
groupCols: groupCols,
needed: needed,
spanPred: tsc.spanPred,
resourcePred: resourcePred,
qualified: qualified,
})
cteFragments = append(cteFragments, fmt.Sprintf("__scoped_traces_total AS (%s)", totalSQL))
cteArgs = append(cteArgs, totalArgs)
limitSQL, limitArgs := outerLimitSQL(query, traceAggs, groupNames, (end-start)/querybuilder.NsToSeconds)
cteFragments = append(cteFragments, fmt.Sprintf("__limit_cte AS (%s)", limitSQL))
cteArgs = append(cteArgs, limitArgs)
exprs := make([]string, 0, len(groupCols))
for _, gc := range groupCols {
exprs = append(exprs, "toString("+gc.expr+")")
}
limitPred = fmt.Sprintf("(%s) GLOBAL IN (SELECT %s FROM __limit_cte)",
strings.Join(exprs, ", "), strings.Join(groupNames, ", "))
}
msc, err := b.newScanContext(ctx, orgID, start, end, keys, spanExpr, "", variables)
if err != nil {
return nil, err
}
perTraceSQL, perTraceArgs := b.buildPerTraceScan(msc.sb, start, end, msc.resolved, msc.maskExpr, perTraceScanOpts{
stepSeconds: stepSeconds,
groupCols: groupCols,
needed: needed,
spanPred: msc.spanPred,
resourcePred: resourcePred,
qualified: qualified,
limitPred: limitPred,
})
cteFragments = append(cteFragments, fmt.Sprintf("__scoped_traces AS (%s)", perTraceSQL))
cteArgs = append(cteArgs, perTraceArgs)
groupBys := []string{}
if stepSeconds > 0 {
groupBys = append(groupBys, "ts")
}
groupBys = append(groupBys, groupNames...)
if len(groupBys) > 0 {
sb.GroupBy(groupBys...)
}
if query.Having != nil && strings.TrimSpace(query.Having.Expression) != "" {
rewritten, err := querybuilder.NewHavingExpressionRewriter().RewriteForTraces(query.Having.Expression, query.Aggregations)
if err != nil {
return nil, err
}
sb.Having(sqlbuilder.Escape(rewritten))
}
if requestType == qbtypes.RequestTypeTimeSeries {
if len(query.Order) != 0 {
for _, orderBy := range query.Order {
if _, ok := traceAggOrderIndex(orderBy, query); !ok {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderColumn(orderBy.Key.Name, query.GroupBy), orderBy.Direction.StringValue()))
}
}
sb.OrderBy("ts desc")
}
} else {
for _, orderBy := range query.Order {
if idx, ok := traceAggOrderIndex(orderBy, query); ok {
sb.OrderBy(fmt.Sprintf("__result_%d %s", idx, orderBy.Direction.StringValue()))
} else {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderColumn(orderBy.Key.Name, query.GroupBy), orderBy.Direction.StringValue()))
}
}
if len(query.Order) == 0 {
sb.OrderBy("__result_0 DESC")
}
if query.Limit > 0 {
sb.Limit(query.Limit)
}
}
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
finalSQL := querybuilder.CombineCTEs(cteFragments) + mainSQL + " SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000"
finalArgs := querybuilder.PrependArgs(cteArgs, mainArgs)
return &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
Warnings: msc.warnings,
WarningsDocURL: msc.warnURL,
}, nil
}
// rendered divides a rate aggregation by the interval (step for time series, window
// length for scalar); the divisor applies to the whole expression, which holds only
// because a rate must be the sole aggregation.
func (ta traceAggregation) rendered(rateInterval uint64) string {
if ta.isRate {
return fmt.Sprintf("%s/%d", ta.expr, rateInterval)
}
return ta.expr
}
// outerLimitSQL ranks groups on whole-window per-trace values, so a non-composable
// aggregate (avg) ranks exactly rather than over bucketed rows.
func outerLimitSQL(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], traceAggs []traceAggregation, groupNames []string, windowSeconds uint64) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
selects := append([]string{}, groupNames...)
for i, ta := range traceAggs {
selects = append(selects, fmt.Sprintf("%s AS __result_%d", ta.rendered(windowSeconds), i))
}
sb.Select(selects...)
sb.From("__scoped_traces_total")
sb.GroupBy(groupNames...)
for _, orderBy := range query.Order {
if idx, ok := traceAggOrderIndex(orderBy, query); ok {
sb.OrderBy(fmt.Sprintf("__result_%d %s", idx, orderBy.Direction.StringValue()))
} else {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderColumn(orderBy.Key.Name, query.GroupBy), orderBy.Direction.StringValue()))
}
}
if len(query.Order) == 0 {
sb.OrderBy("__result_0 DESC")
}
sb.Limit(query.Limit)
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
// traceAggOrderIndex reports whether an order key refers to the i-th aggregation
// (by alias, expression, or index), mirroring the trace builder.
func traceAggOrderIndex(k qbtypes.OrderBy, q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (int, bool) {
for i, agg := range q.Aggregations {
if k.Key.Name == agg.Alias ||
k.Key.Name == agg.Expression ||
k.Key.Name == fmt.Sprintf("%d", i) {
return i, true
}
}
return 0, false
}

View File

@@ -1,75 +0,0 @@
package scopedtracesstatementbuilder
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRewriteTraceAggregation(t *testing.T) {
cols := map[string]struct{}{
"input_tokens": {}, "output_tokens": {}, "total_tokens": {}, "llm_call_count": {}, "max_llm_latency_ns": {},
}
cases := []struct {
name string
expr string
isTrace bool
want string // rewritten expr, only checked when isTrace
used []string
wantErr string
}{
{name: "avg trace col", expr: "avg(trace.output_tokens)", isTrace: true, want: "avg(output_tokens)", used: []string{"output_tokens"}},
{name: "sum trace col", expr: "sum(trace.total_tokens)", isTrace: true, want: "sum(total_tokens)", used: []string{"total_tokens"}},
{name: "count traces", expr: "count(trace.trace_id)", isTrace: true, want: "count(trace_id)"},
{name: "p90 trace col", expr: "p90(trace.max_llm_latency_ns)", isTrace: true, want: "quantile(0.90)(max_llm_latency_ns)", used: []string{"max_llm_latency_ns"}},
{name: "arithmetic between trace cols", expr: "avg(trace.output_tokens + trace.input_tokens)", isTrace: true, want: "avg(output_tokens + input_tokens)", used: []string{"output_tokens", "input_tokens"}},
{name: "arithmetic with constant", expr: "sum(trace.output_tokens * 1.5)", isTrace: true, want: "sum(output_tokens * 1.5)", used: []string{"output_tokens"}},
{name: "ratio of two aggregations", expr: "sum(trace.output_tokens)/count(trace.trace_id)", isTrace: true, want: "sum(output_tokens) / count(trace_id)", used: []string{"output_tokens"}},
{name: "backquoted trace col", expr: "avg(`trace.output_tokens`)", isTrace: true, want: "avg(`output_tokens`)", used: []string{"output_tokens"}},
{name: "bare count is span-level", expr: "count()", isTrace: false},
{name: "span attribute is span-level", expr: "sum(gen_ai.usage.output_tokens)", isTrace: false},
{name: "countIf span predicate is span-level", expr: "countIf(has_error = true)", isTrace: false},
{name: "mixed domains in one expression", expr: "sum(trace.output_tokens) + sum(gen_ai.usage.input_tokens)", wantErr: "mixes trace-level"},
{name: "mixed domains in one function", expr: "sum(trace.output_tokens + gen_ai.usage.input_tokens)", wantErr: "mixes trace-level"},
{name: "output-only column rejected", expr: "avg(trace.span_count)", wantErr: "unknown trace-level aggregation column"},
{name: "unknown column rejected", expr: "avg(trace.bogus)", wantErr: "unknown trace-level aggregation column"},
// a dotted column keeps every segment after the prefix, so it is reported whole
{name: "multi segment column rejected by full name", expr: "avg(trace.service.name)", wantErr: `"trace.service.name"`},
{name: "bare trace identifier is span-level", expr: "avg(trace)", isTrace: false},
{name: "countIf over trace col rejected", expr: "countIf(trace.output_tokens > 1000)", wantErr: "not supported"},
{name: "bare trace col rejected", expr: "trace.output_tokens", wantErr: "must be inside an aggregation function"},
{name: "backquoted bare trace col rejected", expr: "`trace.output_tokens`", wantErr: "must be inside an aggregation function"},
{name: "bare trace_id rejected", expr: "trace.trace_id", wantErr: "must be inside an aggregation function"},
{name: "arithmetic outside an aggregation rejected", expr: "trace.output_tokens + trace.input_tokens", wantErr: "must be inside an aggregation function"},
{name: "trace col beside an aggregation rejected", expr: "sum(trace.output_tokens) + trace.input_tokens", wantErr: "must be inside an aggregation function"},
{name: "aggregation scaled by a constant", expr: "sum(trace.output_tokens) * 2", isTrace: true, want: "sum(output_tokens) * 2", used: []string{"output_tokens"}},
{name: "rate over traces", expr: "rate(trace.trace_id)", isTrace: true, want: "count(trace_id)"},
{name: "rate_sum trace col", expr: "rate_sum(trace.output_tokens)", isTrace: true, want: "sum(output_tokens)", used: []string{"output_tokens"}},
// the interval divides the whole rendered expression, so a second aggregation
// alongside a rate would be divided too
{name: "rate mixed with another aggregation rejected", expr: "rate(trace.trace_id) + avg(trace.output_tokens)", wantErr: "combines a rate with another aggregation"},
{name: "ratio of two rates rejected", expr: "rate_sum(trace.output_tokens)/rate_sum(trace.input_tokens)", wantErr: "combines a rate with another aggregation"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
ta, isTrace, err := rewriteTraceAggregation(tc.expr, cols)
if tc.wantErr != "" {
require.ErrorContains(t, err, tc.wantErr)
return
}
require.NoError(t, err)
assert.Equal(t, tc.isTrace, isTrace)
if !tc.isTrace {
return
}
assert.Equal(t, tc.want, ta.expr)
for _, u := range tc.used {
assert.Contains(t, ta.used, u)
}
assert.Len(t, ta.used, len(tc.used))
})
}
}

View File

@@ -1,144 +0,0 @@
package scopedtracesstatementbuilder
import (
"context"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
qbvariables "github.com/SigNoz/signoz/pkg/variables"
"github.com/huandu/go-sqlbuilder"
)
// traceHaving is the resolved trace-level filter part: a HAVING predicate over the
// per-trace aliases plus the aliases it references (so scans select only those).
type traceHaving struct {
pred string
used map[string]struct{}
}
// resolveTraceHaving runs a trace-level filter through the standard where-clause
// pipeline against the per-trace aliases, so operators, bound args, and __all__ behave
// as in span filters. Returns nil when nothing is left to filter; args bind into sb.
func (b *scopedTraceStatementBuilder) resolveTraceHaving(ctx context.Context, expr string, variables map[string]qbtypes.VariableItem, sb *sqlbuilder.SelectBuilder) (*traceHaving, error) {
if strings.TrimSpace(expr) == "" {
return nil, nil //nolint:nilnil
}
// replaced before validation so variable literals are not mistaken for aggregate
// names; an unresolved $var is left in place and fails validation as an unknown one
if len(variables) > 0 {
replaced, err := qbvariables.ReplaceVariablesInExpression(expr, variables)
if err != nil {
return nil, err
}
expr = replaced
if strings.TrimSpace(expr) == "" {
return nil, nil //nolint:nilnil
}
}
allowed := b.orderableColumnSet()
// upfront targeted errors; the visitor folds them into a combined "Found N errors"
if err := validateAggregateFilter(expr, allowed); err != nil {
return nil, err
}
// both spellings resolve here: the key parser strips the trace. prefix into
// FieldContextTrace, which matches this entry's context
fieldKeys := make(map[string][]*telemetrytypes.TelemetryFieldKey, len(allowed))
for alias := range allowed {
key := &telemetrytypes.TelemetryFieldKey{Name: alias, FieldContext: telemetrytypes.FieldContextTrace}
fieldKeys[alias] = []*telemetrytypes.TelemetryFieldKey{key}
}
cb := &aliasConditionBuilder{allowed: allowed, used: make(map[string]struct{})}
prepared, err := querybuilder.PrepareWhereClause(expr, querybuilder.FilterExprVisitorOpts{
Context: ctx,
Logger: b.logger,
ConditionBuilder: cb,
FieldKeys: fieldKeys,
Variables: variables,
Builder: sb,
})
if err != nil {
return nil, err
}
if prepared.IsEmpty() {
return nil, nil //nolint:nilnil
}
return &traceHaving{pred: prepared.Expr, used: cb.used}, nil
}
// aliasConditionBuilder renders filter conditions directly against the per-trace
// aliases, recording the ones it touches; a key resolving to no alias is an error.
type aliasConditionBuilder struct {
allowed map[string]struct{}
used map[string]struct{}
}
var _ qbtypes.ConditionBuilder = (*aliasConditionBuilder)(nil)
func (c *aliasConditionBuilder) ConditionFor(
_ context.Context,
_ valuer.UUID,
_, _ uint64,
key *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
_ qbtypes.ConditionBuilderOptions,
op qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
matching := keys[key.Name]
if len(matching) == 0 {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregate %q cannot be used in a trace-level filter; filterable aggregates: %s",
key.Name, strings.Join(sortedAliases(c.allowed), ", "))
}
alias := matching[0].Name
c.used[alias] = struct{}{}
col := quoteAlias(alias)
var cond string
switch op {
case qbtypes.FilterOperatorEqual:
cond = sb.E(col, value)
case qbtypes.FilterOperatorNotEqual:
cond = sb.NE(col, value)
case qbtypes.FilterOperatorGreaterThan:
cond = sb.G(col, value)
case qbtypes.FilterOperatorGreaterThanOrEq:
cond = sb.GE(col, value)
case qbtypes.FilterOperatorLessThan:
cond = sb.L(col, value)
case qbtypes.FilterOperatorLessThanOrEq:
cond = sb.LE(col, value)
case qbtypes.FilterOperatorIn, qbtypes.FilterOperatorNotIn:
values, ok := value.([]any)
if !ok {
values = []any{value}
}
if op == qbtypes.FilterOperatorIn {
cond = sb.In(col, values...)
} else {
cond = sb.NotIn(col, values...)
}
case qbtypes.FilterOperatorBetween, qbtypes.FilterOperatorNotBetween:
values, ok := value.([]any)
if !ok || len(values) != 2 {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"between on trace-level aggregate %q requires exactly two values", alias)
}
if op == qbtypes.FilterOperatorBetween {
cond = sb.Between(col, values[0], values[1])
} else {
cond = sb.NotBetween(col, values[0], values[1])
}
default:
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level aggregate %q supports only comparison operators (=, !=, <, <=, >, >=, in, between)", alias)
}
return []string{cond}, nil, nil
}

View File

@@ -32,9 +32,6 @@ type traceQueryStatementBuilder struct {
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.TraceAggregation]
aggExprRewriter qbtypes.AggExprRewriter
skipResourceFingerprintEnabled bool
// traceScope, set only on the per-call copy made by BuildTraceScoped, constrains
// queries to spans whose trace_id is in the __trace_scope CTE.
traceScope *qbtypes.Statement
}
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*traceQueryStatementBuilder)(nil)
@@ -98,33 +95,6 @@ func NewTraceQueryStatementBuilder(
}
}
// BuildTraceScoped is Build constrained to trace_ids selected by traceScope; the
// receiver is copied so the shared builder stays stateless.
func (b *traceQueryStatementBuilder) BuildTraceScoped(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
traceScope *qbtypes.Statement,
) (*qbtypes.Statement, error) {
scoped := *b
scoped.traceScope = traceScope
return scoped.Build(ctx, orgID, start, end, requestType, query, variables)
}
// attachTraceScope adds the trace-scope condition to sb and returns the CTE fragment
// + args to prepend; both empty when no scope is set.
func (b *traceQueryStatementBuilder) attachTraceScope(sb *sqlbuilder.SelectBuilder) (string, []any) {
if b.traceScope == nil {
return "", nil
}
sb.Where("trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)")
return fmt.Sprintf("__trace_scope AS (%s)", b.traceScope.Query), b.traceScope.Args
}
// Build builds a SQL query for traces based on the given parameters.
func (b *traceQueryStatementBuilder) Build(
ctx context.Context,
@@ -549,11 +519,6 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
cteArgs = append(cteArgs, args)
}
if scopeFrag, scopeArgs := b.attachTraceScope(sb); scopeFrag != "" {
cteFragments = append(cteFragments, scopeFrag)
cteArgs = append(cteArgs, scopeArgs)
}
sb.SelectMore(fmt.Sprintf(
"toStartOfInterval(timestamp, INTERVAL %d SECOND) AS ts",
int64(query.StepInterval.Seconds()),
@@ -714,13 +679,6 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
cteArgs = append(cteArgs, args)
}
// skipResourceCTE means this scalar is embedded as a CTE of a time-series query,
// which has already emitted the __trace_scope fragment — add only the condition.
if scopeFrag, scopeArgs := b.attachTraceScope(sb); scopeFrag != "" && !skipResourceCTE {
cteFragments = append(cteFragments, scopeFrag)
cteArgs = append(cteArgs, scopeArgs)
}
allAggChArgs := []any{}
fieldNames := make([]string, 0, len(query.GroupBy))

View File

@@ -46,6 +46,7 @@ var (
GCPServiceComputeEngine = ServiceID{valuer.NewString("computeengine")}
GCPServiceGKE = ServiceID{valuer.NewString("gke")}
GCPServiceCloudStorage = ServiceID{valuer.NewString("cloudstorage")}
GCPServiceCloudSQLMySQL = ServiceID{valuer.NewString("cloudsql_mysql")}
)
func (ServiceID) Enum() []any {
@@ -82,6 +83,7 @@ func (ServiceID) Enum() []any {
GCPServiceComputeEngine,
GCPServiceGKE,
GCPServiceCloudStorage,
GCPServiceCloudSQLMySQL,
}
}
@@ -124,6 +126,7 @@ var SupportedServices = map[CloudProviderType][]ServiceID{
GCPServiceComputeEngine,
GCPServiceGKE,
GCPServiceCloudStorage,
GCPServiceCloudSQLMySQL,
},
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
@@ -31,11 +32,12 @@ func (enum *Signal) UnmarshalJSON(data []byte) error {
}
var (
SignalTraces = Signal{valuer.NewString("traces")}
SignalLogs = Signal{valuer.NewString("logs")}
SignalApiMonitoring = Signal{valuer.NewString("api_monitoring")}
SignalExceptions = Signal{valuer.NewString("exceptions")}
SignalMeter = Signal{valuer.NewString("meter")}
SignalTraces = Signal{valuer.NewString("traces")}
SignalLogs = Signal{valuer.NewString("logs")}
SignalApiMonitoring = Signal{valuer.NewString("api_monitoring")}
SignalExceptions = Signal{valuer.NewString("exceptions")}
SignalMeter = Signal{valuer.NewString("meter")}
SignalAiObservability = Signal{valuer.NewString("ai_observability")}
)
// NewSignal creates a Signal from a string.
@@ -51,6 +53,8 @@ func NewSignal(s string) (Signal, error) {
return SignalExceptions, nil
case "meter":
return SignalMeter, nil
case "ai_observability":
return SignalAiObservability, nil
default:
return Signal{}, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid signal: %s", s)
}
@@ -187,6 +191,18 @@ func NewDefaultQuickFilter(orgID valuer.UUID) ([]*StorableQuickFilter, error) {
{"key": "host.name", "dataType": "float64", "type": "Sum"},
}
// AI observability (builder_ai_query trace explorer), ordered by expected
// usage: env scoping, the LLM identity keys, then service and the rest.
aiObservabilityFilters := []map[string]interface{}{
{"key": "deployment.environment", "dataType": "string", "type": "resource"},
{"key": telemetrytypes.GenAIOperationName, "dataType": "string", "type": "tag"},
{"key": telemetrytypes.GenAIProviderName, "dataType": "string", "type": "tag"},
{"key": telemetrytypes.GenAIRequestModel, "dataType": "string", "type": "tag"},
{"key": "service.name", "dataType": "string", "type": "resource"},
{"key": telemetrytypes.GenAIToolName, "dataType": "string", "type": "tag"},
{"key": telemetrytypes.GenAIAgentName, "dataType": "string", "type": "tag"},
}
tracesJSON, err := json.Marshal(tracesFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal traces filters")
@@ -212,6 +228,11 @@ func NewDefaultQuickFilter(orgID valuer.UUID) ([]*StorableQuickFilter, error) {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal meter filters")
}
aiObservabilityJSON, err := json.Marshal(aiObservabilityFilters)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to marshal ai observability filters")
}
timeRightNow := time.Now()
return []*StorableQuickFilter{
@@ -275,5 +296,17 @@ func NewDefaultQuickFilter(orgID valuer.UUID) ([]*StorableQuickFilter, error) {
UpdatedAt: timeRightNow,
},
},
{
Identifiable: types.Identifiable{
ID: valuer.GenerateUUID(),
},
OrgID: orgID,
Filter: string(aiObservabilityJSON),
Signal: SignalAiObservability,
TimeAuditable: types.TimeAuditable{
CreatedAt: timeRightNow,
UpdatedAt: timeRightNow,
},
},
}, nil
}

View File

@@ -3,10 +3,11 @@ package telemetrytypes
// OpenTelemetry gen_ai semantic-convention attribute keys. Single source of truth
// shared by the AI query builder and the LLM pricing pipeline.
const (
GenAIRequestModel = "gen_ai.request.model"
GenAIToolName = "gen_ai.tool.name"
GenAIAgentName = "gen_ai.agent.name"
GenAIProviderName = "gen_ai.provider.name"
GenAIRequestModel = "gen_ai.request.model"
GenAIOperationName = "gen_ai.operation.name"
GenAIToolName = "gen_ai.tool.name"
GenAIAgentName = "gen_ai.agent.name"
GenAIProviderName = "gen_ai.provider.name"
GenAIUsageInputTokens = "gen_ai.usage.input_tokens"
GenAIUsageOutputTokens = "gen_ai.usage.output_tokens"
@@ -25,10 +26,11 @@ const (
// on, surfaced by the metadata store even before ingestion so the AI gate/columns
// resolve on a fresh install.
var GenAIFieldDefinitions = map[string]TelemetryFieldKey{
GenAIRequestModel: {Name: GenAIRequestModel, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIToolName: {Name: GenAIToolName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIAgentName: {Name: GenAIAgentName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIProviderName: {Name: GenAIProviderName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIRequestModel: {Name: GenAIRequestModel, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIOperationName: {Name: GenAIOperationName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIToolName: {Name: GenAIToolName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIAgentName: {Name: GenAIAgentName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIProviderName: {Name: GenAIProviderName, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeString},
GenAIUsageInputTokens: {Name: GenAIUsageInputTokens, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},
GenAIUsageOutputTokens: {Name: GenAIUsageOutputTokens, Signal: SignalTraces, FieldContext: FieldContextAttribute, FieldDataType: FieldDataTypeFloat64},

View File

@@ -1,16 +1,5 @@
from datetime import datetime, timedelta
from http import HTTPStatus
from fixtures import types
from fixtures.querier import (
Aggregation,
BuilderQuery,
OrderBy,
RequestType,
TelemetryFieldKey,
get_scalar_table_data,
make_query_request,
)
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
@@ -43,10 +32,10 @@ def ai_trace(
*,
now: datetime,
service: str,
user: str,
in_tokens: int | None,
out_tokens: int,
user: str = "user",
cost: float = 0.1,
cost: float,
model: str = "gpt-4o-mini",
environment: str = "production",
) -> list[Traces]:
@@ -85,28 +74,6 @@ def ai_trace(
]
def tool_only_trace(*, now: datetime, service: str) -> list[Traces]:
"""Root + one tool span: passes the gen_ai gate but has NO LLM span."""
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service}
return [
root_span(now=now, trace_id=trace_id, span_id=root_id, resources=resources, duration_s=2),
Traces(
timestamp=now - timedelta(seconds=4),
duration=timedelta(seconds=0.5),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="execute_tool",
kind=TracesKind.SPAN_KIND_INTERNAL,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"gen_ai.tool.name": "get_weather", "gen_ai.tool.type": "function"},
),
]
def ai_trace_mixed_spans(*, now: datetime, service: str, user: str) -> list[Traces]:
"""Root + LLM + tool + agent spans; only the LLM span carries gen_ai.request.model."""
trace_id = TraceIdGenerator.trace_id()
@@ -149,48 +116,3 @@ def ai_trace_mixed_spans(*, now: datetime, service: str, user: str) -> list[Trac
),
child("agent.step", TracesKind.SPAN_KIND_INTERNAL, {"gen_ai.agent.name": "chat-agent"}, 2),
]
def ai_aggregation_query(
service: str,
expression: str,
*,
filter_extra: str = "",
group_by: list[TelemetryFieldKey] | None = None,
alias: str | None = None,
having: str | None = None,
order: list[OrderBy] | None = None,
limit: int | None = None,
step_interval: int | None = None,
) -> dict:
filter_expression = f"service.name = '{service}'"
if filter_extra:
filter_expression += f" AND {filter_extra}"
return BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=filter_expression,
aggregations=[Aggregation(expression=expression, alias=alias)],
group_by=group_by,
having_expression=having,
order=order,
limit=limit,
step_interval=step_interval,
).to_dict()
def scalar_value(signoz: types.SigNoz, token: str, start_ms: int, end_ms: int, service: str, expression: str, filter_extra: str = "") -> float:
"""The single cell of a one-aggregation, ungrouped scalar query."""
response = make_query_request(
signoz,
token,
start_ms,
end_ms,
[ai_aggregation_query(service, expression, filter_extra=filter_extra)],
request_type=RequestType.SCALAR,
)
assert response.status_code == HTTPStatus.OK, f"{expression}: {response.text}"
data = get_scalar_table_data(response.json())
assert len(data) == 1, f"{expression}: expected one row, got {data}"
return float(data[0][-1])

View File

@@ -1,4 +1,116 @@
{
"note": "Divergences of the CURRENT promql serving path from the upstream reference engine, enforced exactly by 01_upstream_corpus.py in both directions. These document shipped defects, not test debt: the dominant class is the v1 remote-read fetch injecting a synthetic 'fingerprint' label into every series (pkg/prometheus/clickhouseprometheus/json.go), which breaks without() grouping and default vector matching. Entries must be REMOVED as the serving path is fixed or swapped.",
"divergences": {}
}
"note": "Divergences of the CURRENT promql serving path from the upstream reference engine, enforced exactly by 01_upstream_corpus.py in both directions. These document shipped defects, not test debt: the dominant class is the v1 remote-read fetch injecting a synthetic 'fingerprint' label into every series (pkg/prometheus/clickhouseprometheus/json.go), which breaks without() grouping and default vector matching. Entries must be REMOVED as the serving path is fixed or swapped. Second class, and the bulk of the entries below: promql_query.go drops NaN and +/-Inf from results, mirroring the builder path in consume.go, so every case whose expected output carries a non-finite value diverges on both legs. That class is a product decision rather than a defect, so it is not part of the burn-down.",
"divergences": {
"aggregators.test:630[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:630[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:633[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:633[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:636[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:636[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:639[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:639[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:642[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:642[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:645[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:645[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:648[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:648[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:661[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:661[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:698[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:698[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:702[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:702[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:706[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:706[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:710[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:710[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:714[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:714[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:717[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:717[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:720[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:720[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:724[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:724[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:862[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:862[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:865[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:865[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:868[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:868[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:873[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:873[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:885[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:885[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:888[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:888[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:891[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:891[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:896[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:896[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:906[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:906[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:909[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:909[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:919[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:919[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:922[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:922[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:925[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:925[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:930[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:930[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:942[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:942[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:945[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:945[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:948[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:948[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:953[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:953[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:963[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:963[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:966[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:966[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"operators.test:533[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"operators.test:533[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"operators.test:539[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"trig_functions.test:13[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:13[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:18[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:18[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:23[base]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:23[instant-coarse]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:28[base]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:28[instant-coarse]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:33[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:33[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:38[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:38[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:43[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:43[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:48[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:48[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:53[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:53[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:58[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:58[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:63[base]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:63[instant-coarse]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:68[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:68[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:73[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:73[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:78[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:78[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:83[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:83[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:88[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:88[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:8[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:8[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:93[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:93[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing"
}
}

View File

@@ -1,17 +1,128 @@
{
"note": "Divergences of the clickhousev2 provider (pinned via X-SigNoz-PromQL-Provider) from the upstream reference engine, enforced exactly by 01_upstream_corpus.py in both directions. This ledger is the rollout scorecard for the provider swap: the default provider cannot be replaced by clickhousev2 while anything is listed here. Entries must carry the defect's cause and be REMOVED as the provider is fixed. Current class: the engine aggregates floats with Kahan compensated summation (sum, sum_over_time) and an overflow-free incremental mean (avg); ClickHouse's sumForEach/avgForEach/arraySum are naive, so extreme-magnitude corpus data (±1e100 cancellation, ±1.8e308 overflow) diverges on transpiled plans. Burn-down candidates: sumKahanForEach for the cancellation class; the overflow class needs an incremental-mean aggregate ClickHouse does not have.",
"note": "Divergences of the clickhousev2 provider (pinned via X-SigNoz-PromQL-Provider) from the upstream reference engine, enforced exactly by 01_upstream_corpus.py in both directions. This ledger is the rollout scorecard for the provider swap: the default provider cannot be replaced by clickhousev2 while anything is listed here. Entries must carry the defect's cause and be REMOVED as the provider is fixed. Current class: the engine aggregates floats with Kahan compensated summation (sum, sum_over_time) and an overflow-free incremental mean (avg); ClickHouse's sumForEach/avgForEach/arraySum are naive, so extreme-magnitude corpus data (±1e100 cancellation, ±1.8e308 overflow) diverges on transpiled plans. Burn-down candidates: sumKahanForEach for the cancellation class; the overflow class needs an incremental-mean aggregate ClickHouse does not have. Second class, and the bulk of the entries below: promql_query.go drops NaN and +/-Inf from results, mirroring the builder path in consume.go, so every case whose expected output carries a non-finite value diverges on both legs. That class is a product decision rather than a defect, so it is not part of the burn-down.",
"divergences": {
"aggregators.test:630[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:630[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:633[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:633[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:636[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:636[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:639[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:639[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:642[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:642[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:645[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:645[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:648[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:648[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:651[base]": "avg over near-max-float64 values: engine's incremental mean never forms the overflowing sum; avgForEach sums then divides, overflowing to +Inf",
"aggregators.test:651[instant-coarse]": "same as aggregators.test:651[base] on the coarse-step grid variant",
"aggregators.test:654[base]": "avg over near-min-float64 values: engine's incremental mean never forms the overflowing sum; avgForEach overflows to -Inf",
"aggregators.test:654[instant-coarse]": "same as aggregators.test:654[base] on the coarse-step grid variant",
"aggregators.test:661[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:661[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:687[base]": "sum over {1e100, -1e100, small}: engine uses Kahan compensated summation; sumForEach's naive summation loses the small terms to cancellation and returns 0",
"aggregators.test:687[instant-coarse]": "same as aggregators.test:687[base] on the coarse-step grid variant",
"aggregators.test:695[base]": "avg over {1e100, -1e100, small}: same Kahan-vs-naive cancellation as aggregators.test:687, divided by count",
"aggregators.test:695[instant-coarse]": "same as aggregators.test:695[base] on the coarse-step grid variant",
"aggregators.test:698[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:698[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:702[base]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:702[instant-coarse]": "expected output is entirely Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:706[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:706[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:710[base]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:710[instant-coarse]": "expected output is entirely -Inf; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:714[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:714[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:717[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:717[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:720[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:720[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:724[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:724[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:862[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:862[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:865[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:865[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:868[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:868[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:873[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:873[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:885[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:885[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:888[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:888[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:891[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:891[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:896[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:896[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:906[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:906[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:909[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:909[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:919[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:919[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:922[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:922[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:925[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:925[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:930[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:930[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:942[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:942[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:945[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:945[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:948[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:948[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:953[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:953[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"aggregators.test:963[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:963[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:966[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"aggregators.test:966[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"functions.test:1084[instant-coarse]": "sum_over_time over a window containing ±1e100: the disjoint coarse-step form's arraySum slide is naive summation, cancelling to 0 (the base variant's W>64 shape falls back to the engine and is exact)",
"functions.test:1087[instant-coarse]": "avg_over_time, same window and cancellation as functions.test:1084[instant-coarse]",
"functions.test:1149[base]": "avg_over_time over ±2.258e220-magnitude samples: engine's Kahan-compensated incremental mean cancels exactly to 0; the bucketed form's naive slide summation leaves a ~1e202 residue",
"functions.test:1149[instant-coarse]": "same as functions.test:1149[base] through the disjoint coarse-step form"
"functions.test:1149[instant-coarse]": "same as functions.test:1149[base] through the disjoint coarse-step form",
"operators.test:533[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"operators.test:533[instant-coarse]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"operators.test:539[base]": "expected output is entirely NaN; promql_query.go drops non-finite values, so no series is emitted",
"trig_functions.test:13[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:13[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:18[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:18[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:23[base]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:23[instant-coarse]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:28[base]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:28[instant-coarse]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:33[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:33[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:38[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:38[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:43[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:43[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:48[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:48[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:53[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:53[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:58[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:58[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:63[base]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:63[instant-coarse]": "2 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:68[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:68[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:73[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:73[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:78[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:78[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:83[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:83[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:88[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:88[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:8[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:8[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:93[base]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing",
"trig_functions.test:93[instant-coarse]": "1 of 3 expected points are NaN; promql_query.go drops non-finite values, so those timestamps are missing"
}
}

View File

@@ -0,0 +1,65 @@
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
from uuid import uuid4
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.metrics import Metrics
from fixtures.querier import get_all_series, make_query_request
HOUR_MS = 3_600_000
SAMPLE_INTERVAL_MS = 60_000
def test_promql_ratio_with_zero_denominator_is_dropped_and_cached(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
) -> None:
# 12h ending on an hour boundary 15m ago — old enough to be cached.
end_ms = (int((datetime.now(tz=UTC) - timedelta(minutes=15)).timestamp() * 1000) // HOUR_MS) * HOUR_MS
start_ms = end_ms - 12 * HOUR_MS
sum_metric = f"job_duration_sum_{uuid4().hex[:8]}"
count_metric = f"job_duration_count_{uuid4().hex[:8]}"
# active_job divides finite; idle_job is 0/0 at every step.
series = {"active_job": (100.0, 4.0), "idle_job": (0.0, 0.0)}
metrics: list[Metrics] = []
for job_name, (sum_value, count_value) in series.items():
for ts_ms in range(start_ms, end_ms + 1, SAMPLE_INTERVAL_MS):
timestamp = datetime.fromtimestamp(ts_ms / 1000, tz=UTC)
metrics.append(Metrics(metric_name=sum_metric, labels={"job_name": job_name}, timestamp=timestamp, value=sum_value))
metrics.append(Metrics(metric_name=count_metric, labels={"job_name": job_name}, timestamp=timestamp, value=count_value))
insert_metrics(metrics)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
promql = f"sum by (job_name) ({sum_metric}) / sum by (job_name) ({count_metric})"
def run() -> tuple[dict[str, dict[int, object]], int]:
query = {"type": "promql", "spec": {"name": "A", "query": promql}}
response = make_query_request(signoz, token, start_ms, end_ms, [query], no_cache=False)
assert response.status_code == HTTPStatus.OK, response.text[:300]
body = response.json()
out: dict[str, dict[int, object]] = {}
for entry in get_all_series(body, "A") or []:
labels = {l["key"]["name"]: str(l["value"]) for l in entry.get("labels") or []}
out[labels["job_name"]] = {v["timestamp"]: v["value"] for v in entry.get("values") or []}
return out, int(body["data"]["meta"]["stepIntervals"]["A"])
# First populates the cache, second must be served from it.
first, step_seconds = run()
second, _ = run()
expected_points = (end_ms - start_ms) // (step_seconds * 1000) + 1
assert set(first) == {"active_job"}, f"the 0/0 series must not reach the response: {sorted(first)}"
assert set(first["active_job"].values()) == {25.0}, sorted(set(first["active_job"].values()))
assert len(first["active_job"]) == expected_points, f"expected {expected_points} points, got {len(first['active_job'])}"
# The cached read excludes end_ms, the one legitimate difference.
assert set(second) == set(first), sorted(second)
for job_name, points in first.items():
expected = {ts: value for ts, value in points.items() if ts < end_ms}
assert second[job_name] == expected, f"{job_name}: got {len(second[job_name])} of {len(expected)} points"

View File

@@ -72,8 +72,8 @@ def test_ai_list_having_aggregate_filter(
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""One filter box splits into WHERE + HAVING; bare and `trace.` spellings behave
identically; an output-only aggregate is rejected."""
"""Span + aggregate condition in one filter box splits into WHERE + HAVING; bare
and `trace.` spellings behave identically; an output-only aggregate is rejected."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-having"
@@ -322,8 +322,9 @@ def test_ai_list_nested_group_span_or_and_aggregate(
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A nested (span OR span) group ANDed with an aggregate must not flatten: span
predicates go to WHERE, the aggregate to HAVING."""
"""service.name = X AND (has_error = true OR gen_ai.request.model = 'gpt-4o') AND
total_tokens > 100: the nested OR group must not flatten, span predicates go to
WHERE, the aggregate to HAVING."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-nested"

View File

@@ -1,528 +0,0 @@
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.querier import (
Aggregation,
BuilderQuery,
OrderBy,
RequestType,
TelemetryFieldKey,
get_all_series,
get_scalar_columns,
get_scalar_table_data,
get_series_values,
make_query_request,
)
from fixtures.querierai import ai_aggregation_query, ai_trace, query_window, scalar_value, tool_only_trace
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
def test_ai_scalar_trace_level_aggregations(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Scalars over per-trace values, and the bare-key span domain through the same request type."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-scalar"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def value(expression: str) -> float:
return scalar_value(signoz, token, start_ms, end_ms, service, expression)
assert value("avg(trace.output_tokens)") == pytest.approx(200)
assert value("count(trace.trace_id)") == 2
assert value("max(trace.total_tokens)") == pytest.approx(330)
assert value("p50(trace.output_tokens)") == pytest.approx(200) # AggreFuncMap -> quantile(0.50)
# arithmetic inside one function and between functions
assert value("avg(trace.output_tokens + trace.input_tokens)") == pytest.approx(220)
assert value("sum(trace.output_tokens)/count(trace.trace_id)") == pytest.approx(200)
assert value("count()") == 2 # the two LLM spans; roots are not gen_ai
assert value("sum(gen_ai.usage.output_tokens)") == pytest.approx(400)
# multiple trace-level aggregations in one query -> one column per aggregation
multi = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)"), Aggregation(expression="count(trace.trace_id)")],
)
resp = make_query_request(signoz, token, start_ms, end_ms, [multi.to_dict()], request_type=RequestType.SCALAR)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and [float(v) for v in data[0]] == [pytest.approx(200), 2], data
def test_ai_scalar_trace_level_filter_qualifies_traces(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A trace-level condition qualifies whole traces before aggregation, on both domains."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-qualify"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
for expression in (
"sum(trace.output_tokens)", # native trace-domain path
"sum(gen_ai.usage.output_tokens)", # delegated span-domain path (__trace_scope)
):
got = scalar_value(signoz, token, start_ms, end_ms, service, expression, filter_extra="trace.output_tokens > 100")
assert got == pytest.approx(300), expression
# the qualification also constrains delegated (span-domain) time series
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
ai_aggregation_query(
service,
"sum(gen_ai.usage.output_tokens)",
filter_extra="trace.output_tokens > 100",
step_interval=60,
)
],
request_type=RequestType.TIME_SERIES,
)
assert resp.status_code == HTTPStatus.OK, resp.text
assert [v["value"] for v in get_series_values(resp.json(), "A")] == [pytest.approx(300)]
def test_ai_scalar_group_by_model(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Trace-level aggregation grouped by a span attribute."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-groupby"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[ai_aggregation_query(service, "avg(trace.output_tokens)", group_by=[TelemetryFieldKey(name="gen_ai.request.model")])],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
by_model = {row[0]: float(row[-1]) for row in data}
assert by_model == {"gpt-4o": pytest.approx(200), "gpt-4o-mini": pytest.approx(50)}, data
def test_ai_scalar_group_by_intrinsic_span_column(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Grouping by an intrinsic must not alias the group column to the span column it reads
(`toString(name) AS name` is a cyclic alias ClickHouse rejects)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-groupby-intrinsic"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=10, out_tokens=300) + tool_only_trace(now=now, service=service))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
ai_aggregation_query(
service,
"count(trace.trace_id)",
group_by=[TelemetryFieldKey(name="name")],
order=[OrderBy(key=TelemetryFieldKey(name="name"), direction="asc")],
)
],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
columns = get_scalar_columns(resp.json())
assert columns[0]["name"] == "name", columns
data = get_scalar_table_data(resp.json())
# the root spans are gated out, so each trace groups under its gen_ai span name
assert [(row[0], int(row[-1])) for row in data] == [("chat gpt-4o-mini", 2), ("execute_tool", 1)], data
def test_ai_timeseries_trace_level_aggregation(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-ts"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
# all spans fall in one step bucket
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[ai_aggregation_query(service, "avg(trace.output_tokens)", step_interval=60)],
request_type=RequestType.TIME_SERIES,
)
assert resp.status_code == HTTPStatus.OK, resp.text
assert [v["value"] for v in get_series_values(resp.json(), "A")] == [pytest.approx(200)]
def test_ai_timeseries_top_n_groups(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A grouped, limited time series ranks groups on whole-window per-trace values in
the requested order."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-topn"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=100, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def top_series(order: list[OrderBy] | None) -> dict:
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
ai_aggregation_query(
service,
"sum(trace.output_tokens)",
group_by=[TelemetryFieldKey(name="gen_ai.request.model")],
alias="total_out",
order=order,
limit=1,
step_interval=60,
)
],
request_type=RequestType.TIME_SERIES,
)
assert resp.status_code == HTTPStatus.OK, resp.text
series = get_all_series(resp.json(), "A")
assert len(series) == 1, f"limit=1 must keep exactly one group, got {len(series)} series"
return series[0]
top = top_series(None) # default ranking: first aggregation desc
assert top["labels"][0]["value"] == "gpt-4o", top["labels"]
assert [v["value"] for v in top["values"]] == [pytest.approx(400)]
bottom = top_series([OrderBy(key=TelemetryFieldKey(name="total_out"), direction="asc")])
assert bottom["labels"][0]["value"] == "gpt-4o-mini", bottom["labels"]
assert [v["value"] for v in bottom["values"]] == [pytest.approx(50)]
def test_ai_timeseries_limit_without_group_by(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A time-series limit without group-by has nothing to rank and is ignored."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-limit-nogroup"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=10, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[ai_aggregation_query(service, "avg(trace.output_tokens)", limit=1, step_interval=60)],
request_type=RequestType.TIME_SERIES,
)
assert resp.status_code == HTTPStatus.OK, resp.text
assert [v["value"] for v in get_series_values(resp.json(), "A")] == [pytest.approx(200)]
def test_ai_scalar_group_order_limit(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Scalar limit is a plain top-N over the grouped rows."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-scalar-limit"
insert_traces(
ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o")
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=100, model="gpt-4o")
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini")
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=10, model="gpt-4")
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
ai_aggregation_query(
service,
"sum(trace.output_tokens)",
group_by=[TelemetryFieldKey(name="gen_ai.request.model")],
alias="total_out",
order=[OrderBy(key=TelemetryFieldKey(name="total_out"), direction="desc")],
limit=2,
)
],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert [(row[0], float(row[-1])) for row in data] == [("gpt-4o", pytest.approx(400)), ("gpt-4o-mini", pytest.approx(50))], data
def test_ai_timeseries_span_time_bucketing(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Per-trace values are clipped per (bucket, trace), so a trace spanning two buckets
contributes each call's tokens to its own bucket, not the total to both."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-buckets"
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service}
def llm(offset_s: float, out_tokens: int) -> Traces:
return Traces(
timestamp=now - timedelta(seconds=offset_s),
duration=timedelta(seconds=1),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="chat",
kind=TracesKind.SPAN_KIND_CLIENT,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"gen_ai.request.model": "gpt-4o-mini", "gen_ai.usage.output_tokens": out_tokens},
)
root = Traces(
timestamp=now - timedelta(seconds=130),
duration=timedelta(seconds=130),
trace_id=trace_id,
span_id=root_id,
parent_span_id="",
name="POST /api/chat",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"http.request.method": "POST"},
)
# two LLM calls two minutes apart
insert_traces([root, llm(124, 100), llm(4, 300)])
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[ai_aggregation_query(service, "avg(trace.output_tokens)", step_interval=60)],
request_type=RequestType.TIME_SERIES,
)
assert resp.status_code == HTTPStatus.OK, resp.text
series = get_all_series(resp.json(), "A")
assert len(series) == 1, series
assert sorted(v["value"] for v in series[0]["values"]) == [pytest.approx(100), pytest.approx(300)], series
def test_ai_scalar_variables_in_trace_level_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Variables resolve inside trace-level conditions with span-filter semantics."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-vars"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
query = ai_aggregation_query(service, "sum(trace.output_tokens)", filter_extra="trace.output_tokens > $threshold")
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[query],
request_type=RequestType.SCALAR,
variables={"threshold": {"type": "text", "value": 100}},
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and float(data[0][-1]) == pytest.approx(300), data
# an unresolvable $var is a 400 today via aggregate validation
resp = make_query_request(signoz, token, start_ms, end_ms, [query], request_type=RequestType.SCALAR)
assert resp.status_code == HTTPStatus.BAD_REQUEST, resp.text
# quotes in the message are JSON-escaped, so match the halves separately
assert "$threshold" in resp.text and "cannot be used in a trace-level filter" in resp.text, resp.text
# a dynamic variable resolved to __all__ drops the condition (both traces count)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[query],
request_type=RequestType.SCALAR,
variables={"threshold": {"type": "dynamic", "value": "__all__"}},
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and float(data[0][-1]) == pytest.approx(400), data
def test_ai_scalar_tool_only_trace_null_semantics(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A tool-only trace (in the gate, no LLM span) follows plain SQL NULL semantics."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-toolonly"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + tool_only_trace(now=now, service=service))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def value(expression: str, filter_extra: str = "") -> float:
return scalar_value(signoz, token, start_ms, end_ms, service, expression, filter_extra)
assert value("count(trace.trace_id)") == 2, "tool-only trace is an AI trace and must be counted"
assert value("avg(trace.output_tokens)") == pytest.approx(100), "NULL tokens are skipped by avg"
assert value("avg(trace.tool_call_count)") == pytest.approx(0.5), "tool-only trace feeds tool aggregates (1 and 0 calls)"
assert value("count()") == 2, "span-level count sees the LLM and the tool span"
# filtering on LLM activity is explicit, not implicit
assert value("count(trace.trace_id)", filter_extra="trace.llm_call_count > 0") == 1
def test_ai_scalar_having_on_aggregation(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""The outer having filters aggregation results per group (by alias)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-having"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
ai_aggregation_query(
service,
"avg(trace.output_tokens)",
group_by=[TelemetryFieldKey(name="gen_ai.request.model")],
alias="avg_out",
having="avg_out > 100",
)
],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and data[0][0] == "gpt-4o", data
def test_ai_aggregation_rejections(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-reject"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def expect_bad_request(query: dict, message: str) -> None:
resp = make_query_request(signoz, token, start_ms, end_ms, [query], request_type=RequestType.SCALAR)
assert resp.status_code == HTTPStatus.BAD_REQUEST, resp.text
assert message in resp.text, resp.text
# span-level and trace-level aggregations cannot be mixed in one query
mixed = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)"), Aggregation(expression="count()")],
)
expect_bad_request(mixed.to_dict(), "cannot be mixed")
expect_bad_request(
ai_aggregation_query(service, "avg(trace.output_tokens)", group_by=[TelemetryFieldKey(name="trace.llm_call_count")]),
"grouping by trace-level aggregate",
)
# a bare per-trace column would emit one row per trace instead of one aggregated row
expect_bad_request(ai_aggregation_query(service, "trace.output_tokens"), "must be inside an aggregation function")
# the rate interval divides the whole expression, so it may not carry a second aggregation
expect_bad_request(ai_aggregation_query(service, "rate(trace.trace_id) + avg(trace.output_tokens)"), "combines a rate with another aggregation")
# order-by is stopped earlier, by request validation
expect_bad_request(
ai_aggregation_query(service, "avg(trace.output_tokens)", order=[OrderBy(key=TelemetryFieldKey(name="trace.total_tokens"), direction="desc")]),
"invalid order by key",
)

View File

@@ -1,32 +0,0 @@
import pytest
from testcontainers.core.container import Network
from fixtures import types
from fixtures.signoz import create_signoz
@pytest.fixture(name="signoz", scope="package")
def signoz_ai_observability(
network: Network,
migrator: types.Operation, # pylint: disable=unused-argument
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
# without the flag the gate keys only resolve once a span carrying them is ingested
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
cache_key="signoz-ai-observability",
env_overrides={
"SIGNOZ_FLAGGER_CONFIG_BOOLEAN_ENABLE__AI__OBSERVABILITY": True,
},
)